Ideally, you wouldn't have vast amounts of off-screen textures just for infinite scrolling. A better approach would be to have just enough texture "chunks" to fill the screen with a little off-screen buffer at both ends. Then, when scrolling, as soon as one of the chunks has scrolled out of view (typically, off the left-edge of the screen), just move it to the end of set of textures (so, off of the right-hand edge of the screen). Just repeat this each time a chunk is no longer viewable.
That way, you just need enough chunks to fill the screen, plus a bit of buffer and you still get infinite scrolling...
Specifically to your question of "is there a cost to all the off screen textures"... While I'm sure the answer is Yes, I can't really quantify what that cost might be...