The solution to your problem really depends on how you want to optimize your game. Since you simply want to generate a 2D terrain, I would recommend using a Camera2D, so that the "removal" of chunks will occur as the camera changes position (that is, as the chunks are no longer being rendered). I would also recommend that you avoid moving the world, as Zylann mentioned before. Instead, you could attach the aforementioned Camera2D to your player, so that it will follow your player and load chunks accordingly.
As for the world generation, I would recommend using a Perlin/Simplex noise algorithm, which are usually used for random terrain generation, as they can create an arbitrarily (essentially "infinite") sized "Grid" with random values. The specifics of how you render this Grid (i.e. color of chunk, items in chunk, etc) is really up to you, but you can find ready to use implementations of both algorithms. If you can build Godot from source, you could use Zylan's module (what a pleasant coincidence!). In case you have difficulties building Godot from Source, you can also use the PerduGames function set, which is written in GDScript, and should be easier to use in your project.
Whichever approach you decide to use, I wish you the best of luck!