Everytime you have issues due to a huge element in the game world, the solution is almost always about splitting it into chunks. Doing that takes advantage of the engine's culling system, so chunks not in view are not drawn, collision shapes far away are ignored, and the extents of it can nearly be infinite because it can be unloaded when too far away.
Note that if you are using a single texture for the whole thing, you will have a limit on some graphic cards above 4096x4096 (if you want to go that far), so if it's really huge, you can split it as well.