So I found a workaround, for those future duckduckGo'ers:
Use the gridContainer, add the child nodes and set up spacing etc. first.
Then I changed gridContainer type to "control" by right clicking the node from the scene list.. This just inherited all of gridContainers spacings without rearranging everything when something becomes hidden or removed.