From the docs:
"There is no dedicated server build for Windows yet. As an alternative, you can use the --no-window
command-line argument to prevent Godot from spawning a window. Note that even with the --no-window
command-line argument, you'll need to have OpenGL support available on the Windows machine." cite
MacOS and Linux headless builds are available, however.