I think I figured it out! I have f.lux installed, which tints the entire screen to reduce the amount of blue light it emits at night. The method that f.lux uses to tint the screen doesn't affect the mouse cursor, so the cursor looks brighter and slightly bluer than everything else on the screen- it can be pretty annoying.
In the settings for f.lux, there is an option to fix this problem labeled "Software mouse cursor when needed (fixes 'white cursor')". I assume that this option is implemented by hiding the system cursor and replacing it with a lookalike "cursor" controlled by f.lux that can be properly rendered behind the tint layer, rather than on top of it.
I'm guessing that f.lux's software cursor renders behind Godot's fullscreen layer. Disabling the software cursor allows both the system cursor and custom cursors to be visible in fullscreen Godot projects, though sadly, neither are affected by f.lux's tint.
This seems likely to be an edge case that the devs just haven't considered before, but just in case: are there any Windows users who have f.lux installed, and can see the cursor in fullscreen even when the software cursor is enabled and the screen is tinted? If not, I guess it would be best for me to submit compatibility with the f.lux software cursor as a feature request, rather than a bug report.