First of all, when you use C#, compiled scripts are not referenced the same way as GDScript. C# uses a project file to list what goes into the assembly (this is general to C#, it's not just a Godot thing, look it up). I think Godot creates it if it does not exist, but it doesn't re-generates it everytime (otherwise it would loose any compilation settings you put in it). This project file is used by Mono to compile, and is very often used by IDEs and code editors like Visual Studio, Mono Develop, Rider or VSCode. Editing from Godot works but it's barely raw text edition.
When it first showed up it was named new_script.cs. I changed that name to Missile.cs and hit run.
This is not how C# works. Changing the name of the file does not change the class name inside it. That's two different things. You probably ended up with a file named Missile.cs but a class inside named new_class.
Now my error is: The namespace '' already contains a definition for 'new_script'
Because of the above, you now have two different files, but they both contain a class named new_script, which causes a conflict. It's a good practice in C# to make sure the class name in a file matches the name of the file. I think Godot also relies on this to find which class to instanciate (since you can have more than one class in a file).
Now it won't compile because it can't find new_script.cs
Check also your C# project (it's a .csproj file). You can do this by opening it with one of the editors I mentionned above. You can also check with a plain text editor (it's XML inside), and make sure it references your script. If not... add it, either from an IDE or by editing the XML. Again, C# is a bigger beast than GDScript and is best edited with an actual IDE.
If you don't want that complexity, just use GDScript.
It seems weird that I can't just delete files that aren't being used by any of my objects. I want to be able to delete files that I'm not using without such a big fuss.
Just delete the files then. You didn't delete any so far, only renamed them.