I don't think (someone correct me if I'm wrong) there is a way to update inside the editor. The Steam version will update just like any other game if you have auto updates enabled.
What I do is just re-download, unzip and use. Click the "scan" button when you launch Godot and browse to find your project at the location you saved it.
I recommend backing up your projects before editing them in a new version (duplicate the whole folder in the directory you saved it, give it a slightly different name for a fast easy way). 3.1 to 3.1.1 shouldn't have any real consequence, but it's still better to be safe. If the project was created in an earlier iteration of Godot like V3.0 then definitely back up because you will probably prompted to convert it to the new version and will likely break a few things, especially in script.