I'm not a networking expert, but I'm kinda experiencing the same problems, since I'm also trying to make a local and online multiplayer game in Godot. Other people, correct me if I'm wrong.
You might wanna check these things:
1º - Port forwarding and firewall permission
2º - Hairpinning
You see, there are so many public IPs to be had with IPv4, specifically 2^32 possible addresses (theoretically). So, there's something called NAT to help with that: instead of giving public IPs to every device, you just give one to every router, and the routers act as middlemen between your local network and the Internet.
In your first example, you were able to connect to your own computer using your IPv4 private IP address. Since both the source and the destination for the packets (basically data) are located in your local network, the communication has no further problems.
But now, you're trying to connect to your own computer using the public IP address. The situation is different: your public IP isn't tied to your computer, it is the router's public IP. How would it know that you want the data to be sent to your computer, if you're not explicitly saying it?
That's where port forwarding becomes useful. You can use it to redirect a communication request from your router's public IP and port number to your computer's private IP and port number. That's something you gotta configure manually, in your router settings, or you can look at UPnP (Universal Plug and Play), which is basically an automated way of doing it.
If you have firewall enabled (probably), you will need to enable the communication for the chosen port too (open the port). I don't know if this is common, but in my router's web settings, there's a page for port forwarding and another one for firewall rules. If I add a port mapping, the firewall rules are automatically added, so I just have to map the port and it's basically done. You will need to check how your router works.
Lastly, for your example, you are trying to communicate from the local network to the local network, but leaving said network (you are using public IP, so you're "going" to the Internet). This is a specific situation, and not every router is able to establish that communication out of the box. You will need to check if your router supports hairpinning (NAT loopback). Also, it seems like some routers don't support it out of the box, but you can still make it work by messing with its settings.
I think the topics I've listed might answer why you aren't able to connect to your own computer using your public IP. For online connections (between different local networks), only the port forwarding and firewall permission will be relevant, the hairpinning is for your specific case.