If you're hosting website on a specific port in IIS like 4321 then you'd have to allow this port through Windows Firewall too. Here're the steps that I followed along with the imanabidi's answer to get it work for me:
- Windows Firewall > Advanced settings
- Inbound Rules > New Rule
- Select Port > Next
- Specific local ports > Add the Port you want to allow
- Allow All Connections
- Enter a name and some description so that you remember it later on
- Done