Accessing your self-hosted web server

Introduction

Something that has been bothering me for a while is that if you run your own self-hosted website you cannot see it on computers on your own local network without editing the hosts file. To me it seems simple, you type in the website address, it goes off to the DNS servers that resolve the address which then calls up the web server and serves the page to the computer you are on. Instead you get a message saying "This site can’t be reached" or something similar, and so you have to edit the hosts file or find another solution.

How the internet Works

Here we are only concerned with the World Wide Web (WWW) not hte rest of the traffic that uses it such as email, instant messaging or anything else.

Networks, and the WWW is just a giant network, uses Internet Protocol (IP) addresses to identify devices on it. Your Internet Service Provider (ISP) has a pool of IP addresses ultimately assigned to it by the Internet Assigned Numbers Authority (IANA). You need at least a modem to connect your oomputer to your ISP. That's all they do, they are fairly dumb machines and can only connect one device to the ISP. Once connected, your ISP will assign an Internet Protocol (IP) address from a Dynamic Host Configuration Protocol (DHCP) server so that you can connect to their network. With a single computer connected to it, that computer gets the issued IP address.

Who now has just a single computer connected to their modem? A router is connected to the modem and that gets the IP address issued by the ISP. That is its external IP address. It also has an internal IP address. This is the address that computers that use it know it by. Routers also have their own DHCP server and issue addresses to the devices attached to it in the range of 192.168.0.0 to 192.168.255.255.

What the router is doing is creating a smaller network, a Local Area Network (LAN), with each device on it having its own IP address. It's then connecting this LAN to the wider WWW using the IP address provided by the ISP. It also keeps track of and controls the traffic beteen the two - it's why it is called a router. Unlike a modem, a router is a clever device.

Names like Google.com or even brisray.com mean nothing to how the internet actually works, what the routing of traffic around the internet needs are the IP addresses of the devices connected to it. This is why no two devices on a network can have the same IP address. When you type brisray.com into your browser' address bar what happens is that the request goes to a Domain Name System (DNS) server where brisray.com is converted to its IP address so it can be found.

As I run my own server which is part of my own LAN, the router knows which of my computers is the web server and routes the traffic to that, which then returns the requested web page.

It's a wonder the internet works at all!

You Can't Access Your Own Self-Hosted Website

So you go through the process of installing a web server, creating a website and doing everything else required to run properly. You view the website on the computer it's on by typing 127.0.0.1 into the browser address bar. 127.0.0.1 also known as localhost, is a computer loopback address looking for a server on that computer. If properly set you will see your website's homepage or at least the web server's splash screen. If the computer does not have a server then all you'll see is a message saying "This site can’t be reached" or something similar.

127.0.0.1

It's an odd situation. You know your website is up and running because sites such as Down for Everyone or Just Me, Freshping, and Is It Down Right Now all say it is up. You can even see it on sites such GeoPeeker, LocaBrowser, and WebPageTest, but no computer on your LAN can see it.

While checking these sites it is also worthwhile checking which ports are open on your server using Shields UP!. Only ports your are using should be open, all others should be blocked or not even discoverable.

What's Going On?

These could be several explanations as to what is happening, the ones I know of are covered in a single Stack Exchange post.

It's Your ISP

One explanation I have seen is that most ISPs do not usually allow outgoing connections from a public IP back to the same address. I couldn't find anything definite on this but I suppose they do this to protect their resources.

Most residential ISPs say they do not allow servers on any kind on their accounts. In practice, I've found they seem not to mind but perhaps some do and not allowing loopbacks like this may be part of enforcing their no server rule. It may be also part of their Distributed Denial-of-Service (DDoS) defence. Imagine an attacking making endless request loops.

It's Your Router

According to a poster on Stack Exchange "it's basically impossible to have a packet exit the router and then reverse direction and go back in." A basic router simply drops the packets of data Oddly enough changing the configuration of the router is one of the solutions.

Router NAT Hairpinning

Many routers support Network Address Translation (NAT) hairpinning or NAT loopback. When you make your self-hosted web server public you need to port forward port 80 (HTTP) and port 443 (HTTPS) to your server. Port forwarding is also known as Destination Network Address Translation (DNAT).

What the router does during NAT hairpinning is check the destination address of the data being sent. If it matches the port forwarding rules the data is directed to the server.

Hosts File

A hosts file is found on almost every operating system and its main main purpose to to act as a kind local DNS lookup table - it enables the translation of host names to IP address numbers. It is a plain text file and the example given in the default hostsfile on Windows is:

102.54.94.97 rhino.acme.com

Location

In the latest versions of Windows the location of the hosts file is at Windows\System32\drivers\etc\host. In most versions of Linux it is located at /etc/hosts

Editing

The hosts file is a system file and such it it can be viewed as a normal user, but can only be edited by those with elevated permissions. I have found the easiest method on Windows is to open Noteapd as an administrator and then use the File menu to navigate to the file and open it that way for editing.

Suppose your site is named mysite.org and the server's IP address is 192.168.1.3, then using the example in the file, the new entry will be:

192.168.1.3 mysite.org

Multiple websites and subdomains can be added to the same IP address entry. This is an entry from my hosts file:

192.168.1.59 brisray.com hmsgambia.org ihor4x4.com www.brisray.com www.hmsgambia.org www.ihor4x4.com

More

The hosts file can also provide another useful service and that is redirecting unwanted host such as advertising sites, even if they are on a page you want to view to another IP address, usually 127.0.0.1 or 0.0.0.0

0.0.0.0 is a non-routable address used to designate an invalid, unknown or non applicable target.

There are sites that contain ready-made lists of sites for use in hosts files such as this one and the ones by Constantin Hofstetter, and Dan Pollock.

Problems

The problem with the hosts file is that they are local to each machine on the LAN. For a small network this is not a great problem as the file is easily copy and pasted between computers.

Microsoft provides the original hosts file text in case it gets damaged.

Your Own Local DNS Server

It is possible to run your own loal DNS server software, Wikipedia has a list of them. One not listed is Pi-Hole. Some are basic LAN IP address resolvers, others are a sort of distributed hosts file and others are capable of much more.

Sources and Resources

0.0.0.0 (Wikipedia)
Blocking Unwanted Connections with a Hosts File - Anti-ad hosts file
Comparison of DNS server software (Wikipedia)
Hosts (GitHub) - Anti-ad hosts file by Constantin Hofstetter
Hosts File (Wikipedia)
How to make the internet not suck (as much) - Anti-ad hosts file by Dan Pollock
Localhost (Wikipedia)
How to reset the Hosts file back to the default (Microsoft Support)
Network Address Translation (NAT Hairpinning) (Wikipedia)
Router doesn't allow loopback, can I use MS DNS to internally forward my .com to the local IP of the IIS server? (Stack Exchange)
Why can't I access my own web server from my local network? (Stack Exchange)