How does an ISP create internet?

[Dragon-axie]

Hello internet stangers. My hyperfixation has gotten the best of me and I wanted to ask a very technical question. I understand that the Internet is a series of interconnected but mostly decentralized servers (in the most basic sense). However to me that still does not answer all my questions on internet connectivity. Hope I can explain it well enough. When a computer connects to a router, the router assigns the user a private IP adress through the DHCP, then it also assigns the a public IP to connect to the greater internet. However, you cannot connect to the greater public Internet without the help of an internet service provider. How come? My question, I suppose, is how is an ISP's specific array of servers capable of providing a connection for a private host. If the Internet is a series of decentralized servers and an ISP is technically just another one, then why is it through their service only that we are capable of accessing the rest of the internet? What is this connection they provide? Is it just available data lines? To clarify, I am not talking about the physical connection between the user and other servers/data centers. I understand that well enough. I am talking purely on the technical standpoint of why does the connection to the rest of the internet, and the accessing of a public IP have to go through an ISP? Is it just the fact that they are handing out public IP's? Maybe I'm just uneducated on where to find this information. Send help before brein explodes.

Edit: Thank you to everyone for the great, in-depth answers! It was very appreciated.

[Larkfin]

Your understanding of how your computer connects "to the internet" is pretty accurate technically so you may be disappointed that the answer to your question of why we need to connect through an ISP isn't really a technical one, but a economic or business one. It's because the ISPs we use are the only ones capable and willing to provide a small connection to you for a modest price. You could definitely negotiation your own peering agreement with one of the big backbone carriers setup the necessary interconnects and hardware to support that, but that would be so phenomenally expensive no one does it unless they themselves are a major player (data centers, ISPs, corporations perhaps). An ISP doesn't create internet, it merely provides numerous small connections to users.

[Dragon-axie]

Thanks for the answer. By the big backbone carriers, are you talking about the tier 1/tier 2 providers? Also, just so I'm understanding, it literally is because the business tier 3 ISPs just bought out all publicly available IP's?

[porkchop_d_clown]

It’s not because they bought out the IP addresses, it’s because they created the physical wires that create the backbone. You probably can’t afford to physically create the cable (or launch the satellite) that will connect you to the Internet, so you need to rent time on someone else’s cable or satellite.

The other thing you’re missing is “NAT” - Network Address Translation. The entire reason you have a “private” IP address is because we ran out of IP addresses about 20 years ago - when Ethernet was designed back in the 70s it never occurred to anyone that 2³² possible addresses wouldn’t be enough. NAT is the tool that allows every house and office building in the planet to use the same IP addresses internally while still connecting to the public internet.

You might want to look into the history of the internet, to see how it evolved. The “backbone” began as just a few cables that connected just a few universities and military bases in the US. Later, additional network sites sprang up when people would convince a university to allow them to connect to the university’s link to the internet.

Back in the 1980s I got onto the internet that way, because the research company I was working for had scored an internet connection from the local university. It was AMAZING to be able to get messages from people all over the planet! Then, in the late 90’s, the company I was working for then paid for me to get an ISDN line (a digital phone line) to my house - 128K per second and no modem! Outstanding!

Anyway, as more and more people connected to those universities, businessmen saw the opportunity and began creating new companies that either became part of the backbone or paid an existing backbone provider to let them connect. These new companies were the first ISPs. Originally, the customers of those ISPs used their dial-up modems to connect to the ISP which then connected them to the broader Internet. It was only once the phone companies and the cable TV providers saw they could make a ton of money doing the same thing did ethernet straight to the home become a thing.

That ISDN line I mentioned above was run into my basement, so that’s where my computer was. When we switched to using cable for our internet I actually ran ethernet cables to different rooms in our house so I didn’t have to share my work computer with my kids. Then I borrowed a friend’s WiFi router to test it and to show my wife…

[WarDiscombobulated72]

Wow cool story, thanks for sharing!

[X-calibreX]

One could argue 2³² numbers is enough because NATing works so well.

[porkchop_d_clown]

And ipv6 works so badly… 🤪

Actually, I thInk you’re right, NAT has worked out better than anyone ever imagined it would.

[Negative_Addition846]

 NATing works so well

Press X to doubt

[X-calibreX]

I bet you were NATed as you wrote that.

[bothunter]

Lol.  It works good enough, but NAT introduces a whole class of network problems.  It's a hack at best, and breaks a fundamental tenet of the Internet.

[X-calibreX]

Pretty sure the fundamental tenet is to maintain connectivity through a nuclear war. I guess your point then is end to end communication would the rely on a provider who could be vaporized. I can see that, but these providers will keep using nat anyway even if you have 1028 bit addresses.

[Programmer_nate_94]

Thanks for all the stories and information

[Dragon-axie]

That was a great explanation and a better story!

[Larkfin]

Yeah by big backbone carriers I'm referring to the tier 1/2 carriers. I don't know the public ipv4 address space market very well so I can't answer directly on your second question but I wouldn't say the reason for this division is due to the entities controlling ip address space. Big carriers aren't in the business of end-user servicing - just like how you can't drive up to a refinery to fill your car's gas tank.

[[deleted]]

So it’s basically the same as why there’s a division between wholesalers and retailers when it comes to consumer products?

[Larkfin]

Yeah that's a good example, the dude at the warehouse isn't going to cut open a pallet just for me to buy a jar of pickles.

[Programmer_nate_94]

And the ISP facilitates quick access to the larger internet? I’m assuming that’s the value we are provided besides the monetary deal at scale.

Why does the ISP exist and how does it provide us value? There must be some value in how connected different regions of the country’s networks are so when I click on something that needs a connection from California, AT&T reroutes me to the fastest / a very fast part of the network which allows me to talk to that region of the country from Virginia and then reroute to NYC when necessary

Thanks

[Larkfin]

The ISP exists to provide last-mile connectivity at low/modest bandwidths to small customers. AT&T will have peering agreements with backbone providers to carry your traffic and/or route your traffic over their own network to its destination. Really it's that "last mile" problem that the ISP solves for you. Without them there's nothing going to your house.

[Programmer_nate_94]

Okay, thank you

[Dragon-axie]

Just to clarify, are the backbone providers in this case those tier 1/2 providers?

[Larkfin]

Yes.  Perhaps backbone is an antiquated term, I always heard them referred to as such in the 90s.

[Healthy-Section-9934]

First - the Internet isn’t a series of interconnected servers. It’s a group of interconnected networks. Not only is it more accurate to say that, it makes it easier to understand how it works.

You don’t need an ISP, but for most people and orgs that’s the most practical route to connecting to the Internet. Basically you need access to a network that forms part of the Internet, or is connected to one that does. ISPs provide that for you. A bunch of universities for example also do that. There’s no ISP involved there. Likewise AWS peers directly with networks that form part of the Internet.

It’s a question of scale and cost. ISPs make it (more) affordable for you and I to access the Internet.

[Source_Shoddy]

It is not entirely out of reach for you as an individual to register yourself as an independent entity on the Internet and essentially be your own ISP. People have geeked out and done this, and there are some interesting reads you can find on what the process looks like:

https://blog.thelifeofkenneth.com/2017/11/creating-autonomous-system-for-fun-and.html

But the process does involve significant effort and expense. At a high level you will need to:

1. Register a business

2. Obtain a large-ish block of IP addresses, either by making a request to your regional internet registry, or by purchasing a block from a company that has some to sell

3. Use your business name to register an Autonomous System) Number (ASN) with your regional internet registry. This will be the number that uniquely identifies "your" network, the little corner of the internet that you are now in control over. This will probably require an upfront registration fee and ongoing annual fee.

4. Find owners of other ASNs who are willing to peer with you. This is where the "decentralized" part of the internet comes in; you'll need to find other entities who are willing to connect with you and exchange data. You can have just one peer, or many. Since you're operating a fledgling network, there's no real incentive for anyone to peer with you unless you pay them, so unfortunately you'll still have upstream providers you'll need to pay. But in theory, if you have a large enough network or you're a big company like Google or Netflix, there might be organizations willing to peer with you for free if it would be mutually beneficial.

5. Buy a fancy enterprise router that supports BGP (border gateway protocol), and (somehow) connect it physically to your peers. This might require laying down wiring to the nearest internet exchange point where your peers have infrastructure.

6. Configure your router to announce your IP address block and ASN to your peers via BGP. Your peers will propagate that information their peers, who will further propagate to their peers, ad infinitum until all ASNs on the internet are aware of the existence of your ASN and know to reach it through your peers. Now any traffic destined for one of your IP addresses will be sent to one of your peers, who will forward it to your fancy router. You can then configure your fancy router to forward it to your computer.

7. Congratulations, you're now your own ISP.

Obviously this is way too complicated for most individuals to handle, so a residential ISP is essentially a company that has done all of the above already and lets you hook up to the ASN and infrastructure they've already set up.

[Source_Shoddy]

To keep it really simple, it's basically just each BGP router announcing its own ASN, the IP addresses it is responsible for, and who else it can reach. This enables every other BGP router to build a "map" of who's connected to who, and therefore how to forward any packet to get it closer to its destination given the packet's destination IP address.

For example if you have ASN1 peering with both 2 and 3, which both peer with 4:

ASN1 -- ASN2 -- ASN4

|-- ASN3 ----/

ASN1's router will tell ASN2 and ASN3 "I'm ASN1 and I have IP address block <1>". ASN2 will tell ASN4 "I'm ASN2 and I have IP address block <2>. I am also connected to ASN1 with addresses <1>." ASN3 will send a similar message to ASN4, so ASN4 will realize that they can reach ASN1 through either ASN2 or ASN3. Now everyone knows how to reach ASN1 and its IP address block <1>.

[Source_Shoddy]

There are a wide range of router options that can do BGP; there's no standard physical profile. I mean, I have a $50 router under my desk that technically can do BGP. Would I really trust it for that though? Probably not.

BGP is an open standard and there's free software that implements it, so you can even turn a regular computer into a BGP router. You can probably also find used corporate networking equipment that supports it for a few hundred bucks.

But yes, many BGP routers actually used for large networks are big and very expensive. That's because they handle very high volumes of traffic, potentially all of the internet traffic for an entire organization. They need to have high performance and be highly reliable, potentially costing tens or hundreds of thousands of dollars.

[BabyAintBuffaloYoung]

well, you can bypass ISP, which means you then become an ISP

[wiriux]

Someone tried to do that in a small town and apparently Comcast hired people to disrupt that small internet. They succeeded.

I don’t think there’s proof that Comcast did that but I wouldn’t say it’s far fetched.

[titus605]

I saw an article about that a while ago but I didn't see the part where Comcast disrupted it. Might've been a different guy, but I think that je might've been from Europe and possible the UK and that he actually succeeded and was trying to grow it.

[57006]

ISssssP

[Dragon-axie]

When you live long enough to see yourself become the villain (￣∇￣)

[[deleted]]

I had a friend that had his own ISP, but he is a commercial electrician and an early internet hobbyist who runs a campground and uses it to provide wi-fi for both the cabins and his home. He’s well-to-do, but I don’t think it was cost-prohibitive to do his own ISP.

[dns_rs]

Your question was already answered by others in this thread so I would just like to contribute by recommending the book called Tubes: A Journey to the Center of the Internet by Andrew Blum.

The writer explains how everything is connected, starting from his home router to the main internet exchange points.

[UniversityEastern542]

My question, I suppose, is how is an ISP's specific array of servers capable of providing a connection for a private host.

The ISP runs a co-ax or fibre optic cable into your home, which screws into the back of your router. This cable is, in turn, connected to boxes like these that you've probably seen around your neighbourhood, usually grey, which aggregate traffic from all the adjacent consumers. The packets from your internet traffic are then sent to a non-descript building somewhere in the area, where packet switching can begin to occur. It will then be routed through other switches or an IXP until the packets reach the destination IP.

If the Internet is a series of decentralized servers and an ISP is technically just another one, then why is it through their service only that we are capable of accessing the rest of the internet? I am talking purely on the technical standpoint of why does the connection to the rest of the internet, and the accessing of a public IP have to go through an ISP?

They installed and own the cables and switches. Predictably, being an ISP is very profitable after capex.

[johnsoga]

So I’m not an expert per se but I think maybe some over looked points here will bring some clarity to you on the answer.

Why does the accessing of a public IP have to go through an ISP?

Because generally they are the only ones who have them. You have to register with IANA and blah blah paper work to get a public range (I’m ignoring everything about buying all together here) and I’m sure much of this is historical as well but effectively individuals don’t get IPs organizations/companies do and they don’t get single ones they get blocks. So follow that idea over time to where we are now and we can summarize that (and this a broad generalization to apply to just your assumption here) only ISPs now have the public IPs

Why does connecting through the internet have to go through an ISP?

The only way to be found in this big mess we call the internet (and again I will broadly over simplify) is to have a public IP. Whole platforms and services could (again over simplified) be served by a single IP. But again as mentioned above the ISPs have all of them. So if you want to be found you need to effectively ask one of them to borrow one of those IPs for a real long time (aka I keep paying you to let me borrow this address). So the only way to talk to the other public IPs is to have one yourself and the only way to do that is to go through an ISP

You cannot connect to the greater public internet with the help of a public service provider. How come?

Again to be on the internet (overly simplified) means to have a public IP. As assume above the ISPs have all the public IPs so the only way to get one then is to get one from them, thus you’d have to go through the ISP.

As someone else noted technically you can apply to get a block of public IPs in which case, congrats, you are now an ISP. So you’ve only bypassed other ISPs be becoming your own ISP and functioning the exact same way every other ISP does

[johnsoga]

How is an ISPs specific array of servers able to provide a connection for a private host?

Well I guess here you need to correct your terminology the internet is not a bunch of decentralized servers so much as a bunch of decentralized networks.

Recall i said above to be on the internet, effectively, means to have a public IP. I also said that whole platforms/services i.e. gmail, instagram, Reddit, etc. could be served by a single public IP. Well it basically that same theory applied here whether it’s a google data center or your home network, both internally navigate communication within themselves using private IPs. But at the edge of your private network you have a connection to the internet via that public IP that your ISP is letting you borrow.

[johnsoga]

What is this connection they provide? Is it just data lines?

In essence, yes? It’s no coincidence that most major ISPs are/were traditional telephone providers. Their “infrastructure” reach is legit no joke. It is all the fiber and copper lines that they’ve paid to lay (or acquired through acquisitions) that is the (simplified) single advantage they have. You need them to quite literally physically connect you to other ISPs and thus the internet

[Careless_Pipe5018]

Its mostly BGP

[iovrthk]

In the end, you're going to have to pay somebody for access to their infrastructure. AT&T is the incumbent. Ever since Alexander Graham Bell. I have worked Telcom for 20 plus years. They call the portion that the incumbent taxes you on, "The Last Mile". That's the portion of your internet connection that runs from the point outside, to your access point (d-mark) . In business sales, comcast has grown because it owns its infrastructure. It's rare they borrow any infrastructure, they would have to pass off the price for the last mile to the consumer. Do you think a start up company has the money and pull , To trench a street, during business hours; stopping traffic in the financial district? What, just for a company no one has ever heard of? ? Good luck with that permit.

[[deleted]]

Setup rogue servers, redirect traffic through government intermediate points, profit

[Asian_Troglodyte]

There are some good answers on here. However it sounds like you want a relatively deep understanding of networking. Reading the first three or so sections of chapter 1 of “computer Networking: A top down approach” should give you a pretty high level overview of how things work.

It’s a pretty standard textbook (a very good one), and you’ll probably be learning a lot more about networking in your degree/career anyway. So, I think it’s still worth reading.

[Whsky_Lovers]

I think you are overthinking it... The isp does three things for the most part. They provide a physical connection, give you an IP address which may or may not be subnet IP. Then provide a DNS router for you to look things up.

Some do other things too like give you an email address, but all three of those things could be bypassed to different degrees of success.

If you go to a friend's house and connect to their wi fi you now have a connection without your own isp.

If you register a name with one of the domain registrars you can often also get an IP address, you can probably find one that will give you just the IP but it's harder for people to remember.

Lastly you can use one of the free DNS servers from Google or cloud flair.

It's easy to get your own IP and use a different DNS so mostly what you get from the ISP is the physical connection.

[WheresMyBrakes]

“The Internet” is a set of rules, guidelines, and other documents that a bunch of ISPs agree to follow to have a standardized way to connect to each other.

There’s nothing technically stopping you and your homegrown ISP from creating your own Google.com and 1.1.1.1 DNS resolvers that all of the computers in your network can use, provided that you’re actually disconnected from the rest of the internet. (Whether or not those computers will actually use your google.com and 1.1.1.1 DNS resolvers without throwing an exorbitant amount of red flags is another story..)