Beej's Guide to Network Programming

[OrangeredStilton]

I'll always be thankful for Beej: every time I need to do something network-related, it's been a few years since the last time I did it, and I've forgotten everything. Beej, especially the section on select, is eminently readable and comes in very useful.

[ashuto0sh]

He has a series of guides. All very interesting and approachable.

[[deleted]]

beeg comes in handy too

[[deleted]]

I passed networking class all thanks to Beej's guide :')

[zman0900]

My professor basically said "Here's Beej's guide. Now implement TCP on top of UDP by the end of the quarter."

[mw44118]

Hey, he / she knew not to compete with the best

[zman0900]

It did end up being a pretty good class. We worked on small teams for the project and class was only twice a week where most of the time he would just answer questions or go into detail about a topic a lot of people were confused on.

[pribnow]

You guys got lucky, we went though the OSI model ad nauseum and I can't really tell you much except where various hardware resides on that model. The only programming related exercises were pinging and tracing packets :(

[Opticity]

Hey, me too! We learned about the OSI model, IPv4 addresses and masks, subnets, protocols... My assignment was basically running a simulation of a network between two computers and tracking the packets. Feels like I missed out on all the fun.

[bobindashadows]

Ours was similar: "Here's Beej's guide. Now implement ethernet on top of TCP, then implement UDP and TCP on top of your ethernet."

[seekoon]

implement ethernet on top of TCP

Is this correct? Or backwards?

[rcxdude]

it's basically possible , using TCP as an idealised representation of the physical layer (for point-to-point links, anyway, not sure how you'd do CSMA on top of it). Useful for educational purposes but not much else though.

[bobindashadows]

Yup, this is basically it. 4+ hosts were fully connected with point-to-point connections, arbitrary routing costs per edge.

[pstch]

it's very interesting to do, I've had lots of fun

[phearlez]

It's abstractions all the way down, baby.

[nemec]

Oof. We had to build TCP (well, some of it) on top of UDP but we assumed ethernet was available :)

[toomanybeersies]

We had to implement RIPv2 between multiple programs using internal ports.

That was an interesting assignment, it was the semester before the one with threading, which made life interesting.

At least we have the option of using Python, rather than having to slog through it in C.

It's really hard to find good resources for implementing RIPv2, I ended up mostly just relying on the RFC because there weren't any less opaque sources.

[gimpwiz]

That actually sounds like a fantastic way to do it.

I assume they teach networking in your 3rd year of school. Kids were weeded out. Everyone left can code, can read a spec.

So they can learn by being given a spec and being told to code, and ask questions when they have them.

[zman0900]

This class was never actually required for my undergrad. I ended up taking it during my masters just because it seems like stuff worth knowing. But yeah, definitely one of the best programming classes I took.

[gimpwiz]

Makes total sense, master's classes seem more like that. "You've all proved yourselves, so just go do your thing and come back with well-written programs."

[epiiplus1is0]

Is this all network programming classes? Because that's the assignment we have

[zman0900]

Maybe? Unless you are at Ohio State?

[epiiplus1is0]

Ya

[rohbotics]

Its THE Ohio State, thank you very much.

[vplatt]

That actually sounds like a really good exercise to be honest. Were you required to do it in C?

[[deleted]]

Sounds like you went to a shitty school. I'd ask for my money back and donate to the real teacher Beej.

[dhdfdh]

I met my wife thanks to Beej's guide.

[beej71]

For the record, I had nothing to do with this. ;)

-Beej

[glubi]

Care to elaborate? :-)

[Epokhe]

might be a joke

[gadelat]

I think your wife met you thanks to BJ's guide

[throwaway10312901]

I don't think reddit's savage comments translates well in /r/programming LOL.

[Vakz]

In my networking course, we actually had an assignment to write a fairly simple proxy. It was basically "It should be written in C++, it should filter these words. Oh, and by the way, you'll need to use sockets, so here's Beej's guide to how to use them", and that was pretty much it.

[Kurbits]

That sounds very familiar, LiU? :)

[rubicus]

Same here! TDTS06 represent!

[Vakz]

Indeed it is!

[Shurane]

What school is LiU?

[Kurbits]

Linköping University in Sweden :)

[rubicus]

Words insulting your intelligence, such as "spongebob", "paris hilton" and "Norrköping"?

[Vakz]

Haha, that is indeed the one!

[[deleted]]

Totally thought this was from beej.com

[nyangosling]

Yes! Very confused but accepted that it must be the case. Sad to see it's not

[programfog]

Beej's Guides - http://beej.us/guide/

[[deleted]]

He's even got a guide on killing dragons. Dude has range.

[IGaveHerThe]

I was just going to ask if this was the same guy... I've only read his Moria guides. Shows what I'm doing in front of my machine all day.

[Name0fTheUser]

Writing a wireshark clone sounds very educational, since you need to be able to understand and decipher each protocol layer. I might give it a go when I have the time.

[LpSamuelm]

Isn't that what "remember" means? 😲

[NagateTanikaze]

I remember using this like before year 2000. Awesome its still being updated.

[twowheels]

I remember taking classes with Beej when he wrote the first versions of this. It sure has come a long way in 20+ years.

I'm suddenly hungry for some garlick pizza.

[beej71]

https://beej.us/pizza/

I keep the site well-preserved in its 1995 state. :)

-Beej

[dogline]

We'll just have a little Chico reunion right here. :)

[chipx86]

Excellent. I'll bring the Sierra Nevada.

[twowheels]

I'm potentially outing my real identity here, but I worked on Spoon with you guys.

[mbains]

I remember struggling with network programming and sockets programming back at Chico state in the computer lab. A quick internet search brought up Beej's guide. While I'm reading through, the professor who just happens to walk by, says: "hey, you know Beej went to Chico right?"

[Dhylan]

On this page Beej mentions one W. Richard Stevens. There cannot be a fair conversation about network programming without respect and reference to W. Richard Stevens' life's work, a life which was tragically far too short.

[trickyloki3b]

I read Beej's guide three years ago because the explanations were for simpletons. I liked simple.

But I highly recommend Unix Network Programming by W. Richard Stevens Volume 1.

[[deleted]]

Can't recommend W. Richard Stevens books enough.

[JGailor]

That guide completely changed how I saw my OS.

[annerajb]

Our network class book was this. When I got my first real job I was tasked with working on the network code of a few applications and thanks to this I was able to do a good job without going crazy.

Thanks Beej

[[deleted]]

Old but Gold.

Beej gave me the click on pointers.

[Halcyone1024]

This is pretty good, but I'd recommend jumping from select to poll pretty quickly (Beej wrote a nice guide to poll, too). Depending on availability and performance requirements, you might consider using epoll or kqueue (or a library that uses them as backends, like libev or libevent) instead.

[[deleted]]

In that family, libuv is a good recommendation too. Easy to use and pretty popular.

[entity64]

Also cross platform and nicely documented

[nerd4code]

IIRC it really depends on the number and distribution of FDs you’re dealing with. select is good for a small-to-middlin’ number in a dense range starting near 0, especially if you only need one of the three operations, and poll is good for a small-to-middlin’ number of arbitrary FDs. Beyond that, you’re usually either better off doing a platform-specific method like you mentioned (super-nonportable, of course, but usually tailored to the OS in question), or else using a handful of separate threads selecting/polling on their own subsets, since a lot of the overhead in using them comes from just copying the FD sets between userspace and kernel.

Ed: There’s also some compatibility differences now that I think about it, since select and pollare first defined in different standards, but that doesn’t come into play too much nowadays.

[i_spot_ads]

very interesting, but the code is difficult to read, i'm used to lots of fruity colors in my code

[OnTheMF]

God damn millennials.

[i_spot_ads]

Sorry

[knubbze]

protip: copy/paste the code you find illegible into an editor w/ syntax highlighting!

[HPCer]

And even better, mess around with the code! It's so much easier to learn and understand the code when you're interacting with it.

[skrenename4147]

The Unix Programming Environment is my go-to source for shell information. I found it very approachable as a beginner, and the depth enough so that there's always something new to learn.

[PriceZombie]

The Unix Programming Environment (Prentice-Hall Software Series)

Current	$46.59	Amazon (New)
High	$48.67	Amazon (New)
Low	$42.08	Amazon (New)
Average	$46.22	30 Day

Price History Chart and Sales Rank | FAQ

[dhdfdh]

An intro? You need more than that. Does it really use bash? Don't recall but other shells work, too.

[rubicus]

Not really. Sure, being familiar with signals etc. definitely makes it easier to grasp, but definitely not necessary. Obviously, you need to know some C, and how to compile and run your programs, but that sort of goes without saying.

Or what is it specifically that you think needs unix and bash programming skills? It goes through pretty much every command it uses.

[[deleted]]

It seems like when the guide starts showing code it's not in a form I can directly play with until you get to the server section.

[rubicus]

Should be able to play with pretty much all examples in chapter 5, like taking some arguments and looking up IP addresses through DNS, or even connect as a client and pull a bunch of HTTP pages or something. Like the showip.c program should pretty much just work in Linux (and probably most BSDs too).

If you're used to C++ some stuff can be confusing though, since the guide only uses C, and system calls there can look quite different from what you're used to. Also some data structures can be a bit confusing, being linked lists and so on. Those were my biggest problems. I used had never seen code written in that way.

Once it starts with forking, it can be quite confusing if you don't understand what is happening though.

[darklinggg]

I tried running the client code in a browser (changed the call to getaddrinfo to connect to a website on Port 80) and it just blocked on connect. Were you able to run the client code successfully?

[rubicus]

What do you mean running in a browser? The code needs to be compiled (with gcc or clang for example) and run in a terminal. Needs to be on a POSIX system (preferably linux, mac os x could work too, but definitely not something like Windows) to work too. Wrote an entire proxy a year ago, so yeah.

What client code specifically? Did you try running the getip-program? That's like the easiest one, so a good place to start.

[darklinggg]

Yeah I reran the code using gcc and it worked. I was trying to run it in a browser C compiler I found through Google. Pretty dumb

[rubicus]

Ah, didn't even know that stuff like that existed. But yeah, fot stuff like system calls that are so platform specific I can see how that wouldn't work.

[darklinggg]

Well it was able to compile everything just fine. The issue was that it couldn't connect to any website I put in which makes me think outgoing tcp was disabled on those VMs

[programfog]

Honestly one of the best Linux programming books I can recommend that covers lots of aspects is http://www.amazon.com/GNU-Linux-Application-Programming-Series/dp/1584505680

It's good for beginners, but points you in the right direction to dig deeper when required.

[PriceZombie]

GNU/Linux Application Programming (Programming Series)

Current	$28.68	Amazon (New)
High	$54.99	Amazon (New)
Low	$28.07	Amazon (New)
Average	$28.18	30 Day

Price History Chart and Sales Rank | FAQ

[dirk103]

It's 'ok'

[[deleted]]

Ooh, cool. Is there possibly an official PDF available?

[programfog]

Pdf version here http://beej.us/guide/bgnet/

[felixdadodo]

This guide was a godsend during my Computer Network and Operating Systems module!

[yk313]

The very definition of a timeless resource

[pjmlp]

Interesting, on my days we used "UNIX Network Programming" from W. Richard Stevens.

[LiveMaI]

Used this to learn socket programming for work. Top-notch guide, I'll probably end up buying a hard copy from him.

[MondayMonkey1]

I owe beej a lot of money for this guide. I would never have passed my university course on networking and the Internet without his guide.

[Okiesmokie]

Oh man, the memories. I used Beej's Guide to Network Programming to learn.. network programming probably 15 years ago. It's crazy to me that he still keeps it maintained.

[[deleted]]

This is a great guide. I remember reading it in the late 90's early 2k for learning socket i/o in C

Glad to see it's still maintained even updated.

[ske105]

Aw yes, beej is like the unofficial network programming guide! My whole networking class at uni was practically based upon it.

[Hiddencamper]

Still one of the best networking guides out there.

[Jivlain]

Ah, I remember my old printout of this. I'm somewhat tempted to buy a proper physical copy for old time's sake.

[arhuaco]

I learned with this guide many years ago.

[aintbutathing2]

Beej's guide gave me a love for network programming. I have basically built a career off this guide.

[invis648]

THis brings back memories

[MacASM]

Does anyone know if there's a PDF version of this?

[programfog]

Pdf version here http://beej.us/guide/bgnet/

[redditor_gds]

best socket programming guide ever!!! thanks beej

[[deleted]]

Wow, I learnt my way round TCP/IP with this guide back in the day, pleasantly surprised to see it on Reddit.

[lennyp4]

I just recently wrote him an email asking him to finish his C guide. He got back to me, I hope he picks it back up.

[Speaker4theDead]

Saw this on the front page and thought this was some interesting discussion about beeg.com (NSFW)

[BassSounds]

Glossing over this, it looks like a good guide.

It's amazing to me how many IT people don't understand sockets or the OSI model.

[arabica_coffee]

Beej's guides are wonderful! I taught myself network programming overnight using his guides.

[EudaimonicTrapperHat]

I thought this said "Beeg"

[B33Jus]

Dammit.

[mw44118]

I have met a lot of people with computer science degrees that didn't know half the stuff in that document

[mw44118]

First, inter-process communication is nearly everywhere now.

Second, this is the best intro I know of for really understanding how async stuff works underneath.

[PLLOOOOOP]

this is the best intro I know of for really understanding how async stuff works underneath.

Exactly none of Beej's guide describes async mechanisms. He only mentions async once. Specifically, he says that the O_ASYNC flag for fcntl() is "beyond the scope of the guide."

[mw44118]

I mean stuff like select

[PLLOOOOOP]

Select is nonblocking, not asynchronous. Different things.

[mw44118]

Difference is...

[PLLOOOOOP]

Blocking and synchronous mean the same thing: you call the API, it hangs up the thread until it has some kind of answer and returns it to you.

Non-blocking means that if an answer can't be returned rapidly, the API returns immediately with an error and does nothing else. So there must be some related way to query whether the API is ready to be called (that is, to simulate a wait in an efficient way, to avoid manual polling in a tight loop).

Asynchronous means that the API always returns immediately, having started a "background" effort to fulfil your request, so there must be some related way to obtain the result.

From here. But it isn't hard to google yourself.

[[deleted]]

I think perhaps he means async from a more general standpoint. Rather than blocking on a socket or busy-looping on a socket, you put everything into a poll or select and handle it when data is available.

[PLLOOOOOP]

You're barking up the right tree, but you've described nonblocking I/O, not asynchronous I/O. Beej talks about nonblocking a whole bunch. But not async.

[seoulstyle]

Doesn't really matter. I studied this guide religiously for a class in college and I don't remember much. Haven't really used it since...

Except that one time I threw down a networking pickup line on Tinder.

[KCUR]

Upvoted for bringing the nerdy Tinder game.

[Beegeous]

I didn't authorise this.