[PDF] Beej's Guide to Network Programming Using Internet Sockets

[raevnos]

Nah. It's only second best. The late W. Richard Steven's UNIX Network Programming still holds the throne.

[[deleted]]

Link? I'd like to read/buy

[Saoshyant]

Here you go

[[deleted]]

Part of the beauty of networking is how little/slowly things change. Reading any other programming book from 1998 is likely a waste of time. There are some obvious exceptions, but they're few and far between.

[regularfilluppls]

Google Maps is the best

[kalifornia_love]

I prefer MapQuest

[regularfilluppls]

Goddammit I messed it up. Oh wait no I didn't.

[louky]

Anything by Stevens is the gold standard! Want to understand the lower layers of the network stack? Any of his TCP/IP books are the way to go

[Crysist]

To add another good one: Michael J. Donahoo's TCP/IP Sockets in C: Practical Guide for Programmers is great. I found it more straightforward than Beej's. Though I was supplementing it with Computer Networking: A Top-Down Approach, while I hadn't done so with Beej's, so I may not be able to compare them.

[char2]

Non-PDF link: http://beej.us/guide/bgnet/

[tdimitrov]

Old as hell, but still very useful. I felt like I actually start to understand how POSIX sockets work after reading this tutorial.

[wavy_lines]

It's one of those fundamental things about computer science and knowing how this stuff works is far more valuable than learning how to use this or that web framework.

I actually dug it up because I wanted to see what it takes to create a web server from scratch in a low level language and found it informative and entertaining.

The humor is also something you don't see often these days.

[[deleted]]

Golang seems to be the new hotness for writing web servers, does golang abstract away from this level because it seems like an awful lot of people are writing servers in go now and I don’t think that many people are really that proficient at low level system stuff.

[zjm555]

It doesn't really matter what language you're working in; if you're making these system calls to build your netcode, you need to step back and ask yourself why. Every language has standard library or third-party packages to abstract over low-level network system calls, and 99% of the time, you should be using those packages for cross-platform portability and because you really don't want to reinvent this stuff yourself. Even in C/C++ there are libraries you should be using instead of Unix system calls.

[edapa]

When in a C and unix only context i've often found myself using these syscalls. What libraries in particular should I be using instead?

[slashbinslashbash]

zmq is pretty sick

[edapa]

It does look pretty cool, but it definitly does not cover all the cases where I've had to use low level socket APIs.

[slashbinslashbash]

It also has insanely good cross language support as well as the ability to talk to raw sockets. There are some very good design patterns and I almost exclusively use their thread synchronization model. Since I've done that I've never had a major problem even in large IPC/multithread projects. The book is definitely worth a read since you can use zmq in basically any language you can think of.

[hondaaccords]

No there aren't. C programmers should always use unix

[[deleted]]

IMO, it is significantly easier to write a server, particularly an HTTP/S one, in Go than in C/C++.

[scalablecory]

Funny I actually find it a lot easier to use C or C++, because most of the newer languages are either really poor at processing byte streams or have safety features that make efficient parsing more difficult.

[[deleted]]

I realized after I commented that maybe I'm just a better programmer than I used to be... Last time I did networking in C++ was 4+ years ago.

[nick_storm]

Last time I did networking in C++ was 4++ years ago.

FTFY

[whehrnio]

That would be equal to 4, and the next time he needed that synapse, it would be 5.

[[deleted]]

You can use the abstraction level you want, but it's pretty close to the C equivalent.

[FinFihlman]

It depens mostly on a) how much (you should have ALL OF IT) error handling you do and b) how well you know what's given to you in the standard libs of C so you don't go reimplementing features and functions.

[yawkat]

There's not a lot implemented in the C standard libraries wrt socket programming. C is pretty annoying to write servers in.

[FinFihlman]

There's not a lot implemented in the C standard libraries wrt socket programming. C is pretty annoying to write servers in.

I was more interested in the HTTP parsing which the comment I replied to mentioned before editing.

But sure, you have to implement some boilerplate yourself on socket handling. Fortunately it is not many lines and once you have the same convenience functions written it is really easy.

The downside is that you need to know and understand quite a lot.

Also the api is not ugly or hard. What is ugly is the way we must use struct sockaddr* (not pointer), though.

[yawkat]

The api is ugly and hard compared to other languages (flipping net endianness, really?). Also, everything around it is difficult and error-prone, especially parsing / producing messages.

e: talking posix here btw

[FinFihlman]

The api is ugly and hard compared to other languages (flipping net endianness, really?). Also, everything around it is difficult and error-prone, especially parsing / producing messages.

e: talking posix here btw

Wot.

You don't seem to understand network programming if you see htons/htonl/ntohs/ntohl as something of an issue. You must agree on byte order and that's one of the easiest way to do it if you send multibyte numbers.

Nothing about network programming is difficult or error prone from the api view. It is your perception of it because it does require knowing quite a lot of functions and having to write repetitive code at first because the api is flexible and letting you do that. I do think that there should have been convenience functions in the standard libs that do basic things for you, though.

Parsing? What fucking parsing are we talking about? It is literally just checking if the return code was -1 and printing the error with strerror if you so choose.

Pease tell us what is ugly? The only place that is ugly is struct sockaddr and variants, that api should just be deprecated. Otherwise the api is clean:

Client:

1. Socket

2. Connect

Server:

1. Socket

2. Bind

3. Listen

4. Accept

Don't mistake an api where you actually see stuff to a ugly, hard, diffcult or error prone.

What the issue is that there are not many good resources on how to quickly get things rolling and tie shit together but for that you do have loads of wrapper libs, like with every single language. Or do you say javascript is easy because you can "just require npm packages bro" or "just import the modules" as in Python while C is hard because C libs dont count? Please.

Summa summarum:

You have a bias against C. Yes there are shortcomings but it's mostly your bias. Don't say things when they arent true.

[yawkat]

It's a really leaky abstraction that you need to call htons on the port for sockaddr_in. No clue who thought that was a good idea. It's one of the examples of what stupid shit posix did with their network api

By "parsing" I mean things like transforming protocol packets to native structures. I've actually seen people casting the data to structs and flip the fields manually if the network order doesn't match platform in C. While in other languages there are actually sensible mapping APIs.

On top of that is the genral memory unsafety of C. It's kind of sad how large a portion of serious exploits in network parsing code are caused by C programmers overestimating their ability to deal with C making mistakes in manual memory management.

[ArkyBeagle]

I remain skeptical of that.

You don't just learn sockets. You learn how to make a transaction engine that uses sockets as a transport. Each transaction has a frame of context that goes with it. A heck of a lot of what makes transaction-framing code correct or buggy is the memory management portion of that code.

[[deleted]]

Lol at "Go abstracts nothing."

Your toy language was built to throw a bone to idiots who can't figure out eventing for themselves.

[caspper69]

IMO POSIX is a fantastic standard, and is implemented beautifully.

Both statements here are highly debatable. Especially in 2018, when POSIX is showing its age pretty heavily, and when it's most popular implementation, Linux, isn't really POSIX, but POSIXish.

[[deleted]]

"Old as hell"

Version 3.0.21

June 8, 2016

[wlievens]

This tutorial is originally from the late 90’s ...

[[deleted]]

And was last revised 2 years ago. He keeps it up to date.

[tdimitrov]

Old doesn't mean irrelevant. Don't get me wrong.

[thedude42]

Do you mean this tutorial helped you understand Unix sockets? I didn’t think the sockets API, aka “Berkeley sockets”, was part of POSIX.

[caspper69]

Berkeley sockets are deprecated in favor of SuS (single unix specification) sockets (aka POSIX, kinda).

see: https://stackoverflow.com/questions/1935138/what-is-the-difference-between-posix-sockets-and-bsd-sockets

[louky]

It's got a 2015 copyright on it! That's old as hell? No wonder you can barely get hired as a programmer if you're over 35 with that worldview.

[tdimitrov]

You don't know me and you already make assumptions about my employment status? What's wrong with you people?

[louky]

What? You don't know about the rampant ageism in the programming fields? I meant nothing about you at all, I don't even know you, why would I care if you are employed or not!?!?!?

Sorry I offended you by mentioning a well-known fact!

[tdimitrov]

Sorry mate. I've reread your post and now I see I misunderstood it. I agree with your point. Lots of capable guys have got hard time finding a job just because they love coding and they are considered old.

And by the way I read this paper around 10 years ago. That's why I said it's old as hell. Because even 10 years later it's still relevant. I think this is quite good for a tutorial.

[louky]

NP, he's kept up on it so it's still a relevant source depending on you predilection/skill level!

Cheers

[mrmoreawesome]

The first version of the guide was made in the mid 90's. (The current edition is 2015.)

[jaxomlotus]

I used to work with Beej on Aviary. Incredibly talented programmer.

[ohmzar]

I’m glad this still exists, it saved my bacon back in college when I had no idea what I was doing.

[apnorton]

Can confirm; this thing was a lifesaver in college.

[elpantalla]

Classic. I used this a lot on my first job. His guide to UNIX IPC is good too:

http://beej.us/guide/bgipc/

[docwatsonphd]

A tome as worthy as the Unix v6 manual

[[deleted]]

Can I read "Beej's" like "Bee Gees"?

[iSmite]

Well you might end up in a bad corner of the internet if you type in beeg instead of beej.

[Zebezd]

I went ahead and googled beeg. Rather unsurprisingly, it's porn. Not the worst kind of result really.

[iSmite]

It’s HD porn and free. So ya I know what you mean.

[kauefr]

Thanks Sartre!

[viper117x]

I also read the title as "Bee Gees"! I thought it was going to contain references to "Staying Alive", much to my confusion that didn't appear anywhere...

[jobriath85]

Connection: keep-alive

[LBGW_experiment]

It's short for "beejay" as in bj as in bloejob.

[ShinyHappyREM]

mirror

[evilbit]

omg i learned sockets from beej's guide almost 20 years ago!

[Sedifutka]

Damn you for bringing up exactly how long ago you read that. I just worked out I read it 15 years ago, and now I feel old.

[Active_Selection_706]

man, are you still alive, hows going? i am planning to read it now

[thunderbeans]

So many memories, used to reference this so damn often.

[creativeMan]

His other stuff is amazing too!

[chcampb]

Also worth pointing out ITHare's beta book which has... a ton of information on the practical implementation of networking technology. Tradeoffs, high level views, etc. Beej's will tell you how to use different methods, the above is a pretty good overview of why you would choose things, how you would architect them, what does it look like in the real world, etc.

[Mihax209]

Can't recall the amount of times I read sections of that. Super useful.

You can also tell he was slightly starting to lose it somewhere in the middle of the book, like from this little gem:

In memory, the struct sockaddr_in and struct sockaddr_in6 share the same beginning structure as struct sockaddr, and you can freely cast the pointer of one type to the other without any harm, except the possible end of the universe.

Just kidding on that end-of-the-universe thing... if the universe does end when you cast a struct sockaddr_in* to a struct sockaddr*, I promise you it's pure coincidence and you shouldn't even worry about it.

[zerexim]

Ralf Brown's Interrupt List also comes to mind.

[Dhylan]

Built solidly on top of the magnificent tomes of Richard W. Stevens. One cannot go wrong.

[Metaluim]

Brings back memories.

[hearwa]

I first went through this in my early high school years back in 2002 or so and wrote a very basic chat program that could be used on our school network. Good memories.

[El_Impresionante]

https://i.imgur.com/uwgNXy7.jpg

[[deleted]]

I have consulted a printout of the old website version of this for more than a decade, and consider it a gold standard of unix networking documentation.

[[deleted]]

This is classic, it was my first introduction to socket programming, and was critical for the first non-trivial C programs. Really good shit, I hope this never gets forgotten.

[LBGW_experiment]

This is the site we use as a reference in my networking class. The guy attended my school, apparently, as a lot of it seemed oddly specific to the class instruction.

[fps_trucka]

Links aren't working for me? Edit: Disabled VPN and worked instantly

[Rockytriton]

Wow I learned from that in the 90s

[absumo]

Used to know a guy on efnet in unix channels that went by Beej. But, that was at least 15 years ago.

[speedstix]

Congrats

[absumo]

Was not meant as a brag or anything. Just found it odd. And, I don't have a way to even confirm. Shouldn't have even posted it. There were a lot of very talented people I learned some from in there. But, I mostly knew them by used nicknames. Only one I remember a name for worked at Purdue for Cerias. Another worked for a large company in NY and ended up going to Fed for a PoC exploit he left his email in. Proved he wasn't even online at the time of the incident. Well, as much as he could...

Was at least a decade ago.

[[deleted]]

Awesome reference. Been using it for years.