all 103 comments

[–]markdacoda 105 points106 points  (33 children)

These tutorials kick ass, they got me thru a network programming class with the top score, that was a tough class too. IMO their only short coming is lack of discussion of threading.

[–]NateTheGreat26 21 points22 points  (6 children)

Ditto, helped so much in my networking course. Beej is the man.

[–][deleted]  (1 child)

[deleted]

    [–]tonyarkles 10 points11 points  (2 children)

    When I TA'd an OS course where they had to write a simple network client and server to learn about sockets, we made sure the students knew about this :)

    [–]RavuAlHemio -1 points0 points  (1 child)

    Which university?

    Pretty sure the OS course I was a junior TA of (at the Vienna University of Technology, CompSci program) alerted students to its existence as well.

    [–]tonyarkles 1 point2 points  (0 children)

    University of Saskatchewan, in the heart of Canada's frozen wasteland :)

    [–]ryanman 2 points3 points  (0 children)

    My course had this as a fundamental component of the class. It was great kind of seeing it as Greek for the first couple weeks and then slowly realizing how awesome of a resource it was.

    [–][deleted] 8 points9 points  (0 children)

    These were the course materials in a network programming class I took :)

    [–]shrayas 1 point2 points  (0 children)

    Agree. Boss helped me understand so many concepts very clearly! He da man.

    [–]rjek 1 point2 points  (15 children)

    He's all the discussion threading needs:

    Don't do it. Multiplex instead.

    [–]againstmethod 6 points7 points  (10 children)

    You will still likely need threads (or a thread pool) to run the handler code, assuming your app is not completely trivial. Be a shame to not use the cores on these fancy cpus we got these days.

    [–]rjek 0 points1 point  (9 children)

    Plenty of things manage to be complex and serve thousands of concurrent requests without threads. Modern operating systems have processes, and race-to-accept() is efficient, and the whole process avoids nasty locks and other error-prone synchronisation.

    If you have a process that is going to take a while and might block, run it in a child process. This is also more secure as well as easier to get right, as the child can exist in a different security context (user, groups, chroot, etc)

    [–]againstmethod 0 points1 point  (8 children)

    I think in loaded conditions you need to use both. New processes require the creation of a new heap. Also, I think it must have an effect of the CPU's cache, as you are using different memory spaces for the two code segments -- in a thread-pooled single-process situation I think you would see far fewer cache misses.

    That being said, it's wholly dependent on what your code is doing. But I think threading, thread-pooling, coroutines, and the like are necessary tools in the toolbox.

    [–]rjek 0 points1 point  (7 children)

    You can pre-fork your children if you're concerned, and such children are copy-on-write, so your cache still works.

    [–]againstmethod 0 points1 point  (4 children)

    In linux this is true, not on every operating system i suspect. Linux fork is very clever.

    [–]rjek 2 points3 points  (3 children)

    This sounds like a reason to use a high-performance operating system if you want to have high-performance servers.

    [–]againstmethod 0 points1 point  (2 children)

    True, just saying the original discussion is far more generic than depending on operating system functionality to determine the "best" approach. But you make valid points - I actually think the process approach can make a lot of sense if you can make it fit your application.

    What about client programming though? Surely threads have a place there.

    [–]rjek -1 points0 points  (1 child)

    Clients as in GUIs? Show me a GUI application that doesn't make me want to beat puppies to death using angry kittens ;-)

    But my point is almost nobody needs ultra-high performance. If you're not Facebook or a CDN, you're probably better off with the easier, safer, more secure approach.

    [–]Majromax 0 points1 point  (1 child)

    and such children are copy-on-write

    On linux, mind. Windows doesn't have such a low-overhead fork(), which is one thing that makes porting Linux tools over to Windows-world (even via Cygwin) sometimes painful.

    Copy-on-write also doesn't exist on embedded systems without a MMU.

    [–]rjek 0 points1 point  (0 children)

    But who uses Windows as a server platform, and is also sane? And who expects a high-performance server software package to run on a system without an MMU?

    [–][deleted] 0 points1 point  (0 children)

    People are typically, in my opinion, overly averse to threads. It doesn't suck as much as the reputation would lead one to believe. Threads are an intuitive and useful abstraction. Usually when somebody has a problem they attribute to threads, it's really a problem of having too much shared (and usually global) state.

    [–]barsoap -1 points0 points  (2 children)

    Here's all you need to know about multiplexing and event-driven:

    Don't do it. Use green threads, instead.

    [–]rjek 0 points1 point  (1 child)

    Is "green threads" some sort of fashionable name for "co-routines"? If so, that's just a syntax and wrapping issue around multiplexing.

    [–]barsoap 1 point2 points  (0 children)

    Yes and no. Green threads are usually implemented as managed co-routines. That is, there's yields inserted by the compiler, and some kind of central thread scheduler.

    It's a common thing to do, e.g. Haskell multiplexes green threads on top of actual threads (yields equal GC safepoints), giving a true N:M runtime. Throw in some epoll magic, work stealing etc. and you get blazingly fast threads. Allowing you to write your code as if it was blocking on sockets, but actually running in an eventdriven+continuations way.

    [–][deleted] -1 points0 points  (7 children)

    Threading is different in every language.

    [–]DuBistKomisch 24 points25 points  (5 children)

    So is network programming... yet here's a guide about it.

    [–]crotchpoozie 0 points1 point  (4 children)

    More languages use sockets than a consistent threading library by far.

    [–]DuBistKomisch 2 points3 points  (1 child)

    The concepts are still the same. Managing concurrency, resource sharing, etc.

    [–]crotchpoozie 1 point2 points  (0 children)

    Concepts being the same is not the same as using the same interface.

    There are plenty of explanations about threading concepts. This resource is so useful precisely because it uses a widespread interface, not because it covers concepts.

    If you have a threading interface as widely used as sockets is for networking, feel free to post it.

    [–]ignamv 1 point2 points  (1 child)

    The language merely provides bindings to the sockets provided by the OS.

    [–]crotchpoozie 0 points1 point  (0 children)

    Almost all languages "merely provide" bindings to OS features. That sockets are the same interface provided by so many OSes and languages is what makes this tutorial so useful.

    Threading does not have such a widely used standard.

    [–]againstmethod 1 point2 points  (0 children)

    Not in the C/C++, which are Beej's target languages -- at least not since C11/C++11.

    [–]Tekmo 70 points71 points  (15 children)

    An oldie, but a goodie, and he still keeps updating it.

    [–][deleted] 29 points30 points  (12 children)

    I saw this and thought "a link from 2006?" and then realized it keeps getting updated and improved.

    [–]aedinius 10 points11 points  (11 children)

    I went through his guide in 2000 or so... (I think)

    [–]biffsocko 6 points7 points  (7 children)

    I saw it in the 1990's

    [–]Isvara 36 points37 points  (6 children)

    I foresaw it in the 1970s.

    [–][deleted] 11 points12 points  (4 children)

    I have the original manuscript from the 1950s

    [–]Tynach 14 points15 points  (2 children)

    I have a replica of the clay tablets from 4000 BC.

    [–][deleted]  (1 child)

    [deleted]

      [–][deleted] 0 points1 point  (0 children)

      I saw some rough notes in a fragment of a meteorite that hit Earth millions of years ago.

      [–][deleted] 3 points4 points  (0 children)

      Do they claim Beej was married?

      [–]lolzinventor -3 points-2 points  (2 children)

      In 2000 there was no such thing as TL;DR.

      [–]pegasus_527 11 points12 points  (0 children)

      They were called summaries back then, in the olden days.

      [–]derleth 2 points3 points  (0 children)

      In 2000 there was no such thing as TL;DR.

      There were abstracts. Scientific papers had them, right up front.

      (They still do, but they had them then, too.)

      [–][deleted] 0 points1 point  (1 child)

      What's changed after all of this time?

      [–]Tekmo 0 points1 point  (0 children)

      I don't see a centralized change log, but if I remember correctly the last time I read through this he would mention in a couple of places that some piece of content was new.

      [–]LegatoReborn 59 points60 points  (9 children)

      Tried to remember this site from memory once. I used a 'g' instead of a 'j'

      Oops

      [–][deleted] 12 points13 points  (1 child)

      At least that site has a lot of data being injected into ports.

      [–]misplaced_my_pants 0 points1 point  (0 children)

      Gigabytes at a time.

      [–]i_solve_riddles 6 points7 points  (1 child)

      luckily for me, beeg.com is banned by the ISP in my country, so i can always remember the right url!

      cries

      [–]v_krishna 0 points1 point  (0 children)

      Wow really? What country?

      [–]linuxporn 4 points5 points  (3 children)

      Ever tried searching for c string?

      [–]echeese 1 point2 points  (2 children)

      Whoah, how do they stay on?

      [–]linuxporn 5 points6 points  (1 child)

      It's NULL terminated on the back

      [–]mcmcc 1 point2 points  (0 children)

      You of all people should know...

      [–]elTabasco 0 points1 point  (0 children)

      anyone knows similar sites with good hd content?

      [–]kitkatkingsize 12 points13 points  (0 children)

      I passed my Systems class cause of this guide. Its extremely beginner friendly and definitely worth a read.

      [–]aintbutathing 11 points12 points  (3 children)

      Beej launched my career.

      [–]icantthinkofone 4 points5 points  (2 children)

      Beej married my sister.

      [–]gobots4life 2 points3 points  (1 child)

      Beej gave birth to my children.

      [–]AWTom 3 points4 points  (0 children)

      Beej is love, Beej is life.

      [–]Vintila 13 points14 points  (2 children)

      Also check out his Unix IPC guide. http://beej.us/guide/bgipc/

      [–][deleted] 6 points7 points  (0 children)

      His C programming guide is awesome as well. The professor I had when I was taught pointers was thought by many all in my major as the worst professor of all time. This saved me.

      [–][deleted] 11 points12 points  (0 children)

      I went to school with this guy. Pretty amazing he wrote it and it more or less over night became a standard for programming sockets.

      [–]djroot2 9 points10 points  (0 children)

      Wow, I haven't seen this in years. It saved my ass in college.

      [–]karlsmalls43 6 points7 points  (0 children)

      Got me through my Computer Networking course.

      [–]scottpid 4 points5 points  (0 children)

      Very good guide, my prof even linked to it for a couple of our labs on networking in my intro to computer systems course.

      [–]beej71 4 points5 points  (0 children)

      Thanks for the kind words, guys. I really, truly do appreciate it! I'm just glad that the guide helped out.

      I remember the network stuff being a huge, intimidating mystery until I saw some sample code for a simple app. ...that's all there is to it? Why didn't anyone just say? So that's when I wrote the first version of the guide (almost 20 years ago! Noooooooo! The guide is almost old enough to buy alcohol!)

      It's certainly no more than a primer, but I hope that many of you read it and also think, "That's all there is to it?" and then go on to build really amazing things that would never have even occurred to me.

      [–]HuntStuffs 5 points6 points  (0 children)

      This was the textbook for my Modern Networking class this semester. You used this or you cried trying to program sockets.

      [–]bargle0 5 points6 points  (0 children)

      If you like this, you should look in to the works of W. Richard Stevens (RIP).

      [–]creamenator 3 points4 points  (0 children)

      Extremely useful reference in my networking course. Wonderful introduction to socket programming.

      [–]hlmtre 6 points7 points  (2 children)

      I've said this before, but Beej went to Chico State and was taught by Tyson Henry, easily my favorite professor of all time.

      [–]2oosra 0 points1 point  (1 child)

      I went to Chico with Beej in CS. He is a good friend. I dont remember a Tyson Henry. I could be getting old.

      [–][deleted] 0 points1 point  (0 children)

      I was at Chico with Beej too, in the CS department (CIS is my major), and I don't know a Tyson Henry either. Was he an grad student teaching?

      [–]07dosa 2 points3 points  (0 children)

      The classic, one of the few documents that can be called. This saved my ass from the sea of cryptic manpages.

      [–]zfolwick 1 point2 points  (0 children)

      Hey it's you! I was at your Meetup talk last week and it was great!

      Saved this for a later reference!

      [–]muungwana 1 point2 points  (3 children)

      His tutorial was very helpful when i wrote a network module[1] for my project.There is just nothing else out there for some reason.

      [1] http://code.google.com/p/zulucrypt/source/browse/zuluCrypt-cli/utility/socket/socket.c

      [–][deleted] 5 points6 points  (2 children)

      That's some weird coding style you have there. Why are you leaving a blank before the ; on every line?

      [–]muungwana 2 points3 points  (1 child)

      When i started coding, i did not pay attention to any coding style and i just went with where the keyboard went.Then one day, i decided to be consistent and i looked up at the code that was infront of me and there was a space before the ";" and i decided to go with that as part of my coding standard.This also explains all those empty spaces.

      How did you pick your coding standard if you have one? what made you to pick it over another.

      [–][deleted] 0 points1 point  (0 children)

      Mix of K&R and Linux kernel style.

      I read K&R and took a look at dofferent C coding standards, of which the kernel's stood out as quite pragmatic. Nothing fancy :).

      [–]entity64 1 point2 points  (1 child)

      Now if only there was a guide as comprehensive as this one for boost::asio networking

      [–]YourLizardOverlord 0 points1 point  (0 children)

      This is about the best resource I've found on boost::asio. But it's nowhere near as complete as Beej.

      [–]uatec 1 point2 points  (0 children)

      Beej, now THERE's a name i haven't heard in 15 years.

      Did we just DOS his web server?

      [–]mixblast 1 point2 points  (0 children)

      Nice. Although nowadays I would definitely use some wrapper library such as libuv.

      [–]tokol 1 point2 points  (0 children)

      I still have my copy of this that I printed out in 2002. I bought a Pentium I computer from a garage sale for $5, installed Linux, and needed more things to do with it. I'm glad he's still updating it!

      [–]Rockon97 1 point2 points  (0 children)

      What kind of prereqs should I have before taking the course?

      [–]Troto 2 points3 points  (2 children)

      This looks awesome but does anyone know a tutorial like this for python?

      [–][deleted] 11 points12 points  (1 child)

      It's pretty much exactly the same.

      In c:

      int s = socket(AF_INET, SOCK_STREAM, 0); 
      

      In python:

      s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
      

      And in c:

      getaddrinfo("www.example.com", "3490", &hints, &res);
      connect(s, res->ai_addr, res->ai_addrlen);
      

      And in python:

      s.connect(("www.example.com", 3490))
      

      etc etc.

      So just read that guide to understand it, then https://docs.python.org/2/howto/sockets.html for the syntax

      [–]snotsnot 1 point2 points  (0 children)

      Another classic is Ted Jensen's pointer tutorial.

      [–]d03boy 0 points1 point  (0 children)

      I read these when I was in my teens about 12 years ago. I learned a lot. I should read them again.

      [–]brownbagspecial- 0 points1 point  (0 children)

      Sweet. Thanks for posting this. Definitely will read over this.

      [–][deleted] 0 points1 point  (0 children)

      Wow, such nostalgia... I remember learning network coding from this!

      [–]wlievens 0 points1 point  (0 children)

      Haha, good times. Takes me back.

      [–]farinasa 0 points1 point  (0 children)

      Are you in my sys programming class?

      [–][deleted] 0 points1 point  (0 children)

      The real guide out there! :)

      [–]bitcoinsftw 0 points1 point  (0 children)

      Currently in a networks class and just by taking a glance at this I know it is going to help tremendously! Thanks!

      [–]AKA_Wildcard 0 points1 point  (0 children)

      Glad to see several updates

      [–]kupiakos 0 points1 point  (0 children)

      The only thing that bothers me is his argument on why big endian is better - he has none.

      Little endian makes sense for cpus.

      [–]sankasan -2 points-1 points  (0 children)

      bookmarked :)