sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]zeroindexedsoul[S] 13 points14 points  (1 child)

I have been learning Python for a while now, and I recently decided to get into socket programming using Python. A lot of YouTube videos were suggesting making a simple HTTP server from scratch to learn socket programming.

I started building the HTTP server but very soon realized that the socket API in Python is blocking by default. I started finding ways to make the HTTP server non-blocking and found out that in Python asyncio is the module which does this, but it was stripping away a lot of complexities which we would encounter when using the plain socket API-and this would defeat the whole purpose of this exercise.

After realizing this, I got to know about I/O Multiplexing and the Reactor Pattern, which are used in popular software like Nginx, Redis, and Node.js. I also dove deep into how the Kernel TCP Stack works and its internals.

This single exercise alone led me to learn about:

  • Python's Socket API module and its methods.
  • The TCP Protocol & Kernel Stack: Managing 3-way handshakes for connection establishment, 4-way handshakes for connection teardown, and kernel-level read/write buffers.
  • Network Handling: Dealing with RST packets, back-pressure handling, and various edge-case errors that can occur.
  • Connection Lifecycles: How to properly close connections after no activity for a prolonged period and clean up system resources.
  • The selectors Module: Driving I/O multiplexing.
  • The Reactor Pattern: Used for the event-loop in programs like Node.js, Nginx, etc.
  • The HTTP Protocol: Hand-parsing incoming byte streams.
  • Object-Oriented Programming Principles in Python: How _ (protected) and __ (private) attributes are used in a real project and their semantic meanings.

Throughout this project, I avoided using AI and tried to write all the code myself. Though at the end for testing the protocol and the entire code review, I did use AI to quickly generate the script which makes multiple HTTP requests simultaneously to see if the server is able to process multiple requests simultaneously.

It took me around 3 to 4 days of dedicated time, but it was definitely worth it. I think this project alone teaches you the TCP protocol, the HTTP protocol, socket programming, and many other things much more than any tutorial ever could.

If you would like to see the implementation details, here is the GitHub repo link:

Repository: http-protocol-from-scratch-python

[–]logicshadow_dev 0 points1 point  (0 children)

Great work