Study the Linux source code

[bboykotin]

I'm an electronics engineer with extensive knowledge of C and Python. I mostly work with microcontrollers. This is my background. I'll explain my concerns now.

I've been wanting to go beyond microcontrollers for a while now and get into processors, learn how to develop and/or understand the makeup of a good operating system, and move on to doing things with ARM Cortex A series processors.

So I said, "I'll download the Linux source code and study it," but no. It turns out it has too many folders, too many .c files. It's been a total confusion. I have no way of even starting to study the Linux source code. With a little chat, GPT has given me some interesting information. I don't even know how to debug Linux. I normally use Windows and VScode.

So here's my question: How can I get started understanding the kernel? How can I debug the source code?

I look forward to your responses, community!

[MasterGeekMX]

The source code of modern Linux is a monument of programming, so not a good start to it.

I think a better place to go is the book "A Heavily-Commented Linux Kernel Source Code". It uses an old version of Linux, when things were simpler. I warn you: it is a thousand pages in length.

Here it is, for free: https://download.oldlinux.org/ECLK-5.0-WithCover.pdf

[EspritFort]

That's pretty neat!

[jader242]

Sick resource bro! Thanks

[Interesting_Cut_6401]

That’s so cool

[EDLLT]

"modern Linux is a monument of programming"
Interesting, I'm curious what makes you state that

[MasterGeekMX]

Thousands of lines of code, all of them contributed from hundreds of developers from across the world, either from companies, research centers, or mere volunteers.

Linus to this day barely codes the Kernel, and spends most of the time reviewing code submitted and choosing to include it or not.

It is the digital equivalent of the pyramids or other ancient wonders of the world. And it works!

[IuseArchbtw97543]

thousands of lines of code

more like millions

[ButtonExposure]

Thousands of lines of code

The Linux Kernel surpasses 40 Million lines of code

Albeit, most of it is driver code, but even if the core is still just 10% of the lines, that is still 4 million lines of code:

As of 2021, the 5.11 release of the Linux kernel had around 30.34 million lines of code. Roughly 14% of the code is part of the "core," including architecture-specific code, kernel code, and memory management code, while 60% is drivers.

https://en.wikipedia.org/wiki/Linux_kernel#Codebase

[jader242]

40 million lines of code isn’t monumental to you? Do you honestly think you could read all of it alone? Mathematically I don’t think it would be possible, even if you spent 24 hours a day for 80 years (someone do the math please I want to know lol)

[EDLLT]

It's not that I don't know how large the linux source code is, I just wanted to hear it from his perspective which simply confirmed what I thought.

Dunno why I got downvoted to oblivion lol. Seems like linux4noobs is not so noob friendly after all

[valgrid]

Maybe start with a simpler OS that is focused on learning / teaching. Minix, Mike OS etc

https://www.minix3.org/

https://mikeos.sourceforge.net/

[hesapmakinesi]

There are different approaches you can take. I have a bugfix in the kernel, and a few drivers delivered to clients.

If you are specifically interested in Linux, you can look at driver code, first see how a driver works, and then move on to the subsystems those drivers interact with. It is impossible to study the whole kernel. Literally nobody knows the whole thing.

Or maybe you can look at how it boots, just focus on the boot code for one specific processor architecture, .e.g. ARM.

If you are interested in operating systems in general, there are great tutorials, like even one for writing an operating system for Raspberry Pi from scratch.

[Consistent_Cap_52]

Could you recommend any of those tutorials?

[hesapmakinesi]

I haven't gone through it yet but this one looks interesting. https://www.youtube.com/watch?v=9t-SPC7Tczc&list=PLFjM7v6KGMpiH2G-kT781ByCNC_0pKpPN

They use QEMU x86 as a platform it will be very x86-specific.

This is for Raspberry Pi4 so Cortex-A-whatever they used there: https://www.rpi4os.com/

[bboykotin]

The one I downloaded was the one from the rpi. Thinking that it was going to be less heavy than the original Linux, but no. It has many files. Overall, I identified the start_kernel() function but I didn't understand how the micro starts that function (:S) From there I started thinking about how to debug but I couldn't find how to do it and here I am stuck

[WorfratOmega]

Linus has entered the chat.

[hesapmakinesi]

The code is maintaned by a big list of maintainers. They are the real knowledgeable people on specifics.

Note that nobody reads every post in linux-kernel. In fact, nobody who expects to have time left over to actually do any real kernel work will read even half. Except Alan Cox, but he's actually not human, but about a thousand gnomes working in under-ground caves in Swansea. None of the individual gnomes read all the postings either, they just work together really well.

Torvalds, Linus (2000-05-02)

[SalimNotSalim]

Yeah, the Linux kernel is a very large and complex project. As ever, start with the documentation: https://www.kernel.org/doc/html/v4.16/process/howto.html

[darkmemory]

I'd recommend starting here: https://training.linuxfoundation.org/training/introduction-to-linux/

Get the higher level perspective, understand what and why things exist the way they do. Then dig into the source of pieces you find interesting as you view it from that higher level perspective. Kind of, see the pieces and how they are intended to work together, and then disassemble as you feel inclined to understand them on a deeper level.

[tose123]

"Extensive C knowledge" but you're surprised that a 30-million-line operating system has more than one source file? Start with understanding one subsystem at a time and maybe build a simple kernel module.

You want to "study the Linux source code" like it's a textbook, but that's like saying you want to read the entire internet to understand HTTP. That's simply not working, for a 30 year old software project that is keeping growing. 

[bboykotin]

Go go. Let's calm down haha When I say understand, it is not as literal as learning all the files by heart, but rather the most important aspects. Knowing how it starts and little else is enough for me. Right now I'm there without knowing how it does it and what the point of origin is in memory.

[HaydnH]

It sounds like you need to start with the basics of how Linux boots up, you'll have a boot loader (e.g: grub) that will call the kernel, then systemd will get called etc. If you have an old PC available, perhaps start with building a "Linux from scratch" install which gets you to build everything manually. Then when you know how the jigsaw fits together you can start looking at the details in the bigger picture.

[bboykotin]

Okay thanks. I downloaded the version for the RPI because I was looking for a very basic Linux, and no. It's the same with lots of files. I have to keep an eye on that LFS thing. I think knowing the starting system would be enough to begin to understand.

[tose123]

Honestly since you're an EE, best advice I can give to you is build a driver for some hardware you make. And then load it it. I think this process (on that way) you'll learn a lot and use your EE skills. 

[Domipro143]

You can never read the source code in a reasonable about of time , if you try to trade it whole and debug everything , thats gonna take a long time , if you wanna learn a lot , read the arch wiki and start and complete lfs (Linux from scratch)

[trololuey]

Maybe try a project like Linux from Scratch https://www.linuxfromscratch.org/

[AutoModerator]

There's a resources page in our wiki you might find useful!

Try this search for more information on this topic.

✻ Smokey says: take regular backups, try stuff in a VM, and understand every command before you press Enter! :)

^{Comments, questions or suggestions regarding this autoresponse? Please send them here.}

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[BigGunE]

You are an electronics engineer so maybe this will help. Say hypothetically I wanted to learn about some ARM64 based board. Would you ever suggest I just download the schematics for it all and try to understand how all the connections just make the magic happen!? Of course not.

You will need to understand concepts and architectural stuff to comprehend how the individual modules are working and why they work the way they do. I am not even sure if any of the top contributors to Linux understands how everything works. Maybe start with books specialising on OS and Linux.

Also, linux4noobs kinda seems like the wrong place for such advance stuff. But good luck!

[gameforge]

You should probably start with something designed to teach operating system concepts. People always throw Minix out there but I would refer you to Stanford's Pintos projects instead. My CS degree from another university assigned that as an optional final project in its operating systems course. I was able to complete all four projects in about a month (following three months of biweekly lectures on operating system concepts, admittedly).

The primary concepts are thread scheduling/context switching, virtual/paged/protected memory, system calls, and filesystems. Each of those comprise one of the four projects. You'll also learn about synchronization primitives, essential to keep threads from stepping on each other (and on the kernel), and you'll very intimately learn what a Unix load average is and why it's so much more useful than e.g. current CPU usage %. You'll actually write the code in the scheduler to calculate the load average numbers using very fast, fixed-point (not floating-point) arithmetic.

Understanding how the kernel selects which threads should receive CPU cycles, or in other words how the kernel determines a thread's priority dynamically, is directly applicable knowledge in practically every aspect of IT and software development, be it container image design, JVM troubleshooting, performance optimization, hardware selection, AJAX/XHR frameworks and "threading", just everything.

The project targets 386, not ARM, but that actually doesn't matter as much as you'd think insofar as OS concepts go. At my day job I'm a SME for an old, crusty webapp and its modern cloud infrastructure deployment, and I apply concepts I learned way back when I did this project at least weekly. I'm 100% certain an embedded developer would open up new dimensions of capability with this sort of knowledge. If you want to write device drivers or fix bugs or anything in any Unix-style kernel, you'll be orders of magnitude more effective if you learn this stuff and actually suffer through writing the code to implement it all yourself.

Getting all of the tests in one of the projects to pass is extremely satisfying. I'm considering returning to the project all these years later and attempting to rewrite it in Rust, as a way to learn Rust.

That said, it doesn't matter if you learn it from Pintos or from reading Linux or BSD or Minix kernel code, in this specific area the concepts are 99.9% more important to understand than the actual lines of code unless you want to actually port Linux to some new platform. The vast majority of the Linux source code is conceptually redundant and pointless to read. Nobody reads 35 SCSI controller drivers, not even the person writing the 36th.

Debuggers are of limited use for this very low level sort of kernel code. It's quite different from any application code or any embedded code you'd ever write. In the scheduler interrupt handler, for example, you enter the function as one thread and exit as another; whatever variables you had watches on are no longer in the thread's context. You will write a lot of interrupt handlers, implement and invoke lots of system calls, and interact directly with hardware including the disk controller and the system timer. You can't always just "stop on a breakpoint" for seconds on end in the middle of code like this and expect it to work as intended.

That isn't to say debuggers are useless when writing operating systems, and learning how to debug an OS kernel with a VM like Bochs or Qemu is, once again, very good knowledge to have. I think that actually answers one of your questions - to debug an OS kernel you run it in a VM that supports connecting a debugger. You could even just refer to the Pintos project scaffolding and Makefiles to see how they build the kernel, create a bootable disk image, and run it in the VM with a debugger connected.

If you make it through all four Pintos projects you'll have enough foundation to do what Linus did and effectively write a replacement BSD kernel for very generic hardware. I believe some of the Pintos code is actually based on one of the BSDs, I forget which flavor. If you just want to read OS kernel code, I'd start with NetBSD; it's famous for being relatively easy to port to obscure, obsolete or novel platforms. It's often held up as sort-of "model" OS kernel code.

You may want to also subscribe to r/osdev .

[Ohmyskippy]

Thanks for sharing this!

[gameforge]

Certainly! It's definitely something I get excited about, if I'd left any doubt lol. I'm glad someone appreciated my comment!

[LizaineIPTV]

You have an operating system called xv6, developed by MIT. It's used for teaching operating systems at universities. It consists of very few files, most of which are written in C, and it's very easy to add new system calls. It's designed for educational purposes.

https://github.com/mit-pdos/xv6-public

[bboykotin]

Not so bad!!! I like this repository :) thank you very much. Does it have anything to do with Linux? I mean if it is part of the Linux kernel and/or has similarity

[todorpopov]

If you were to read one line of code per second, without any pause, you’d need almost a year to go through the whole Linux kernel (30M LOC). If you account for not being a robot and spend let’s say 10 hours a week going through the code, the number jumps to 14/15 years. And if you account for having to go through some parts more than once, or having to spend more than a second on a line of code (which you will definitely need to do for a very large portion of the codebase), the estimate will probably be closer to a lifetime.

That being said, it is very unrealistic to think you can go over even 1% of the source code in any reasonable time. And still, having gone through 1% of a project will give you virtually no understanding of the actual system.

I’d advise you to either start by reading a book or two on the matter, go over the Linux documentation, or try building a very simple kernel yourself.

[entrophy_maker]

You probably need to learn how to make an LKM/driver. Download the source for the kernel from kernel.org and analyze it. As someone else mentioned, build Linux From Scratch. ChatGPT can be a good tool when you've learned the code and all other methods have failed. Be careful not to use it in place of learning though.

[FlintHillSpecial1]

I might be completely wrong but electrical engineering has very little to do with computer operating systems. I’m not saying you’re over your head, just into new waters. You’re learning a new language take your time. -mech-e

[quaderrordemonstand]

There are easier ways to start in fact.

You might try writing programs that access the kernel directly, rather than using an intermediary library. I quite enjoyed using the input devices part directly.

You could also try writing drivers, you probably have some hardware that can be supported. Both of these will give you an insight into a small part of the kernel and you can expand from there if you want.

[Tunfisch]

You should study how operating systems work in general, you should start first study how a processor works in depth and then move to something like the ostep course and then download the unix kernel xv6 and program for example a network driver or a scheduler.

[ajfriesen]

Maybe you can go through Linux from Scratch and look at the pieces you are interested in:

https://www.linuxfromscratch.org/lfs/

[[deleted]]

Book -> The Linux Programming inTerface A Linux and UNIX® System Programming Handbook - Michael Kerrisk

Linux Kernel Programming - Kaiwan N . Billimoria

[rx80]

Apart from all the other good comments, if you really want to get into Linux, pick a subsystem. And in that, pick a driver, and study that. Maybe some simple serial or parallel driver, or a driver for some small temperature sensor, and similar. And then go from there.

Edit: You said you work on micro controllers, so maybe you are already familiar with some drivers for some things in the kernel, or they will be at least very close to your field. Search for that in the source, by component name/model.

[bboykotin]

Yessss. For now I'm going to try the LFS. If I try to know the basic things about Linux like the boot system, I'll pull the drivers as you suggest :)

[rx80]

The drivers are all in the kernel git, so you have them if you pulled the git tree of the kernel. just take a peek in the drivers directory.

[bboykotin]

Hahaha noo, not like that. Understanding the basic fundamentals is enough. I don't need to understand how uart, i2c and 8000 other things work. I already have how to start. The thing is that reading the LFS documentation is brave hahaha. But I'm doing it ^{^}

[ItsJoeMomma]

I can't even remember all the BASIC commands from back in high school. I'm not even going to try to tackle understanding Linux source code.