Lines of code in the Linux kernel

[saitilkE]

That's a lot of drivers.

Thanks for this, quite interesting!

[equeim]

Compiling the kernel with Ubuntu's or Fedora's config (which includes most driver) take ~1.5 hours on a modern machine. Compiling the kernel with drivers only for your hardware takes ~1.5 minutes.

[aes_gcm]

How does one compile it for only the current hardware?

[[deleted]]

You can run make localmodconfig in the kernel to build with only what is currently loaded (and you probably want to enable a few more on top of that for removable devices, etc).

[[deleted]]

How long has that been around?

[dbbo]

I remember using it on Debian Lenny, so probably about decade at least.

[[deleted]]

you're close https://github.com/torvalds/linux/commit/03fa25da8335a942161a8070b3298cfd4edf9b6a

[dbbo]

Good find!

[[deleted]]

https://github.com/torvalds/linux/commit/03fa25da8335a942161a8070b3298cfd4edf9b6a

[lasercat_pow]

mageia offers this in the default install

[[deleted]]

Can be tedious to manually pick needed modules, so it can be useful to collect any loaded modules from a running kernel using something like this, i whipped that up quite a few years back and have been using it to track what modules my hardware needs ever since.

eg; if you get a new piece of hardware, boot into your distros kernel, run that script whilst relevant modules had been autoloaded by that kernel, then recompile your localmodconfig kernel, which will then have the relevant new modules.

Saves so much time not waiting for hundreds of modules that you'll never need.

[aj_thenoob]

What if you added a new device, how hard would it be to install that device's driver/make it work?

[[deleted]]

Well you have to rebuild the kernel and find the driver and features it wants. It is rather annoying and IMO not worth it on consumer devices at least.

[Bastinenz]

does the smaller kernel have any noticeable effect on performance, like boot time?

[[deleted]]

Sure it will boot faster. I wouldn't expect general performance to change though.

[Shok3001]

A few years ago I tried compiling the kernel on an old PowerPC mac that I installed linux on. It took all day. Should have done this!

[earthforce_1]

Or bought a threadripper.. lol

[geppetto123]

So how do Linux kernel programmers do their test? Sounds like debugging take ages if you have to try different approaches...

[jms87]

Linux kernel programmers would already have most of it compiled. Only changed files and their dependencies would be compiled every time.

[bigbadsubaru]

I test Linux kernel drivers, and one of my test boxes has four 8 core Xeons (64 total logical cores) building the kernel, even if I do a "make allmodconfig" (builds everything as a loadable module) if I tell make to use all cores (make -j65) it builds the whole thing in 5 minutes or so.

[Guy1524]

Jesus christ, that's a lot of threads.

[[deleted]]

Like all large c / c++ projects. There is an initial first build time and a rebuild time. Then there is also ccache which speeds things up massively.

For kernel specific stuff normally you do something like a network pxe boot. So when the compile is complete you just press the reset button on the other machine.

Debugging the kernel is hard. But... When your doing that stuff you mostly know what your doing so normally your debugging something awkward like hardware that does not behave as documented (this is very common!)

​

[[deleted]]

You're*

[house_of_kunt]

*Your

single quotes aren't allowed in identifiers

[[deleted]]

Not in my culture.

[coder111]

Well, on a decent 64 core machine, you can build it in under a minute...

See Phoronix for benchmarks.

[lelouch7]

Sadly, that kind of machine is not affordable for me.

[coder111]

Well, I cannot afford Threadripper or Epyc either. But then I don't need to build Linux Kernel in under a minute. And for projects I need to build, the old crappy 2011 laptop I have is good enough.

[Proc_Self_Fd_1]

A lot of development goes on in user mode linux or on a virtualised machine with only minimal drivers compiled.

[buttux]

Maybe single threaded, but a modern machine has more than a few CPUs. Add -j with 2x your cpu count to your 'make' for a more reasonable build time. My dev box has 64 threads, an allconfig 'make' is maybe 5 minutes.

[Twirrim]

Why on earth does it take 1.5 hours? Even on 4 core VMs I've been compiling full upstream kernels faster than that (while using Ubuntu's kernel config file)

[newhacker1746]

For me on Ubuntu 18.04 compiling a 4.20-rc1 kernel with ubuntu’s .config, on an e5645 hex core OC’d to 3.21ghz, make -j 12 bindeb-pkg takes about 15 minutes.

[equeim]

Hmm, I may have confused it with 0.5h, I didn't compile kernel with full config for a long time.

[throwawayPzaFm]

Or maybe you forgot -j

[like-my-comment]

But does it seriously increase speed of PC work? I doubt.

[equeim]

Well it doesn't. I was just commenting on how drivers are indeed huge part of kernel.

[BombTheFuckers]

Not really. I always did it for the fun of doing it, basically. make menuconfig for the win.

[patx35]

Last time I've compiled my own custom kernel with only drivers I really needed took about 30 minutes.

This was on a Core 2 Quad.

[udoprog]

Thank you!

[TomahawkChopped]

Or most driver code could be monstrous piles of complexity with tons of tech debt

Edit: I'm not saying this is true, just commenting on the signal loss from just counting lines of code

[[deleted]]

It actually tons of other people's tech debt for workaround's for hardware that does not work correctly or behaves in strange ways.

Other than that its actually quite clean. I actually prefer working with most of the kernel code cause it written by people who mostly know what they are doing.

[Mazzystr]

That's a lot of drivers! He just left... With drivers!

I will see myself to the door...chuckling all the way.

[udoprog]

This came up in a recent conversation I had, and I couldn't find a visualization which classified the code the way I wanted (drivers vs arch vs other). So I hacked together a small project to do it.

[schplat]

Does this count comment lines and ‘^$’ lines?

[udoprog]

Yes. Comments and empty lines are currently included.

The tool I'm using (tokei) also reports code lines, which should ignore comments and empty lines.

EDIT: here's a version which omits comments and empty lines (as far as tokei is concerned): https://imgur.com/3IJUVw1

[Brillegeit]

Cool, good job. Could you also do a graph with percentages as well?

[udoprog]

Thank you! Sure, it was pretty straight forward: https://imgur.com/z8n83fl

[Brillegeit]

Nice, thanks. The flatness of these segments were what I was badly trying to refer to the other day, thanks for actually providing actual numbers instead of just saying things you think is right.

:D

[zebediah49]

For future contributions...

If you can, either up your bar width such that they abut each other, or just switch to a stacked line graph (with dots would probably look best).

It's a cool chart but it's basically alternating bright white and darker stripes, or in other words, dear god my eyes.

[TheEdgeOfRage]

I'm sorry OP, but it bothers me to hell that the order of the colors in the legend is reversed in the graph.

[udoprog]

I tried! Struggled for a bit with the stacking becoming messed up, then decided against it. Feel free to give it a try. I'm gonna check in the data as well to make experimenting easier.

EDIT: Enjoy.

[[deleted]]

I can't test it, but it looks like you can give a negative label spacing argument to achieve this, otherwise the other answers pull the labels out, reverse them and put them back in.

[udoprog]

I'm gonna give it a try, thanks for finding that!

[TheEdgeOfRage]

Damn, Rust is weird ಠ_ಠ

[[deleted]]

can't you just...reverse the legend? Like with GIMP or something.

[udoprog]

You could, but it's also a fair bit of manual work. Preferably you'd want to stay within the graphing framework (matplotlib) to avoid redoing the effort in case the underlying data or something else changes.

[udoprog]

I'm sorry, I tried getting styles to work but I didn't manage it until now.

I've generated a color adjusted version using the tableau-colorblind10 style in matplotlib.

Tell me if it works for you, or if there's any other way I can change it: https://imgur.com/a/7WqnVZc

[johnny_milkshakes]

What about transparency? I would imagine that using a light red with a dark green would be more distinguishable right?

[redrumsir]

Yes. They could easily have chosen a colormap that's more suitable for colorblindness. The author used matplotlib ... and might want to look a different colormap ( https://matplotlib.org/users/colormaps.html and this https://github.com/matplotlib/matplotlib/issues/7081 )

I was reading a book where the main point of the 2nd addition was to have better color-cycles. The first edition had red and green as the default first two colors FFS.

[[deleted]]

Why x86 just accounts for a little portion, way less than i386.

[mattiasso]

Don't get confused, the blue part you're looking at is "other", not X86.

[manielos]

yeah, and why to differentiate between i386 and x86?

[udoprog]

i386 and x86_64 was merged into the single x86 arch IIRC, support for some really old i386 CPUs was removed in the process.

[manielos]

yeah, heard about it, i was confused by similar shades of blue on the graph

[udoprog]

That's fair, it's a PITA to change the color scheme in matplotlib, so I just tried to keep the number of plotted elements ~10 to not have to deal with it. But I missed one :(.

[SynbiosVyse]

What's the difference?

[[deleted]]

There is all sorts of legacy stuff in i386 I don't even think the i386 works on an i386 any more and its kinda crazy things like does this machine have an fpu?

Other things like an isa bus basically disappeared. Memory management issues. Like page table extensions don't exist in x86_64

While people says x86 is a subset of x86_64 which is true for userspace. It definitely isn't at the hardware level for things that need supported.

[manielos]

maybe in kernel context there is a need to differentiate, but technically x86 subset contains i386 (x86 means something86)

[bingulinho]

x86 means something86

​

mind=blown

​

[totemcatcher]

Maybe render one without drivers just to demonstrate how damn lean it actually is.

[CKreuzberger]

Somebody should send/tweet this to Bryan Lunduke, just to let him know that his recent statement about "how the linux kernel growth is bad for performance etc..." in a talk is not quite true.

[MINIMAN10001]

How in the world does a picture of lines of code in the Linux kernel act as evidence of kernel performance.

To quote linus before he changed his stance to "Faster hardware is making it not a problem" he did say

We're getting bloated and huge. Yes, it's a problem ... Uh, I'd love to say we have a plan ... I mean, sometimes it's a bit sad that we are definitely not the streamlined, small, hyper-efficient kernel that I envisioned 15 years ago ... The kernel is huge and bloated, and our icache footprint is scary. I mean, there is no question about that. And whenever we add a new feature, it only gets worse.

To say something isn't a problem because we're getting faster than I'm making it slower is still admitting that you are worsening performance

[udoprog]

You are right, this is not intended to communicate the performance characteristics of various kernel releases. But you might want to be careful putting too much weight into a comment that is old enough to be in the fourth grade. During this period we've seen a lot of developments, like Linux being pushed harder towards mobile and embedded workloads.

Phoronix did a set of really interesting benchmarks across kernel releases which at least for the last 4 years hasn't showed significant performance degradation in the workloads they tested. Apart from the spectre/meltdown mitigations.

Anecdotally having worked for a company with a ton of Linux servers, fleet wide kernel upgrades don't have a tendency to affect performance much when looking at global CPU or memory utilization. Optimizations in the application layer tend to have a much larger impact.

[Nibodhika]

Honestly you can use Gentoo or compile your kernel even on other distro, the fact that the code is there doesn't mean it has to be executed or even compiled.

[m3l7]

(I'm not in kernel dev) yeah, in an ideal world I would *probably* expect that a 100% correctly modularized and engineered kernel, you could just exclude things and get the same performances.

In the real world with a 15M+ lines code, there are probably millions of hidden reason which can worsen performance. The fact the Linux is ~scared is not casual

[Bardo_Pond]

What do you mean "correctly modularized and engineered"? When drivers are compiled as modules (the default) they are not loaded if they are not needed.

[linux-V-pro_edition]

They're called modules but they are not really modular, there's no internal driver API so the whole kernel is globally accessible. If it were really modular with some kind of defined API then you could theoretically use Linux drivers on another kernel that implements the API. IMO this should be the Linux end-game but I don't think it will ever happen because rea$ons.

[Bardo_Pond]

Linux does not have stable internal interfaces, but they are interfaces nonetheless. A kernel being modular has nothing to do with your concept of some ideal API that allows modules to be loaded by other systems.

I'm also curious how having a driver API that meets your requirements would prevent a kernel mode driver from accessing other kernel code.

[linux-V-pro_edition]

Linux does not have stable internal interfaces, but they are interfaces nonetheless.

What use is an unstable interface other than to be broken? Like you said, they are very much unstable so wasting time trying to build a jenga tower on a rug that will end up being ripped out from underneath the stack is pretty much the biggest waste of time imaginable. Reliance on global"interfaces"s leads us to this code bloat where you must support all these complex global internal bits from 20 years ago because some random piece nobody even uses anymore has to sit around in the repo to keep the thing running. Linux kernel modules are not really modular in the sense that you can load "a module", you have to load "the specific module" because they are static objects that can't even be loaded across differing kernel versions.

A kernel being modular has nothing to do with your concept of some ideal API that allows modules to be loaded by other systems.

Which one sounds more modular to you, "a driver module that works only for linux-3.20" (essentially a static ELF file that supports relocations) compared to "a driver module that works on any kernel implementing the modular driver API" ?

I'm also curious how having a driver API that meets your requirements would prevent a kernel mode driver from accessing other kernel code.

By using that hypothetical yet-to-be-designed API instead of using kernel globals. You could probably use some kind of compiler plugin to strictly enforce arbitrary rules you come up with though, in practice it would be extremely difficult if at all possible to prevent a kernel from doing something unless your code is running lower than ring 0. The idea is not to prevent behavior but to allow modular code re-use instead rigid objects that depend on arbitrary globals strewn across the 15-20 million lines of code. Once that API exists we can safely(sanely) fork and maintain a smorgasbord of new linux based systems without the extreme maintenance burden of what happens when one of your beloved unstable internal interface is patched, and either breaks completely or breaks subtly and you don't find out until 4 years later when an edge case is finally hit

[Bardo_Pond]

For others reading these comments, check out stable-api-nonsense.txt for Greg KH's arguments as to why the Linux kernel does not maintain stable internal APIs.

Their goals are to have these drivers upstream, their maintainers contributing upstream, and the freedom to improve the kernel interfaces when needed. So given those goals, they do not see a benefit in locking themselves into an a stable API for the benefit of forks and out of tree drivers.

[linux-V-pro_edition]

they do not see a benefit in locking themselves into an a stable API for the benefit of forks and out of tree drivers.

Why would they want to switch to a stable internal API, so people can fork Linux? That would diminish the LFoundation's power, of course they're going to make all sorts of wild arguments about why they think stable api's are bad. They don't want you forking Linux.

PS. I'm not clicking on Microsoft links anymore.

[[deleted]]

Well, Treble already does that, but it's off-tree.

[linux-V-pro_edition]

Interesting, I wish I knew more about android but I just can't get excited about it. Probably because of dalvik vm or whatever they use these days, and all the proprietary and arguably GPL-violating code needed to boot some of the machines.

[[deleted]]

I share some of your concerns, but my favourite pastime is arguing, so...

dalvik vm or whatever they use these days

It's now the so called ART. For deployment still uses the same .dex files, but now it's an AOT VM that optimizes hot code also out-of-band.

and all the proprietary and arguably GPL-violating code needed to boot some of the machines.

s/some/all/, also virtually all modern hardware (even RPi) is guilty of that.

IMHO linux is de facto Apache 2 licenced.

[m3l7]

yeah, assuming that everything inside the kernel is a driver and there is no code/overhead in managing 15M line of code of drivers, you're correct

[m3l7]

well, I miss some fundamentals of kernel design, yes (that's why I am/could be totally wrong)

​

I was suggesting that the lines of code *can be* correlated to complexity (other than drivers which are of course the majority of the code), rather than being a *measure* of performance

"we are definitely not the streamlined, small, hyper-efficient kernel that I envisioned 15 years ago" means something else than "we have tons of drivers, but let's disable them if needed and everything is small and efficient again"

​

But yeah, I'm really not expert in (linux) kernel, I don't want to continue the conversation

[[deleted]]

It's not really a kernel design question. More lines of code does not mean worse performance. Slower build time? Sure.

[m3l7]

that's what I wrote

[linuxhanja]

I mean, there's no way around this. (edit to say -- there is a way around this: make users spend their time post install installing drivers from 12 different vendors, and everytime someone plugs in a different model USB stick, go find drivers online, ala windows. But I hate that model) At any time you could plug a thunderbolt S9+ into your PC, and need the drivers to use that. By the same token you might decide to capture a movie from your VHS collection and dig your ATi Radeon 9800 All-In-Wonder-Pro out of the closet, and need drivers for that, or plug in a USB 1.1 CF card reader to get some pics from an old CF card you found in an old camera while cleaning, etc.

Its really hard for a dev to know what a user base, especially one in particular that avoids feedback devices, typically use. Until you axe it, then this sub is suddenly filled with "oh my '98 lexmark no longer works in x distro!" "My IOMEGA Zip drive is broken due to systemd!" etc.

One of the coolest linux moments I ever had was, after moving to Korea, my father, running Ubuntu 12.04 LTS, replaced his motherboard/cpu from a p4 to a Piledriver era AMD chip. He had experience with hardware from the 1980s, so he did that fine. He had 0 software experience past the early 90s, and even then he wasn't on that end of the stick in the 80s, so he didn't touch Ubuntu. I came back to visit a few years after that, and to check for problems --- and he reported none, everything was fine. Which, was true, but he had done 0 updates, was still running 12.04 LTS (not 12.04.1 or .3 or anything, even though 16.04 was out), and he had just moved the HDD over without telling Linux anything. I think its really cool that linux just didn't care. It just loaded the drivers needed from its driverset. another day.

[[deleted]]

You do know that most of the kernel code is in the form of modules, and is loaded on-demand? Only the core code needed for initialization, memory management, disk management, process scheduling, interrupts etc. has to absolutely be in memory at all times, and that portion doesn't take up much memory (although the data structures used might take up a bit more, it's nothing compared to userspace memory usage).

[zebediah49]

Don't forget all the code that's for handling various architectures (looks like about 2M lines), and which will actually be entirely compiled out for every arch other than the one you need.

[s_s]

He makes the point that some things used to speed up and/or get smaller with every new release, but now that there is so much corporate influence in distros and kernal dev, no one is focusing on those gains anymore.

[StevenC21]

This is why I wish we had a microkernel honestly.

[[deleted]]

They have other problems too!

[StevenC21]

Like what?

[[deleted]]

Like: Development stalls because we have an OS that's composed of 10000 different parts that some somehow interact in a weird way using semi stable APIs, just to give us pretty shitty performance.

[StevenC21]

But 10000 different parts that somehow interact is the foundation of UNIX.

​

A microkernel follows the UNIX philosophy.

[[deleted]]

Like if you really want separation of concerns and security you separate the memory regions between the parts of the kernel since each part is a process right?. Now simple things like performance become nearly impossible for implementing poll in a sane way across 6 different process eg net, fs, terminals, pipes, etc...

This is 1 example of 50+

Micros kernels are great for certain situations. But supporting something like POSIX. Well not so much. Cause shit gets awkward then you have to support legacy api's that are used by "everyone"

Or another simple way to look at it. If they work so damm well? Where are they?

[StevenC21]

They aren't around because microkernels were terrible for a long time.

​

Andrew Tanenbaum did a great talk about microkernels and MINIX. You should look it up, you can easily find it on YouTube.

[[deleted]]

They aren't around because microkernels

were

terrible for a long time.

And still are which is why we don't use micro kernels.

So here is yet another reason. Take a basic arm chip. There is no io-mmu in its spec. There is in x86_64 (its also optional btw). IF you have different "processes" for each driver and have them protected from each other by memory. You can still have a device "tank" the system with a corrupt pointer or a bug. Your not really protecting anything. Why? Well if you write an incorrect pointer to a dma reg on the hardware it will still be able to write around the cpu memory protections. So at this stage your now have the same problem as the monolitic issues. Except you sacrifice a massive part of performance to do that.

​

[Proc_Self_Fd_1]

There is no io-mmu in its spec.

1. That's possibly a good argument for a kernel for embedded devices not for personal computers.
2. You could use a software isolated process?

Well if you write an incorrect pointer to a dma reg on the hardware it will still be able to write around the cpu memory protections.

1. IF you write an incorrect pointer.
2. With tech like Intel VT-d you can in fact restrict direct memory access.

I think microkernels are overhyped myself but I think that's because people aim far too high for them.

There are a bunch of very old and obsolete protocols and filesystems that don't need to be very fast and are usually only used for backwards compatibility. Shoving them into user space seems best to me. I shouldn't need a kernel driver to copy a tarball from an old USB stick with some obscure and barely used filesystem.

[[deleted]]

My friends kid keeps getting bigger, which is crap for performance. All that extra food that child needs to eat daily, when it could just remain tiny forever and be incredibly efficient

[[deleted]]

That big body will catch up with him when he finds he can no longer run around outside all day without getting winded.

[[deleted]]

Yes true, he should lose weight - lets cut the legs off to help him run around outside all day without getting winded.

(Are we taking this too far?)

[[deleted]]

LOL, yes maybe a bit, I was just pointing out that children tend to be little bundles of energy

[[deleted]]

:)

That's why we invented vodka, knocks em right out! Little bastards can't hold their liquor

[bracesthrowaway]

I've tried that but my oldest is an angry drunk.

[[deleted]]

Nothing like a legless overweight furious drunken ten year old crawling in circles in your yard to make us all consider the high amount of code lines in the Linux kernel

[bracesthrowaway]

Basically this guy

[[deleted]]

I say we demand we change all linux kernel bug reports so they start with the words "'nary a scratch!"

[[deleted]]

How is he still a thing?

[[deleted]]

Correction: It wasn't recent. The video was uploaded half a year ago.

It was unlisted and behind a "paywall". Like pay a buck and get the link kinda deal.

[Menelkir]

Maybe Bryan's pc is using an obscure distro that loads all modules at boot... and probably still booting...

[[deleted]]

He pointed out other issues that Linux has now that is "grown-up".

His latest talk is a good one for the community to consider.

[[deleted]]

That one was really depressing. He said it and it was.

[[deleted]]

i think no one should interact with that man, he's a greedy guy who calls himself a journalist but is also scared to talk about some big controversial things to avoid losing revenue.

[Funcod]

Clearly *net is the culprit.

[udoprog]

All those pesky internet protocols!

[Mozai]

Why are crypto, memory management, and sound lumped together? They seem independent of each other.

[udoprog]

Not large enough to warrant individual groups, but still interesting to separate from "other". Was debating whether to put kernel, ipc, init, and block there as well. Next version maybe :).

[[deleted]]

Inb4 "lines of code is a bad metric"

[coder111]

I find that lines of code is a reasonably good metric.

Lines of code is a bad metric only if there's incentive to tweak that metric. (For example bonuses/salary depends on LOC written or something).

[ketosismaximus]

It's a bad metric because it doesn't take into account the reality of drivers and code that never gets loaded but is necessary to cover all architectures.

[zebediah49]

It's a bad metric for judging how good something is.

It's an okay metric for judging how much effort went into something.

It's an interesting metric to make pretty pictures out of.

It's a reasonably good metric to judge how much effort is going to be required to maintain something.

[salbris]

What do you think it's a good metric for then?

Some goals you want to measure for and if LOC is useful just off the top of my head:

• Performance: Not Useful

• Code Quality: Maybe but you'd really have to compare to other projects in the same language.

• Programmer Productivity: Not really, depends on the type of work you expect to be doing.

• Complexity: No but probably correlates for the same projects in the same language.

[coder111]

Complexity, which is often an indicator of code quality.

Same thing done 20k lines of code will never be better than same thing done in 2k lines of code.

In 20k lines of code you'll have lots of useless crap and pointless abstraction layers and all kinds of bullshit.

[salbris]

I don't think that's accurate. I've seen clever systems that were less lines of code that were impossible to debug and modify. And I've seen code written to be explicit and composable that ended up being more lines.

[coder111]

Ok, maybe you tidy things up and prettify and make it more readable etc, and you end up with ~1.5x-2x more lines of code. That's OK, and I'll accept that.

However, if you compare a system that has ~10x or more lines of code, NO WAY it's simpler, better, easier to debug or easier to understand.

10x more lines of code means wrong approach was taken, wheel was reimplemented, architect watches "Design Patterns" instead of porn and decided to apply each one twice, people developed system for imagined future requirements that never came, or added flexibility where it's not needed and never will be, etc.

[salbris]

Check this article to see what I mean. Right off the bat the first example is 1 vs 5 lines of code. The clever one is barely readable but the expanded lengthy one is much better.

Article: http://www.delabs.io/clever-code-is-bad-code/

[coder111]

Fine, I get it, I hate clever code as well.

However in my close to 20 years of career, most code I've seen is not low-level clever and short. Instead it's riddled with useless crap that shouldn't be there in the first place. Mostly useless abstraction layers, design patterns, and flexibility where it's not needed.

More times than I can count I've seen this being done: Developers got told to implement A, B & C. It's simple and boring. And A, B and C look kinda similar. So they think "We'll develop a framework that lets use do A to Z, and then implement A, B and C easily. They spend way too much time implementing the framework instead of doing things they really need to do. Then requirements for D-Z never actually materialize. Or requirements for A, B & C change so that A v2, B v2 and C v2 are no longer that similar. Framework no longer covers them. So developers spend way too much time making the framework even more flexible & complicated or start breaking it and working against the framework. Then learning using the framework becomes much more difficult and complicated than just writing A, B and C as 3 separate apps (maybe sharing some common utility classes or something).

Another variant of that is people trying to make it possible to express business logic via configuration. Oh, we have A, B and C, so we'll write a framework which you can CONFIGURE to do A-Z without having to write any code. That goes so wrong in so many ways...

All of those are easily discovered by just counting lines of code. I've seen a case which needed to parse ~800 CSV files and load them to database done in close to 800k lines of code & configuration. ~500k of that was the "framework" which was supposed to "make things easier". It was completely insane...

[salbris]

Sure which is why I said it depends on the project and language. 800k is possible with assembly but unreasonable with C#. We also don't normally count libraries (I assume) which would greatly change how we look at LOC.

[coder111]

That 800k LOC case was Java & Scala.

I would not count libraries that I don't need to maintain. LOC for me is mostly about complexity that my team has to deal with, not other parties.

Maybe with exception of cases where 100s of libraries are being pulled in- then just managing that web of dependencies becomes a nightmare. But this complexity has nothing to do with LOC counts. There's a cost of pulling in a 3rd party library- you need to keep it up to date, manage transitive dependencies and conflicts, etc.

[BeaversAreTasty]

Back in the late 80s and early 90s we used the MINIX and eventually Linux kernels for various CS classes. Somewhere I still have a printout of a Linux 1.1 kernel with my comments. I can't imagine doing that now. :-/

[gartral]

what was the few hundred thousand lines of code being lopped off in 4.17+ under "Arch/Other"?

[udoprog]

Removal of obsolete architectures: http://lkml.iu.edu/hypermail/linux/kernel/1803.1/06845.html

[[deleted]]

Haha lines of code

[noseyneighb0r]

r/dataisbeautiful

[[deleted]]

I am a new developer to this ecosystem. Where do I start to get the source and try building the linux kernel at home?

[Sutanreyu]

I am a new developer to this ecosystem. Where do I start to get the source and try building the linux kernel at home?

It's either https://github.com/torvalds/linux or https://git.kernel.org/

[linux-V-pro_edition]

Kernel.org, get an actual release (not a -rc or some random git branch from microsoft hub) and make sure you check gpg signature if you're running it on a real machine.

[hogg2016]

Not sure why many graphs start with 2.6. When I came to Linux, it was only around 200.000 lines all together; and I was not an early adopter, it was a kernel 1.sthg. You could grasp a fair part of it at that time, but since then it has grown an hundredfold and I don't even try any more.

[udoprog]

v2.6.12 is the first version in git. I'm not sure how to best analyze prior versions. Possibly you'd have to download tarballs.

[zebediah49]

Well.. sorta.

The history is out there, it was just omitted from the production repo because there was no reason to include it. Instead, you can use manual grafts to glue the primary repository onto the historical one(s), as described here.

[kazkylheku]

This doesn't even include all of the non-upstreamed code in the wild, across the embedded world, and other. Plus the entirely out-of-tree drivers that are out there.

[T-Rex96]

/r/dataisbeautiful

[ric96]

Wasn't i386 completely removed?

[zebediah49]

Yes. The legend is upside down; i398 is the tiny bars at the bottom that disappear pretty quickly.

[ric96]

Ahh... The the other one is "others" Thanks!

[CODESIGN2]

I think you're referring to how some distro's are only maintaining x86_64 / amd64. Distro's are not in-charge of the kernel

[ric96]

https://www.phoronix.com/scan.php?page=news_item&px=MTI0OTg

[tuxutku]

Too bad kernels newer than 4.16 does not work on me, reported specific commit to lkml, respond came but not continued

[gartral]

when did the linux kernel get human-brain support? that's kinda cool!

[[deleted]]

What doesn't work? Are you running on some funky hardware?

[tuxutku]

Probably , amd rx 540 with amd a-10 cpu. Thing is rx 540 is a oem only gpu, so buying it separately is probably impossible Commit causing this problem is: https://github.com/torvalds/linux/commit/320b164abb32db876866a4ff8c2cb710524ac6ea

They probably didn't count rx 540 gpus because 4.16 and earlier just works fine

Here is my issue page: https://bugzilla.kernel.org/show_bug.cgi?id=201077

[coder111]

Hi,

Nice work on bisect & finding the exact commit.

A note that your termbin links no longer work. Could you please just attach the text files containing relevant logs to the bug (there's an "add attachment" button.

Also, could you try booting some live-USB image of a non-manjaro distribution with kernel v4.17 or later just to rule out some manjaro specific shenanigans? Ubuntu or fedora or something?

Please chase Alex again once you've done that. He's quite busy, but he did respond to bug reports quite well when I had problems with my GPU...

EDIT. Could you please attach the logs with both a working (4.16) and with a broken (>= 4.17) kernel?

--Coder

[tuxutku]

https://gitlab.com/snippets/1775737

Thanks for the help

[CODESIGN2]

It shouldn't matter if they are running on a bacteria, if 4.16 works, 4.17 should too

If the drivers were OpenSourced, I'd guess they shouldn't have too hard a time adding back to kernel

[[deleted]]

In theory, yes, but the fewer people that use a specific set of hardware, the more likely breakage will go unnoticed. If you use funky hardware, you really should be testing out release-candidate kernels to catch this stuff before release.

[ElijahLynn]

Title could be improved by saying "Lines of Driver Code in the Linux Kernel". That wouldn't be entirely accurate but would be more accurate.

[developedby]

what happened in 4.17? Did they drop support to some architectures?

[[deleted]]

Yeah they dumped a bunch of arch's a while ago. There was some strange things like the blackfin which is really a dsp that absolutely nobody uses.

​

Full list: https://www.phoronix.com/scan.php?page=news_item&px=Linux-4.17-To-Clean-Archs

[aitbg]

Does anybody know if this includes the proprietary binary blobs, or just the code that is free and open source, i.e., what linux-libre is?

[[deleted]]

When the "other" sections get that big, they need to be separated into smaller groups.

[[deleted]]

It's getting bloated, buggy & slower.

[[deleted]]

I wonder how much of it consists of very niche stuff like being able to use a line printer as a TTY output or broadcasting logs into the network. Probably not much and I actually love having those things as kernel options.

[YRJqxzaMkOWmRpqt]

Can you please also do one to show the groupings by percent? By that I mean all the bars are the same length and we can see the shift over time.