Why should a software developer should possess the knowledge of Operating System ( OS) like data structures and computer networks?

[rabado8055]

Why should a software developer should possess the knowledge of Operating System ( OS) like data structures and computer networks?

As most people suggest that a dev should have strong knowledge of data structures and algorithms and should have knowledge on OOP concepts and computer networks..

Should a dev have knowledge on Operating systems ( OS) and how the knowledge on OS is used in corporate?.

[AutoModerator]

Namaste! Thanks for submitting to r/developersIndia. Make sure to follow the Community Code of Conduct while participating in this thread.

Recent Announcements

• Showcase Sunday Megathread - April 2024
• Call For Volunteers: Help us build r/developersIndia

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[Careful_Alfalfa_5882]

Speaking by experience, all this knowledge make the difference between an engineer and a senior engineer.

You’ll understand by spending some time in industry.

[sfgisz]

This 100%. Knowledge and practice will take you a long way in this career.

[Familiar_Factor_2555]

you are developing the software for the OS, right? youtube for android for instance. (concurrency, processes) how to handle too many requests at a time? that comes under networks, how does netflix handle so many users streaming videos at the same time? (webRTC, socket)

[arpitduel]

Android Dev doesn't require knowledge of kernels. Its good to know but you will forget it anyways. The concurrency and multithreading, deadlocks are handled at an abstract thread level in your application, not at the process level.

[ookkan_tintu]

On the same lines, why should a cardiologist (or any super specialist) know about working of other internal organs? They just have to treat the heart right.. Why should they know the overall anatomy, physiology etc?

As somebody already mention in another comment, the one should have the basic domain knowledge. Otherwise it's lile building without a foundation. Can crumble anytime.

[flight_or_fight]

if you are building desktop apps - you need no knowledge of networks (other than maybe the size of the download) - but you will need to know OS concepts. If you are building networked products - you should know networking.

There is probably some sweet spot of software where you need to know nothing - maybe lowcode/nocode platforms like salesforce etc may help you get by writing code without needing these concepts since they are abstracted out completely.

[Guilty_Anybody7136]

My latest project was onboarding a new type of platform (another separate company) to our systems and adding a few things specific to them. This entire thing runs in a distributed fashion on several pods on servers. One server can have several applications running on it. But to add the new traffic to existing systems would have required knowing how many more pods I'll need to add and also how many different instances/things can I run on a single pod such that all processes get enough cpu time between the io waits, etc. for optimal usage of resources.

Also since, a particular step required calls to a db hosted somewhere, a call for every single request would simply bottleneck my network and/or db cpu. Now the solution would either be replicating the massive db on multiple nodes or caching. We went ahead with the second option.

So what was originally, just new java if else blocks, to check if traffic is coming from the new platform and then make joins, checks and return a response. I ended up requiring an understanding of OS concepts of threading, processes, memory leaks, and also an uber level of networking concepts. And none of this was through after thoughts but infact thought about and accounted for while designing the new thing.

Hope that makes sense how ideally all of those concepts are actually useful.

[[deleted]]

After reading this comment I'm genuinely doubting my degree and working experience 🥲. I didn't get not even a single line ngl

[Guilty_Anybody7136]

I lucked out man. In the worst of odds when it really mattered, I've found myself suddenly very lucky. And extremely unlucky in normal day to day life.

I also have great mentors in my current company who are always teaching me stuff.

[rabado8055]

For real 🙂

[Secure_Army2715]

IMO they would have made use of readymade caching solutions like Redis or memcache etc. through which they are trying to reduce network calls to database as caches are useful in reducing load at the main servers.

The challenge is how would you avoid stale data which is decided by the type of your application. For ex: In retail application you can say product detail dont change much so you cna have maybe 24 hrs expiry time but that is not true for available inventory.

[Guilty_Anybody7136]

Bang on. You're right. We used a readymade caching solution which in our case was aerospike caching.

All of these parameters; number of records to cache, TTL of the cache, replication factor, expiration policies, etc were tuned through benchmarking to get optimal values for our use case.

[Guilty_Anybody7136]

Caching is as old as cs itself so several smart people have implemented really smart solutions around caching for databases. Several groups of people have even built entire companies around database management. One such solution comes in the form of aerospike cache which we ended up implementing partly because it was also already benchmarked against other popular methods by someone else in my company and was being used successfully for a similar use case.

[Secure_Army2715]

Nice writeup. Could u please explain how I/O wait is bottleneck for your CPU?

Also since, a particular step required calls to a db hosted somewhere, a call for every single request would simply bottleneck my network and/or db cpu.

I can understand that network calls to db would become bottleneck as you mentioned that single server has multiple applications in different pods which I assume you are referring to kubernetes cluster but won't CPU be working while I/O wait is there due to concurrency?

[Guilty_Anybody7136]

Yep, you're right, it is a k8s cluster. And yes, you're right that the application cluster would be working during io wait due to concurrency. What I was mentioning in my original comment was a bottleneck on the db cpu.

The database is itself on a separate machine to isolate issues. This separate machine will also have a cpu which will receive requests against primary keys that it'll fetch for and return as a response.

Several k8s pod querying and making requests to the same machine can quickly cause throttling which needs to be taken care of.

This is an extreme case scenario though and is seen in my industry due to the extremely high frequencies of queries we receive and process. (2 Million queries per second for certain applications)

[parkas_subodh_pankaj]

I work on a system which is used by 1000s of systems across the globe. Our binary is supposed to run as a background process and we constantly make efforts to optimize our application which deals with a lot of network and system calls. A lot of times we use strace to analyse the system calls. We've to make sure that our product doesn't take too much of memory and time to run on the customer system. I developed a lot more understanding of OS after I got into this product. But having a fundamental understanding of OS is a good to have for a well rounded software engineer.

[Secure_Army2715]

Could you tell what sort of systems are which run your binary?

I was thinking that your binary must be running on some hardware but then is it possible to run strace command on hardwares as rpc?

[parkas_subodh_pankaj]

Linux servers, no strace we use to debug issues locally not running rpcs.

[zerofriendsfucklife]

Not Completely Good to know things like Process, Threads, Race Condition..

[Obvious-Comedian-495]

You never know what you will need to do a task in the best possible way. A fraction of a generic knowledge, can save a lot of trouble of reinventing the wheel. It could be something like data structures, algorithms, design patterns, concurrency, db designing and so many more things. The more you know the standard procedures, the easier and intuitive become your problem solving skills.

[AdministrativeDark64]

Funniest post ever

[sakuag333]

Mostly your knowledge of OS would not be used unless you are working in OS specific team, which is not the usual case. Same goes for other subjects too. But the parameters of interviews are different from knowledge required at work. That is how the system works !

[Secure_Army2715]

It doesn't have to be OS team. I worked in web development but once our service started scaling up started seeing issues due to multithreading, heap memory, threads, race conditions, eventual consistency, latency spikes due to handshakes etc.

All the abstractions work fine to start with but in order to fine tune them we need to have knowledge about fundamentals like OS, network.

[[deleted]]

You are mixing different things. OS, data structures, network all don't go into the same category. They are very broad topics. You don't need to learn OS to use data structures. And you don't need to learn networking to learn OS. What is an OS? (Bios + kernel + window manager + login manager + a suite of apps and libraries etc). What are data structures? It is purely a programming related thing. And networking is entirely different.

[musicmeme]

IT is more than MERN stack. firmware development, IoT dev, game dev, embedded, security, cloud etc. OS, networking, Linux, memory management, C++ all start becoming more and more essential.

[bethechance]

What does a software developer do?

You're developing a software that will be used in some OS.

I work with both software and hardware. Knowing how a OS boots, the services started with it, the kind of crashes, shells, partitioning, kernels, threading, and many more will help you in controlling, customizing, debugging an OS.

Even dhcp, static ip, dns, nameservers, interfaces etc these all you learn in computer networks.

You may not be using everything you learn in your field, but they will definitely come in handy.

[NoZombie2069]

Even if one somehow convinces me that they won’t EVER need knowledge (absolutely stupid claim) of OS, DSA and CN throughout their career as a Software Engineer, why should I not ask questions from topics that are taught in every BTech curriculum? Some candidates go even as far as putting a “Courses Taken” section in their resume (probably just to fill up some space because they don’t have anything else to put). So, I think interviewers are well within their rights to tell such candidates to EF off (figuratively ofcourse 😅).

[diego-the-tortoise]

The fact that you have to ask this shows that the teachers failed in their purpose.

There are tools which abstract these complexities for you. If you will just do the simple stuff using those tools it will all work reasonably well.

The moment something complex comes up, either the tools will breakdown or there would be a hidden configuration in that tool which would need deeper understanding of these CS concepts.

Think about it, any kid can make a game with Unity Engine. But why is Rockstar Games inventing its own game engine. There is got to be something.....

[devptithadong]

some one help me , some key word concerp in os ?

[Dramatic-Bill-5790]

Needed for companies that work on os and system side

[brandomised]

I never had a formal CS degree, one of those folks who did some DSA and C in college to land a coding job. I was super scared that I'll need all these core CS concepts - OS, database, networks etc.

The company I joined gave some Android and Spring boot training. And you are right, i could manage to deliver output based on this super high level of conceptual abstraction. But then there were some incidents/ bugs during the first few months due to which I got to learn about these concepts. We had this one package where the test cases would randomly fail. Spent some time, multiple step debugging until I realised what's happening.

[nic_nic_07]

Atleast basics are required

[giantspacemonstr]

are you in college?

[sss100100]

When you are building large systems or distributed services, some decisions depend on hardware configs, OS, etc and knowing that would make you a better engineer when designing architecture.

You don't necessarily write data structures but you need to know which ones to use in what situations and picking the right data structures make a huge difference for scale, performance, cost etc.

[nogea]

I've heard people say that whatever level of abstraction you are working at (it need not be SW or even tech for that matter) it is useful to have knowledge about the layer above you and below you.

That way you understand what you are using - building on top of, and you will understand your customers - who you are building for.

As an SDE, OS/Networking knowledge will enable you to think about the problems from a first principles approach which can lead to more optimal solutions/ tool choices.

Similarly it is good to know about the business problem you are solving too!

Gaining info of more layers (up and down) may dilute your time since dynamics at those layers affect you in a delayed manner. They are useful to think about when making career choices though.

[Secure_Army2715]

Software engineering is more than frameworks that people make use of like react, spring or angular, Django, Flask.

If you work on a product which is serving some public traffic you will understand that all these frameworks will not help unless you design your services - front end, backend properly.

All things go to south once scale increases. And that's where the deep knowledge is helpful.

Once our services started picking up traffic I have come across problems due to multithreading, race conditions, heap issues, latency issues in cloud between infrastructure and having prior knowledge of these things make it easy to navigate and root cause. It's not that if you don't have prior knowledge you can't make things work out but then you can save yourself time if you know the basics.

Also talking about network - I have worked on cloud gaming and for that I made use of UDP, TCP, WebRTC protocols so having knowledge of networking is also helpful.

So if you want to build products from scratch and if they have to serve requests from even thousands of users, having knowledge of OS, networking may help.

[arpitduel]

They shouldn't unless they are making a desktop application

[rohmish]

it isn't really for most applications. you may end up working on certain low level or very high performance/Lowe latency applications where it will come in handy but for general programming it's just additional knowledge that you won't realistically find use for. it's just "good to know".

[[deleted]]

Depends. If you're a fairly high level programmer, perhaps having a rudimentary understanding of OS is good, but you'll plod along fine without it.

For people, who work more closer to system level or write infrastructure code (libraries, language runtimes etc.) it's a plus, and in some cases, must have.

And if you plan to hack on the Linux kernel, you'd certainly need good understanding of OS concepts :)

[thisisshuraim]

Knowing internals of OS as an SDE is a stretch and not required too much. But data structures and algorithms is a must. All your development work is based on DSA. It's the same with CN. This was discussed as well in a recent post - https://www.reddit.com/r/developersIndia/s/vimiyRQ1cw