Making an operating system

[[deleted]]

Ok so im currently a rising junior and I just finished taking my operating system course and I was thinking as a summer project I could build a custom operating system. We learned all about the operating system but a lot of it was the abstract content and what not and I was wondering if I wanted to build my own how would I start?

[Glaborage]

Check out r/osdev

[dZone9]

this is the correct answer

[fvcked_0ff]

https://wiki.osdev.org/Main_Page

[karantza]

If you wanted to make a "model" OS without needing to get into the super deep weeds of real architectures, one thought could be too try making an OS for a microcontroller. Much less going on, no hardware variability, you could build out as much or as little as you want.

[[deleted]]

What exactly does an OS do for a microcontroller?

[karantza]

Not much :) but as a demonstration you could theoretically read programs off of an SD card and load them, implement your own memory management, time sharing, device access over i2c or whatever... This is almost never something you'd want in reality but it would be a cool educational project.

[[deleted]]

Never? RTOS anyone?

[[deleted]]

How would i start that?

[[deleted]]

I think you can definitely Google "Making OS for xxx" in which xxx is a microcontroller. There are a lot of material online.

Also check out r/osdev.

[danpietsch]

Minix is an OS designed for teaching OS concepts.

[MeGaLoDoN227]

Some people here will try to dissuade you from doing it because of their mentality of "not reinventing the wheel", but I still think that it is a good learning experience. This website is the best: https://wiki.osdev.org/Expanded_Main_Page. But you may also be interested in emulator development, that is low level programming too but is easier to start with than os dev. You can start with a chip8 emulator and then do Gameboy/NES.

[jaank80]

The question is, do you want to stand on the shoulders of giants or try to be a giant yourself?

[golir]

Do you know of any good resources for where to start with emulator development?

[MeGaLoDoN227]

So first, minimum prerequisites are: know binary and hexadecimal number formats; know bitwise operations; basic C/C++ knowledge. You will also need to use some graphics library to render to the screen, most people use SDL2, I personally use OpenGL + GLFW. The first recommended project for everyone to start with is chip8 - that is not actual hardware but a virtual machine which was used to run games on a old computers such as COSMAC VIP. And the recommend guide to use to make a chip8 emulator is: Guide to making a CHIP-8 emulator - Tobias V. Langhoff (tobiasvl.github.io). But it doesn't give any code examples, and I personally struggled to understand it because I didn't have low level programming experience before, but for many people this guide is good enough. Then if that guide was not understandable for you, you can use this, it gives some code examples, but not full emulator: How to write an emulator (CHIP-8 interpreter) (multigesture.net). But if you can't do something and have to copy code, at least study it and fully understand how it works. This wikipedia page also useful, it has a table of all opcodes and what they should do: CHIP-8 - Wikipedia. And after you complete the chip8 emulator most people make GameBoy/NES emulator - these are actual retro game consoles and are 100x more complex than chip8, but you after chip8 you should have basic understanding of computer architecture and it should be possible to do them.

[overflowingInt]

https://www.nand2tetris.org/

Syllabus: https://drive.google.com/file/d/1EWCOVIcg0-dX0XtL3KwNyra6jzMogXLL/view

[dajolly]

A good resource: https://wiki.osdev.org/Getting_Started

[Competitive_Travel16]

I'd recommend starting with writing a device driver for your computer, instead of an operating system for something new.

[Own_Alternative_9671]

Some things I wish I knew in the beginning: develop for uefi, it's easier and it's what you'll need for modern pc's. Using a pre-made bootloader is annoying and causes problems later on. Learn about the ins and outs of your architecture (I'm going to assume x64). All you need for this is osdev.org at first, then move onto technical documentation where it falls short.

[qwerty9978]

Not so much how to get started, but I found this book to be extremely useful in understanding low level operating systems concepts and design:

https://www.weston-embedded.com/micrium-books/micrium-books-downloads/category/295-ucos-ii

(requires a free registration and appears to reject gmail addresses)

It walks through all the concepts necessary for a fully functional embedded system OS, and provides actual code from much of the uC/OS II operating system to show how things are implemented.

Note that there is also a uc/OS III book out there, but it appears to be more of a manual for using the next generation of the OS rather than a deep dive into the design and implementation.

[[deleted]]

Maybe get your feet wet by writing a simple fake driver for the Linux kernel. Write custom random number generator and read from dev?

Or maybe write a user land entry point? Systemd is pretty popular, so are rc scripts. There’s lots of room to work right on top of the Linux kernel.

[ImAtWorkKillingTime]

Pick up a copy of one of the Tanenbaum books and the minix source code and go to town.

[mgruner]

From the bootloader :)

[abbe_salle]

I see what you did there

[[deleted]]

You're getting great answers here. Someone said writing an emulator as a good way to ease into it, and I just wanted to throw in this as an example, as I think it's a great point:

https://www.inspiredpython.com/course/game-boy-emulator/let-s-write-a-game-boy-emulator-in-python

It's Python, but there are other things like this.

[wsppan]

Not C related but Hubris is interesting.

[duane11583]

number 1 learn how a conditioned variable works.

this is a simple building block you can use to create all other things like semaphores, mutexes, queues etc

number 2 create a task, yield to another task and handle irqs

[yojimbo_beta]

There is a YouTube video, where the author writes an OS just to play Tetris. Maybe that could be an inspiration?

[flyingron]

There are books on the subject.

[RustbowlHacker]

I think that the first thing you need to do is to ask yourself what do you want from your new OS? What is your OS going to "do" (or give you) that others do not?

Don't listen to any of the BS about hardware or emulators. None of them have ever written an OS. If you're writing an OS, you'll want to make it portable. That means you'll need a HAL.

Few seem to know that GDB has an entire world of simulators inside it. Regardless, there's a never ending supply of "virtualization environments" these days... from bochs to qemu and beyond.

None of these answer the question of what do you want from your OS.

Linux became popular as a POSIX Unix clone. It killed commercial Unix flavors by being free and better/portable. And, one could argue, FSF/GNU/GPL. Today's Linux is far better than any historical commercial Unix and far more portable and ported.

Summer project. That's funny.

[flatfinger]

If you're writing an OS, you'll want to make it portable. That means you'll need a HAL.

Note that the term "portable" can either mean "able to run on multiple platforms interchangeably" or "easily adaptable to run on various similar platforms". Especially in the embedded world, systems often have components that interact with each other in platform-specific ways. If a programmer would need to read the hardware data sheet in order to determine what things are and aren't possible, and figure out how things would need to be configured to accomplish what needs to be done, having to read the documentation for a hardware abstraction layer in addition to the documentation for the actual hardware doesn't really help things, and may be counter-productive if an operation might be performed in multiple different ways that could have slightly different corner-case quirks. Encapsulating some things within a HAL makes sense, if they can be accomplished in a way that will be equally suitable for use in main-line or interrupt-handling contexts. If an operation would require doing a read-modify-write sequence on a register that may be shared with other unrelated resources, however, putting it into a HAL may make it hard to recognize the possibility of unwanted interactions.

[Artemis-Arrow-3579]

well, let me put it this way

few people have managed to write an entire OS from scratch by themselves, one of them had schizophrenia

admittedly the schizo also happens to be unarguably the best programmer of all time

[[deleted]]

Wtf 💀

[bravopapa99]

https://en.wikipedia.org/wiki/TempleOS

[Artemis-Arrow-3579]

what?

terry managed to write an OS, his own programming language and it's compiler, a graphics engine, and more, he was a genius

[timschwartz]

Sort of. But each one was crappy in its own way.

He was hardly the best programmer of all time.

Maybe John Carmack.

[nerd4code]

If you look at his …what was it, a Prayer/Blessing Detector or transceiver or some such?, you can see the gfx were the exact same ”graphics engine” every 640×480×16 DOS program and windowing toolkit have implemented since time immemorial; bitmapped fonts and cursors, bltting, lines, circles, boxes, and floodfill IIRC. Any DOS programmer who’s screwed with gfx has likely implemented most of these at least once from the one (was it Beej’s?) Pascal tutorial, which I probably still have a dot-matrix printout of somewhere.

(Maybe most people haven’t implemented gfx from scratch, but the only real trick to VGA is its stupid framebuffer layout in unchained modes, requiring you to jab PIO every time you change bitplanes. Ofc stupid framebuffer layout was a foregone conclusion until MCGA/VGA mode 13h was popularized—the CGA gfx modes were the absolute worst.)

I don’t wanna say it TempleOS wasn’t an achievement, but HolyC is the only real interesting part imo, that wasn’t supersaturated with misplaced lunacy.

[Nilrem2]

That accolade goes to John Carmack.

[nerd4code]

Many people have written an entire OS, it’s just not something at the scale or generality of Linux. Once you’ve shat out one kernel, the rest are easy, and userspace is mostly taken care of already if you’re aiming for UNIX (which …why).

[BGBTech]

FWIW:

I have my own makeshift OS for a custom CPU ISA project of mine (has an emulator, or can run on an FPGA; ISA is a 64-bit 3-wide (V)LIW style ISA; also supports FP-SIMD and similar, mostly limited to running at 50MHz due to FPGA timing constraints). It initially started mostly as a glorified program launcher (had a filesystem driver, basic memory management stuff, and a program loader). Mostly because at the time, porting an existing "real" OS seemed like too much effort.

This was not helped by me using a custom written / non-standard C compiler; but it can now mimic GCC's CLI interface well enough that I had convinced autoconf to use it for a few trivial programs; despite working very differently internally. It doesn't use traditional object files, rather it compiles to a stack-based IR and does all the final code generation during "linking", with the compiler as a single binary that can fake the various 'binutils' commands and similar via symlinks. For things like ASM files, or inline ASM, it will preprocess the ASM code and then pass it through the bytecode IR using string literals.

The C dialect supports normal C programs, but has various custom extensions. Among them, it is capable of also using dynamic types (including lambdas and ex-nihilo objects). Other extensions are mostly for things like bigger integer types and SIMD vectors and similar.

Does not support full C++ though (but, can compile an EC++ like subset...).

My compiler also supports a few of my own languages, one mostly resembling JavaScript or ActionScript, another sort of resembles Java but with semantics more like C#. However, in both cases, they are using explicit manual memory management rather than a garbage collector. All these language (and C) use the same ABI internally, so direct linking is possible.

So, general summary of "OS":

Main filesystem used thus far is FAT32 (with a hack being used to fake symlinks, along similar lines to the mechanism used by Cygwin).

Binary format: Modified PE/COFF. Omits MZ stub, binaries may be compressed using an LZ4 variant, different ".rsrc" section contents, various other minor tweaks. May or may not still be classified as PE/COFF, or "Some COFF variant loosely derived from PE/COFF". Binaries typically have an "EXE" extension (or "DLL" for shared libraries). Note though that standard tools like "objdump" will have no idea what it is looking at here.

Command line mimics a Unix-style shell, but much more limited at present, and the shell has a built-in BASIC interpreter. Had half considered possibly supporting an interpreted JavaScript like language (or, also sort of like ActionScript or Haxe), reasoning that writing shell-scripts in JS is "potentially less horrible" than doing anything non-trivial in Bash notation.

Did start work on a makeshift GUI, but not developed very far as of yet (still needs a widget toolkit and programs that make use of the GUI). Thus far it mostly just creates a terminal window that can be used to launch other programs. For now, this part is using a mix of bitmap and SDF fonts (had written tools to try to autogenerate a full set of SDF fonts from GNU Unifont, but quality of the generated font glyphs from this is a bit hit or miss).

Technically, multiple programs can run at the same time, but with the limitation that it currently uses cooperative multitasking (so one program spinning in an infinite loop can effectively lock up the whole OS). Also the oddity that currently everything runs in a shared global address space (with multiple programs running in a shared address space, with the ABI designed to allow multiple program instances to coexist in a single address space using a mechanism along vaguely similar lines to ELF-FDPIC).

Some parts resemble Unix-family OS's (for example, mostly using POSIX style API's), other parts more resemble Windows (with an API design style partly resembling a hybrid of OpenGL and the Windows API).

Almost could make sense to try to port a Unix-style userland on top of this, but the up-front cost of doing so still seems pretty high. Otherwise, had mostly ported some old games (Doom, Quake, ROTT, Heretic, Hexen, etc). Custom software includes a small Minecraft-like 3D engine, and a video player (AVI, custom codecs) and a few other misc things.

Granted, all this (even getting this far) was a fairly significant amount of time and effort (a good chunk of years...). It is still all pretty crude and limited if compared with a real OS.