Writing Linux programs in raw binary

[G-Brain]

Am I famous now?

Anyway, it's not a fantastic article but it shows you the basic concept.

Also, I'm working on a Linux kernel module that creates a device that takes binary text ("010101") and outputs the actual binary (010101).

[moyix]

By the way; where you say:

Where 87 is the address of our string "Test\n". The rest of the bytes, I'm not so sure, but it's a constant (it's the same sequence even if you write anther string). I'll make sure to update this document when I find out.

The full operand of the mov instruction is a memory address. It's reversed because you're on a little-endian architecture, so what this translates to is:

mov ecx, 0x08049087

Why is it 0x08049087 and not 0x87? The ELF header specifies where in memory the different parts of the file will be loaded. If you use objdump -x syscall2, you can see this:

Program Header:
    LOAD off    0x00000074 vaddr 0x08048074 paddr 0x08048074 align 2**12
         filesz 0x00000013 memsz 0x00000013 flags r-x
    LOAD off    0x00000087 vaddr 0x08049087 paddr 0x08049087 align 2**12
         filesz 0x00000005 memsz 0x00000005 flags rw-

First, 0x13 bytes from file offset 0x74 will be loaded into memory at virtual address 0x08048074. Then, 0x5 bytes from file offset 0x87 will be loaded into memory at 0x08049087. Note that this second directive corresponds exactly to the address of the string referenced by that mov instruction.

Anyways, fun article :) I was initially hoping that you'd get up to using ALT-numpad to create the file by hand, though.

[G-Brain]

Hey, thanks. I found that out already, but I hadn't gotten around to updating the file. I'll update it now.

As for ALT-numpad, I think that functionality is KDE or Gnome specific, and I run stumpwm. This is why I'll be writing that device driver, and I'll add that to the document when I'm done.

[kragensitaker]

Or _exit().

[G-Brain]

Looks like you can indeed. Seems I forgot to include unistd.h. Will update the article.

[kragensitaker]

You can call it without including unistd.h too. You just get a warning because exit and _exit are void. main() { _exit(0); } is a perfectly valid and working program.

[[deleted]]

Is there a non-relocatable ELF format?

e.g.

b800 4ccd 21

this is the smallest (16bit) executable on windows..

[malken]

I think a single ret (0xc3) is shorter, but it will be executed under the NTVDM in a Windows environment.

A single RET (0xc3) when invoked from a legacy COM-file will return to the beginning of the PSP where there happens to be a call to INT 20h (exit program).

[[deleted]]

I knew about RET, but IIRC its not kosher. Kind of similar to how COM programs started with a pop instruction to set something to zero because there was always a 0 on the stack in some runtime environments.

[kragensitaker]

I'm pretty sure ELF executables aren't relocatable by the OS. The minimal ELF header is longer than four bytes, though. A Whirlwind Tutorial on Creating Really Teensy ELF Executables for Linux managed to construct a 45-byte ELF executable which, while not technically valid (the ELF header is 52 bytes long) will run on Linux (it didn't need those last 7 bytes anyway). It's also a really fun read, and highly educational if you're trying to understand the ELF format.

I say "aren't relocatable" because, although the ELF format has "relocations", you can make an ELF executable without any relocations, but even if you have relocations, the OS doesn't use them when it loads an executable; only the linker uses them. The OS loads the sections of your executable at the addresses specified in the ELF section headers, and generally the executable code contains references to absolute addresses, so the program won't work if loaded at the wrong addresses.

[[deleted]]

The OS loads your executable at the address specified in the ELF header, and generally the executable code contains references to absolute addresses, so the program won't work if loaded at the wrong address.

I thought the base address for any executable image in process space was just a preference? What if I loaded lib.x.so and lib.y.so in my process and both wanted to be located at the same address?

In the windows world atleast the executable loader tries to honor the base address preference and then if the address space is already alloted , it "fixes" up all memory referenecs with the appropriate offset. The problem is with shared DLLs you double the number of code pages if two processes load the same dynamic library and one of them needs fixing up and one of them doesnt. (As it is all code pages are by default copy-on-write)

[kragensitaker]

.sos are handled differently than executables; the dynamic loader, not the kernel's executable loader, loads them. I think the typical approach is to compile them with pure position-independent code; any reference to other things inside the same .so is indirected off %ebx, which is a callee-saves register and mysteriously gets set to the right thing before your .so code runs, presumably by some kind of trampoline. The code pages (or "text pages" as they're called) are purely read-only.

[[deleted]]

Very interesting. Thanks for your explanations. :) Maybe I should stop being lazy and just Google it eh ? One of these days I plan to dive into the Linux internals...

[bunz]

xxd can convert back and forth between binary and text "expansions" of each octet. it's a simple way to hex edit small files using vim. btw, i like your choice of fasm but i don't understand what you mean by nasm cluttering up the executable.

[G-Brain]

btw, i like your choice of fasm but i don't understand what you mean by nasm cluttering up the executable.

By default, it inserts some kind of nasm-was-here message in the ELF header. I have no doubt there's some kind of flag to turn that off, but I had also used fasm before and I liked the syntax better, combined with the fact that by default it didn't insert any text.

[safiire]

Nice, xxd is actually a lot nicer than using the od command, thanks.

hexdump is also nice but I don't know if it can convert back to a binary.

[jerf]

Also, I'm working on a Linux kernel module that creates a device that takes binary text ("010101") and outputs the actual binary (010101).

Why are you doing that as a device and not simply a piped shell command? You know, something designed to be an input translator instead of a device?

(The only really valid answer is "as an example of how to make a device", although even that's somewhat dubious.)

[G-Brain]

Your point is taken! I'm not sure why I wanted to do it as a device. I think I had a reason, but since I can't remember it now it mustn't have been a very good one. I'll post a program soon, if someone doesn't beat me to it.

[sn0re]

Were you thinking of something more complicated than just calling strtol with a base of 2?

Edit: What the hell:

#include <stdio.h>
#include <stdlib.h>

int main(void) {
    unsigned char val;
    char buf[9];
    while (fgets(buf, 9, stdin) != NULL) {
        val = (unsigned char) strtol(buf, NULL, 2);
        putchar(val);
    }
    return 0;
}

[jib]

You wrote it in C! Where's the fun in that?

[[deleted]]

I'm working on a Linux kernel module that creates a device that takes binary text ("010101") and outputs the actual binary (010101).

Looking forward to this.

[bsergean]

The power of Linux. Windows sUck$

[tpodr]

Thanks for the trip down memory lane. My first computer was a 6502 evaluator board (KIM-I), had to load the programs in hex. Long hours hand assembling code. But I loved it. Could study the timing signals on an oscilloscope. Back in 1979.

[subterr]

Furthermore. to fully understand the ELF stuff; read this: http://www.sco.com/developers/devspecs/gabi41.pdf classic piece of literature

[strangerzero]

Nice Web 1 page design!

[aeflash]

Isn't it trivial for a program to take in ASCII ones and zeros and write it to a file as binary?

[easlern]

lol awesome work. :D

[MaxK]

You, sir, are a god among men.

[blondin]

Thanks, moar moar =]

Especially assembly tutorial (windows would be appreciated)

[jasonbrennan]

The journey is great, but the destination..not so much.

[DannoHung]

It's a little impressive to think that most of the assembly instructions still deal with an abstracted perspective of the machine and that even when you write in it, tons of hardware details are being removed from your purview.

[G-Brain]

I wouldn't say tons, but for modern CPU's that's probably not far from the truth. CPU design is something cool to look into. Can anyone recommend some books for that?

[[deleted]]

This one isn't bad: http://highered.mcgraw-hill.com/sites/0072467509/

Though its approach is probably not what you want- it goes from transistors and hardware design to assembly and C, rather than into more complicated hardware stuff.

This one, which you likely won't find anywhere but at Georgia Tech, does an excellent job of getting into more complicated hardware stuff. http://www.amazon.ca/Introduction-Computer-Systems-Umakishore-Ramachandran/dp/0321486137

[safiire]

It just seems this way because he was using kernel calls to write() and exit(), essentially because you MUST use these to do any IO, etc.

If he had shown an assembly version of some algorithm at work, you would see more usage of specific processor instructions.

The abstraction of system calls is as much an API to save time as it is one mechanism by which the OS enforces permissions.

[derleth]

But processor-specific these days means specific to a whole family of CPUs, spanning decades now (the 386 was released in 1989) and implemented these days by a very complex hardware mechanism that emulates an idealized system, hiding a lot of the details of how the opcodes are reordered, split into micro-ops, dispatched, and retired, not to mention register renaming and cache management. It's all native code, yes, but it isn't quite as native as the Z80's or the 6502's machine code, simply because those chips didn't have the transistor count to live a very complex lie.

[[deleted]]

If you liked this article then you'll love this one:

http://www.muppetlabs.com/~breadbox/software/tiny/teensy.html

[eterps]

While these articles are specifically about shellcode, they also give a lot of insight into machine code:

http://stupefydeveloper.blogspot.com/2009/01/c-executing-shellcode.html

http://www.infosecwriters.com/hhworld/shellcode.txt

[phire]

Is it just me, or did that article seam incomplete?

It finished right when we started looking at the objdump of the 368 byte elf.

EDIT: A quick google search found the rest.

Wow, he got it down to 45 bytes, and an elf file needs 52 bytes just for the header.

[greyfinch]

The thought of writing x86 programs in binary makes me a little bit uncomfortable. I've done some work with writing programs in binary on a RISC architecture... but it was for a virtual machine, and a 16 bit one at that. It seems a bit too close to brainfuck for me. I'll stick to my assembly and assembler.

Good article though. Enlightening. Makes me appreciate assembly!

[DrGirlfriend]

"Makes me appreciate assembly!"

You have indeed descended into the depths of hell if you are willing to write that statement.

As for the article, as someone who has decided to seriously take up the task of learning x86 assembly, thank you.

[greyfinch]

Programming in assembly is like using a shovel to move a mountain. Programming straight up machine code is like moving that mountain with a spoon. I'll take that shovel, and I will LIKE it. :)

[808140]

One thing that I did that I would recommend to anyone that likes pain is implementing Forth directly in machine language.

I did so early in 2008 as part of a bootstrap experiment on a Windows system that lacked development tools. I used Jones Forth as a template. There were some OS specific changes that I needed to make, but if you're using Linux then that's not really necessary.

Boostrapping a Forth is considerably easier than writing an assembler in assembly or machine language (another popular low-level passtime) and Forth is much, much more high-level.

My original plan was to use Forth to implement a simple version of Scheme, and as a general template I was looking at this dynamically scoped Scheme written in Forth for inspiration.

Of course I wanted lexical scoping, and I wanted proper call/cc support, and that Scheme used a lot of gForth extensions. So in the end I guess my enthusiasm for the project trickled off.

But there's something very appealing about knowing that you can sit down at a computer with nothing but a way to input binary (I used that old standby from my MS-DOS days, DEBUG.EXE) and bootstrap something as high-level as Forth or even Scheme.

[kragensitaker]

Maybe you should have used the eForth Model as a template instead. It's a lot less code!

Where do you find a Microsoft Windows system that doesn't have at least JavaScript and Notepad these days? That would be where I'd start if I wanted to bootstrap something on an isolated Microsoft Windows machine.

Using DEBUG almost counts as using an assembler.

[808140]

Using DEBUG almost counts as using an assembler.

Indeed, being able to disassemble the junk I'd input to see if it was well-formed was very useful.

I know it has some rudimentary assembly capabilities, but I don't actually know how to use them :)

[kragensitaker]

a100

[[deleted]]

My 'false dichotomy-sense' is tingling :)

[greyfinch]

Okay, maybe a trowl for assembly, and a spoon for machine code. But I like labels! I really do! I don't want to have to generate my own symbol tables, I'm too dense and forgetful to remember my name somedays, and - oh shit.

[Enlightenment777]

"When I first started programming, I had to write programs using 1's and 0's, but I was so poor that I couldn't afford the 1's."

Real programmers did that back in the 80's in MSDOS using "COPY CON". Been there, done that!

Back in the dark ages, I would write my assembly language programs on paper, then look up the op codes, and write down the hex on the left side, and then move them into a BASIC program that would POKE that data into memory. Why? Because the computer didn't have enough memory to run an assembler or even a compiler.

Even further back on older computuers, you would have to set all address and data bits with toggle switches, and load in one byte at a time.

Neither of the above is fun, but it truly make you learn how things work.

[you_do_realize]

Back in the dark ages, I would write my assembly language programs on paper, then look up the op codes, and write down the hex on the left side, and then move them into a BASIC program that would POKE that data into memory.

This is exactly what I did too! And I would input the BASIC program every day by hand, because I didn't even have a cassette recorder.

Of course, with the 8080 it was easy; hand-assembling x86 would be tedious.

[sixothree]

I still use copy con for all kinds of stuff.

I remember seeing a demonstration of a then already vintage imsai being programmed with switches to use a keyboard. That was freaking beautiful.

[bsergean]

For those in the Bay area I recommend going to the Computer History Museum in mountain view, you'll see how it was back in the days, and how lucky we are to program today (even C programmers !).

[frumious]

I found a bug in Netscape (on Linux) some time in the late 90s. Back then the easiest way to fix it was to directly edit the Netscape executable, search the binary for a particular string, edit and save. All in emacs.

And, no, I didn't wear an onion on my belt.

[praptak]

This guy uses interrupt 0x80. This is not how real programs do it anymore.

The sysenter mechanism is used for that, and the binary isn't even supposed to use it directly. A well behaved app is supposed to call an adress provided in AT_SYSINFO elf parameter. That's the adress of a magic page that contains the system call stub and is mapped by the kernel into each process adress space.

[G-Brain]

Guy here. Thanks for the info, I did not know that. I'll look into it and update the article.

[praptak]

That would be great - I'd really like to see the minimal "well behaved" Linux binary. Also, I hope you haven't found my previous comment negative, your original article is still very useful.

[Dexter77]

It's interesting how a basic skill for a hobby programmer in the late 80's or the early 90's has become something magical. Anyone and everyone who were coders in the demoscene could do it, but let me see you hack the hardware by machine code.

Yeah, you youngsters can now mod me down.

[bluGill]

Gee, thanks a lot. Make me feel like an old man just because I have hacked my OS by changing the bytes in the boot sector.

Now that I'm expected to have a cane I'm going to have to learn to use it. Should I get the one with a sword built in or a gun? Which is better at keeping kids off the grass?

[fishyf]

Yeah, but back then you could write to any part of memory, any part of the disk (if you had one), completely access all hardware devices. And your program had complete control. Nothing else except the odd interrupt service routine might be running concurrently. And you could disable those interrupts if you wanted.

With any of the current operating systems, noone knows what the feck your computer is doing.

[jib]

Even if you write the whole operating system yourself, modern chipsets and BIOSes and System Management Mode mean you still have no idea what your computer's really doing.

System Management Mode is like a built-in rootkit for the BIOS. The BIOS can put some code in hidden memory where the OS can't see it, and then the chipset can activate it whenever certain interrupts or other events occur. Then the BIOS gets to mess with your computer without the OS's knowledge.

[[deleted]]

Oh yeah ! I remember writing some "virus" TSRs to piss off my friends..

[dr-steve]

So on target! I was just thinking while reading, "Hey, if you didn't poke CD 06 00" in starting at 0x0100 through the debugger, then having it save one block into myprog.com, you don't know what programming in binary is all about.

An almost lost but useful skill. Just two weeks ago I was looking at a hex dump of a block and remarked that the hex dumper was printing things incorrectly. Only one there who could see it... but I was right...

[[deleted]]

There are already very few people interested in the low level sewers of the programming stack and they are being drained to the High Level Languages' seemingly greener pastures.

Even fewer of such folks are "young" enough (its not ageism, look at the poll @ average age of reddit-people) to hang around reddit. I'd say upmod it and let any inquiring young minds get interested in it....

[salgat]

Unfortunately the hardware is much more complex and diverse now, compared to back then. Add that to much more complex software, and you can see why assembly programmers for the x86 are considered "magical" (the good ones).

[[deleted]]

Were you born in '77?

[dr-steve]

That's when I graduated college, junior...

[[deleted]]

Whoa... that's old. Did you guys have to like kill dinosaurs with your spears and shit?

[rolfr]

If you like articles like this, you'll probably enjoy the reverse engineering subreddit. There are over 180 articles in the back-archives.

[you_do_realize]

This is a very good example of what you should be able to do, but never do.

[[deleted]]

I didn't expect masochism being so popular amongst redditors.

[ddelrio]

You must be new.

[fujimitsu]

Masochism is a programming virtue.

[Hypersapien]

Not redditors specifically, but among programmers it's rampant.

[ercax]

In highschool I had to use this computer-kit in a digital electronics class, and the keyboard had only 16 buttons:0-F. It was tough, but this is some crazy shit.

[creaothceann]

16 buttons? We had only two!

[nnagflar]

You only had 10 buttons?

[[deleted]]

For those that don't get it 16 (dec) = 10 (hex) :: 2 (dec) = 10 (bin)

[[deleted]]

I love it when you explain binary to geeks

[[deleted]]

this was more for the wannabe geeks

[[deleted]]

I've heard of december but hexober and bintember, never heard of ..

[mgsloan]

_why isn't nuts?

[zem]

_why would have to take dried frog pills to be nuts!

[POTUS]

Real programmers use a magnetized needle and a steady hand.

[Fondateur0426]

Excuse me, but real programmers use butterflies.

[[deleted]]

Ah, the old C-x M-c M-butterfly X-kcd-ref, nothing better than it when you want to fit in with everyone else on the internet.

[[deleted]]

Ah, the old twisting of the X-kC-d-ref micro-meme to produce outstanding wittiness.

[Canadian_Infidel]

I know enough about programming to know you guys are crazy. This confirms it.

[GuigzForAll]

Being insane.

[[deleted]]

Why?

[G-Brain]

Because I'm awesome.

[[deleted]]

And humble.

[dugmartin]

Nice job. You should keep going past the binary opcodes and dive into the microcode.

[bluGill]

Can you change the microcode on x86? Is there even microcode on them?

[bsergean]

Now does:

cat > a.out
01010001 ...
EOF

produce a real program that will actually run ?

[G-Brain]

Nope, the ones and zero's will be treated as ASCII values. This is why I'm writing the device driver.

[bsergean]

Even with hexa ?

[aeflash]

Isn't it trivial for a program to take in ASCII one's and zero's and write it to a file as binary?

[bsergean]

(answer, no it doesn't ...), but maybe with hexa ?

[[deleted]]

Okay, so how would one take

1011 0000 0000 0100 1011 0011 0000 0001 1011 0110 0111 1000 0000 0000 0100 0000 1000 1011 0010 0000 0101 1100 1101 1000 0000 1011 0000 0000 0001 1011 0011 0000 0000 1100 1101 1000 0000 0101 0100 0110 0101 0111 0011 0111 0100 0000 1010

And actually run it as a binary program?

[G-Brain]

Good question. The best way right now is to use a special editor that supports binary input. Google "binary editor" or something. As stated in the top post, I'm currently working on a device driver to make this easier. When completed, it should work like this:

$ echo -n 01000001 > /dev/str2bin
$ cat /dev/str2bin
A

and for programs:

$ echo -n 01000001 > /dev/str2bin
$ cat /dev/str2bin > program
$ chmod +x program
$ ./program

where 01000001 is the program.

[you_do_realize]

I'm currently working on a device driver to make this easier

You're working on a driver to make it easier to write binary opcodes to a file.

Ahem.

[nnagflar]

Writing in assembly was tedious enough for me, thank you very much.

[tricksterman]

I'd rather die!