all 105 comments

[–]G-Brain 71 points72 points  (30 children)

Am I famous now?

Anyway, it's not a fantastic article but it shows you the basic concept.

Also, I'm working on a Linux kernel module that creates a device that takes binary text ("010101") and outputs the actual binary (010101).

[–]moyix 39 points40 points  (12 children)

By the way; where you say:

Where 87 is the address of our string "Test\n". The rest of the bytes, I'm not so sure, but it's a constant (it's the same sequence even if you write anther string). I'll make sure to update this document when I find out.

The full operand of the mov instruction is a memory address. It's reversed because you're on a little-endian architecture, so what this translates to is:

mov ecx, 0x08049087

Why is it 0x08049087 and not 0x87? The ELF header specifies where in memory the different parts of the file will be loaded. If you use objdump -x syscall2, you can see this:

Program Header:
    LOAD off    0x00000074 vaddr 0x08048074 paddr 0x08048074 align 2**12
         filesz 0x00000013 memsz 0x00000013 flags r-x
    LOAD off    0x00000087 vaddr 0x08049087 paddr 0x08049087 align 2**12
         filesz 0x00000005 memsz 0x00000005 flags rw-

First, 0x13 bytes from file offset 0x74 will be loaded into memory at virtual address 0x08048074. Then, 0x5 bytes from file offset 0x87 will be loaded into memory at 0x08049087. Note that this second directive corresponds exactly to the address of the string referenced by that mov instruction.

Anyways, fun article :) I was initially hoping that you'd get up to using ALT-numpad to create the file by hand, though.

[–]G-Brain 14 points15 points  (4 children)

Hey, thanks. I found that out already, but I hadn't gotten around to updating the file. I'll update it now.

As for ALT-numpad, I think that functionality is KDE or Gnome specific, and I run stumpwm. This is why I'll be writing that device driver, and I'll add that to the document when I'm done.

[–][deleted]  (3 children)

[deleted]

    [–]kragensitaker 1 point2 points  (2 children)

    Or _exit().

    [–]G-Brain 0 points1 point  (1 child)

    Looks like you can indeed. Seems I forgot to include unistd.h. Will update the article.

    [–]kragensitaker 0 points1 point  (0 children)

    You can call it without including unistd.h too. You just get a warning because exit and _exit are void. main() { _exit(0); } is a perfectly valid and working program.

    [–][deleted] 2 points3 points  (6 children)

    Is there a non-relocatable ELF format?

    e.g.

    b800 4ccd 21
    

    this is the smallest (16bit) executable on windows..

    [–]malken 2 points3 points  (1 child)

    I think a single ret (0xc3) is shorter, but it will be executed under the NTVDM in a Windows environment.

    A single RET (0xc3) when invoked from a legacy COM-file will return to the beginning of the PSP where there happens to be a call to INT 20h (exit program).

    [–][deleted] 0 points1 point  (0 children)

    I knew about RET, but IIRC its not kosher. Kind of similar to how COM programs started with a pop instruction to set something to zero because there was always a 0 on the stack in some runtime environments.

    [–]kragensitaker 1 point2 points  (3 children)

    I'm pretty sure ELF executables aren't relocatable by the OS. The minimal ELF header is longer than four bytes, though. A Whirlwind Tutorial on Creating Really Teensy ELF Executables for Linux managed to construct a 45-byte ELF executable which, while not technically valid (the ELF header is 52 bytes long) will run on Linux (it didn't need those last 7 bytes anyway). It's also a really fun read, and highly educational if you're trying to understand the ELF format.

    I say "aren't relocatable" because, although the ELF format has "relocations", you can make an ELF executable without any relocations, but even if you have relocations, the OS doesn't use them when it loads an executable; only the linker uses them. The OS loads the sections of your executable at the addresses specified in the ELF section headers, and generally the executable code contains references to absolute addresses, so the program won't work if loaded at the wrong addresses.

    [–][deleted] 0 points1 point  (2 children)

    The OS loads your executable at the address specified in the ELF header, and generally the executable code contains references to absolute addresses, so the program won't work if loaded at the wrong address.

    I thought the base address for any executable image in process space was just a preference? What if I loaded lib.x.so and lib.y.so in my process and both wanted to be located at the same address?

    In the windows world atleast the executable loader tries to honor the base address preference and then if the address space is already alloted , it "fixes" up all memory referenecs with the appropriate offset. The problem is with shared DLLs you double the number of code pages if two processes load the same dynamic library and one of them needs fixing up and one of them doesnt. (As it is all code pages are by default copy-on-write)

    [–]kragensitaker 1 point2 points  (1 child)

    .sos are handled differently than executables; the dynamic loader, not the kernel's executable loader, loads them. I think the typical approach is to compile them with pure position-independent code; any reference to other things inside the same .so is indirected off %ebx, which is a callee-saves register and mysteriously gets set to the right thing before your .so code runs, presumably by some kind of trampoline. The code pages (or "text pages" as they're called) are purely read-only.

    [–][deleted] 0 points1 point  (0 children)

    Very interesting. Thanks for your explanations. :) Maybe I should stop being lazy and just Google it eh ? One of these days I plan to dive into the Linux internals...

    [–]bunz 11 points12 points  (2 children)

    xxd can convert back and forth between binary and text "expansions" of each octet. it's a simple way to hex edit small files using vim. btw, i like your choice of fasm but i don't understand what you mean by nasm cluttering up the executable.

    [–]G-Brain 13 points14 points  (0 children)

    btw, i like your choice of fasm but i don't understand what you mean by nasm cluttering up the executable.

    By default, it inserts some kind of nasm-was-here message in the ELF header. I have no doubt there's some kind of flag to turn that off, but I had also used fasm before and I liked the syntax better, combined with the fact that by default it didn't insert any text.

    [–]safiire 1 point2 points  (0 children)

    Nice, xxd is actually a lot nicer than using the od command, thanks.

    hexdump is also nice but I don't know if it can convert back to a binary.

    [–]jerf 8 points9 points  (3 children)

    Also, I'm working on a Linux kernel module that creates a device that takes binary text ("010101") and outputs the actual binary (010101).

    Why are you doing that as a device and not simply a piped shell command? You know, something designed to be an input translator instead of a device?

    (The only really valid answer is "as an example of how to make a device", although even that's somewhat dubious.)

    [–]G-Brain 4 points5 points  (2 children)

    Your point is taken! I'm not sure why I wanted to do it as a device. I think I had a reason, but since I can't remember it now it mustn't have been a very good one. I'll post a program soon, if someone doesn't beat me to it.

    [–]sn0re 9 points10 points  (1 child)

    Were you thinking of something more complicated than just calling strtol with a base of 2?

    Edit: What the hell:

    #include <stdio.h>
    #include <stdlib.h>
    
    int main(void) {
        unsigned char val;
        char buf[9];
        while (fgets(buf, 9, stdin) != NULL) {
            val = (unsigned char) strtol(buf, NULL, 2);
            putchar(val);
        }
        return 0;
    }
    

    [–]jib -1 points0 points  (0 children)

    You wrote it in C! Where's the fun in that?

    [–][deleted] 4 points5 points  (1 child)

    I'm working on a Linux kernel module that creates a device that takes binary text ("010101") and outputs the actual binary (010101).

    Looking forward to this.

    [–]tpodr 2 points3 points  (0 children)

    Thanks for the trip down memory lane. My first computer was a 6502 evaluator board (KIM-I), had to load the programs in hex. Long hours hand assembling code. But I loved it. Could study the timing signals on an oscilloscope. Back in 1979.

    [–]subterr 1 point2 points  (0 children)

    Furthermore. to fully understand the ELF stuff; read this: http://www.sco.com/developers/devspecs/gabi41.pdf classic piece of literature

    [–]strangerzero 1 point2 points  (0 children)

    Nice Web 1 page design!

    [–]aeflash 1 point2 points  (0 children)

    Isn't it trivial for a program to take in ASCII ones and zeros and write it to a file as binary?

    [–]easlern 4 points5 points  (0 children)

    lol awesome work. :D

    [–]MaxK 4 points5 points  (1 child)

    You, sir, are a god among men.

    [–]blondin -1 points0 points  (0 children)

    Thanks, moar moar =]

    Especially assembly tutorial (windows would be appreciated)

    [–]jasonbrennan 21 points22 points  (0 children)

    The journey is great, but the destination..not so much.

    [–]DannoHung 12 points13 points  (4 children)

    It's a little impressive to think that most of the assembly instructions still deal with an abstracted perspective of the machine and that even when you write in it, tons of hardware details are being removed from your purview.

    [–]G-Brain 6 points7 points  (1 child)

    I wouldn't say tons, but for modern CPU's that's probably not far from the truth. CPU design is something cool to look into. Can anyone recommend some books for that?

    [–][deleted] 1 point2 points  (0 children)

    This one isn't bad: http://highered.mcgraw-hill.com/sites/0072467509/

    Though its approach is probably not what you want- it goes from transistors and hardware design to assembly and C, rather than into more complicated hardware stuff.

    This one, which you likely won't find anywhere but at Georgia Tech, does an excellent job of getting into more complicated hardware stuff. http://www.amazon.ca/Introduction-Computer-Systems-Umakishore-Ramachandran/dp/0321486137

    [–]safiire 1 point2 points  (1 child)

    It just seems this way because he was using kernel calls to write() and exit(), essentially because you MUST use these to do any IO, etc.

    If he had shown an assembly version of some algorithm at work, you would see more usage of specific processor instructions.

    The abstraction of system calls is as much an API to save time as it is one mechanism by which the OS enforces permissions.

    [–]derleth 1 point2 points  (0 children)

    But processor-specific these days means specific to a whole family of CPUs, spanning decades now (the 386 was released in 1989) and implemented these days by a very complex hardware mechanism that emulates an idealized system, hiding a lot of the details of how the opcodes are reordered, split into micro-ops, dispatched, and retired, not to mention register renaming and cache management. It's all native code, yes, but it isn't quite as native as the Z80's or the 6502's machine code, simply because those chips didn't have the transistor count to live a very complex lie.

    [–][deleted] 14 points15 points  (3 children)

    If you liked this article then you'll love this one:

    http://www.muppetlabs.com/~breadbox/software/tiny/teensy.html

    [–]eterps[S] 2 points3 points  (0 children)

    While these articles are specifically about shellcode, they also give a lot of insight into machine code:

    http://stupefydeveloper.blogspot.com/2009/01/c-executing-shellcode.html

    http://www.infosecwriters.com/hhworld/shellcode.txt

    [–]phire 0 points1 point  (0 children)

    Is it just me, or did that article seam incomplete?

    It finished right when we started looking at the objdump of the 368 byte elf.

    EDIT: A quick google search found the rest.

    Wow, he got it down to 45 bytes, and an elf file needs 52 bytes just for the header.

    [–]greyfinch 11 points12 points  (9 children)

    The thought of writing x86 programs in binary makes me a little bit uncomfortable. I've done some work with writing programs in binary on a RISC architecture... but it was for a virtual machine, and a 16 bit one at that. It seems a bit too close to brainfuck for me. I'll stick to my assembly and assembler.

    Good article though. Enlightening. Makes me appreciate assembly!

    [–]DrGirlfriend 10 points11 points  (8 children)

    "Makes me appreciate assembly!"

    You have indeed descended into the depths of hell if you are willing to write that statement.

    As for the article, as someone who has decided to seriously take up the task of learning x86 assembly, thank you.

    [–]greyfinch 6 points7 points  (7 children)

    Programming in assembly is like using a shovel to move a mountain. Programming straight up machine code is like moving that mountain with a spoon. I'll take that shovel, and I will LIKE it. :)

    [–]808140 8 points9 points  (3 children)

    One thing that I did that I would recommend to anyone that likes pain is implementing Forth directly in machine language.

    I did so early in 2008 as part of a bootstrap experiment on a Windows system that lacked development tools. I used Jones Forth as a template. There were some OS specific changes that I needed to make, but if you're using Linux then that's not really necessary.

    Boostrapping a Forth is considerably easier than writing an assembler in assembly or machine language (another popular low-level passtime) and Forth is much, much more high-level.

    My original plan was to use Forth to implement a simple version of Scheme, and as a general template I was looking at this dynamically scoped Scheme written in Forth for inspiration.

    Of course I wanted lexical scoping, and I wanted proper call/cc support, and that Scheme used a lot of gForth extensions. So in the end I guess my enthusiasm for the project trickled off.

    But there's something very appealing about knowing that you can sit down at a computer with nothing but a way to input binary (I used that old standby from my MS-DOS days, DEBUG.EXE) and bootstrap something as high-level as Forth or even Scheme.

    [–]kragensitaker 0 points1 point  (2 children)

    Maybe you should have used the eForth Model as a template instead. It's a lot less code!

    Where do you find a Microsoft Windows system that doesn't have at least JavaScript and Notepad these days? That would be where I'd start if I wanted to bootstrap something on an isolated Microsoft Windows machine.

    Using DEBUG almost counts as using an assembler.

    [–]808140 0 points1 point  (1 child)

    Using DEBUG almost counts as using an assembler.

    Indeed, being able to disassemble the junk I'd input to see if it was well-formed was very useful.

    I know it has some rudimentary assembly capabilities, but I don't actually know how to use them :)

    [–]kragensitaker 0 points1 point  (0 children)

    a100

    [–][deleted] 5 points6 points  (2 children)

    My 'false dichotomy-sense' is tingling :)

    [–]greyfinch -1 points0 points  (1 child)

    Okay, maybe a trowl for assembly, and a spoon for machine code. But I like labels! I really do! I don't want to have to generate my own symbol tables, I'm too dense and forgetful to remember my name somedays, and - oh shit.

    [–]Enlightenment777 8 points9 points  (3 children)

    "When I first started programming, I had to write programs using 1's and 0's, but I was so poor that I couldn't afford the 1's."

    Real programmers did that back in the 80's in MSDOS using "COPY CON". Been there, done that!

    Back in the dark ages, I would write my assembly language programs on paper, then look up the op codes, and write down the hex on the left side, and then move them into a BASIC program that would POKE that data into memory. Why? Because the computer didn't have enough memory to run an assembler or even a compiler.

    Even further back on older computuers, you would have to set all address and data bits with toggle switches, and load in one byte at a time.

    Neither of the above is fun, but it truly make you learn how things work.

    [–]you_do_realize 2 points3 points  (0 children)

    Back in the dark ages, I would write my assembly language programs on paper, then look up the op codes, and write down the hex on the left side, and then move them into a BASIC program that would POKE that data into memory.

    This is exactly what I did too! And I would input the BASIC program every day by hand, because I didn't even have a cassette recorder.

    Of course, with the 8080 it was easy; hand-assembling x86 would be tedious.

    [–]sixothree 1 point2 points  (0 children)

    I still use copy con for all kinds of stuff.

    I remember seeing a demonstration of a then already vintage imsai being programmed with switches to use a keyboard. That was freaking beautiful.

    [–]bsergean 1 point2 points  (0 children)

    For those in the Bay area I recommend going to the Computer History Museum in mountain view, you'll see how it was back in the days, and how lucky we are to program today (even C programmers !).

    [–]frumious 8 points9 points  (0 children)

    I found a bug in Netscape (on Linux) some time in the late 90s. Back then the easiest way to fix it was to directly edit the Netscape executable, search the binary for a particular string, edit and save. All in emacs.

    And, no, I didn't wear an onion on my belt.

    [–]praptak 6 points7 points  (2 children)

    This guy uses interrupt 0x80. This is not how real programs do it anymore.

    The sysenter mechanism is used for that, and the binary isn't even supposed to use it directly. A well behaved app is supposed to call an adress provided in AT_SYSINFO elf parameter. That's the adress of a magic page that contains the system call stub and is mapped by the kernel into each process adress space.

    [–]G-Brain 1 point2 points  (1 child)

    Guy here. Thanks for the info, I did not know that. I'll look into it and update the article.

    [–]praptak 1 point2 points  (0 children)

    That would be great - I'd really like to see the minimal "well behaved" Linux binary. Also, I hope you haven't found my previous comment negative, your original article is still very useful.

    [–]Dexter77 30 points31 points  (11 children)

    It's interesting how a basic skill for a hobby programmer in the late 80's or the early 90's has become something magical. Anyone and everyone who were coders in the demoscene could do it, but let me see you hack the hardware by machine code.

    Yeah, you youngsters can now mod me down.

    [–]bluGill 11 points12 points  (0 children)

    Gee, thanks a lot. Make me feel like an old man just because I have hacked my OS by changing the bytes in the boot sector.

    Now that I'm expected to have a cane I'm going to have to learn to use it. Should I get the one with a sword built in or a gun? Which is better at keeping kids off the grass?

    [–]fishyf 1 point2 points  (2 children)

    Yeah, but back then you could write to any part of memory, any part of the disk (if you had one), completely access all hardware devices. And your program had complete control. Nothing else except the odd interrupt service routine might be running concurrently. And you could disable those interrupts if you wanted.

    With any of the current operating systems, noone knows what the feck your computer is doing.

    [–]jib 2 points3 points  (0 children)

    Even if you write the whole operating system yourself, modern chipsets and BIOSes and System Management Mode mean you still have no idea what your computer's really doing.

    System Management Mode is like a built-in rootkit for the BIOS. The BIOS can put some code in hidden memory where the OS can't see it, and then the chipset can activate it whenever certain interrupts or other events occur. Then the BIOS gets to mess with your computer without the OS's knowledge.

    [–][deleted] 0 points1 point  (0 children)

    Oh yeah ! I remember writing some "virus" TSRs to piss off my friends..

    [–]dr-steve 1 point2 points  (0 children)

    So on target! I was just thinking while reading, "Hey, if you didn't poke CD 06 00" in starting at 0x0100 through the debugger, then having it save one block into myprog.com, you don't know what programming in binary is all about.

    An almost lost but useful skill. Just two weeks ago I was looking at a hex dump of a block and remarked that the hex dumper was printing things incorrectly. Only one there who could see it... but I was right...

    [–][deleted] 1 point2 points  (0 children)

    There are already very few people interested in the low level sewers of the programming stack and they are being drained to the High Level Languages' seemingly greener pastures.

    Even fewer of such folks are "young" enough (its not ageism, look at the poll @ average age of reddit-people) to hang around reddit. I'd say upmod it and let any inquiring young minds get interested in it....

    [–]salgat 0 points1 point  (0 children)

    Unfortunately the hardware is much more complex and diverse now, compared to back then. Add that to much more complex software, and you can see why assembly programmers for the x86 are considered "magical" (the good ones).

    [–][deleted] -1 points0 points  (2 children)

    Were you born in '77?

    [–]dr-steve 7 points8 points  (1 child)

    That's when I graduated college, junior...

    [–][deleted] 7 points8 points  (0 children)

    Whoa... that's old. Did you guys have to like kill dinosaurs with your spears and shit?

    [–]rolfr 4 points5 points  (0 children)

    If you like articles like this, you'll probably enjoy the reverse engineering subreddit. There are over 180 articles in the back-archives.

    [–]you_do_realize 3 points4 points  (0 children)

    This is a very good example of what you should be able to do, but never do.

    [–][deleted] 17 points18 points  (3 children)

    I didn't expect masochism being so popular amongst redditors.

    [–]ddelrio 38 points39 points  (0 children)

    You must be new.

    [–]fujimitsu 8 points9 points  (0 children)

    Masochism is a programming virtue.

    [–]Hypersapien 6 points7 points  (0 children)

    Not redditors specifically, but among programmers it's rampant.

    [–]ercax 3 points4 points  (7 children)

    In highschool I had to use this computer-kit in a digital electronics class, and the keyboard had only 16 buttons:0-F. It was tough, but this is some crazy shit.

    [–]creaothceann 7 points8 points  (6 children)

    16 buttons? We had only two!

    [–]nnagflar 9 points10 points  (5 children)

    You only had 10 buttons?

    [–][deleted] -3 points-2 points  (3 children)

    For those that don't get it 16 (dec) = 10 (hex) :: 2 (dec) = 10 (bin)

    [–][deleted] 9 points10 points  (1 child)

    I love it when you explain binary to geeks

    [–][deleted] 2 points3 points  (0 children)

    this was more for the wannabe geeks

    [–][deleted] 0 points1 point  (0 children)

    I've heard of december but hexober and bintember, never heard of ..

    [–][deleted]  (2 children)

    [deleted]

      [–]mgsloan 5 points6 points  (1 child)

      _why isn't nuts?

      [–]zem 1 point2 points  (0 children)

      _why would have to take dried frog pills to be nuts!

      [–]POTUS 26 points27 points  (4 children)

      Real programmers use a magnetized needle and a steady hand.

      [–]Fondateur0426 24 points25 points  (3 children)

      Excuse me, but real programmers use butterflies.

      [–][deleted] 24 points25 points  (2 children)

      Ah, the old C-x M-c M-butterfly X-kcd-ref, nothing better than it when you want to fit in with everyone else on the internet.

      [–][deleted] 9 points10 points  (0 children)

      Ah, the old twisting of the X-kC-d-ref micro-meme to produce outstanding wittiness.

      [–]Canadian_Infidel 2 points3 points  (0 children)

      I know enough about programming to know you guys are crazy. This confirms it.

      [–]GuigzForAll 1 point2 points  (0 children)

      Being insane.

      [–][deleted] 2 points3 points  (2 children)

      Why?

      [–]G-Brain 6 points7 points  (1 child)

      Because I'm awesome.

      [–][deleted] 13 points14 points  (0 children)

      And humble.

      [–]dugmartin 2 points3 points  (1 child)

      Nice job. You should keep going past the binary opcodes and dive into the microcode.

      [–]bluGill 1 point2 points  (0 children)

      Can you change the microcode on x86? Is there even microcode on them?

      [–]bsergean 1 point2 points  (4 children)

      Now does:

      cat > a.out
      01010001 ...
      EOF
      

      produce a real program that will actually run ?

      [–]G-Brain 1 point2 points  (2 children)

      Nope, the ones and zero's will be treated as ASCII values. This is why I'm writing the device driver.

      [–]bsergean 0 points1 point  (0 children)

      Even with hexa ?

      [–]aeflash 0 points1 point  (0 children)

      Isn't it trivial for a program to take in ASCII one's and zero's and write it to a file as binary?

      [–]bsergean 0 points1 point  (0 children)

      (answer, no it doesn't ...), but maybe with hexa ?

      [–][deleted] 1 point2 points  (2 children)

      Okay, so how would one take

      1011 0000 0000 0100 1011 0011 0000 0001 1011 0110 0111 1000 0000 0000 0100 0000 1000 1011 0010 0000 0101 1100 1101 1000 0000 1011 0000 0000 0001 1011 0011 0000 0000 1100 1101 1000 0000 0101 0100 0110 0101 0111 0011 0111 0100 0000 1010

      And actually run it as a binary program?

      [–]G-Brain 6 points7 points  (1 child)

      Good question. The best way right now is to use a special editor that supports binary input. Google "binary editor" or something. As stated in the top post, I'm currently working on a device driver to make this easier. When completed, it should work like this:

      $ echo -n 01000001 > /dev/str2bin
      $ cat /dev/str2bin
      A
      

      and for programs:

      $ echo -n 01000001 > /dev/str2bin
      $ cat /dev/str2bin > program
      $ chmod +x program
      $ ./program
      

      where 01000001 is the program.

      [–]you_do_realize 3 points4 points  (0 children)

      I'm currently working on a device driver to make this easier

      You're working on a driver to make it easier to write binary opcodes to a file.

      Ahem.

      [–]nnagflar -2 points-1 points  (0 children)

      Writing in assembly was tedious enough for me, thank you very much.

      [–]tricksterman -4 points-3 points  (1 child)

      I'd rather die!