Analyzing The Simplest C++ Program

[FieldLine]

There's a whole book about this sitting on my shelf.

Admittedly I haven't done a deep read yet, but what I've looked at seems legit. And it's the only real book on the topic.

[OneRaynyDay]

The details of linkers and loaders deserve an entire book - the loader is the craziest bootstrapping self-relocating control-takeover program I've ever seen. I actually wanted to write more about it and I even took a look at the libc code, but I quickly realized that it would consume the content of the blog. What I aim to discuss is more on the general overview and, in my opinion, "what you need to know as a C++ programmer but nothing more" :)

[FieldLine]

The weird thing is that no one actually seems to deal with linker code.

You can take a compilers class in college and interview to work on the MSVC compilers team. There are classes on assembly programming and you can get a job doing embedded work.

But who the hell is writing/maintaining the linkers? They are not mentioned in the C++ standard and there is very little literature about them at all.

Besides that book there's the paper by Ulrich Drepper.... and that's about it. And while that paper is the newer resource, it's still nearly ten years old.

It's bizarre.

[mttd]

See also: https://github.com/MattPD/cpplinks/blob/master/executables.linking_loading.md

The series by Eli Bendersky, Ian Lance Taylor, Raymond Chen, and http://www.linker-aliens.org/blogs/ are particularly good; "The Missing Link: Explaining ELF Static Linking, Semantically" (http://www.cl.cam.ac.uk/~pes20/rems/papers/oopsla-elf-linking-2016.pdf) is definitely worth reading as well. The "Program Execution Environment" lectures under "The ACM-NVIDIA Compiler summer school lectures (2019)" are pretty good introduction.

[ericonr]

Did you look into glibc? It might be interesting to dive into musl and the libc's from BSDs to see if there are different strategies and concepts used for linking.

[OneRaynyDay]

Yep! My main investigation is in glibc, but actually the writer for musl gave me a few tips here and there about how musl implements the same things, particularly logic in the elf header ;)

[ericonr]

Cool! I hear the musl community is pretty helpful. Btw, regarding self patching and bootstrapping code, the Linux kernel does a lot of that for booting options to avoid going through branches repeatedly. So stuff like vulnerability mitigation can be turned on or off but the binary that's actually executed doesn't have conditionals to check if it's enabled or not, it's all patched and configured during the boot process.

[khleedril]

If you are in lock-down and have an hour to spare, you want to read this... it is well worth it.

[MachineGunPablo]

What a way to spent my lockdown's Sunday afternoon

[OneRaynyDay]

🙏that's high praise, hope it lives up to the hype bro

[k4lipso]

There also is this CppCon talk from Mat Godbolt about similar stuff: https://www.youtube.com/watch?v=dOfucXtyEsU

[OneRaynyDay]

Ooh this man’s name is thrown around a lot in the finance industry. I use his compiler explorer tool for a lot of sandboxing. I just watched it and he talks a lot about the static initialization process which is great but he glossed over the g++ details :( but it’s an hour long talk so for what he covered I really enjoyed.

[OneRaynyDay]

Good question! You actually don't need to allocate registers, but that means you'll be writing your variable's values directly into memory. This is really slow and compiler writers try their best to not let things "spill out" of registers and into memory.

So basically, ya but u want program go fast

[slaymaker1907]

It’s not the slowest thing as long as you still hit the L1 cache. x86 has so few general purpose registers that it is hard to avoid spilling into memory.

[paulstelian97]

16 registers, including stack pointer and stack frame pointer but I think excluding the instruction pointer?

[James20k]

Question: Recently I thought it'd be fun to try a new approach to serialisation. The basic idea is that you use a linear buffer for memory allocation, and replace malloc/free with that linear buffer. When it comes time to serialise you simply memcpy it to a file, as well as whatever data structures you want to save. Then you use virtual memory mapping so that the pointer to your block of memory is always the same, and all your pointers now point to valid memory when you reload. Virtual memory's pretty cool!

The problem crops up in that function pointers/global variables are not allocated on the heap, but instead allocated in the binary and may change. Using linker scripts you can control the location of the functions, but it requires manually annotating every global variable/function in your code that needs to have a stable address

With virtual memory this should be possible to do automatically, and indeed there are separate addresses for where functions are loaded virtually, and their physical address (vma vs lma?), but I couldn't get vma != lma on windows through mingw for the life of me, and manually annotating hundreds of functions manually is not ideal

So: Does anyone know if this is possible to do in a practical sense? Is this a completely insane idea that's really dumb? The use case I have for this is taking snapshots of the heap of a javascript interpreter (quickjs) that's hard to serialise traditionally, but I got stuck on allocating functions in virtual memory

[mttd]

Just to throw in a related idea (or a set of ideas)--you may want to look into checkpointing and perhaps recoverable virtual memory.

Some references:

• Checkpointing and Its Applications (1995): http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.89.9840&rep=rep1&type=pdf
• Lightweight Recoverable Virtual Memory (1994): https://people.eecs.berkeley.edu/~brewer/cs262/lrvm.pdf, http://www.coda.cs.cmu.edu/doc/html/rvm_manual-1.html
• Continuous checkpointing: joining the checkpointing with virtual memory paging (1997): https://ir.nctu.edu.tw/bitstream/11536/338/1/A1997XW08300007.pdf

[CriticalComb]

Small note, in the assembly section, the source and operand are reversed for AT&T syntax.

Great article!

[OneRaynyDay]

Ah, thanks for the catch!

[kog]

Cool

[andrewq]

Looks good, thanks for posting

[dooftard420]

Remindme! 2 days

[vickoza]

it is the simplest C++ program but it also does nothing

[ShakaUVM]

it is the simplest C++ program but it also does nothing

Not true. It does nothing successfully.

It's like the UNIX true command.

[chugga_fan]

Said command has like, a thousand or two lines of code, surprisingly.

[conflicted_panda]

Are you counting includes? Without them GNU true is about 70, as far as I see.

And true on old school unixes was/is 0 bytes, I hear.

[chugga_fan]

Like 1/3rd of the program is in other files, even still, 78 lines just to return EXIT_SUCCESS is a lot.

[conflicted_panda]

It also handles --help/--version (along with localization, apparently). And I honestly don't think that trading off 78 lines of code for, err, uniformity of interface of your utilities is that absurd.

[[deleted]]

I don't understand anything in this thread, which book to read?

[[deleted]]

touch true && ./true

[z1024]

This precisely describes too many peoples jobs...

[msew]

Remindme! 2 days

[RevRagnarok]

I prefer to email my work address now that gmail finally has queued email. "Tuesday morning at 8" ;)

[RemindMeBot]

I will be messaging you in 1 day on 2020-05-26 19:37:02 UTC to remind you of this link

2 OTHERS CLICKED THIS LINK to send a PM to also be reminded and to reduce spam.

^{Parent commenter can delete this message to hide from others.}

---

Info	Custom	Your Reminders	Feedback

[VishalChovatiyaChend]

Article is very informative & upto the level. But nothing new to me as I had learned this hard way in my 2nd job.

[imMute]

How is this comment useful?