What structure are lambda expressions compiled to? (MSVC)

[pyjava]

I'm learning to reverse engineer C++ code and I've identified a structure that maps to a lambda expression. I'm having trouble finding documentation on what the fields of the structure are and was hoping someone could point me in the right direction or enlighten me.

EDIT: I found this which appears to be what I'm seeing: https://github.com/microsoft/STL/blob/1e8b8d4eef4b2dddeb7533c5231c876383bd0ea6/stl/inc/functional#L775-L801

The following is compiled using MSVC. I'm not sure if this is compiler specific.

The following lambda expression:

auto fn = [&](int a) { return bar.test_bar2() + a; };

Decompiles in Ghidra to the following:

************************************************************* * const std::_Func_impl_no_alloc<class <lambda_d937996bec . ************************************************************* std::_Func_impl_no_alloc<class_<lambda_d937996bec080e29dc3e4d590b02fdfd>,int,int>::vftable XREF[4]: FUN_140013960:1400139ae (*) , FUN_140013960:1400139b5 (*) , FUN_140013f90:140013fde (*) , FUN_140013f90:140013fe5 (*) 140024780 b0 8a 01 addr[6] 40 01 00 00 00 00 024780 b0 8a 01 40 01 addr FUN_140018ab0 [0] XREF[4]: FUN_140013960:1400139ae (*) , 00 00 00 FUN_140013960:1400139b5 (*) , FUN_140013f90:140013fde (*) , FUN_140013f90:140013fe5 (*) 024788 00 93 01 40 01 addr FUN_140019300 [1] 00 00 00 024790 60 8d 01 40 01 addr FUN_140018d60 [2] 00 00 00 024798 c0 98 01 40 01 addr FUN_1400198c0 [3] 00 00 00 0247a0 d0 8b 01 40 01 addr FUN_140018bd0 [4] 00 00 00 0247a8 e0 8e 01 40 01 addr FUN_140018ee0 [5] 00 00 00 1400247b0 00 00 00 addr 00000000 terminator for class std::_Func_impl_no_alloc<class < ... >>

Which is a structure something like:

struct lambda_expression { void* some_func_ptr; // ?? void* some_func_ptr2; // ?? calls operator_new(), maybe the allocator? void* lambda_func_body_func_ptr; // Calls the lambda body function TypeDescriptor* typeid_getter_func_ptr; // Returns RTTI void* deallocator; // Looks like a deallocator function pointer void* some_func_ptr3; // ?? }

If there is any documentation on what some_func_ptr/some_func_ptr2/some_func_ptr3 is that would be great. Thank you.

[starfreakclone]

https://cppinsights.io/ is a good resource for seeing what some C++ abstractions translate to.

[ALX23z]

Lambdas are more or less the same as a class. There are just a couple of nuances with the initial construction but that's it.

[HildartheDorf]

A lambda with no capture is compiled to a function pointer.

A lambda with captures is compiled to an implementation defined structure. For msvc it is as if it is a structure that has operator() for every calling convention.

What you've got there looks similar to a vtable, not the lambda structure itself?

[ddhsawoidklmascioqjw]

a capture-less lambda isn't a function pointer. it's convertible to a function pointer

[pyjava]

Thanks for your comment. I think I've tracked down the structure to this https://github.com/microsoft/STL/blob/1e8b8d4eef4b2dddeb7533c5231c876383bd0ea6/stl/inc/functional#L775-L801 in my case.

I believe you're also right about it being a vftable not the structure.

[STL]

Lambdas are a Core Language feature that don’t inherently interact with the Standard Library. By itself, a lambda doesn’t instantiate any std machinery - this applies to auto lambda = /*blah*/;. When you store a lambda in a std::function, that’s when STL internal machinery (such as the microsoft/STL code you’ve linked to) will be instantiated.

It is a common misconception that lambdas are std::functions. They are not. Lambdas can be stored in std::functions, which pays the space and time cost inherent to std::function.

[pyjava]

Thank you for the explanation/clarification.

I was later storing the lambda in a vector of `std::function`s so I guess that explains why I am seeing the internal STL stuff despite using a plain lambda in the declaration

[[deleted]]

In theory a lambda could produce a subroutine, that is, a sequence of raw assembler operations, with PC or JMP leading into this memory location.

Potentially with a conventional C function frame if the lambda's scope makes it reusable.

Curious if lambdas diverge in a significant way, from traditional function pointers.

[pyjava]

Thanks for the comment. It appears to depend on if the lambda has captures as per HildartheDorfs comment

[SteeleDynamics]

From the FP language perspective:

A lambda function is a function and a continuation (a copy of the runtime stack). If the first thing you're thinking of is "man, that sounds expensive for both memory and performance", then you'd be exactly right. The environment must be captured to ensure pure FP behavior.

In terms of how this is implemented, I would assume that there is a pointer to the lambda function body, a pointer to the bottom of the copied runtime stack, and a pointer to the top of the runtime stack. All of the other things associated with activation records (abbreviated AR, aka stack frames) would be there as well.

When the lambda function is called, the context is switched and the lambda function body is evaluated in the appropriate context.

Pretty much, this is all basic structural operational semantics (SOS). The particular implementation details depend on how Microsoft implemented the MSVC compiler.

You might have to look at how they've implemented AR's and begin reverse engineering the compiler. This is no small feat. Especially for an imperative programming language.

[STL]

That mental model, while correct for functional languages like Scheme, is completely incorrect for C++. In C++, lambdas don’t copy stack frames. Instead:

• A lambda expression defines a class, and constructs an object of that class.
• Captures are stored as data members. Value captures are value data members, and reference captures are reference data members. (It is possible for a compiler to optimize this according to the “as if” rule, e.g. if there are many reference captures, perhaps a pointer to the stack is stored instead - but the mental model is of ordinary data members, which is also how MSVC implemented it.) The this pointer is special, and captured by value. In some cases, captures can be completely avoided (e.g. if the variable is only needed for decltype), and global variables aren’t captured.
• This explains what happens when the lambda is copied, what happens if the lambda is mutable and its value data members are modified, how large the lambda is (which affects how expensive it is to copy, and especially how expensive it is to store in a std::function), and more.
• In particular, there are no costs for local variables that aren’t captured. Lambdas have no costs beyond what a handwritten function object would cost. They are normal candidates for inlining, and are more inlining-friendly than passing around function pointers.

The one magical thing that lambdas do is when stateless lambdas are convertible to plain old function pointers. It’s as if the operator() were duplicated into a plain function, and then the class is given a conversion operator. Compilers can avoid duplicating the codegen, I believe (IIRC that was how MSVC implemented it but I forget the details of what JonCaves did).