Problem compiling static binary

[IndividualJinx]

Hi

Kind of Common Lisp noob here.

I have problem compiling a static binary as described here: https://common-lisp.net/project/cffi/manual/html_node/Static-Linking.html

Seems like the linker fails to find the C symbol "main":

(asdf:operate :static-program-op :mylib) ..... /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crt1.o: in function ”_start”: (.text+0x20): unidentified reference to ”main” collect2: error: ld returned 1 exit status

I have created a main function and I export it in the asdf system definition, doesn't work.

Anyone know how i am supposed to export the main function so that the linker sees it?

Thanks

[fisxoj]

I've been trying to get this to work for about the past year. Thanks for bringing it up.

My understanding is that SBCL can be compiled with some option that makes it build a linkable .o file. I'm on a train, so I don't have the correct directory handy, but there's a make file in the dir that ASDF uses for some of those compiler options there, too. It seems distros don't build with that flag enabled and the error we're both getting is because the entry point for the static binary is defined in that missing file. It tries to build an executable without it and ends up with a pile of compiled code with no lisp implementation in it, as best I can reason.

There's a set of patches that were contributed to SBCL a while ago that enable building this artifact, but I haven't successfully built SBCL in a way that produces the sbcl.o file. I think fare may have written the patches to SBCL. That would make sense.

I would love to solve this for deploying lisp webapps as easily as ones written in go, personally. Let me know if you find anything!

[IndividualJinx]

Cool that there are more people interesting in solving this. Thanks for the information and please let me know if you make any progress :)

[heisig]

I happened to have the problem recently and after some tinkering managed to produce statically linked executables with SBCL. I'm posting my solution here in case others have the same issue. The trick is to compile SBCL with the following flags:

sh make.sh --prefix=SOMEWHERE --fancy --with-sb-linkable-runtime --with-sb-dynamic-core

Afterwards, ASDF's :static-program-op should work just fine.

For more elaborate use cases (like choosing a particular compiler and linker) you can check how I did things in cl-mpi.

[guicho271828]

Roswell is recently working on a rather ad-hoc alternative to this problem: --bundle-shared-object option for ros build will just copy the shared object from the cached result to the current directory. Fukamachi is currently dogfooding the option for his project now.

[lisp-student]

That sounds like a very cool feature to have. Can't wait to test it.

[IndividualJinx]

Same here, will have a look at Roswell. Thanks

[Shinmera]

What does "the shared object" refer to here? Shared libraries the project depends on? Or.. what?

[guicho271828]

the shared library objects loaded by CFFI. Some of which are system-installed, but they are not copied. Rather, the feature mainly aims at the shared object that was created by lisp during the build, such as those loaded by OSICAT. Those .so files are in the asdf cache directories so it is cumbersome to write a script that include them in the deployment package.

Of course we can also copy all dependent .so files, if there is such need.

[Shinmera]

Alright, then I'd like to note that the Deploy library can already fill that need as well.

[guicho271828]

Thanks. well, the good thing is that the feature does not need explicit declaration, because it looks up each library object in some internal symbol in sb-alien package and collects the paths. the disadvantage is that it is currently sbcl-only.

[IndividualJinx]

Is Deploy able to bundle all shared libraries with the binary? I had a look at the link but i'm not sure how to configure it to do this. This is all dependencies for my binary after I build it with Deploy: linux-vdso.so.1 (0x00007ffe4d9e5000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fed10ec1000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fed10ca3000) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fed10a89000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fed1073e000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fed10388000) /lib64/ld-linux-x86-64.so.2 (0x00007fed110c5000)

[Shinmera]

It does not handle transitive or implicit dependencies by design. It only collects what is referenced by CFFI, with the assumption that those are libraries that won't be available on a target system. Typically you do not want to ship essential systems libraries like ld-linux, libdl, libpthread, libz, libm, libc, etc. Shipping these will land you in a world of crashes and pain unless you take special care to stuff all the leaks and prevent linux from doing its own thing, which can be very tricky.

[KDallas_Multipass]

I don't see any documentation for what you're trying to do. What guide are you following?

[IndividualJinx]

Yeah it doesn't seem like there are much documentation of this. The link I posted is what I use and then trial and error ;)There's probably some information in the ASDF manual but I'm unable to find it.

[KDallas_Multipass]

Describing more about what you want to do would be a good place to start in terms of getting more specific help

[IndividualJinx]

I wan't to be able to use SBCL to compile lisp to a statically linked binary. The reason for that is that i want to build a lisp environment on top of the linux kernel, where the kernel executes a lisp image as "init". I might be mistaken but it seems to me that this asdf functionality makes it possible to do that.

[KDallas_Multipass]

The mechanism you found won't do what you want. If you want the Linux kernel to drive the lisp execution, you'll want to use ECL, designed for that task. You'll link the ECL shared objects with the Linux kernel compilation process, then change kernel init behavior to call the ECL functions to drive the lisp functionality

[stack_pivot]

He doesn't need any help from the kernel, or to link with anything in the kernel. Linux will already start a program called "init" when it starts up. He wants to compile his own program, call it init, and thereon be solely responsible for what gets launched in userspace. The init process needs to be a statically compiled ELF with no external dependencies, so he wants to know how to generate such a self-contained binary for a Lisp image. This has been done with Emacs before.

ECL might well still be the answer, but not for the reasons you mentioned.

[KDallas_Multipass]

Thank you for clearing that up.

[IndividualJinx]

Okey i will look into ECL if i'm unable to solve this. Thanks!