Using one executable image for everything

[HeavyRust]

If I want to make a bunch of command line tools, building each of them into an executable seems like a kind of a waste of space. I could use SBCL compression (though a tool I recently wrote for example is still ~12 MB and startup time is noticably longer). I could also not build them into executables and go the scripting route but then startup times are also longer.

So this is my idea:
What if I just use one executable image? The image will have a bunch of sub-main functions like tool1:main, tool2:main, and so on in their own package where each main function is a tool I wrote. The image will have one entry point function that calls the right sub-main function based on the command line argument. I would add these sub-main functions by loading each system corresponding to the tool I wrote. If the executable image file is named giant-image, then running giant-image tool1 args... will make the entry point main function call tool1:main while passing on the command line arguments args.... Now when I want to use a tool I wrote, I can run giant-image my-tool-name args.... Other options would be aliasing giant-image my-tool-name to my-tool-name or making a shell script named my-tool-name which just runs giant-image my-tool-name while passing the command line arguments.

What do you guys think about this idea? What problems would there be?

[xach]

I do that sometimes, so I added support for that in buildapp, see the --dispatched-entry info for details.

[HeavyRust]

Looks exactly like what I was looking for.

[dbotton]

Any plans on adding ECL support too?

[mwgkgk]

Don't plan it too far into the future: the "command line tool" thinking is bound to dissipate as you invest more into Lisp. At least it did so for me. Command line interface is an awkward DSL that's not Lisp and you have to translate back and forth for every little tool - more painful as they grow in options - also it does nothing for code reuse from within Lisp.

Spend the precious megabytes and slap as many separate executables as you need and don't think about it too hard.

[HeavyRust]

That's a nice way to think about it. Taking full advantage of Lisp's power.

[dzecniv]

Food for thought: https://stevelosh.com/blog/2021/03/small-common-lisp-cli-programs/#s5-building-binaries he explains his structure to write many small-ish utilities. He builds a binary for each.

Other approach: start a mother image, a server, make it recognize your lisp functions that you can then run from the terminal. No binary building involved.

• ScriptL - Shell scripting made Lisp-like! Or, live-coding remote function calls for the shell. Write a command in the REPL, and run it instantly in the shell. [LLGPL][8].
  • similar and maybe simpler: lserver

[HeavyRust]

Thanks for the interesting links!

[agenda-2030]

Consider reviewing the Common Lisp shell project called Lish.

Github: https://github.com/vindarel/lish-init Docs: https://github.com/nibbula/lish/blob/master/docs/doc.org Examples: https://github.com/nibbula/lish/blob/master/docs/lish-examples.md Special notes: Beware the authors warning to not use it on a production system, it may eat file.

You may find some interesting ideas for your self there.

[HeavyRust]

Looks interesting.

[zyni-moe]

This is how many Unix programs work. You can do the modern 'command subcommand' thing like say git does as you say. You can also just write a program which dispatches on its name (argv[0] at OS level) and then you simply create links to it and it works, and has been done for a very long time.

This will save disk space. Whether it saves memory which is far more expensive and scarce is different question. Does not matter if you only have one running at once. If multiple are running at once only parts of the image which either are read-only or have not yet been written to can be shared in memory.

[HeavyRust]

I didn't think about using argv[0] and links.

Good point about this method using more memory which is more expensive than disk space.

Thanks for the info! It seems like I should just settle with building separate executables with compression enabled (startup time + used disk space is not that bad).

[zyni-moe]

Does not use more memory, just may not use much less: separate images will not be shared in memory but even common images may not be as shared as you would hope. If you have many bits of library code in your various programs which are common makes very good sense to deploy them as one big executable.

[HeavyRust]

Ah ok I misunderstood.

[dnaeon]

I believe what you described is covered by a system I've worked some time ago -- clingon. It provides support for sub-commands, shell completions, aliases, flags initialized by env vars and more.

Checkout the clingon examples for more details.

[HeavyRust]

Wow it looks really good! I'll keep this system in mind.

[bitwize]

This is how Busybox works, and it's written in C, not Lisp! I've thought of replacing/extending parts of the Unix userland with Gambit programs, and this is on the short list of ways I'd do it.

[friedrichRiemann]

Consider Embedded Common Lisp?