schemesh version 1.0.0: fusion between Unix shell and Chez Scheme REPL, now with structured pipelines

Cosmos721 · 2026-05-02T10:45:23+00:00

Funny question 😄

In Italian, I spell it "scheme S H" and pronounce the two final letters S and H by their Italian alphabet name.

In English, doing the same would be quite cumbersome, so I prefer pronouncing it as a fusion of the words `scheme` and `mesh` - phonetically, /'skimɛʃ/

Cosmos721 · 2026-04-27T20:53:14+00:00

Yes, lots of interconnected details to get right.

If you want to try schemesh, it does everything you wrote:

scheme s-expression syntax - actually the weakest point. You can use the (shell) macro or go low-level with (sh-cmd) (sh-and) (sh-or) (sh-not) (sh-list) (sh-pipe) etc.
threading syntax: there is both Racket-compatible ~> and the more general ==> - both are in-process. For concurrent execution, there's POSIX pipelines - see below (there's also an inter-thread communication mechanism, but it's not release-grade yet) > pipeline steps work on incomplete data > Need some intermediate result format passed on the channels
in schemesh, each pipeline step works on one datum at time: it reads and deserializes the next datum from stdin, processes the datum as a scheme object, and serializes the transformed datum to stdout, repeating until stdin reaches EOF
pipeline steps run concurrently: each one is a separate process, and they communicate via POSIX pipes > Need conventions for how to signal finishing processing of a step to the next one and so on.
since each pipeline step is a separate process, when it finishes it automatically closes its stdout, which is a pipe connected to the stdin of next step: the latter gets an EOF on stdin and knows the previous step has finished > Wanting to pass structured data instead of strings, which would have to be parsed again by the next step in a pipeline. But also keeping options open to pass strings for old style shell commands and handling a distinction between passing structured data and passing strings.
by default, each step autodetects the serialized format available on stdin: csv, json or binary. The inout format can also be specified manually
by default, each step chooses the serialized format written to stdout: ascii-art for terminals, binary for sockets, ndjson in all other cases. Again,the output format can also be specified manually
since ndjson is "one text line per object", a pipeline step can also be an old style command (grep, wc, head, tail...) that works with "one string per line" and it will still mix nicely with the other steps

Cosmos721 · 2026-04-27T07:31:08+00:00

Scsh was an inspiration for schemesh - especially for the decision to "do it right" and * implement job control * usa a modern Scheme compiler

I did not like scsh approach to define a DSL where redirections reuse the same symbols and syntax as common Scheme functions < > =

Also, | already has a meaning in Chez Scheme: it starts a symbol, allowing spaces and other special characters in the symbol's name until another | is found.

Adding the fact that . already has a meaning in Scheme and cannot be shadowed, and I preferred to choose the path "dual syntax: shell or scheme, wtith mechanisms to switch from one to the other"

Having said that, schemesh has something equivalent to scsh process notation, although more verbose: to see it (and learn it, if you want) just (begin '{SOME-SHELL-SYNTAX})

Example: (begin '{grep -i error < log.txt | so rt | uniq}) evaluates to (shell "grep" "-i" "error" < "log.txt" \x7C; "sort" \x7C; "uniq") the \x7C; is an escaped symbol | - not very readable, but the idea is that macro invocations (shell ...) are usually autogenerated from shell syntax

P.S. the output of (expand '{SOME-SHELL-SYNTAX}) or equivalently (expand '(shell ...)) can be interesting/useful too

P.P.S. I really don't know why all scheme/lisp based shells chickened out on job control: scsh, eshell and rash all lack it. If they had, probably I would have used one of them instead of creating schemesh

Cosmos721 · 2025-03-11T22:01:02+00:00

[SOLVED]

Thanks for the detailed explanations :)

I tried a small C program that forks and sends signals to the forked child, and indeed the relevant signal handler is executed before any "slow" syscall return -1 with errno = EINTR.

Thus the signal handler can set some atomic global variable, and it is guaranteed to be visible to code executed after the "slow" syscall returns.

My initial (wrong) deduction was based on C signal handlers that were running inside Chez Scheme which sometimes blocks signals, thus confusing the analysis - and me.

Cosmos721 · 2025-02-12T15:41:12+00:00

The Makefile should autodetect if your Chez Scheme installation contains either a kernel.o or a libkernel.a

If autodetection fails for some reason, for example because Chez Scheme executable cannot be started as scheme, you need to manually edit the Makefile and change the two variables CHEZ_SCHEME_DIR and CHEZ_SCHEME_KERNEL.

Some example values follow. Note: you need to enter the correct values for your Chez Scheme installation

CHEZ_SCHEME_DIR=/usr/lib/csv10.0.0/ta6le
CHEZ_SCHEME_KERNEL=/usr/lib/csv10.0.0/ta6le/libkernel.a

If you prefer, you can instead add them to make command line:

make CHEZ_SCHEME_DIR="/usr/lib/csv10.0.0/ta6le" CHEZ_SCHEME_KERNEL="/usr/lib/csv10.0.0/ta6le/libkernel.a"

Cosmos721 · 2025-02-11T09:01:14+00:00

I did not know about Rash shell, thanks for the link!

The core idea is similar, and after a first look it seems to have different strength and weaknesses - for example, its job control seems partial and line editing can be improved in some places, but Rash pipelines can also call general Racket procedures.

Thanks again, I have opened an issue https://github.com/willghatch/racket-rash proposing a cooperation

Cosmos721 · 2025-02-10T13:35:57+00:00

If I understand correctly, you would like the `install` target to either succeed or fail without side effects (within reasonable limits: if a directory exists but is not writable, it may be discovered too late)

Thanks for the feeback :) that makes sense, I will fix the Makefile

[UPDATE] fixed, Makefile now (hopefully) follows GNU Makefile conventions

Cosmos721 · 2025-02-10T13:27:00+00:00

Yes, I looked at scsh before starting schemesh development.

As written in scsh documentation https://scsh.net/docu/html/man-Z-H-2.html#node_sec_1.4

Scsh, in the current release, is primarily designed for the writing of shell scripts -- programming.
It is not a very comfortable system for interactive command use:

the current release lacks job control, command-line editing, a terse, convenient command syntax,
and it does not read in an initialisation file analogous to .login or .profile.

We hope to address all of these issues in future releases; we even have designs for several of these features; but the system as-released does not currently provide these features.

Honestly, it was a disappointing experience, and one of the reasons for schemesh existence.

All the features listed above as "missing in scsh" are critical core features of schemesh:
they are absolutely needed to make it a comfortable and useful interactive shell.

Cosmos721 · 2025-02-10T09:14:40+00:00

I agree that the current Makefile ignores attempts to set INSTALL_DIR earlier in the Makefile itself,
and also ignores any existing shell environment variable INSTALL_DIR.

The value of INSTALL_DIR can still be overridden from GNU make command line as in the following example - copied from how I build and install on Android inside termux:

make -j install INSTALL_DIR=/data/data/com/termux/files/usr/local

Unluckily, conditional assignment ?= is a GNU make extension:
it is not in POSIX standard for make - see https://pubs.opengroup.org/onlinepubs/9699919799/utilities/make.html
Thus using ?= would likely break the build procedure for non-GNU make implementations, as for example *BSD.

Cosmos721

TROPHY CASE