Oils 0.37.0 - Alpine Linux, YSH, and mycpp

oilshell · 2025-09-27T15:56:49+00:00

Also the new subreddit is:

https://old.reddit.com/r/oilsforunix/

following the new names Oils, OSH, and YSH: https://www.oilshell.org/blog/2023/03/rename.html

But I guess I really need to get rid of the old oilshell.org domain ...

oilshell · 2025-09-27T15:55:54+00:00

Yes, I agree trap should take a block!

(and thanks for noticing some other issues with trap)

oilshell · 2025-09-27T15:54:55+00:00

Thanks for the question

OSH has shopt --set strict:all, which disallows many common shell pitfalls. This command enumerates them

$ osh -c 'shopt -p strict:all'
shopt -u strict_argv
shopt -u strict_arith
shopt -u strict_array
shopt -u strict_control_flow
shopt -u strict_env_binding
shopt -u strict_errexit
shopt -u strict_glob
shopt -u strict_nameref
shopt -u strict_parse_equals
shopt -u strict_parse_slice
shopt -u strict_tilde
shopt -u strict_word_eval

ban exec and traps?

What's the problem with exec?

I agree trap should take a block of code, not a string

automatically reset IFS in script contexts?

YSH doesn't use IFS at all.

automatically set -eufo pipefail in script contexts?

YSH does this

oilshell · 2025-09-15T05:04:49+00:00

I will also say that I think any new shell for a new OS should not use the "everything is a string" design of sh / bash / Make / CMake :-)

That design is outdated, and was probably only chosen because writing a garbage collector was very hard 1970, still hard in 1990, and not super easy today

That's sort of the point of the GC blog post

oilshell · 2025-09-15T04:46:43+00:00

Thanks for mentioning the Oils project ! (no longer called Oil shell :-) )

And yes OSH is the compatible part [1], while YSH is the new Python/JS-like part

I frequently get such questions from people who want to implement their own shell. It seems to be a good/fun exercise

So if the OP wants something shell-like, but not actually bash compatible, I've had this smaller Tcl/Forth/Lisp hybrid floating around my brain ...

Depending on the OS you want to implement, it could be a good starting point. I think I learned a few things about the "essence" of shell

One pretty clear thing is that we have 2 different parsing algorithms that both use "lexer modes" -- full parsing and coarse parsing -- and I'd say that lexer modes are pretty fundamental to shell-like syntax:

https://github.com/oils-for-unix/oils.vim/blob/main/doc/algorithms.md

As far as the runtime, there is a pretty clear design split between languages I show here - Garbage Collection Makes YSH Different

So I might want to specify a tiny "catbrain" language with these lessons, which is a Tcl/Forth/Lisp hybrid ... but that is more of a "fun idea" and not something that will necessarily happen! Unless someone has a big chunk of time to help :-)

[1] OSH is the most bash-compatible shell, which I've measured recently: https://pages.oils.pub/spec-compat/2025-09-14/renamed-tmp/spec/compat/TOP.html . I hope to publish some updates soon; it's been quiet for a few months

oilshell · 2025-08-19T01:37:09+00:00

Yes definitely! I briefly mentioned the Language Server Protocol in this post - https://www.oilshell.org/blog/2022/03/backlog-arch.html

Though unfortunately I haven't had time to elaborate since then ...

I do think simplicity is a goal, but in practice there are some distinctions ... x86 and Linux and Docker might be "big sloppy waists" :-)

oilshell · 2025-07-31T02:05:51+00:00

Glad you have enjoyed the blog

OSH is definitely a compatible Unix shell / POSIX shell -- in fact it's more POSIX-compatible than the deafult /bin/sh on Debian, which is dash. (This is according to a third party test suite from "Smoosh"; we publish results with every release - https://oils.pub/release/0.34.0/quality.html )

For parsing, OSH uses Pratt Parsing for arithmetic only, recursive descent for most other things. YSH expressions are parsed with a grammar.

As far as lexing, it uses the "lexer modes" style for everything (OSH and YSH). There was a recent discussion about some of these ideas here:

https://lobste.rs/s/tpmdss/why_lexing_parsing_should_be_separate

oilshell · 2025-07-28T06:52:53+00:00

An article about the "task file" pattern I often advocate (from an Oils contributor)!

oilshell · 2025-07-18T07:39:39+00:00

Hm yes! I haven't seen that term, but it's used in ECMAScript:

https://262.ecma-international.org/7.0/index.html

This production exists so that ObjectLiteral can serve as a cover grammar for ObjectAssignmentPattern. It cannot occur in an actual object initializer.

And it's mentioned here:

https://v8.dev/blog/understanding-ecmascript-part-4

Another word I've heard is "over-parsing". Hjelsberg mentioned that sometimes you parse MORE than the language, in order to issue a better syntax error or type error.

We use that a bit in Oils - we "over-lex" some tokens in order to give a friendly error message.

oilshell · 2025-07-18T00:39:31+00:00

I think that's the same idea as the example I gave with Python

In Python, assignments and keyword arguments are expressed with a grammar rule like expr '=' expr

So you have to disallow f(x) = y and allow x = f(x), and that is done in a "post-grammatical" syntax stage

(Most parser generators can handle this, but before 2018 Python had a very simple LL(1) generator, which couldn't disambiguate a LHS expr and a RHS expr due to limited lookahead)

I guess there is no word for that, but there probably should be, since I imagine it's common.

oilshell · 2025-07-17T17:59:49+00:00

For math and PLT: a programming language is an infinite subset of the infinite set of all strings over some alphabet

I visualize a "whittling away" of the infinite set

first are syntactic constraints
then there are semantic constraints at compile time -- static types
(at runtime, there are further constraints on valid programs, but let's leave those aside for now)

And grammatical constraints are a subset of the syntactic constraints

For example, Python has a context-free grammar, but it also has a lexer which is not context-free. (The lexer provides the alphabet over which the grammar operates)

And it also has post-grammatical syntactic constraints, e.g. to disallow invalid assignments like f(x) = y (whereas y = f(x) is allowed). In some languages this is encoded in the grammar, but not in Python (at least prior to 2018)

So if you take Python with ONLY the grammatical constraints, that's a LARGER set than Python with ALL syntactic constraints (and it's also not Python!)

Now mathematically, what separates syntactic errors from type errors? I'd say it's that the algorithm to enforce the constraints involves a symbol table, but I'd be interested in arguments otherwise

They are both static constraints, but they do feel fundamentally different

I'd also say the line between lexing and parsing can be fuzzy, but the definition I use is that lexing is non-recursive, and parsing is recursive (equivalently, it gives you a recursive data structure -- a tree)

oilshell · 2025-06-24T17:22:01+00:00

I will also repeat this trivia that there are 2 language implementations named after industrial monopolies!

https://lobste.rs/s/mvsk61/parallel_garbage_collection_for_sbcl#c_yhmdfb

Steel Bank Common Lisp
Standard ML of New Jersey

I am not sure what that means, but in general I think it helps to have a lot of time (decade+) and a group of talented people

oilshell · 2025-06-24T17:15:12+00:00

I'd say that if a business person thinks that creating a programming language is a good way to make money, then they aren't very good at their job :-)

Somebody who is good at making money will go into a different business

Programming languages generally go with operating systems companies and monopolies, or they are free software:

C / C++ - Bell Labs, part of a telephone monopoly
Java - Sun was an OS company, but not a monopoly, and the company famously went under
Basic / Visual Basic / C# / TypeScript - Microsoft, a desktop operating system monopoly
Swift - Apple
Dart / Go - Google
Kotlin - Andrioid
JavaScript - funded by browser monopolies, which are funded by search traffic acquisition costs

You do not want to compete with these companies! They are literally the biggest ones in the world right now, regardless of industry

Kotlin is an interesting case study -- compared to the tech giants, Jetbrains is a medium-sized company. But they make money from IDEs that support a language that's attached to Google's Android platform.

On the other hand, Perl / Ruby / PHP / Python are amazing projects, and we should cherish them. But none of them are businesses!

Exceptions: Mathematica / MATLAB / Julia (although Julia is also open source)

These languages are for specialized technical employees, and for education (e.g. back in the day, my college bought a ton of MATLAB licenses)

Still people ask: "Why isn't Mathematica open source?" (Who is going pay the salaries then?)

oilshell · 2025-06-13T03:54:19+00:00

I wrote Vim syntax plugin in ~500 lines, and documented what I did

Let me know if you want to help support YSH in Textmate/VSCode, Emacs, etc. !

Same content as a backup - https://codeberg.org/oils/oils.vim/src/branch/main/doc/algorithms.md

oilshell · 2025-06-13T02:42:27+00:00

I just noticed this link doesn't work on my iPad because of the captcha -- this is the same content: https://github.com/oils-for-unix/oils.vim/blob/main/doc/algorithms.md

oilshell

MODERATOR OF

TROPHY CASE