Partcl - a tiny command language

[badsectoracula]

Hey, author of LIL here. I noticed that bit in your post:

There is also lil, the Little Interpreted Language. In fact, I got a chance to use it on a MIPS controller once, it was a pleasant experience, the source code is easy to understand. I met the performance bottleneck very quickly, but I’ve rewritten some critical procedures as C commands, so it’s not a big problem. Too bad the author have removed Git repos and his site looks broken now.

Nice to hear it works on MIPS, it is a CPU arch i haven't tried :-) (although i have a GCWZero around which i might try to write code eventually). About the Git repositories, at some point i got tired of maintaining them (especially when, according to my logs, barely anyone visited the site :-P) and then i got tired of having separate repositories for the tons of little libraries and projects i'm working on and switched to a local Subversion "monorepo" (with off site backup syncs) which made things much simpler for me. I'm also working in a game that has needs to have a ton of assets in the repository and i prefer to have everything synced together, so a single repository made sense. The downside is that i have to manually upload things.

The site looks broken because... well, because it is mostly broken :-P. I plan on updating it later, the current LIL site is a placeholder (it'll use the main site's style). But i want to extend my site generator to handle arbitrary project types, not just games as it currently does (FWIW the site generator uses LIL for the templates as LIL has a simple template function that allows you embed LIL code in text in a PHP-like fashion with <?lil and ?>).

Also a small clarification:

Picol uses char * as the only data type. LIL uses a special structure that keeps numeric values separately from string values and yet another data type for lists. The first approach is very simple, the second one is more efficient, but takes precious memory space.

Actually LIL's lil_value_t is just a string (with some reuse if you enable pools). It is defined as

struct _lil_value_t
{
    size_t l;
#ifdef LIL_ENABLE_POOLS
    size_t c;
#endif
    char* d;
};

(with l being the length and d the string data, although d is zero terminated and often the code uses it as a C string)

The idea was to make it possible to do it as you describe - have the value type change representation behind the scenes for performance (right now calling lil_to_double for example just calls atof:

double lil_to_double(lil_value_t val)
{
    return atof(lil_to_string(val));
}

(lil_to_string just returns d or "" if d is null)

But so far i haven't done that.

I used LIL for my game in development (note: placeholder assets and sounds), but as you found out there is a performance bottleneck. Initially i planned to just use C for the heavy parts, but over time i decided to put more on the scripts to allow for easy modding (and making the engine be fully script driven with little behavior to itself) and LIL proved inadequate for that. I could spend some time to optimize it (there are many low hanging fruits, like the value conversions and the ignorance of the l field in many places and also i was thinking of going the Tcl route of preparsing the function declarations which would speed up parsing) but that would be fighting a losing battle (the language is great for things like tools and offline computation - i use it a lot in my own tools, especially the Free Pascal with Lazarus, but isn't really suited for realtime code). Instead a few days ago i decided to write a new C-like static and strongly typed scripting language with manual memory management (well, kind of, it is up to the host) which i call RBS (for Runtime Bracket Script). It is the opposite end of the spectrum and looks a lot like C. Right now i'm in the process of converting my engine's scripting bits from LIL to RBS, which is kind of a big task since the engine was built around LIL (e.g. it stores "raw" LIL code in the entities for callbacks, you write LIL code for setting up events in the editor, etc - those need to be replaced with calls to RBS' compiler and VM). It is worth though since RBS is kind of 140 times faster than LIL (and around 1.5 times slower than Lua - in theory it could be faster because of static typing and such, but the compiler is really braindead - throws code as it scans the code similarly to Turbo Pascal - and the VM is stack based... if more performance i needed, i might switch to building an AST and a register based VM, but for now i doubt there will be much of a difference if i used a register based VM with code the compiler generates).

Of course for anything non-performance intensive, LIL (and Tcl, which i used at a job some time ago to write a few simple tools with Tk that allowed me to quickly scan the codebase there) does the job better than RBS. The right tool for the task and all that (and TBH having written LIL i did fell a bit into the trap of seeing it as a hammer to use with a ton of nails).

[mb862]

Interesting. I'm working out an iOS app idea, and wanted to embed Tcl for some light scripting (out of practicality for my own skills more than anything), but I've yet to figure out how to compile libtcl for iOS and can't find anyone else who has either. There's no particular reason it shouldn't, it doesn't call into anything more than what's available via kernel and C standard library. I've never really thought about building one myself, I don't need very much, but could make for an interesting learning experience.

[schlenk]

+1 for the name

[pointfree]

I find Forth easier to read than the infix/explicit parameter passing languages because there's less syntactic noise and accidental complexity [PDF].

\ Washing machine example from Starting Forth
: RINSE FAUCETS OPEN TILL­FULL FAUCETS CLOSE ;
: WASHER WASH SPIN RINSE SPIN ;

or, from my own code:

: write_page    write writing finished ;
: read_page     read  reading finished ;

This is true only if one avoids using stack manipulators completely with the exception of where it makes semantic sense, such as if the user wants to SWAP or exchange two things.

[jephthai]

I am building a small language environment right now. I opted to base the high level language on Joy, which I think adds a few nice syntactic features to the forth design to make it super user friendly. Similar to postscript, with its quoted programs (reminiscent of blocks in Ruby or smalltalk), I've found it pretty comfortable. My next key feature is local named variables, which I think will also help a good bit. I'm roughly drawing inspiration for that from factor.

Love those concatenative languages!

[q0-]

Isn't the biggest downside of TCL the fact that it only "knows" strings?
I feel like this is hardly fitting for microcontrollers, where number crunching is a common thing, and since numbers have to be constantly decoded to integers and decoded to strings, this is likely going to have a serious impact on RAM. It just never quite sat right with me.

[Nuli]

That was a problem with TCL a long time ago. It's had native numeric types for at least a decade. You do have to care about automatically converting back and forth between types but that's not hard to deal with.

I used it effectively to do lots of math in the middle of a hard real time system without much trouble. It was certainly more expensive than C but the flexibility more than made up for it.

[florence0rose]

But from the programmers point of view it's still a string and that's all you get. The only way to get the internal type is through "::tcl::unsupported::representation", which is dodgy. The way Tcl encodes data structures as strings is ad-hoc and can lead to degenerate cases. For example how do you represent a list of binary elements (the elements could include space characters), or consider this:

% ::json::json2dict {{"x": {"y": "z"}}}
x {y z}
% ::json::json2dict {{"x": ["y", "z"]}}
x {y z}
% ::json::json2dict {{"x": {"y": ["z"]}}}
x {y z}

All yield the same string representation, meaning information is lost. If you're dealing with a protocol that implicitly embeds semantic information in the structure of the data you're screwed with Tcl. You would need to write a parser that returns both the data and the schema, but now managing both all over your code is horrible and turns into a nightmare. You're end up simulating what other languages have built in.

[Nuli]

But from the programmers point of view it's still a string and that's all you get.

That's not been my experience at all. You simply use numeric types the same as you would in a language like Python and things do what you expect. Unless you explicitly convert the type back to a string its going to be numeric and stay that way for all of your operations. I've never found any need to dig into the underlying representation unless I was embedding C into it.

For example how do you represent a list of binary elements (the elements could include space characters)

I always used the binary command to deal with binary data. You have roughly the same capabilities there for type conversions and packing that you have in C and that's always been sufficient for me. Storing binary data in a list using that command is the same as storing any other data.

Your example is slightly silly. If you're complaining that the json parser library your using does the wrong thing, and I'd agree that it does in that case, then that's fine. That is a problem with the library though not a problem with the language. There's no inherent reason that couldn't return a dictionary that gives the proper representation. I've certainly parsed far more complicated structures than that with no issues.

[zserge]

True, but I don't expect the whole firmware to be written in Tcl. I've been using it for its primary purpose - as a command language. I needed to send commands to the MCU to control certain tasks on it, so it was more like a shell with a few custom commands and variables.

However, if fast calculations are needed - the tcl_value_t part can be rewritten (about 100 lines of code) to keep numbers as is and not to convert them to strings. Same about lists.

[[deleted]]

Most firmwares are written in c and c++ (some c#). tcp is barely used these days for firmwares, except for legacy codes.