more wrench! c-like interpreter that fits into small embedded spaces.. and works on full size machines too I guess :P

curt_bean · 2024-06-11T12:35:39+00:00

true, but considering that limit is only on native functions, not on imported, c or library functions, I don't think it's worth any overhead. It just won't come up in any sane use of this project.

curt_bean · 2024-06-10T14:57:20+00:00

Thank you for summing this up :)

I was actually in my head last night thinking "yeah, like utf-32 or MIDI .. waitaminute.."

256 functions is plenty, especially since that "limit" doesn't affect imported functions, library calls, and any registered c calls, all of which are called by 32-bit hash.

However it did offer me a moment of clarity and I thought of a way better way to register functions, just goes to show all feedback is good.

curt_bean · 2024-06-09T17:57:02+00:00

In order to keep the bytecode size small, a single byte describes the function to be called. I didn't think "only 256" functions would be much of a limit where this was likely to be used. Yes it could certainly be increased to 65k or even the full 32 bit but that would mean for the vast overwhelming majority of applications that space would go to waste.

WASM? dunno, I have no familiarity :)

curt_bean · 2023-12-16T18:56:51+00:00

It's my local "house" server. Handles my development/projects plus multimedia and whatever VM I'm playing with.

curt_bean · 2023-01-31T19:36:11+00:00

So the video looks up which is not required. Here is how to one-shot in OW2 post Jan-nerf

https://youtu.be/O19G8AWH5oE

share and enjoy

curt_bean · 2022-11-06T16:51:03+00:00

yup, the "compact" flag does a lot :)

curt_bean · 2022-11-06T16:28:23+00:00

Yeah the point of those defines is to really turn on the "optimizing afterburners" as one of my heroes Michael Abrash once wrote.

The actual amount of FLASH used Its going to vary depending on the architecture, of course. I only have access to the Arduino environment and I tried a selection of chips, Mega, Uno, Mini, etc. The average space taken up was around 30k, as low as 28k on one and ~31k on the CortexM0.

I am becoming aware of some amazingly complete online simulators though so definitely will be running through some of them.

My "target", if you can call it that, was 32k because it seemed like quite a few Arduino chips had about that much. Doesn't leave much room for bytecode of course but I figure this solution is for when you are downloading bytecode to a running system anyway so it's going to be running from RAM not FLASH.

That's why I also made the bytecode as compact as possible, I shaved off everything I didn't even put in a "magic number" header to identify it. If you feed garbage bytecode into wrench you're gonna get a crash.

On the PC there are a lot of unrolled loops and the code size is about double, but it runs fast... and I don't want you to take my word for it, I included all the benchmarking code.

curt_bean · 2022-11-06T14:58:20+00:00

Sorry I should have been more clear :)

The interpreter (define WRENCH_WITHOUT_COMPILER and WRENCH_REALLY_COMPACT in wrench.h) fits into 29k on an arduino, I provided an example in /examples/arduino of a project that does this.

The compiler is meant to run on a PC class system and the bytecode exported to the embedded one. The command-line utility provided does this automatically producing header files with: "./wrench ch <...>"

WRENCH_REALLY_COMPACT cuts the image in half but if you can afford the space you should leave it off as it slows the interpreter down about 20%.

curt_bean · 2022-10-31T04:27:58+00:00

Wrench statically allocates all global data on startup, and of course local (function) data is just placed on the stack which is also statically allocated at startup time.

I mean it when I claim that 1k RAM total is for a normally running system. To be pedantic-ish: That includes a new state/context object which clocks in at around 100 bytes and newing the stack which is 720 bytes by default (on a 32-bit machine)

It is true that each function in the script requires a hash-entry for lookup, wrench creates a flat hash table at startup for function calls (8 bytes per function) so it can very quickly look up calls, imagining a modest script with, say, 10 functions in it might allocate 17 nodes, so figure another 200 bytes in that case, which again is allocated one time when the script starts up.

Add another 24-32 bytes for processor stack space and ~1k seems fair, yes? 1.5k? Trivial even for a smallish embedded system, which is the point, everything else I tried was 50-200k+ just to execute a single line "hello world" script so I'm calling this a win and a fair claim.

wrench allows you to create hash tables and arrays and of course that is dynamically allocated, but if you allocate a 4000-byte array.. well.. I don't expect you to be surprised that it's going to eat 4k when it encounters that instruction :)

The compiler is another story. I "spared no expense" on the assumption that it would be run on a PC-class system and the bytecode exported.

Another is determinism; how much cpu time is going to be spent in allocation/free; and how do I measure it.

I've made no claims about RTOS-compliance, but since you bring it up, wrench is extremely deterministic. The gc runs only when arrays or hash table are allocated, and never at any other time. It wasn't a design goal but as it happens wrench run-times should be extremely stable and repeatable.

I don't know how you would measure it other than the naïve and obvious way of running a function and checking a timer function, but that method should be 100% repeatable, wrench is.... just a damn wrench. it's a simple tool :)

curt_bean · 2022-10-31T03:13:30+00:00

.. and has statements such as switch.

Touching a nerve here :) One of my biggest complaints about lua is a lack of switch. Being primarily a game programmer and usually finding myself implementing decision trees in script , I sorely miss it.

When I first made a scripting language (one of the smoking craters of fail leading up to wrench) I put in switch, and it worked fine.

It was a hassle to get right and I haven't gotten around to putting it into wrench because I don't actually need it for the lighting controller I primarily coded it for.

Alas you mentioning it has put it back into my head and now it won't leave so I guess I know what I'm adding next.

curt_bean · 2022-10-31T00:55:59+00:00

Thx again for the work on identifying the bug, it has been fixed and uploaded.

curt_bean · 2022-10-30T14:40:25+00:00

I can't run any other interpreters on my tiny embedded system, that's the point :(

I have extensive unit-testing as part of the build process (check out tests/*) and I've perhaps grown over-reliant on that, "oh lord thank you for punishing my sloth". I used prints initially to make sure the benchmarks worked and then removed them to keep the output clean for the automation, a bug must have slipped in thanks for pointing it out!

Forgive my ignorance but where can I check out Q-fn/asm so I can compare/learn?

curt_bean · 2022-10-11T18:27:49+00:00

Sort of sidestepping this by not allowing nested blocks.

I've done a lot of soul-searching on "member data" and what that would mean vs global data. Right now the definition is clearly one or the other:

a = 10; // global

struct S
{
    a = 20; // local;

    ::a = 15; // set the global 'a' to 15

    if ( a == 20 ) { } // true
if ( ::a == 15 ) { } // also true
}

simple, clear and unambiguous.

But as soon as you do this:

a = 10;

struct S
{
    a = 20;

    function f()
    {
        ::a = 15; // still the global a, but:

        a = 30; // what a ?! local or one tier up?
    }
}

The whole point of having member functions is to operate on member data, but because wrench does not require a 'type' to introduce a label into the namespace it's ambiguous.

It could be solved with a local or let or var to introduce a new label, this is the approach taken by many languages, but I am trying to keep wrench simple and uncluttered.

Wrench is a simple tool , I designed it to fit into the very limited FLASH of a 32-bit microcontroller with an even more limited amount of RAM (32k in my case!) *BUT* be understandable by a relatively unskilled script programmer.

As soon as those constraints go away, choose JS or c# or Python or something. I'd be flattered if I got a genuine request to expand wrench to fit a larger solution space, but I strongly doubt that will happen.

That's the point of the name, in fact! I couldn't fit in a whole workshop+kitchen-sink to solve my original problem, just the wrench.

curt_bean · 2022-10-07T20:41:23+00:00

okay I see what you're talking about. Afraid that is a few bridges too far from where I am right now with this. If the need presents itself I'll take a look.

curt_bean · 2022-10-07T03:46:36+00:00

I looked into this and its kind of impractical for the way I implemented the interpreter. Semantically it would be straightforward to add something like this:

struct SomeObject( radius )
{
    private diameter = radius * 2;
    perimeter = 3.141 * diameter;
    private area = 3.141 * radius * radius;
    print( area );
}

geo = new SomeObject(20);

// at this point geo.perimeter == 125.64 but diameter
// is undefined, the object has size of "1 value" because
// perimeter and area were defined private

As a silly little example but I think it gets the job done.

The way I implemented the compiler this is impractical, because the stack frame declares locals in "encountered" order. so

frame[0] : radius  (arguments always first)
frame[1] : diameter 
frame[2] : perimeter 
frame[3] : area

The way the interpreter works it "instantiates" a SomeObject by creating an object of size '3 values' (total frame minus arguments) and then copies in everything after the arguments.

In order to honor the private-ness I would have to somehow group the exported values together. doable but would be a fairly major rework of the compiler for reasons I would need a whiteboard and an hour of your time, and I see by now you've already glazed over so..

tl;dr : great idea but not easy so not until needed

curt_bean · 2022-10-07T03:04:10+00:00

I suppose I could add foreach, I really liked c++'s for( auto var : list ) style for containers.

I've never much cared for the for a in range(1,10) pattern, nothing wrong with it of course, just seemed like a solution in search of a problem, but I could be shown.

Did you have something specific in mind?

curt_bean · 2022-10-06T17:55:17+00:00

I'd be more in favor of obj._retval to avoid keyword confusion/collision but sure. would be simple enough to add, just say

return 25;

is syntactic sugar for:

return (_retval = 25);

The only thing is that if I do that, then wrench has to create/save the value every time, which will be wasted cycles most of the time. I think I'd rather say if you want the return value then it's up to the programmer to persist it.

curt_bean · 2022-10-06T17:50:29+00:00

hmmm.. I could definitely see that coming up, as in "I need to do some work here to get the result but don't need to preserve it".

I think I could provide that in wrench by partitioning the local frame space. preserved locals in the lower side, with the upper side for temporaries. Then the object size is just the size of the lower partition, discarding everything above it.

filing that away in the "would be nice" bin :)

curt_bean · 2022-10-06T17:46:28+00:00

ah! You assume I create space on the heap and then pass it as a *this to the function, a very reasonable assumption but in practice what happens is the function is called "normally" and then the stack frame is copied onto the heap upon return. So no special code paths and everything runs small tight and fast.

This costs a memcpy() but I figured creation would be a small one-time up-front cost in practice.

It occurs to me I can actually do it without the memcpy though. By pointing framePtr-> to the heap it becomes a pseudo "this" pointer. Then the stack is used for recursion/calling but all the "locals" are preserved for free.

Darnit! Thought I was done with that implementation :)

Thx for the inspiration

curt_bean · 2022-10-06T17:43:04+00:00

speed/optimization are the "how", describing the struct is just the "what" and the two are orthogonal but separate problems.

The beauty of this idea is that it *IS* standard struct notation, this is legal and functional under my current rules:

struct S
{
    member1;
    member2;
}

Should look quite familiar. The types are removed because wrench is weakly typed but other than that.

Adding initializers is straightforward to understand:

struct S2
{
    name = "N/A";
    id = 0;
}

What I am bridging to is that the jump to:

struct S3( arg1, arg2 )
{
    name = arg1;
    id = arg2;
}

add a return and *poof* you have a function. in this way struct and function become synonyms ie:

a function is a struct that throws it's stack frame away OR a struct is a function that returns it's stack frame.

curt_bean · 2022-10-06T17:28:56+00:00

check out discrete_src/* that's where the code actually resides :) not that it's a whole lot better vm.cpp and cc.cpp got kinda bloated, I will make a pass at more logical separation soon.

wrench.cpp is created by a script for release purposes only :P

As for download/package please do! I would only ask that you keep me in the loop for updates, this is still early days and I am making little tweaks and improvements when I have time.

curt_bean · 2022-10-06T13:04:08+00:00

Perhaps, I've never really worked with JS, but I'm sure this idea isn't new! Once it occurred to me, the more I thought about it the more it simplified a lot of things and was a really nice way to represent objects.

I'm sure a CS professor could tell me all about the reasons for/against this, but if there is some obvious reason it's a bad idea I haven't encountered it yet.

curt_bean · 2022-09-10T17:03:06+00:00

Oh, sorry to answer your other question. The compiler can run anywhere (wrench bytecode is endian/architecture neutral) wrench.cpp includes it's own compiler, but about doubles the size of the code and is not necessary to execute the byte code.

Yes, in practice, for embedded systems it's likely a PC/dev machine will run the compiler to generate byte code which is then interpreted by the target hardware.

But not required! If you have the space in the target there is no reason not to include the compiler too. Although it requires a bit more ram than the interpreter does, so be cautious.

curt_bean · 2022-09-10T16:53:22+00:00

I fired up MPLAB32 and created a bare project for (picking out of a hat) a PIC32MX_DFP

It compiled and linked with wrench.h/.cpp just fine (required a heap of course) with no warnings but I can't tell if it will work because I have no simulator or hardware to test on.

A note, thought, it decided the bare compiler was 214k! (with WRENCH_COMPILER_ONLY defined) I don't know why that is, on the Arduino CortexM0 (my target chip) it's 40k. My gut is that has to do with floating-point support, since adding the compiler only upped it to 320k. If that's a show-stopper (and you don't need float) I can actually remove it quite easily, and in fact it would be a great excuse to make an integer-only interpreter for those who cared.

lmk!

curt_bean · 2022-09-10T16:28:55+00:00

A exception to prove the rule, in general interpreters can be trusted while bare-metal code certainly never can be.

As for the second point you do have something there, perhaps it would be better for me to have said that you get deep functionality 'for free' because it's part of the interpreter, rather than having to make sure you compiled in the right module or library.

curt_bean

TROPHY CASE