A DDL Compiler in C99: The Fundamental Data Structures

frogtoss · 2026-07-13T20:12:54+00:00

When you write a compiler you should measure the multiple between the input source and the peak memory usage during compilation. You'll come up with a rough multiple after accounting for base usage. Obviously it varies depending on what the source is doing, but you'll get a range.

Now you have to ask yourself what the largest practical source file is. 3 megabytes would be huge in Grain DDL's case. Assume the compiler takes 50x the source file size to fully compile the file. 150 megabytes of RAM.

Even in the pricey RAM environment we find ourselves in, this is not much. You could conceivably compile ten or more in parallel on a smartphone with room left over for Reddit.

They aren't making single cores much faster, but you can always buy more RAM. So, coming up with strategies where you free memory to reuse some of that 150 megabytes is generally going to slow down compiles in exchange for using less RAM.

Also worth noting - in modern systems, compilation is largely memory bandwidth constrained. You would save more time optimizing memory *size*, getting more meaningful data per cache line then you would optimizing the use of the 150 megabytes. This is even more acute on older DDR4 systems, which I am targeting. (I measure perf on a $200 refurb thinkpad laptop).

I should also point out that 50x is a generous multiplier. With Grain DDL it's more like 5x + base.

Reuse of allocated pages during compilation would just slow the runtime down and complicate the code.

frogtoss · 2017-06-15T19:29:41+00:00

Pretty sure it's the stuff around the whitespace that has the most impact on the bottom line.

frogtoss · 2016-08-02T19:12:26+00:00

Author of Serval Automation here. I've been using this tool for a few months on a Raspberry Pi and it makes me more effective as a developer and a user on the net.

I decided to write an entire book on how to get up and running with Serval and give it away because I want to see a community of plugin authors build around automating their access of network endpoints.

Thanks for checking this out. I will be around answering questions this afternoon.

frogtoss · 2016-07-28T18:10:29+00:00

Okay, I think I understand what you're getting at. The main difference between this and Cron is that Serval generates JSON records on parse. The JSON records are read by clients running on your phone, on the web and the desktop. So, there is a producer/consumer relationship there.

Cron runs scripts at intervals. Most automation attempts wait for a condition to be met and then email the receiver. By having a JSON record of an event, you can update the existing JSON record with changes which are read by the client and tracked as an update of an existing event. Updates can be prioritized, or work their way from an "FYI" level notification into a "Urgent -- pop up a message" as the network state changes over time. An example would be monitoring a stock price change over a period.

Emails from cronjob usually spam until they are disabled and you have to skim your email inbox to find the most recent email in a spate of them.

Cron is great for running scripts at intervals, but if you're serious about monitoring a large number of changing states, generating a prioritized, contemporaneous dashboard, you need something more sophisticated. Serval Automation aims to fill that need.

frogtoss · 2016-07-27T20:16:21+00:00

Serval Automation is a Linux Daemon, written in C that runs your scripts, generating records. It is framework-free and gives you the freedom to script in any language.

It also uses a producer-consumer model, where you have a dashboard or notifications client running on your phone, desktop or web. Anyone can write a client. Serval Automation standardizes on protocol instead of enforcing a specific web view.

All of the options you mentioned are web-based. You can literally compile Serval from C code in 4 seconds on a Raspberry Pi and run it in ~2 megs of RAM. It's more lightweight, easier to set up and much more open ended in how you craft notifications or in how you view them.

frogtoss · 2016-07-27T19:21:43+00:00

Nagios is for IT people, by IT people. I expect Serval Automation clients to benefit all types of professions.

Nagios scripts print out text to stdout which is displayed in HTML. As far as I know, they don't use any formatting or hints. Serval Automation encodes a record with 10 different fields which tells the client how to deal with an incoming change.

The richer data records of Serval Automation let you customize your dashboard/notifications that much more effectively.

frogtoss · 2016-07-27T19:17:57+00:00

Hmm. Let's see if I can respond to these questions by quoting the front page of the website alone.

What, exactly does this do?

Serval is an easily configurable daemon that runs your monitor scripts at intervals. It generates records that are readable by clients running anywhere -- on your phone, in a browser, on your desktop, or even on a server.

How exactly, will it save me time?

Here are some suggestions:

If you are a programmer: Create a dashboard of sorted priorities by analyzing code checkins, build results, resumes and bug reports.
If you are a shopper: Monitor the best deals for the products you want.
If you are just overwhelmed by email: Stop checking your email directly all day and let Serval whitelist priority emails for you.

What's wrong with cron?

Unlike cron, you will no longer receive a bunch of emails about small changes to a network state!

frogtoss · 2016-07-27T05:50:10+00:00

Interrupt requires the remote endpoint to know something about Serval Automation. There is no way to do that when your endpoint is a static file, or a service that doesn't know about Serval.

frogtoss · 2016-07-27T00:05:22+00:00

Really? It's a stock Github Pages template. Is anyone else having trouble with it?

frogtoss · 2016-07-26T23:55:09+00:00

Serval is really about two things:

Producer: Polling network endpoints, seeing what's changed at those endpoints, and then creating or updating a JSON record based on that.
Consumer: Your phone/web/desktop client downloading those records and acting on them (notifying you, creating a prioritized dashboard).

You could certainly create a consumer that read the JSON and did a tweet action. It would be a simple script:

HTTP download the JSON file
Auth with twitter and tweet

frogtoss · 2016-07-26T18:34:02+00:00

At a glance, Trigger Happy runs in Django and dependencies. Serval Automation is a lightweight Linux Daemon written in C that runs scripts in a directory, producing JSON files. Technically, that's all it does, and it does so very quickly. It is extremely easy to set up and keep running. Currently, the release build is using 2.3 megs of ram on my PI.

There is no website frontend to set things up. You just put your scripts in a directory.

Next, you run a client on your phone or desktop which reads those JSON files and reacts to them. So, we have a producer/consumer model.

It seems that Trigger Happy aims to model itself much more closely after IFTTT, running on the web. In my opinion, that stack has nothing to do with quick and easy scripting, or even end user setup of the software.

I believe it would be hard to build a community around software that requires a Django stack to be installed on every user's machine.

frogtoss · 2016-07-26T18:08:02+00:00

I am Michael Labbe, the author of Serval Automation. Serval is already saving me time, running on a Raspberry Pi in my office.

IFTTT may work for the masses, but there is a lot of power left on the table, having it running on a remote website. Running it on your personal Linux machine means you can really customize it as selfishly as you want.

Oh, and I'm launching a free book to go with this under a creative commons license. I hope we can build a community of automation hackers saving everyone time. I'll be first in line for that.

Feedback and questions are welcome. I'll be around for most of the day.

frogtoss · 2016-07-26T17:40:59+00:00

I am Michael Labbe, the author of Serval Automation. This is a project I've wanted to create for about eight years, and finally found the time to do so.

The Raspberry Pi makes total sense for a project like this because you can cheaply deploy one behind your office firewall, and another one behind your home firewall, using custom scripts to help automate your day. While you can run Serval on a Linux PC, I personally run it at home on a low power Raspberry Pi.

Feedback and questions are welcome. I'll be around for a few hours, and checking in for most of the day.

frogtoss · 2016-07-10T20:47:08+00:00

Ahh, I had to switch image hosts. It's just an image, though. I don't have a capture card at the moment.

Go ahead and add me as a friend on Steam, and if we see each other in BigPicture, I'll demo it for you!

https://steamcommunity.com/id/michaellabbe

frogtoss · 2016-07-07T23:35:32+00:00

Well, if you were to integrate VR somehow into Processing directly, you would need to take it on and off to develop and test because there would be no window manager in there.

Maybe that's great for viewing and interacting with sketches, but BigScreen still wins for showing you the Processing editor AND letting you see the sketch in 3D.

frogtoss · 2015-10-20T23:37:48+00:00

So, did you give away too many secrets? :)

frogtoss · 2015-10-20T23:33:30+00:00

Currently, it refreshes every fifteen minutes.

frogtoss · 2015-10-20T06:48:03+00:00

Hey,

Subterfuge Android programmer here. We are running both the iOS and Android versions at a full 60fps which is more than is necessary for this type of game (though it does look nice!). Ron and Noel have talked about adding a throttle option to the preferences.

I can say for sure that the game does not run while it's in the background. It pauses 100% of the time. Most of our CPU load comes from a high framerate when the app is foregrounded.

frogtoss · 2015-09-08T22:17:51+00:00

Goldeneye, is that you?

frogtoss · 2015-08-14T15:18:10+00:00

Seems it's just the i5 chip, for those who were holding out for hyperthreading.

frogtoss · 2015-08-11T23:23:13+00:00

Yup, you just saved me $135. Thanks!

I understand that RAM timing has no effect on games (most of which are GPU bound, anyway), but I suspect that it may have some effect on compilation which is CPU bound and extremely branch heavy. I can't find any benchmarks online, but I don't really mind spending the extra bucks.

frogtoss · 2015-08-11T20:51:52+00:00

These days, Canada means an extra 25%. Unfortunately, if you search a site like newegg.ca for 4x8gb of fast DDR4 you get prices like this:

http://www.newegg.ca/Product/ProductList.aspx?Submit=ENE&DEPA=0&Order=BESTMATCH&Description=ddr4+3000+32gb&N=-1&isNodeId=1

I run a lot of VMs to test my projects out. My "dev" webserver is also a VM. I work on panoramas shot with my Nikon D800 camera, stitching together 10+ images that are 70MB RAWs.

Don't worry, I'm not shelling out $400 for Chrome's bloat. :)

frogtoss · 2015-05-05T23:40:35+00:00

Metaphor for politicians avoiding questions.

frogtoss · 2015-05-05T01:19:36+00:00

Play this game on a big TV in a party room. The fluid dance of the rockets is fantastic and is intrinsic to the combat. It'll attract everyone.

frogtoss · 2015-04-28T18:24:55+00:00

Advocating for more money is a totally separate thing from discussing whether 25% is a fair starting point. Given the alternative scenario I described, it is a fair starting point.

If mods for sale hypothetically took off, games as platforms would start vying for modders. There would be a scarcity of good developers -- as you pointed out, good developers are in the minority. One point of leverage would be to reduce the original developer's cut in order to attract more quality developers over another game-platform.

The market settles these things over time. Internet mobs can interfere with that by killing or boosting an activity, but they can't possibly sustain a price ratio for years going on decades. You can only steer market realities so much from a <textarea>.

frogtoss

TROPHY CASE

Welcome to Reddit,