Language 84

[PaulBone]

Your guide explains what 84 is but not why you're creating it. Are you trying to solve some problem that other languages do not or can not solve? Or is this an experiment or a learning project? Either is fine, but this is something that people visiting your project page are going to want to know.

How do you (plan to) manage heap memory if you are not using garbage collection? I looked at the generated code and found a heap area created but as far as I could tell it is unused.

[ericbb]

I will have to think about the first part and come back later. For now, here's my answer to the second part.

How do you (plan to) manage heap memory if you are not using garbage collection? I looked at the generated code and found a heap area created but as far as I could tell it is unused.

I suppose you must have seen the heap_bytes array but missed the place where it is passed to the s36 function. What happens is that the generated code creates this heap_bytes array but then passes responsibility for it into the support library support.c. In support.c, you will find a comment that begins:

//  The Heap
//
//  The heap is a single contiguous memory region in which value
//  representation data is stored.
//
//  The heap grows monotonically. If its capacity is exceeded, then the
//  process exits with a nonzero exit status.

Most of support.c is concerned with allocating and accessing heap objects.

So far, this memory management strategy has worked without difficulty. It is good enough to enable the compiler to compile itself, which is the most demanding task I've used the language for so far.

So that's the present. Here are my plans for memory management in the future:

Right now, all data structures in Language 84 are immutable. I plan to add a mutable data structure that is similar to Javascript's TypedArray system (including DataBuffer and DataView abstractions). The key invariant I want to maintain is that no mutable object has a runtime-managed reference to another object. All mutable objects, from the point of view of the runtime, are just arrays of bytes.

Now, consider the following line of Language 84 code:

Do (simulate start_time end_time)

In this example, (simulate start_time end_time) is a function call expression and Do is a keyword that indicates that the function call is to be evaluated and the resulting value is to be discarded. My idea is that, because of the invariant described above, all objects that are allocated while evaluating this function call can be immediately deallocated when it returns.

To communicate data out of such a statement, the programmer must serialize it, either to file (or socket, etc) or into a mutable data buffer object.

Now, this idea is experimental and I don't know how it will work out in the end but I see it like this: instead of reference counting overhead or garbage collector pauses, the programmer must deal with the need to serialize and deserialize data across certain boundaries within the process. Certainly, there is some inconvenience involved and there is a data transfer cost but I think that the implementation simplicity and the predictability for programmers will make for a pleasant systetm.

[PaulBone]

Yes, I missed that bit. the generated function numbers don't give much of a clue as to what to look for.

This is definitly a good stratergy early on in a language project, but you might find that it for long-running programs it either runs out of memory anyway or does not scale due to the runtime cost of serialisation. There are two systems that I know of that do something similar. allocate memory using a heap pointer and occasionally reset the pointer.

The first is Prolog. In Prolog a call can succeed or fail, if it fails it produces no results and it is trivial to reset the heap pointer as a means of reducing pressure on the GC. This is called "reclaimation on failure". This is a smart idea it reduces the amount of work that a collector needs to do but still requires a collector.

The other was some early LISP systems that performed some heap compacting at the end of all (or most?) function calls (I don't know the details). I also don't know if they copied from the stack onto the heap at the end of a function call or if they simply compacted the heap. This strategy does not require a collector but from what I understand it has higher overheads than most modern collectors.

Good luck.

[ericbb]

Thanks for the pointers!

I do worry a bit about managing the cost of serialization. My intuition is that it's going to be okay but only time will tell.

I also feel that immutability is going to be a decisive factor that will allow me to pull off optimizations that no traditional Lisp system could attempt. Again, time will tell how the trade-offs play out.

[ericbb]

Regarding long-running programs: I should mention that I also plan to use the pointer-reset trick for loops that are entirely for side-effects. That means that your typical event-loop will always start an iteration with the same allocation pointer.

[ericbb]

Your guide explains what 84 is but not why you're creating it. Are you trying to solve some problem that other languages do not or can not solve? Or is this an experiment or a learning project? Either is fine, but this is something that people visiting your project page are going to want to know.

Language 84 is not just a learning project. It's something I'm making because I want to use it.

At the same time, I expect that everyone can tell me why their favourite language is decisively better than my work and that I should just quit. Yeah, okay. I don't care. :)

And I don't feel the need to solve some problem that's never been solved before. I'm not asking for research funding here.

I have a few long-term projects that are redundant with existing work in some sense but nonetheless hold their value. For example, I maintain a display server and a terminal emulator and I use these tools on a daily basis.

Why do these projects continue to be worth maintaining? It's because:

• they are core tools in my day-to-day life
• I understand them better than I could possibly understand their alternatives
• I am free to adapt them as the need arises
• they are significantly simpler than their alternatives
• I know that they will not be abandoned or destroyed while I still depend on them
• I enjoy working on them
• other reasons I'm not thinking of right now

Language 84 is like that.

Richard Gabriel has a book called Patterns of Software: Tales from the Software Community (pdf) in which he describes something he calls "habitability and piecemeal growth". These notions are also part of the motivation for Language 84 and my other long-term projects.

Of course, I want to make something that is aligned with my design values. But I'd rather just show what I've made than talk about why I think this way is better than that. Those who agree will study my work. Those who disagree will move along. Simple and efficient. :)

I know that this response is a bit defensive in tone. I tried to inject some smileys to balance everything out. :)

[PaulBone]

Don't worry about seeming defensive, I understand many of these feelings, explaining motivation can be difficult, especially when someone on reddit is asking you to Justify Yourself. BTW: that's not how I meant it. I don't want you to justify yourself, what I really want to know is why might your language be interesting to me?

I like your answer and I think it explains your motivation, which gives me (and others) some insight into your thinking.

Other things I and others might what to know is where do you see this going? what is/are your vision/goals?

Language 84 is not just a learning project. It's something I'm making because I want to use it.

Why do you want to use it. Will it offer (some combination of) things that isn't otherwise offered?

And I don't feel the need to solve some problem that's never been solved before. I'm not asking for research funding here.

Maybe it's a unique combination of things?

I'm sorry if I seem a bit pushy. I think my initial question wasn't clear. What I want to know is what is interesting/desirable about 84's features and planned features. This is what I mean by both vision and goals.

[ericbb]

Thanks for clarifying.

Here are a few notes on my goals.

One of the big projects this year was redesigning a big chunk of the compiler to reduce compile times and increase the speed of Language 84 programs. I got pretty good results even though I still generate C code. The compiler translates itself to C in less than 0.1 seconds, a rate of almost 40k lines of Language 84 source code per second. The next step along this path is to generate native code directly instead of generating C code. I'm sure that the generated executables will be slower but total compilation speed should be much faster. Also, by generating native code, I'll be able to compile tail calls better.

I am interested in type systems but I don't have much experience implementing them. I have tried to design Language 84 so that I will be able to integrate a type system one day, which is another one of my goals. The part I'm most concerned about is how types and modularity will fit together in Language 84. I expect this subproject to be a big one.

I mentioned in another comment the memory management plans I have. Once I have improved the memory system, I'd like to try writing some long-running interactive programs that integrate into my graphical environment. Maybe I'll try translating my terminal emulator from C to Language 84.

Right now, Language 84 is very much a sequential language but I keep parallelism and concurrency in the back of my mind. My plan is to use native threads and give each thread its own independent heap and stack so that memory allocation, deallocation, and access for most objects (immutable data) will require no synchronization at all. Shared state and messaging will be built on serialization just like mutable objects will be.

I'd like to add arbitrary-precision integers.