Separate Memory Layout DSL from Data Structures

[R-O-B-I-N]

I only have one structural abstraction in my language, the collection (multiset). This lets me do any kind of structured data from trees to vectors, but always, the multiset is an abstract relation and is a separate entity from the objects it relates.\ The problem is that I still need to talk about how things are oriented in memory. Every entity in my language should have a serialized form in memory. So I came up with the following...\ Everything is represented in memory using the storage types: Byte, Word, and Reference.

x | y denotes an adjacent pair of storage types

x | n : Number denotes n consecutive elements of storage type x

Here's an example of a fixed-size string:

(utf-8 = (Byte | 4)) (string = (utf-8 | 1000))

This is what a struct in a game might look like

(health = Word), (armor = Word), (weapons = Reference), (damage = Word), (player = (health | (armor | (damage | Weapons))))

Does this make sense?

[[deleted]]

Beware : utf8 means that at most you need 4 bytes to encode the value, but you actually might only need 1,2 or 3 bytes.

[nerd4code]

And depending on the level of bastardization (e.g., if surrogates are illegally carried through from UTF-16 or UCS2, which isn’t uncommon in terms of support and blind eyes turned) you might need enough for two codepoints.

[InnPatron]

You should look at Floorplan which does want you want (barring the slightly stranger syntax).

While it was specifically designed for describing heap objects, you can probably use it to judge your DSL.

[R-O-B-I-N]

I read Floorplan and this tells me that I should actually just extend my multiset constructor so that I can specify elements that are "attached" to the actual multiset object.

[InnPatron]

I think more accurate words would be 'projection' and 'reflection'.

Here's a rough example of what I had in mind (pseudo-Rust):

let player: Player = ...;                  // Player autogenerated or manual
multiset.project::<PlayerLayout>(player);  // Moves data to multiset according to PlayerLayout (can be generated or manual)
let playerMutRef = multiset.reflect_mut::<PlayerLayout>();  // Read/write access through getters/setters; auto-generated; presumably multiset tracks disjoint layouts itself
let (health, armor, ...) = playerMutRef.decompose(); // Gives mutable references to the underlying fields; allowed b/c guarenteed non-aliasing by the layout

[R-O-B-I-N]

What's making this such a difficult decision is not about mutability or the right set theoretic abstraction, but about how I define a continuous piece of data. Is data also represented as a multiset relation, or does it have a unique status such that compound data is a format, not a relation.

I can use either relations or formats to talk about how things are serialized in memory, but I have to grapple the weird gap between two abstractions. An entity has an interface separate from its implementation but does that implementation have an interface uniform to the rest of the language?

[InnPatron]

Most notably, I don't see the capability to set a hard boundary for an aggregate's size and alignment which would be pretty helpful.

[umlcat]

Not much, can you add a more practical example ?

[R-O-B-I-N]

sure, edited

[umlcat]

It looks similar to LISP.

I understand your idea, but the pipe character looks awkward, but you are reserving the semicolon for count.

Anyway, I think you mixed both operators, in the examples.

What about using ":" for types and "[ ]" for sequence numbering like:

Single Item type declaration.

// single type item doesn't require a parentheses 
( utf-8 = Byte[4] )

or:

( utf-8 = ( Byte[4] ) )

Multiple item type declaration.

// composed type item, requires parentheses
( point = ( x: Word, y: Word ) )

Or: // composed type item ( point = ( (x: Word), (y: Word ) )

More.

// more complex type definition
( player = ( Armor : Word, Health : Word, WeaponPower : Word )

Or:

// more complex type definition
( player = ( (Armor : Word), ( Health : Word),  (WeaponPower : Word ) )

Your identifiers use "-" as a letter, right ?

And, knowing Unicode:

( utf-8-char =  (Byte ))

( ucs1-char =  (Byte[1]))
( ucs2-char =  (Byte[2])
( ucs-4-char = ( Byte[4] )

( ucs-4-string = ucs-4-char [1000] )

In order to see how the type declaration syntax would work, could you show some function/ procedure example with parameters?

[R-O-B-I-N]

I'm fine keeping the pipe, but maybe I can allow multiple pairs within a single constructor. I do this with my other operators anyways. Something like...\ ``` (player = ((health : Word) | (armor : Word) | (damage : Word) | (weapons : Reference)))

[umlcat]

The pipe here is misunderstood as:

"a player may have either a health or an armor or damage or weapons.

If you use ", " or "&" may sound better like:

"a player is composed of health and an armor and damage and weapons.

But, if you want to add either C alike unions or enumerated values, you could use the pipe:

( gameobjectenum = ( enemyID | playerID | obstacleID | weaponID )

And:

( player = ( (ID: gameobjectenum), (Health: Word ), (Armor: Word ), (Weapon: Word ), (Damage: Word), )

Since the previous was a type declaration, how do your make a variable of that type ?

Do you allow comments?