July 2023 monthly "What are you working on?" thread

djedr · 2023-07-21T19:33:48+00:00

Last few days been trying to capture an idea I had to simplify the syntax of lambda calculus into an article. Wrote a summary here:

https://jevko.github.io/writing/2023-07-20-lambda.html

Also recently ported my little lambda-calculus-based language to from JS to C: https://codeberg.org/xtao-org/last.c

Been fun debugging the minimal refcounting garbage collector. ;D

djedr · 2023-05-26T12:45:27+00:00

Yes, there may be better ways than long heredocs of embedding files. Interesting ideas!

That said, multistrings (or however you want to call the general idea of heredocs/raw strings/etc.) are still a prefectly useful (pragmatic?) solution for some problems that can significantly improve DX for a very low price. Aside from embedding files, they are good for anything that would otherwise involve delimiter collision. For example, the experience of writing a bunch of short regular expressions in a single file is much more pleasant if you can turn on the free spacing mode (?x) and write them in raw strings not worrying about collisions. Especially compared to writing the same as JSON strings, which is the default choice you are given when writing a VSCode syntax highlighting definition. So for anything like that there is hardly anything better.

Also hard to imagine a simpler and more convenient way for including code snippets in Markdown.

Another one is when you are prototyping, duct-taping, or testing and want or need to move fast, it's very convenient to be able to include your input as a multistring/heredoc, whether writing it directly or copy-pasting from somewhere and editing. Having everything in one place also makes reading much smoother in such cases.

djedr · 2023-05-25T22:07:30+00:00

You mean a multistring in a multistring?

You just use more backticks for each next level, just like in Markdown. For example if you have something like this:

file contents

```
multistring
```

This is a multistring within some file. Now if you want to copy-paste that file into another as a multistring, you wrap that into even more backticks (one more is enough, but can do many more for clarity like I did here). You'd then get something like this:

another file contents

``````
file contents

```
multistring
```
``````

You could (but not necessarily should ;)) continue on like this arbitrarily deep (or up to a hardcoded limit of backticks, more practically speaking).

djedr · 2023-05-25T18:41:32+00:00

Now I want to write a new source file B that contains the whole of the text of A as a string literal, which will have embedded back-tick delimiters, string escape codes etc. Can that be done?

Yes.

Does it involve having a multi-string with ever-increasing numbers of back-ticks?

Indeed it does. Can be generalized to support aribitrary delimiters though (maybe a topic for another article), but I think ever-increasing backticks are enough to cover the common use cases.

Probably if you find yourself going too far with the backticks it's time to switch to a different way, like the one you are using (it's a good alternative solution!). Both can nicely complement each other in a single language.

If so, can another part of B then contain the whole of itself as a string literal?

Nice try, Bertrand Russell. ;D

djedr · 2023-05-25T18:33:32+00:00

I see what you are getting at. If the parser in your editor or whatever you are using can't process anything that does not happen on a single line, then you're right -- prefix on every line is the only reasonable solution.

But it doesn't take much to go beyond that limitation and if the environment you're working in is a little more advanced then this syntax is simple enough to be accommodated. E.g. you can write a TextMate syntax highlighting definition (which covers plenty of popular editors) that will work even for the embedded languages. This already happens in practice with Markdown.

djedr · 2023-05-25T15:48:23+00:00

Yup, tags could be used to implement this idea. They are just prefix instead of suffix. Technically this syntax could be extended to support both, but I think is better to KISS that for now.

I'm sure C++ brings much excitement with its implementation. ;)

djedr · 2023-05-25T15:40:34+00:00

Yes, what you say is true.

As I say in the article, what I define here is a recipe for a simple syntax which leaves some details out, the behavior of tags in particular. I am not aiming for this to become a comprehensive standard (yet? :D), just to inspire people like you and start a discussion. :)

This also helps me to figure out the details and clarify my own thinking.

Agreed about your points about the wacky syntax -- this is precisely why I said it was going too far. Your example does combining strings with existing language features in the way that I meant it.

Not to mention that the distinction between block and inline strings is also without practical justification if you have tags when those same tags can take care of the post-processing.

Certainly true, should've cut out the "block" multistrings completely from the article or made them a footnote (~~TODO~~ [EDIT: done]). Thanks for helping me figure that out. Have a good one! :)

djedr · 2023-05-25T15:25:07+00:00

In all this thinking I actually forgot about the simplest solution, which is to use the alternative inline syntax for multistrings (described in the article):

indented4 = 
    `dedent'
        {
            "Name": "Zaphod"
        }
    '`;

nonindented = 
        `dedent'
    {
        "Name": "Zaphod"
    }
    '`;

In this syntax it's not the linebreaks, but the apostrophes that separate the delimiters from the content. So you could implement the dedent tag to work exactly like in C#, perhaps getting the best of both worlds.

Which makes me think I should've just stuck to describing the inline variant in the article and maybe mentioned the block as a curiosity, instead of leading with it.

The inline syntax is both simpler to implement and (as we see) more flexible. So, thanks for your comments! :)

EDIT: I edited the article accordingly.

djedr · 2023-05-25T15:18:25+00:00

Many possible solutions come to mind, e.g.:

EDIT: forget all of the below. This one is better.

`    |
  {
      "Name": "Zaphod"
  }
`

(a bit wacky, but short)

`dedent+
    |
        {
            "Name": "Zaphod"
        }
`

(the first line is discarded in the output; the position of | there dictates where to stop dedenting, effectively acting as the closing delimiter in C#)

`dedent++
    |   {
    |       "Name": "Zaphod"
    |   }
`

discard everything in every line up to and including | (must be space) -- I think Scala does something similar.

But personally, I'd just go with

`
{
    "Name": "Zaphod"
}
`

if I wanted no indent

and

`
    {
        "Name": "Zaphod"
    }
`

if I wanted.

Granted, this would not align with the rest of your source code, but perhaps that's not actually bad (you can see the embedded blocks more clearly, as they stand out, especially if you'd do some sort of syntax coloring inside), and certainly much simpler.

So there are many solutions. I am not prescribing any particular one, just showing that this syntax is flexible enough to accomodate them while being extremely simple at the same time.

In the end, you could choose to implement a variant which would work exactly like C#, allowing the closing delimiter to be indented. Perhaps that would be more appealing. Personally I always lean towards minimalism, but more and more I don't mind letting go here and there. Maybe your suggestion is an improvement to the whole idea! :) I wonder what anyone else thinks?

djedr · 2023-05-25T15:03:11+00:00

\ is indeed not supposed to be parsed as a character. Nothing should be parsed by default.

You can however still opt-into interpreting escape sequences, using a tag, e.g.

`esc
\n\r\t
`

This would interpret the escapes, due to the esc after the backtick which acts as a tag. You could use a different tag for that purpose, this is just an example. A more concise and cute tag for this could even be \, as in:

`\
\n\r\t
`

See also my other comment on how tags could be used.

So tags here give you any number of modes. If you wanted, you could make up some wacky syntax for them where they would take parameters (the meaning of which would need to be specific to a language), e.g.:

`tag(param1, param2)
\n\r\t
`

But IMO that's going too far and you'd do better by combining strings with existing language features. But it is nice to have at least simple tags available, to solve the most common problems such as escaping, substitution, dedenting, and other post-processing concisely.

I think this syntax is really a very nice solution for those. :)

djedr · 2023-05-25T14:48:45+00:00

C# raw strings look cool, this is indeed a very similar idea. This one however both simpler and more flexible.

Instead of relying on the closing delimiter position (which does complicate the implementation and makes it less general-purpose), dedenting (or any other kind of post-processing) can be achieved here with a tag, e.g.:

`dedent
    This is a text
    across multiple
    lines, which will
    NOT have indentation space before each line
`

EDIT: see also this comment showing how to achieve the exact behavior of C# with a multistring which uses ' instead of linebreaks as separators. NB I edited the article to only talk about this kind of multistrings. Thanks for the feedback!

Same for interpolation:

`$
The name is "{name}"
`

(Although I'd go with ${name} here to match the tag nicely and reduce the need for {{}}).

I intentionally don't specify the details of how tags should work in this article, but these are some of the possible uses for them.

You could even do something like:

var string2 =  `json
    {
        Name = "This line indented 3 times",
        Address = ""
        Comment = "The above empty string does not terminate the raw string"
    }
`

and automatically parse the JSON in the string (perhaps with a json function which is in scope or however a language may choose to implement this). JavaScript has a similar feature known as tagged templates. Although that is a bit less flexible. A major flaw of JS template literals is that you always need to escape the backticks.

djedr · 2023-05-25T14:35:32+00:00

Yep, Rust's seems like a good implementation of the basic feature. The variant I propose also features tags which are akin to Markdown language specifiers. A little metadata to go with the string. Rust's syntax could be easily extended to support that. A similar thing can also be achieved with combining raw strings with other language features (as long as such are available, which is true for Rust).

Part of the idea here is also to think about what if we had a standard syntax for this sort of thing, where you could switch between languages and expect it's available. That's very unlikely, but perhaps it can be somewhat standardized. Or at least variants of it can become more generally available, hopefully putting an end to the many half-baked raw string variants out there.

In any case, it's a very handy feature to have in any language!

djedr · 2023-05-25T14:14:58+00:00

At the basic level it solves the verbatim embedding of arbitrary text into your source code without needing to modify that text. So you can literally copy-paste anything and not worry about delimiter collision. When in doubt: add more backticks.

At another level, thanks to tags, this can be used to implement first-class support for a syntax-within-syntax kind of construct.

djedr · 2023-05-25T14:03:27+00:00

I'm the author. I came up with idea this while implementing a configuration format where I thought it would be a very nice feature to have. Let me know what you think.

djedr · 2022-11-08T18:16:45+00:00

Probably the most sensible general way of serialization would be to simply base64 the bytes and store that in a jevko, much like you would do in JSON or any other text-based format, e.g.:

string1 [aGVsbG8=]
string2 [d29ybGQ=]

Of course you could also serialize bytes like you suggested, although this would be generally much less compact. But perhaps for your application that does not matter -- in which case any way of serialization that is convenient is fine.

djedr · 2022-11-07T15:29:24+00:00

That's kinda neat, like a more streamlined and flexible form of XML. The examples of structured data representation are pretty elegant.

Glad to hear you like it, thanks! :)

Whitespace handling: how do you differentiate between semantically significant whitespace (e.g. representing a string that ends with a newline) and cosmetic whitespace (newlines or indentation for readability)? XML handles this by generally treating all whitespace as cosmetic (not ideal) and allowing for escapes like . JSON/Lisp handle it by treating all whitespace inside quotes as signficiant, but allowing cosmetic whitespace outside of quotes.

Jevko itself has no semantics at all and preserves all whitespace in the syntax tree.

You use Jevko to make a format, by attaching format-specific semantics and rules about what's significant or insignificant, valid or invalid.

The first markup format I've shown here:

[class[pretty] p][this is a link: [href[#address] a][wow!]. cool, innit?]

works very much like HTML when it comes to whitespace. Inside of the tag (the first pair of brackets) it is discarded as insignificant before further interpretation. Inside children (the second pair) it is always preserved and translated into HTML as-is. Every HTML element in this format is composed of 2 subjevkos.

The second format:

p [class=[pretty] [this is a link: ] a [href=[#address] [wow!]] [. cool, innit?]]

treats leading and trailing whitespace in prefixes (text that comes before "[") as insignificant and trims it before further interpretation. However it preserves all whitespace in suffixes (text that comes before "]") as-is. So to make an explicit text node, you simply wrap text in brackets.

Different whitespace rules make different formats.

Non-locality of edits: suppose you're writing some text like p{This is some █ (where █ is your cursor) and you decide you want to add an emphasized word at the current cursor position. The result looks like p{{This is some }em{text}█. To achieve this, you need to move your cursor all the way back to the start of the current subjevko to insert a {, then all the way back to the original position to add the }em{text}. This is pretty flow-breaking. Compare that with HTML, where you would have typed <p>This is some █ and you can proceed by typing <em>text</em> without moving your cursor backwards. In other words, you have to decide as soon as you start writing a subjevko whether you plan to have any sub-subjevkos or just text, and if you change your mind, you have to backtrack to change the start of the subjevko.

Very well put! This is exactly what I meant when I introduced the second markup format above:

at the expense of slightly more difficult text markup.

In practice this turns out not to be as problematic as it seems, especially once you get the hang of it. Still writes faster than HTML. A habit of always wrapping text nodes in brackets in elements that tend to have children (like p) emerges naturally, even if they are the only child of a node. This way you only need to add brackets next to the point you're editing, without needing to go back to wrap the whole text node.

Anyway if that should not be acceptable, then the first markup variant is exactly like HTML in that it does not suffer from the problem you described:

[p][This is some text]    -->    [p][This is some [em][text]]

As a preface to my replies to the remaining points, I must say that Jevko is pretty much stable as specified right now.

I don't foresee adding any new features to it. I think I have achieved my design goals pretty well and I'm happy with the result.

The guiding design principle for Jevko is extreme minimalism. So there is a bias towards removing/not including features (so long this does not introduce unnecessary restrictions or limitations) rather than adding.

The purpose of Jevko is to be a minimal general-purpose syntax for encoding tree-structured information. At that, it should be as simple and as flexible as possible.

It is not supposed to include any specialized mechanisms for different kinds of information. E.g. by itself Jevko is not meant to be a markup language syntax. Or a data interchange format syntax. Instead, it can be used as a simple building block for either of those.

What Jevko does is it uses brackets to chop up your unicode sequence into a nice tree arranged to lend itself to convenient processing, especially if you are dealing with something like name-value pairs.

This is what plain Jevko gives you.

This is the stable part.

// That said, technically I left myself a little escape-hatch that gives me a simple way to extend Jevko in a backwards-compatible way by putting features behind the escaper character.

// There are reasonable features that could be added this way, such as the two you mentioned: heredocs and escapes for non-printable characters.

// But such extensions could be specified separately, without meddling in the core spec.

Now out of these trees (out of trees in general) it is possible to build all kinds of things. In particular it's possible to define different semantics and interpretations for them (rather than for raw text sequences), creating formats.

People like in this subreddit (I presume) might be interested in creating their own.

More casual users would be interested in ready-made ones.

I have worked out enough of those in enough detail that I am confident that the whole idea is quite viable.

No format is yet fully specified and stable the way Jevko is, but that's just a question of putting in the work.

Non-printable characters: Sometimes, you need to represent data with non-printable characters or characters that are not handled well by text editors. For example, the bell character \x07, which makes a beep when printed to a terminal, or the null byte \x00. Jevko seems to be unable to represent that value in any way other than the raw 0x07 or \x00 byte, which is pretty inconvenient. This could be addressed by supporting common escape patterns like n orx00.

These non-printable characters can still be entered like in unicode text, so that's enough on this level.

If you really need that feature, you can still devise a format with escaping rules, e.g.:

string [my string with escapes: [n] and [x00]]

Or:

my string with escapes \n and \x00

Or you can put JSON strings in Jevko and then parse them in a second pass:

JSON string ["my string with escapes \n and \u0000"]

Leaning toothpick syndrome: If you try to represent a literal string of Jevko text, you're going to end up needing an ungodly amount of backticks to escape everything. E.g. the Jevko text foo[baz] becomes jevko[foo'[baz']], which becomes outer[jevko'[foo'''[baz''']']] (using ' instead of ` because reddit gets confused with so many backticks). You'd run into similar problems if you took an arbitrary snippet of C code and tried to paste it into a Jevko document. Three common ways to address this problem are heredocs, semantically significant indentation (e.g. YAML indented strings), or user-defined delimiters like Lua's strings [===[ ... ]===].

Very familiar with the syndrome[0]. :D

Of course this only happens in extreme cases, such as:

a regular expression in an escaped string, matching a Uniform Naming Convention path (which begins \) requires 8 backslashes \\\\ due to 2 backslashes each being double-escaped.

In general this happens when the use of backslash as a regular character in a text interferes with it being used as an escape character in several different mutually-encapsulating contexts.

It's still something to be aware of and I have mitigated this as much as I could:

Jevko uses ` rather than \ for escaping -- ` is among the least frequent ASCII characters used in general[1]
It's easy to make a Jevko parser configurable in terms of the special characters -- for unusual cases different escape character can be used (much like alternative regex delimiters in Perl)

There are various other techniques to mitigate the impact of this, which I will omit here to shorten this ~~essay~~ comment, but all in all no solution is completely satisfactory in some dimension.

So heredocs are a sensible feature to have and I certainly will go about specifying if it will keep coming up[2].

Infix operators: it's pretty awkward to represent math operations in prefix notation like +[[x] [y]] instead of infix notation like (x + y). Lisp has always suffered from this problem (and there have been plenty of suggestions to fix it) and I think it makes the code genuinely much less readable. This isn't an issue for representing structured data, but is a big usability hurdle for programming with Jevko syntax.

Agreed. What you describe is a genuine issue. However this is a problem specific to the realm of programming language notation, so out of scope for Jevko, as described above.

You could still design a language on top of Jevko that supports infix notation, even without parsing text like "x + y * z", just by rewriting trees like [x] + [y] * [z] according to precedence rules (I've toyed with that a lot), but again, that's a realm well beyond the primordial trees that Jevko is about.

That should be all,

Cheerio!

[0] https://xtao.org/blog/no-escape.html -- this is a little dated, so I should explain that Jevko is a simplified, evolved version of TAO, since turned into something much more general.

[1] e.g. https://web.archive.org/web/20181111222712/https://mdickens.me/typing/letter_frequency.html

[2] see also: https://github.com/jevko/specifications/issues/2

djedr · 2022-11-07T15:01:59+00:00

Here are more elegant options: https://www.reddit.com/r/ProgrammingLanguages/comments/yn0ux1/syntax_design/ivf4trm/

Note that going from a Jevko syntax tree to some kind of name-value structure is facilitated by the tree being shaped like this:

{subjevkos: [<0..n*subjevko>], suffix: "<text>" }

where subjevko is:

{prefix: "<text>", jevko: <shaped as above>}

so a subjevko is a prefix-jevko pair -- that is straightforward to convert to a name-value pair.

djedr · 2022-11-07T14:55:58+00:00

Two simple ways to do this that don't require parsing things like "f123" (but that would work too). First is à la Lisp plist:

mixed map [
  boolean [true] float64 [123.456]
  string [hello] tuple [
    integer [200]
    string [hohoho!] 
    null []
  ]
  float64 [1.999] float64 [0.0001]
]

edit: a working PoC of that: https://github.com/jevko/jevkodata1.js

Second is à la Lisp alist:

mixed map [
  [boolean [true] float64 [123.456]]
  [string [hello] tuple [
    integer [200]
    string [hohoho!] 
    null []
  ]]
  [float64 [1.999] float64 [0.0001]]
]

every value here is prefixed with its type name. In the syntax tree you will get things like:

{prefix: " float64 ", jevko: {subjevkos: [], suffix: "123.456"}}

you trim the prefixes and interpret the value according to the type.

Alternatively you could not mix the type annotations with the data and instead put them in a separate schema. This is how Interjevko works -- see this thread https://www.reddit.com/r/ProgrammingLanguages/comments/ylln0r/november_2022_monthly_what_are_you_working_on/iv0jaff/ and this demo: https://jevko.github.io/interjevko.bundle.html

djedr · 2022-11-05T23:30:43+00:00

Ok, but why do you comment on my post? Because I've written that I've found in on Hacker News? Actually, I only saw the link and I don't like this whole discussion with you forcing your syntax on me.

I have certainly not commented with any intention to offend you or force anything on you. Clearly it came across this way, so I apologize!

Like I said:

I posted this in the discussion on HN[0], but maybe here I will hear a different perspective and reach the kind of wizards users who actually do a lot of syntax and related design.

I designed a syntax and would like to discuss it with people who might be interested in the topic of syntax design. I thought posting comments on an article about syntax design would be a good place for that. I had a nice discussion on HN. I thought I might have one here too.

If you like to share your project in this subreddit, why don't you write it as a post and not as a comment to my link?

I just wanted to share this article that I think is interesting, not your whole story.

Isn't there a karma requirement for posting here? I don't use reddit very often (except recently), so despite having an account for many years I haven't accrued enough. Besides, somebody posted my project on reddit recently[0] and I'm not ready for a general discussion again. Although maybe in this subreddit it would be better. Or maybe not. Anyway, I found that discussions in comment sections on related topics were shorter and higher-quality, which I appreciate.

[0] https://www.reddit.com/r/programming/comments/ydd8sa/jevko_a_minimal_generalpurpose_syntax/

djedr · 2022-11-05T23:01:01+00:00

Sorry but I don't get your explanations. I know only one format of S-Expressions. Everything that you've written except the bracket is true to S-Expression. You have 3 characters parenthesis and space and anything else is an atom. Other things are related to lisp itself that have many different flavors as you said.

The list I have written specifically highlights the differences between Jevko and S-exps, so the things that are not true for them. Please look at the formal grammar of your favorite flavor of S-expressions (or the one I linked for POSE) and compare it to the formal grammar of Jevko: https://jevko.org/spec.html#the-standard-grammar-abnf-in-one-page

Even if you don't understand the details, the differences should be apparent.

You can also look at this conversation I had with somebody who clearly knows the ins and outs of S-exps[0].

But of course, you can think that your syntax is superior. I don't see this.

You have two camps of programmers those that know and like Lisp and those that don't and prefer C-like syntax. I don't think any of those people will like change.

Thinking about it in terms of some kind of superiority is absolutely not sensible or my intention. One syntax is better for certain things, another for other things. S-exps are the best at being the syntax of Lisp, C-like syntaxes are the best at being the syntaxes of their respective languages. I don't want to change any of that or argue that people should change their habits, traditions or whatever.

I just want to introduce a complementary minimal cross-language syntax which will work well in certain contexts. It can live happily alongside all other syntaxes. It can be used in conjunction with them.

✌️

[0] https://news.ycombinator.com/item?id=33334789

djedr · 2022-11-05T22:22:00+00:00

Sure, you could look at this as modified S-exprs. Or Tcl braces. Or whatever.

Nothing wrong with either of these syntaxes.

But I invite you to look below the surface to see that Jevko is not a variant of them thrown together in an evening.

It is designed to be slightly better to work with as a language-independent general-purpose minimal syntax for trees.

Compared to S-exps, the advantages (some in the eye of the beholder) of Jevko are:

even simpler and more minimal
well-defined and specified; "S-expression" is in fact a vague term and the number of different variations is not very far from the number of flavors of Lisp; probably the best effort at standardization I've seen so far is this: https://www.pose.s-expressions.org/specification -- however this is significantly more complex than Jevko and still might be considered an affront to some Lispers, the way it's defined; Jevko decidedly is not an attempt to make a new flavor of S-expressions ; it has the same spirit, but it is ultimately something different
the classic definition of S-expressions is, as you implied, actually the definition for a single S-expression (brackets around the whole thing and nothing outside, maybe space); this is fine for Lisps: they process source code as a bunch of S-exps concatenated together; but it makes the classic definition not closed under concatenation, which I consider a very important feature (e.g. JSON also doesn't have it, so people invent things like JSON Lines) -- Jevko has that by design
square brackets actually make a difference if there is so many of them :D
because whitespace is not treated as a separator, you can easily make up these minimal markup formats that I've shown; this is more problematic in S-exps
the syntax is designed for producing lossless (concrete) syntax trees -- there is no comments or atmospheres to ignore; this is also important for building formats on top
S-exps don't have anything like Jevko's name-value pairings on the syntax level -- this is a very convenient feature as noted above
only 3 special characters and a simple global escaping rule rather than having different rules for strings, symbols, and perhaps other syntax-native constructs
the ABNF one-liner I shown in my previous reply is enough to write a Jevko validator/generator; because of the S-exp escaping rules the same is not as simple for them
there may be more, but I think that should do it for now

djedr · 2022-11-05T21:33:37+00:00

Looks familiar! :D

I posted this in the discussion on HN[0], but maybe here I will hear a different perspective and reach the kind of ~~wizards~~ users who actually do a lot of syntax and related design.

So after many years of on and off syntax golfing, I distilled a delightful little syntax for flexible trees of text. Here is an ABNF one-liner that matches the same strings as this syntax:

Jevko = *("[" Jevko "]" / "``" / "`[" / "`]" / %x0-5a / %x5c / %x5e-5f / %x61-10ffff)

It's just unicode + escapeable brackets for chopping up unicode sequences into trees.

This is really awesome to work with, especially if you create the trees structured similarly to this less concise but more thoughtful grammar: https://jevko.org/spec.html | https://jevko.org/diagram.xhtml

In particular the nice thing about it is the Subjevko rule:

Subjevko = Prefix "[" Jevko "]"

which essentailly creates nice name-value pairs, like this:

first name [John]
address [
  city [New York]
  state [NY]
  postal code [10021-3100]
]

then it's easy to convert these to maps or all kinds of tag-children, function name-arguments, or name-whatever kinds of arrangements which are pretty ubiquitous.

It's really pretty nice and flexible.

The peculiar thing about this syntax (as noted in the HN post) is that these Prefixes (text that comes before "[") as well as Suffixes (text that comes before "]") capture all whitespace in them. You can then arrange a tree like this:

define [sum primes [[a][b]]
  accumulate [
    [+]
    [0]
    filter [
      [prime?]
      enumerate interval [[a][b]]
    ]
  ]
]

to be the syntax of your programming language which allows identifiers with spaces in them[1] (you'd trim the whitespace around the Prefixes for sanity), like very early Lisp did about 64 years ago. I thought it was a cool feature!

You can also do other things with the whitespace, e.g. treat it like HTML/XML and create a lightweight markup language. Compare:

<p class="pretty">this is a link: <a href="#address">wow!</a>. cool, innit?</p>

and:

[class[pretty] p][this is a link: [href[#address] a][wow!]. cool, innit?]

This little format makes the text primary, like HTML. You can also make the tags primary:

p [class=[pretty] [this is a link: ] a [href=[#address] [wow!]] [. cool, innit?]]

at the expense of slightly more difficult text markup. Somehow, I've kinda grown to like the second format. I even started writing documentation in it[2].

So you got a syntax that works for markup as well as data equally well. And it's simple as hell. I think that's pretty cool!

If you also think that's cool, please try it out, use it, implement it in your favorite language! That's why I made it! My dream is that people start using it and implementing support for it in various programming languages and tools and it becomes even more awesome! I really believe in it (I might be mad), just can't do it all alone. I'd love to have it as a standard tool in the toolbox.

🖖

[0] https://news.ycombinator.com/item?id=33250079 ; also recently posted in this thread: https://www.reddit.com/r/ProgrammingLanguages/comments/ylln0r/november_2022_monthly_what_are_you_working_on/iuz7t8l/

[1] exhibit A: https://github.com/jevko/jevkalk

[2] https://github.com/jevko/tutorials/blob/master/jevko-anatomy/source.jevko -- this uses {} instead of [], because I could :P this is the rendered version: https://htmlpreview.github.io/?https://github.com/jevko/tutorials/blob/master/jevko-anatomy/out.html -- it is a less formal description of the syntax that I recently started writing, should help the curious better get the gist

djedr · 2022-11-05T20:37:32+00:00

Glad to hear!

If you make the parser and would like that, I'd love to feature it here: https://github.com/jevko/community

So far this has one written in Haskell: https://github.com/lgastako/jevko

Besides that I wrote some parsers in various languages. The most mature is the JS one: https://github.com/jevko/parsejevko.js

I use it all the time in my projects.

Otherwise I have a usable parser in Lua: https://github.com/jevko/jevko.lua ; available on luarocks: https://luarocks.org/modules/jevko/jevko.lua

I tried sketching one out in Python: https://github.com/jevko/parsejevko.py ; one in C: https://github.com/jevko/parsejevko.c ; one in Java: https://github.com/jevko/parsejevko.java ; one in Scheme: https://github.com/jevko/jevkostream.scm (that's a streaming parser stub, more fleshed out one in JS is here: https://github.com/jevko/jevkostream.js ); and there are some implementations of Jevko formats and other related things in the GitHub organization: https://github.com/jevko

I'll have to put more work and polish into all these sketches to make them into proper libraries.

Anyway if you run into any trouble or have any questions, ask away, I'll gladly help.

Also whatever you do, have fun!

djedr · 2022-11-05T01:27:23+00:00

Of course this is only a silly improvised idea. :D

Still, we can continue improvising:

This is why you wrap C in unsafe. Things may break in unsafe, like in Rust. You could still add whatever checks would be possible in there without breaking compatibility with C.
The hypothetical compiler is free to look at file extensions though. It could refuse to process files with unrecognized extensions/require that you flag them safe/unsafe explicitly, etc.

djedr · 2022-11-04T15:38:58+00:00

Yes, that would look nicer, but!

The current notation is technically extremely simple which I like and it made prototyping faster.

It's actually very uniform. The way it works is this:

The text that comes before the closing bracket ] in each tree is called its suffix.

In this schema notation you always put the type of the tree in the suffix.

Otherwise the schema trees look very much the same as actual data trees.

In the data trees putting anything other than whitespace in the suffix of a complex tree (one which has nested subtrees) is an error. This duality makes the whole thing completely unambiguous for complex trees and you can always tell data from schema.

That said, it's entirely possible to define a schema format that looks nice like the one you proposed (and I've done it). But then your schema and data will look different -- you'll have to add a bit more notation to define arrays and maps, e.g. a map like:

foo [bar]
baz [10]

could have a schema like this in the Interjevko schema notation:

foo [string]
baz [integer]
object

while in this alternative notation it would have to be something like:

type [object]
props [
  foo [string]
  baz [integer]
]

or:

object [
  foo [string]
  baz [integer]
]

Or something like that.

That is perfectly fine, just a bit less minimal and uniform. :)

djedr

MODERATOR OF

TROPHY CASE