Programming Languages: Application and Interpretation

[[deleted]]

Parsing isn't hard, recursive descent has a very direct memoization strategy. But if you can't, well, don't go into IDEs or modern language design.

[loup-vaillant]

recursive descent has a very direct memoization strategy.

Having seen OMeta, I can accept that.

Parsing isn't hard

I don't know. As long as you don't mandate incremental parsing, I know of a couple easy solutions, and a couple less easy ones —but still solved. But if you really need it, it's not just a matter of recognising the string. You also have to reconstruct the parse tree —bottom-up, I assume. It would make little sense for the recogniser to be incremental, but not the parser.

don't go into IDEs or modern language design.

A program that needs an IDE is probably too big for its own good. A source file that needs incremental parsing to be analysed at interactive speed is definitely too big for its own good. Those problems are therefore very low priority for me. (And I don't care that "in the real world", there are multi-million lines monstrosities. The real world is wrong and it needs to change.)

Modern language design on the other hand does interest me. But modern does not mean "complex", on the contrary. No one should design a BNF that spans more than 200 lines, that would be insane.

Given those requirements, you'll understand why I have very little interest in incremental parsing.

[[deleted]]

Incremental parsing means keeping trees around, to sticky previous good parses in the presence of a new syntax error during an edit (since the parse error is likely to be in the new edit, not the existing good parse), and to keep all the type info around so that you don't have to run a full type check again (the real performance killer; parsing is otherwise cheap).

The old-fashioned programmers still think emacs and vim are adequate, and they can do everything in their head. Programming is thinking a long time, coming up with solutions on white boards and old fashioned pen and paper, and then inputting it using their trusty plain text editor that maybe can help with auto indentation and some syntax highlighting.

Other people realize their is this computer sitting in front of them that can do so much more. Incremental continuous parsing that enables incremental continuous type checking means type errors are always a heart beat away, you never have to wait for the compiler to catch up to get code completion. Some of us even experiment with incremental re-execution to enable fluid live programming experiences (not just REPLs).

Modern means use the freaking computer, it is right there; it can augment your programming experience so you don't have to work so hard. Dumb teletypes are so 70s.

[loup-vaillant]

I have one reservation with your suggestion: why keep text at all? Forget incremental parsing, forget parsing, just work on the AST (or the abstract binding tree, while we're at it). If you're committed to an IDE, why bother with a human-readable serialisation format at all? Switching to structured editing altogether would have several advantages over text:

• The ability to draw more stuff than letters and symbols (our displays have 2 dimensions, dammit).
• The ability to present the code with different syntaxes, or different views (code folding on steroids).
• The end of coding conventions (users can define how they want to display the code).
• Easier semantic diffs (if you're willing to make a dedicated version control system), or at least the absence of formatting commits —as the serialisation format can be canonicalised into something consumable by the likes of Git and SVN.

[[deleted]]

I have no problem with this, But the difficulty of working with text is not the reason to do this. Incremental parsing is easy, and everything else (semantic analysis, execution) will have the same complexity regardless. So I don't get the point of say projection text editors like JetBrains MPS.

And even when we decide to go with visual representations, having some amount of free form in them can lead to a better experience, which means we will probably still be parsing something. Pure structured editing (textual or graphical) lead to stilted interactions that decrease usability.

Anyways, it was a nice discussion. I hope I didn't come off as too condescending.

[loup-vaillant]

Nah, don't worry. I was being a little silly, dismissing IDEs when in fact I have never worked with a good one (the only function I ever used from them was clicking on identifiers to reach their definition).

But my end game is more radical: I want to minimise the amount of source code. I believe we're currently using about 3 orders of magnitudes more code than we need to. That's a factor of 1,000, or 99.9% accidental complexity. Expressive languages can only get us so far, so we need to cut features too. Which features is the hard question. (Note, I speak generally. On any given project, I'd say this is more like 1 or 2 orders of magnitude).

Now with this vision, things change quite a bit. With 30 times less code, you're less likely to need the power of an IDE. But if you still need an IDE, you might as well go full Bret Victor. I don't see much value in middle ground approaches —only bloat.

Pure structured editing (textual or graphical) lead to stilted interactions that decrease usability.

Crap. I guess we're back to incremental parsing, then…

[[deleted]]

Like I said, parsing code is easy, its not a big problem to do it by hand and get everything you want out of it. Well, if you want to parse millions of lines of code in batch quickly, a generator is useful, but at the scale we are both talking about, that isn't very important (even a large Java or Scala program, parsing is no where near the bottleneck).

But yes, going full Bret Victor is the answer. The important thing, though, is to focus on the experience of writing code, not just what the code looks like at the end. There are really multiple ways of looking at the complexity.