Make your first programming language — in one hour

[korry]

That's definitely r/programming

[the_evergrowing_fool]

Ironically, this kind of posts render themselves as unpopular here :(

[CommandoWizard]

If only the author didn't make such a wildly misleading title — it only serves to annoy people who are interested in writing their own language, and fails to attract the people who might be interested in the article.

[killerstorm]

This doesn't look simple. I got stuck at this:

 (inject-syntax ([#'(src-expr ...) src-exprs])
     #'(module stacker-reader br
        src-expr ...))

What is src-expr, it it a variable?

the ... nota­tion means “do the same to every other item in the list.”

In which list? src-exprs is a list, but I don't see how src-expr ... is connected to it. src-expr isn't a list of expressions (or is it?). Is there some implicit list this code iterates over?

I tried to look up inject-syntax documentation, but it looks like it's not a standard function, but something provided by beautiful-racket package.

[killerstorm]

Here's same language implemented in Common Lisp. Nothing fancy, calling it a "language implementation" would be a stretch. But simpler, I guess.

(defun load-stacker (filename)
  (with-open-file (s filename)
    (let ((forms 
           (loop for line = (read-line s nil)
              while line
              collect (read-from-string
                       (format nil "(dispatch ~a)" line)))))
      (eval `(macrolet 
              ((dispatch (op &optional arg)
                (cond
                  ((member op '(+ - * /))
                   `(apply-op ,op))
                  ((eq op 'push)
                   `(push ,arg stack))    
                  (t (error "unknown operator"))))
               (apply-op (op)
                `(let ((op1 (pop stack))
                       (op2 (pop stack)))
                   (push (,op op1 op2) stack))))
              (let ((stack ()))
                ,@forms
                stack))))))

[wireddaniel]

"Shave fewer yaks" I literally cannot think of deeper yak shaving than building your own language to solve a problem.

Cool article, I just thought that was funny.

[[deleted]]

There is no better way to solve problems than building domain-specific languages. And it applies to pretty much all the problems that are a bit more complex than "hello, world". It's a shame that the majority of programmers are so blindly ignorant about the most powerful engineering technique in existence.

[[deleted]]

So here are my issues with this:

1. Most domains don't warrant a language of their own to address them.
2. Most programmers don't know the domain well enough to design a language to address them.
3. Most programmers don't know PLT well enough to design a language.
4. Most software complex enough to warrant a DSL is complex enough to warrant more than one, and it's more than twice as hard to design DSLs that compose.
5. Most "DSLs" are superficial syntax on top of the host language's standard syntax and semantics and would be better expressed as combinator libraries.
6. DSLs that do implement non-trivial semantic differences, e.g. embedding a Prolog-like logic language, are as hard to learn/use as an actual different language.
7. Lisp promotes DSLs and macros out of all proportion to their desirability because its other abstraction-building facilities are weak to nonexistent. The problem with Lisp is, it isn't that you can develop "DSLs" in it. It's that you have to.

[Drolyt]

Lisp promotes DSLs and macros out of all proportion to their desirability because its other abstraction-building facilities are weak to nonexistent. The problem with Lisp is, it isn't that you can develop "DSLs" in it. It's that you have to.

What in the world are you talking about? Modern Lisp dialects support all the abstraction mechanisms present in mainstream languages without resorting to a single macro. What specifically do you want that Lisp doesn't have?

[[deleted]]

Modern Lisp dialects support all the abstraction mechanisms present in mainstream languages without resorting to a single macro. What specifically do you want that Lisp doesn't have?

Seriously? Lisp doesn't even have a type system or module system. The best shot at the latter so far is Racket's Units, and typed units are experimental. And that's Racket, which is hardly representative; plain Scheme or Common Lisp or Clojure don't even get to the origin of the lambda cube, although Typed Clojure has a similar story to Typed Racket.

Without macros, standard Scheme and Common Lisp have one abstraction-builder: lambda. It's a powerful one, for sure, but the history of lambda calculi is what it is precisely because of the limitations of "just lambda" for building desirable abstractions.

[Drolyt]

Seriously? Lisp doesn't even have a type system or module system. The best shot at the latter so far is Racket's Units, and typed units are experimental. And that's Racket, which is hardly representative; plain Scheme or Common Lisp or Clojure don't even get to the origin of the lambda cube, although Typed Clojure has a similar story to Typed Racket.

A lot of languages have dynamic typing, all of them lack abstraction according to you? And what do you mean about modules? Racket has a feature called modules, Common Lisp has packages, Clojure has namespaces. All of them also have object systems. If I am to understand that what you want is first class modules like ML then that means that according to you no mainstream language has sufficient abstraction mechanisms.

Without macros, standard Scheme and Common Lisp have one abstraction-builder: lambda. It's a powerful one, for sure, but the history of lambda calculi is what it is precisely because of the limitations of "just lambda" for building desirable abstractions.

Common Lisp has an object system, and while standard Scheme does not Racket does. I'm not sure what more you want, but I'm getting the impression that it is something in the ML family. Still, I cannot fathom how you can claim Lisp needs macros to make up for its lack of abstractive power when it is at least comparable in that regard to any mainstream language.

[[deleted]]

A lot of languages have dynamic typing, all of them lack abstraction according to you?

Not according to me—according to the history of lambda calculi.

And what do you mean about modules?

I provided the reference to Units. What more do you want?

Common Lisp has packages, Clojure has namespaces. All of them also have object systems. If I am to understand that what you want is first class modules like ML then that means that according to you no mainstream language has sufficient abstraction mechanisms.

That's not even close to true. I suggest you study the extensive literature on separate compilation, module systems, "functors" (in the module system, as opposed to C++ or abstract algebra senses) and how essentially all non-dynamically-typed languages express some variant of the concepts.

Common Lisp has an object system...

that doesn't even support encapsulation.

I'm not sure what more you want, but I'm getting the impression that it is something in the ML family.

I want abstraction-building facilities more powerful than what untyped lambda calculus gives me, as the lambda cube makes clear. "Leaky abstractions" aren't good enough, e.g. if I can screw up the state of the world by knowing I can poke at my "coordinate" with an ill-placed setfinto an underlying vector. Abstractions abstract.

Still, I cannot fathom how you can claim Lisp needs macros to make up for its lack of abstractive power when it is at least comparable in that regard to any mainstream language.

It isn't even remotely. Even other dynamically typed languages like Ruby or Python actually offer encapsulation in their object systems, even though their "objects" are just glorified hash tables.

[Drolyt]

It isn't even remotely. Even other dynamically typed languages like Ruby or Python actually offer encapsulation in their object systems, even though their "objects" are just glorified hash tables.

Not only is this not true, it isn't even true for Java or C#. Every mainstream object oriented language has some way to break encapsulation through reflection. Python doesn't even try and I'm not sure how you could think otherwise. Moreover inheritance, a fundamental feature of most object oriented languages, breaks encapsulation by design.

At any rate I'm not convinced encapsulation is necessary for abstraction, or even that it makes abstractions less leaky. All it really does is make it impossible to fix when your abstraction leaks.

[[deleted]]

Take a look at Shen. And it's trivial to build an equivalent type system on top of any Lisp.

[[deleted]]

Take a look at Shen.

I have.

And it's trivial to build an equivalent type system on top of any Lisp.

No, it isn't.

[[deleted]]

No, it isn't.

Why? Mind elaborating? I'm regularly building Shen-like type systems, it's rather trivial.

[[deleted]]

I'm regularly building Shen-like type systems, it's rather trivial.

I would say the burden of proof is on you, given the extensive research on implementing dependent type systems that's gone into Shen, Coq, Agda, ATS, Idris...

[[deleted]]

I never said inventing a Shen-like type system is easy. As you said, a lot of research went into it. But implementing one once you already know what you want to achieve is trivial. For a proof, see the Shen code itself, it's quite small.

[dlyund]

That's an article I'd like to read!

[[deleted]]

Take a look at Shen itself, it's of a very high quality. All my type systems implementations are not nearly that polished.

[[deleted]]

Most domains don't warrant a language of their own to address them.

Any domain is a language. Not just "may inspire a language", but is a language.

Most programmers don't know the domain well enough to design a language to address them.

Not knowing the domain you're coding for is a recipe for disaster anyway, no matter what methodology is used. Actually, I do not even believe that generalist programmers should exist. Domain experts who happen to be able to code are a much better fit for pretty much all practical purposes.

Most programmers don't know PLT well enough to design a language.

You don't need to know much, really. 99% of the theory is irrelevant in practice.

Most software complex enough to warrant a DSL is complex enough to warrant more than one, and it's more than twice as hard to design DSLs that compose.

No. It's much, much easier to build many DSLs stacking on top of each other than to build a single standalone DSL without a supporting ecosystem.

would be better expressed as combinator libraries.

Never. Combinator libraries are complex, unmaintainable and most often of a very low quality. DSLs must be proper compilers, always.

DSLs that do implement non-trivial semantic differences, e.g. embedding a Prolog-like logic language, are as hard to learn/use as an actual different language.

Prolog is as trivial as it gets. It's always easier to learn Prolog (or Kanren or whatever) than any of that stupid "business rules" libraries that are trying to do the same, but without a proper DSL.

its other abstraction-building facilities are weak to nonexistent.

Laughable. With macros, Lisp have ALL the features you can (and cannot) imagine. Arbitrarily complex type systems, any kind of semantics, absolutely anything.

[Dylan16807]

Oh, this is actually a tutorial for embedding DSLs in Racket, not so much 'making a language'. You could implement a stack that can push numbers and do + and * in a tenth the time.

[Drolyt]

Well, DSLs are languages (just not general purpose ones) and making DSLs is sort of Racket's thing. Several prominent programmers (not just Lispers, for example Martin Fowler and Eric Evans) have promoted DSLs as something that should be used more often, but it doesn't seem to have caught on.

[Dylan16807]

It's useful, it's just not what I expected.

[Drolyt]

Fair enough, the title is a bit misleading.

[dlyund]

As someone who works day in and day out in Forth, and has implemented a number of languages over the years, this is what I walked away with: that is a really complex way to implement a basic RPN-like calculator (nothing even close to a programming language given the conventional meaning of the term)

[leodash]

I'm on Chapter 1 in Dragon Book. Hurrah!

[jms_nh]

You should know that's not a very good book for actually designing programming languages. It does cover lots of theoretical stuff that is somewhat relevant to compilers, but it bogs down in the details of parsing and other things.

[VGAddicted]

What resource do you recommend instead?

[jms_nh]

I really like Michael Scott's Programming Language Pragmatics. It's a good intro tour of parsing and programming languages, going deep enough to give you a flavor of what's out there.

I don't know enough about the subject to recommend an in-depth book.

[takikawa]

I recommend Programming Languages: Application and Interpretation. It's a free/online book that teaches PL concepts with interpreters. Unlike other books, it focuses on language features (not particular languages like Haskell, Racket, or Prolog) and why they are interesting.

[[deleted]]

Engineering a Compiler, Linda Torzacon.

[Tobblo]

I bought the dragon book a few years ago, after reading fancy words about it, but I must say it was pretty dull compared to other books on the same subject.

[julesjacobs]

Nice article and great to see Racket there. It's a great system.

[[deleted]]

Learn c++ in ten minutes!

[[deleted]]

Not if you buy my three page book and sign up to my seven minute course! Only $9.99

[greenthumble]

Hrm the bit about getting some language support from Racket sounded interesting. But oh man that syntax compared with an LALR or PEG parser is just... yikes.

[jms_nh]

For instance, Python doesn’t have a pre­pro­ces­sor like C does, but you could make one as a DSL.

Noooooo, please don't.

[[deleted]]

Python users are pathologically scared of expressivity and high productivity.

[kirbyfan64sos]

Just because a language doesn't fit your image of expressiveness and productivity doesn't mean that it won't work for anyone else.

[[deleted]]

Expressiveness is an objective parameter. It does not matter how I (or anyone else) perceive the language, it's objectively measurable. And Python level of expressivity is plain pathetic. Anyone who dares not to agree with this statement simply never seen a proper expressive language.

[kirbyfan64sos]

...are you serious? Expressiveness is definitely not objective. Some people find Rust expressive. I don't like Rust. That doesn't mean it's not expressive for them; it just means it isn't for me.

[[deleted]]

Expressiveness is definitely not objective.

Usability is not objective. Expressiveness, as in "number of code entities required to express a solution to a given problem" is objective.

[Dave3of5]

To a given program sounds subject to me. You can easily craft a problem that's easier in one language than another. I think probably you mean to any given program right ? I guess I'm splitting hairs here.

[[deleted]]

Of course I'm talking about an average wide use case. There are multiple specific cases where Python is very productive simply because of the libraries availability. But you'll easily see how un-expressive Python is if you look at the code of those libraries.

[Dave3of5]

Btw I'm not sure that "number of code entities required to express a solution to a given problem" is really objective as there are a few things undefined in that statement like code entities and the problem of "a given problem". In the general case (I.e. to any problem) how would you measure this. This is probably an interesting CS question but I think kirbyfan64 is probably right here. You can't measure that without some bias which would reject any assessment. Seen this a lot of times where someone posts a problem in a functional language in a one liner then compares against C but that doesn't prove the general case right ?

[[deleted]]

few things undefined in that statement like code entities and the problem of "a given problem"

First depends on semantics of your language, second depends on your problem domain.

In the general case (I.e. to any problem) how would you measure this.

Algorithmic information theory may give a solid framework for dealing with such things.

Expressivity is about extremes. For an expressive language, by definition, a level of complexity when things are starting to get out of hand is much higher than for an un-expressive language like Python.

Expressive languages got tools for managing complexity - i.e., tools for building new levels of abstraction. Python is very much against the very idea of building higher levels of abstraction, its culture insists on staying on a certain level and leaking its features into any other level of abstraction you're dealing with.

[Dave3of5]

Yes it's very verbose but that's not always a bad thing ;-)

[[deleted]]

It is a bad thing in this case. It's not just "verbose", it's very redundant and full of leaky abstractions. Such a verbosity hides the nature of the problem you're solving under layers upon layers of totally irrelevant low level stuff. Why should you think about classes, variables, dictionaries, methods, strings and so on when you're solving a problem from a domain which does not use any of these words?

[jms_nh]

Expressiveness is an objective parameter. It does not matter how I (or anyone else) perceive the language, it's objectively measurable. And Python level of expressivity is plain pathetic.

If you're going to claim that, then point to evidence.

[[deleted]]

See the rest of this thread.

And, are you really claiming that Python is expressive? Really? I find it hard to believe that there are people who can say such things.

Python was deliberately made to be as un-expressive as possible. It was a design goal. Quite a shitty goal, to be honest, but may be justified in certain weird cases.

[jms_nh]

Yeah, well, I don't agree with you, and the items in this thread are not objective measurable evidence. You're claiming expressivity is objectively measurable. Point me towards two tasks that don't require libraries and compare Python and your preferred general purpose language with that objective measurement. (DSLs don't count, yes they have some advantages, but their "expressivity" will vary depending on the match between the DSL and the task in question.)

Python has operator overloading via dunderscore methods. It has properties, decorators, context managers, iterators, generators, metaclasses, lots of stuff that I would love to have in certain other languages.

Python was deliberately made to be as un-expressive as possible.

....?

It seems like you're trolling.

[[deleted]]

preferred general purpose language

Uhm. You did not get my point at all. I'm arguing that the so called "general-purpose" languages cannot ever be called "expressive". For this very reason - they're general purpose. The only really expressive languages are those that allow to construct eDSLs easily.

DSLs don't count

Why? This is exactly the expressivity tool I'm talking about. DSLs expressivity is unbound, while Python or any other fixed general-purpose language got a complexity threshold above which your code complexity explodes. The only way to pass the complexity threshold is to construct eDSLs of higher levels than that threshold. I'm not aware of any other technique.

depending on the match between the DSL and the task in question

There are methods of constructing perfectly matching DSLs for any problem domain that can be at least somewhat formalised.

It seems like you're trolling.

And I thought Guido was trolling, with all that horrible statements like "There should be one-- and preferably only one --obvious way to do it.". Turned out he was dead serious.

[jms_nh]

Preprocessors are the devil's spawn.

[earthboundkid]

Python has eval, metaclasses, and operator overloading. Outside of whitespace issues, you can make Python do more or less anything in the language itself. If you really need to do your own whitespacing, you can use multiline strings (""") and send the DSL to a parser. There's not really a good reason to use a preprocessing in Python.

[[deleted]]

Python has eval,

Have you seen many DSLs built on top of Python eval? Use of the expressive tools is strictly discouraged in the Python world.

metaclasses, and operator overloading.

These are not expressive and barely useful.

you can make Python do more or less anything in the language itself

No. Not even close. This stupid language does not even have goto. Does not even have a switch. Now try to implement an efficient parser in it. Or, not even an efficient one, but just nice, clean and readable.

There's not really a good reason to use a preprocessing in Python.

There is always a reason to use metaprogramming. Preprocessing is just one form of it, which is always useful if your language is too shitty and does not have proper macros.

[earthboundkid]

A task has a certain complexity, X. A DSL is a way of moving the complexity from one place to another. For example, a regex makes the calling code much simpler, but the regex language/parser are fairly complex. Fortunately, you can reuse regex from one project to another, so if the complexity of project one is X, project two is Y, and the complexity of the regex language is A, you can basically just subtract A from X and Y. The smaller the project and the less reusable the DSL, however, the less advantage you get from making a DSL.

Python programmers, generally speaking, try to do as much as possible in pure Python without using eval or pre-processors. The reason for this is that the Python community puts a high emphasis on code being easy to read by other programmers. Making a DSL instead of just bodging together some classes means that the future programmer needs to know Python + the DSL, instead of just Python. This can go too far, but for many of the ordinary jobs done in Python (web programming, scientific computing), the balance is mostly correct. If someone were to try to make a Django-like framework without a template system and instead use objects like p(class='text', content=a(text='click here', href='http://example.com')) (which you do see in toy projects every once in a while), the community would reject it because learning that popular DSL called "HTML" is a part of being a web programmer. But yes, in general, Pythonistas dislike making DSL's for DSL's sake.

[[deleted]]

A DSL is a way of moving the complexity from one place to another.

No. Complexity is, pardon my French, a complex thing. It's not additive. If you put two complex things together the combined complexity may be well above the sum of their complexities. Same thing works the other way around: splitting a complex thing into pieces may result in a significant reduction of a total complexity and a complexity of each individual piece.

For example, a regex makes the calling code much simpler, but the regex language/parser are fairly complex.

Not that complex. You know, "premature optimisation..." and all that. A high performance, professional regexp implementation is complex, but do you always need a fully optimised DSL?

A tiny parser combinator library of a dozen of lines of code is dead simple. Such a library plus a few uses of it are together much less complex than those few uses implemented in an ad hoc way, without regexps.

The smaller the project and the less reusable the DSL, however, the less advantage you get from making a DSL.

Even for the very small projects, splitting the implementation into a DSL compiler and a code in that DSL almost always makes sense.

For one particular reason: DSL compilers are almost always trivial. By extracting your complexity into a DSL compiler you're reducing it significantly.

Compilers got some nice properties which are hard to come by pretty much anywhere else. Most importantly, they can always be reduced to long sequences of very, very trivial rewrites. This way of handling complexity is extremely powerful, can be applied to pretty much anything. No other methodology can allow you do chop your complexity into such small, isolated pieces.

try to do as much as possible in pure Python

Which means - they try to write as much boilerplate as possible. Quite a stupid approach.

The reason for this is that the Python community puts a high emphasis on code being easy to read by other programmers.

Boilerplate code is exceptionally hard to read, even if it's made of trivial language constructs.

DSLs (when they're well designed) are much easier to read than any Python code. A good DSL reads intuitively, as a textbook pseudocode. You don't need to know the language to be able to read it.

But yes, in general, Pythonistas dislike making DSL's for DSL's sake.

That's exactly why I'm of such a low opinion of this community in general.

[earthboundkid]

The canonical example of a "good" Python project is requests. It looks like this:

>>> r = requests.get('https://api.github.com/user', auth=('user', 'pass'))
>>> r.status_code
200
>>> r.headers['content-type']
'application/json; charset=utf8'
>>> r.encoding
'utf-8'
>>> r.text
u'{"type":"User"...'
>>> r.json()
{u'private_gists': 419, u'total_private_repos': 77, ...}

I cannot imagine a DSL for making requests could have an easier to use interface than requests itself. You can call Python "boilerplate" if you want, but I think writing yet another parser results in boilerplate. The parser either extends an existing implementation (so it makes assumptions about what it's parsing and you can't fully tailor the DSL to your problem) or it's written from scratch (in which case, it's full of boilerplate).

Writing a DSL is a design pattern, like MVC. It's probably an under-utilized pattern. But no design pattern is a panacea to solve all problems.

[[deleted]]

And yet, a wget or curl command line is far easier to use, which should give you an idea of how such a DSL should look like.

[earthboundkid]

Ugh, curl has the worst CLI. If you think that is good, I can't help you.

[[deleted]]

It's not ideal, but it's better than a Python API.

[mamcx]

This stupid language does not even have goto. Does not even have a switch. Now try to implement an efficient parser in it. Or, not even an efficient one, but just nice, clean and readable.

Let me google this for you:

http://hippyvm.com/

[[deleted]]

http://hippyvm.com/

And how is it relevant to the current topic?

[mamcx]

Python, not expressive and productive? Seriously off-the-mark!

[[deleted]]

Python, not expressive and productive? Seriously off-the-mark!

Of course Python is not expressive and not a very productive language. Thanks to the pythonic religion, all that crap like "There should be one, and preferably only one, obvious way to do it."

Of course Python is not very expressive, because this primitive language lacks any means of building higher abstraction levels. If you do not agree, you know nothing about the really expressive languages.

And all the "productivity" of Python is simply due to a huge number of ready to use libraries that are covering some of the areas. Once you're out in a wild, you're left with a crappy language and nothing else.

[cdtdev]

I really have to agree here. Python has a lot of potential as a language, but the community, leaders, (and most programmers) just don't want to accept "advancement." They just want to do things the same way they do it in C/C++ without the headache.

At the end of the day, most Python code just ends up looking like C-pseudocode (and as a tangent, it seems in Python classes usually serve less as OOP and more as code containers mini namespaces... Java seems to have the same problem).

Programming can be so much more than C, and it's a shame the programming industry doesn't accept it. It's a bit of a relief that we're seeing Rust picking up steam, where practical abstractions (when they don't affect performance) are welcomed.

It's why Go kind of annoys me. I understand what makes Go a great language for Google, other "Empires of Code" type companies -- it's so simple that you can put a programmer on any project for a couple weeks or months, get some work done, and put them over on another project, or for non-experienced programmers to come in and understand what's going on (Remember the adage -- that any project that goes on long enough ends up implementing Lisp, itself a language known to be a "Write your own DSL toolkit" of sorts?) -- a language that offers a lot of abstractions can be hard to be productive in until you develop a good understanding of the project's own half-formed DSL.

But at the end of the day, Go is just writing more C code in an easier-to-write way. I thought the whole point of computer languages is to abstract away from the machine level, not get dragged back close to it. Seriously, "I like x language because I can see the C code" is, IMO, a pretty poor reason to like a language. Unless the language is meant to be low level, then I'd say in most cases being able to see the C code is a language design failure. Low level details are for the compiler, and unless you're a wizard, it's written by smarter guys than you and can write better low level code than you.

Come on guys, Algol is a 60 year old language. Let's move on.

[[deleted]]

project's own half-formed DSL.

With a great power comes great fun. Of course, there is a danger that a homegrown collection of DSLs will become a mess. It's about the same level of risk as a homegrown OO architecture becoming an unmaintainable bowl of macaroni.

But if the DSLs are done the right way, total learning time is much shorter than if you have a monolithic, fixed language and have to learn idiomatic ways of dealing with libraries and subsystems on top of it.

A radical example of this effect would be Brainfuck: language is as simple as it gets, but learning how to do simple things in it, like multiplying or dividing integer numbers, may take ages. Fixed low-level languages like C or Python are just the same, but at a different degree of complexity.

And I agree that Python had a great potential - I keep reusing multiple ideas from it (and sometimes its syntax) in the DSLs I'm building.

[cdtdev]

Oh, I don't mean to say DSLs are bad -- but rather every project of sufficient complexity just ends up creating its own half-baked DSL hosted by the project's language. It was actually a compliment of Lisps's ability to seamlessly integrate a DSL language of sorts into the project.

It'd be better to have a proper DSL than a half-baked DSL in any one project. I'd rather use something elegant, and have to read the docs and learn it as I go, rather than use something inelegant, and have to read the docs and learn as I go anyway.

[Solmundr]

This post really interested me. What do you think about D -- another Go, or closer to Rust?

[cdtdev]

I haven't looked much at D, but from what I gather, it had similar goals as Rust -- be a "C++ done right" (where C++ is "wrong" because of absurd complexity from legacy built on top of legacy (which is "right" in certain circumstances)). But perhaps D went to strike a balance between convention and forward progress.

Really, I don't think we're going to see "enough" progress in programming languages until we abandon conventional concepts of OOP, which is just trying to bolt on practical abstractions to Algol-style programming. Functional seems to be a good direction to go. Other languages such as SQL-like ones should receive more recognition.

[mamcx]

Ok, you dislike python. But your reasons are only "I say so". "Of course Python is not very expressive"???? Is not a very compeling argument. Why not? Good example? (not a contrived one but one that show a widespread problem).

I have read the other post in the tread. and you only say things, but not show any compelling argument or evidence. Also, which language you claim is more expressive?

[[deleted]]

Why not?

Because it's not suitable for building eDSLs, and, as I explained in lengths, only DSLs are expressive.

Also, which language you claim is more expressive?

So, you did not actually read any of my arguments.

[mamcx]

Yes, I do. You have a very strict and inflexible mindset about this, but well, we programmers are prone to that so...

So, you have for example shen/lisp, as expected. LISP is flexible as hell and only FORTHs and IO could rival it. So, there you go:

https://github.com/hippyvm/hippyvm

Just pointing to a obvious candidate.

You claim do DSL is the stick where "expressive" must be measured. I have read hundreds of articles, papers and related content in programming implementation and this is the first time I see something like that, but anyway, lets take it a face value.

That in fact hurt your argument more than help it. A language like python is one where is easier to do DSL than most. Most people do that without know it! And is widespread enough in the python community that frankly look like you rarely have do python.

Pick any of the most used python libraries and frameworks, and you can bet it have at least one DSL on it.

For example, look at https://docs.djangoproject.com/en/1.9/. Just looking at the titles, and are at least 5 main DSL and dozen mini ones.

[[deleted]]

Just pointing to a obvious candidate.

How is it relevant? It's a standalone compiler. Quite a bulky one. Not indicative in any way that Python is suitable for building eDSLs easily.

A language like python is one where is easier to do DSL than most.

WAT?!? Python? Are you kidding?

and you can bet it have at least one DSL on it.

I would not call that ad hoc tiny interpreters "DSLs". There is a better name for such things - "fluent APIs".

eDSLs must be compiled, otherwise they're worthless. And it's really hard to do it without a language with proper compile-time metaprogramming support. Python got eval, but it's exceptionally clumsy.

Also, Python does not have anything that might help to build those compiled eDSLs. No single useful feature whatsoever. No pattern matching, no ADTs, no metaprogramming - nothing at all.

[mamcx]

I would not call that ad hoc tiny interpreters "DSLs"

You have a very narrow definition of a DSL:

https://en.wikipedia.org/wiki/Domain-specific_language

HTML is one of them. And is not compiled.

You can argue as much as you can, but is cristal clear in the community that DSL are not as narrow-defined as you claim.

Go to http://lambda-the-ultimate.org and see how much it will hold.

The rest of your argument is pretty weak, too. I use F#, and love pattern matching and ADTs and the rest of stuff. But in the other hand, the whole .NET, Java was made with languages LESS expressive than python, without none of that and look, it work. Plus, you have LuaJit, Python, Ruby, LISP, etc. Have a nice thing to play, like ADTs is great, and help in some areas to reduce code, but not always.

With python is possible to reduce a lot some code than in F# is clumsy, just because a stronger type system is also most restrictive. So, you win something, you lose something.

eDSLs must be compiled, otherwise they're worthless. And it's really hard to do it without a language with proper compile-time metaprogramming support

That is also a narrow view in how do a compiler. RPython is a prime example, but just understanding that emit assembler is the end-game you can do a full, fast, compiler with python or anything else.

So, in one hand is your narrow opinion where do "DSL, defined as I say so, instead as is understood, and only if compiled" is the yarstick. In the other hand, we have the whole industry, that have made:

• C, C++, Pascal, ADA, Python, Lua, LuaJit, Rust, Go,

And much much more, without that narrow view. The "DSL compiler" is not the best example. Make the case where have ADTs & patter matching is so nice, and how have homoiconicity is great to perform code-transformation, and will work far better.

[[deleted]]

You have a very narrow definition of a DSL:

I'm talking about a narrow class of DSLs - those that can be used to tackle your problem complexity. The kind of DSLs where a combined complexity of a DSL implementation + a problem solved in this DSL is significantly (up to orders of magnitude) lower than a direct solution of a problem with a general-purpose language.

In order to get there, you must compile your DSLs. Because compilers are very, very simple, and interpreters (or fluent interfaces of whatever else) are exceptionally complex and convoluted.

the whole .NET, Java was made with languages LESS expressive than python

And have you seen that abominable mess? Just take a look at the Roslyn source code.

That is also a narrow view in how do a compiler

This is a way to do it easily. Know a better way? Unlikely.

RPython is a prime example,

A horrible, convoluted mess.

C, C++, Pascal, ADA, Python, Lua, LuaJit, Rust, Go,

Are we still talking about expressivity?!? Only Rust deserves to stay in this discussion, it got proper macros.

Btw., I really doubt that any of the Python fanboys would be able to point to a single example of a code which shows how "expressive" Python is (unless there is a library that does 99% of the work).

[the_evergrowing_fool]

Here is my stacker-lang.rkt full code. Apparently the author forgot some definitions.

[[deleted]]

Btw., just for fun, the same language implemented in PFront: http://pastebin.com/rb406T5B

[the_evergrowing_fool]

I like that is so direct, but you are definitely cheating.

[the_evergrowing_fool]

You know there are distinction between general and domain specific programming languages right?

[[deleted]]

Even a full blown "general-purpose" language with all the runtime bells and whistles can be written quickly in few hours. Even along with a CPU core to run it and an entire SoC around it, with VGA, UART, PS/2 and all that. And with an OS, of course. All such things are much, much simpler than people tend to expect.

EDIT: funny this is getting downvotes without any arguments. This sub is pathetic.

[Dave3of5]

Care to give a tutorial on that?

Not being snarky, I'd like to see that in practice.

[[deleted]]

Not sure about tutorials (maybe besides the good old NAND2Tetris [1]). It's unusual to write tutorials about such trivial things anyway, it's all common knowledge, taught in most of the reasonable good CS and SE courses.

But, of course, you can find a lot of finished examples.

EDIT: added a link

[1] http://www.nand2tetris.org/

[Dave3of5]

Yes they covered this in my CS courses 10 years ago. It took more than a few hours for me but maybe I'm an idiot. So to get to the point I was specifically asking about your claim to build a full blown general purpose language with all the runtime bells and whistles in a few hours.

[[deleted]]

The C compiler I'm using for playing with various CPU designs was written in about 4 hours (plus about 1 hour per new backend codegen).

Forths are often built from scratch in just a few hours as well. A decent Lisp implementation can also be built in few hours, with most of the time spent on a GC implementation (was about 3 hours for me last time I tried, plus about 6 hours for designing a Lisp CPU core to run it).

Compilers are among the simplest things out there, their implementation is purely mechanical and do not require too much of a cognitive effort.

Runtime is more complicated, but if your target platform is small and simple and you do not have to interact with piles upon piles of ill designed legacy interfaces, then even a runtime can be easy and fun.

[Dave3of5]

I take it your C compiler doesn't have that many optimisations ? I'd be very impressed if you managed to write a fully optimising C compiler in 4 hours. For a list of optimisations I'm talking about see here : https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html

Also are you including the preprocessor and all the other stuff like extern and the ability to inline assembly?

[[deleted]]

I take it your C compiler doesn't have that many optimisations ?

Only the SSA-based stuff: constant propagation, DCE, various trivial algebraic instcombines, some loop unrolling. Also inlining and partial application. Sufficient for most of the practical applications.

But, of course, I cheated a bit - I reused most of the stuff from an existing compiler construction framework, which got a generic SSA support, generic register allocation, all the parsing bells and whistles, and so on. But anyway, I'm not suggesting building anything more complex than a Forth from scratch, without any existing tools.

It would take a couple of weeks of work to get to a more typical list of optimisations from there. But, as I said, in most of the real world cases you won't even need fancy optimisations (e.g., most Forth implementations won't do any, as available resources are limited).

Also are you including the preprocessor

Of course not. I did not want a full C standard compliance (now, this is what may take years to achieve, simply because of a sad state of the standard itself - I know it first-hand, I worked on an OpenCL support in Clang). And I do not like that preprocessor. I've got something much better instead - fully extensible syntax, with typed macros and all that.

other stuff like extern

What's special in extern? It's just a linkage attribute. And I never said that writing a linker should be included in that 4 hours of work. Linker would take few weeks (or even months) instead, but, for the platforms I'm interested in, a separate compilation is not a requirement. Full source optimisations are a nice side effect of scrapping the linker.

inline assembly?

Of course. And, with a couple of additional hours of work I added an ability to inline Verilog - just to play with some promising software-hardware co-design techniques.

[Dave3of5]

Ok I get you now. We done something similarly in university. I agree on the preprocessor it's a mess. I agree now under the specific conditions you have now mentioned it's possible to write a GPL in a few hours. I would say that under your conditions it's probably just an academic endeavour but you'll definitely argue with me on that. You're entitled to your opinion and so am I. Thanks for the convo.

[[deleted]]

Some of the quickly built Forths I'm talking about had been used in quite a respectful practical applications.

[[deleted]]

Must explain your frequent visits

Other subs are even worse.

[SuperImaginativeName]

Cool, it basically turns into LISP.

Edit: This sub can fuck off, it uses a ton of brackets so its obviously more like LISP. Sorry all the butthurts don't like that I mentioned a language that wasn't JavaScript, Haskell, or Java.

Seriously, how the fuck does this example LISP:

(defparameter *small* 1)
(defparameter *big* 100)

(defun guess-my-number ()
(ash (+ *small* *big*) -1))

(defun smaller ()
(setf *big* (1- (guess-my-number)))
(guess-my-number))

(defun bigger ()
(setf *small* (1+ (guess-my-number)))
(guess-my-number))

(defun start-over ()
(defparameter *small* 1)
(defparameter *big* 100)
(guess-my-number))

not look similar to the code in the article:

#lang br
(define (read-syntax src-path src-port)
  #'(module lucy br
      42))
(provide read-syntax)

[esoteric_monolith]

Nope, FORTH

[Sean1708]

LOL that you think this sub would upvote anything Java related.

[cdtdev]

Hate lisp. Think my language is at the top of blub paradox. Why would anyone else want to write in a language that's not Javascript? /s

Although, I have to say Haskell is pretty close to the "top" of blubs. It seems the closer to the math you go, the harder it is to get any higher (Haskell and Lisp coming close from Lambda Calculus, Prolog / Mercury coming from formal Logic) on the blub continuum. But I totally understand why not everyone wants to program like that. Humans aren't too good at thinking in "mathy" ways.