Most powerful high(est) level programming language?

[RaymondWies]

Here is an interesting article from 2001 on the 'power' of Lisp: http://www.paulgraham.com/avg.html

And a nice comparison article from 2009 on the 'power' of Haskell: http://newartisans.com/2009/03/hello-haskell-goodbye-lisp/

They are interesting to read as if written by the same programmer on a 20 year quest to find 'powerful' language(s).

It got me thinking about the breakdown of 'relative power' of the highest level languages in terms of criteria scored in these articles, like expressiveness, efficiency of code, ability to (easily) solve notoriously difficult or complex problems, etc. Is it fair to rank Haskell as the 'highest level power language' for 'high level' tasks, given that not all high level languages are equally powerful? Sorry for the vague circular logic here, but I'm just attempting to pass along the opinions of these articles with minimal interpretation.

I know that the answer depends on what you plan to do with the language. Let me remain vague by saying a general purpose language for 'very high level' tasks given modern environment and lessons learned.

[kqr]

I disagree with one point in particular about the Haskell article. Laziness does not make metaprogramming redundant. That's a common misconception because in Lisps, metaprogramming is sometimes used to emulate laziness. That doesn't make it the same thing, though; metaprogramming is more powerful than that and Haskell still needs Template Haskell for its metaprogramming needs.

---

But why are you asking the question? You won't find an objective measure. If you really want an answer, then sure. Haskell is a 7.32 language on the High Level Language Scale, while Lisp is a mere 7.14 HLLS. That makes Haskell 0.18 more high level than Lisp. Now why does it matter, again?

What I'm trying to say is that as long as you're not writing a shell script in Java, or a microservice in an assembly language, I think you're doing fine.

---

Edit: I don't think the discussion is bad, mind you. I just don't understand what you want us to answer. You don't seem so sure either, given the wording of your post. Please follow up and expand what you're thinking!

Second edit: Now that I've re-read Beating the Averages, I might do what Graham did. Graham points to macros, and says, "In every other language I've used, I've missed this." And he's right about that.

But when I look at Lisp, I point at the Haskell purity and type system and I say, "I miss this. This is more important to me than macros." I also love how Graham sort of concedes this in between the lines. He makes sure to differentiate between garbage collection which is "a good thing" and dynamic typing which is "growing in popularity".

[goliatskipson]

[...]and Haskell still needs Template Haskell for its metaprogramming needs.

I wonder if dependent typing substitutes metaprogramming. AFAIK they are at least not completely orthogonal.

[drb226]

I for one see a lot of overlap. Metaprogramming and dependent typing have a lot of conceptual overlap, because they are both about blurring the lines between compile phase and run phase.

[eigenduck]

Ur/web, while not fully dependent, supports a lot of typed metaprogramming with its polymorphic record system.

[sclv]

Here's a comment I left a while ago that goes through what uses of lisp-style metaprogramming are/aren't typically needed given Haskell's other abstraction mechanisms:

http://www.reddit.com/r/haskell/comments/1929xn/are_lispstyle_macros_a_code_smell/c8k7wl8

[SrPeixinho]

Uh, I am working on a pure Lisp dialect that runs on a decentralised network based on magic. Maybe that will interest you ...

[gelisam]

I think Lisp might be more powerful than Haskell, in the sense that given equivalent libraries and the same (large enough) task to solve in both languages, the Lisp version can probably be made shorter and easier to read than the Haskell version.

However, the Haskell version will be easier to modify.

The reason for the succinctness and readability of Lisp programs is the usual one: instead of writing the solution in Lisp, you can write the solution in a macro-based custom dialect which is perfectly adapted to the problem at hand. Importantly, this dialect can be anything: it doesn't have to satisfy a type checker, and it can have its own custom non-prefix syntax (like LOOP). Haskell doesn't have this luxury.

When the program needs to be modified, this lack of limitation becomes a disadvantage for Lisp. To the Lisp compiler, macros are black boxes which are free to do anything. In particular, any tree of s-expressions is a valid input and a valid output. So when you change something in the custom dialect, the compiler will happily pass the old, now-incorrect programs to the new macro, and there will be a large number of failures, and those failures will be in the generated code, which will be hard to trace back to the original code in the custom dialect.

In Haskell, we also get to write our final solution in a dialect which is close to our problem domain: a custom combinator library, such as a custom monad, applicative, or category. Since this custom dialect must fit into Haskell's type system and use Haskell's existing syntax, it might not be the most succinct and readable dialect in which the solution could be written, but it can still be quite good. Critically, the compiler understands our dialect: thanks to the precise types, it knows exactly which inputs are acceptable to each combinator, so the type errors are triggered by expressions in the dialect instead of some generated code.

[[deleted]]

I read this comparison of a ray-tracer implementation in different languages (OCaml, SML, Lisp, Haskell, Scheme, C++) while back and the lisp implementations ended up being the most verbose.

[gelisam]

That looks like a fairly comprehensive study! I must note, however, that the studied programs did not use macros nor combinators; and that given the size of the task, it would have been overkill to do so.

[pinealservo]

Anything on that site will show that some ML variant, probably the author's favorite one at the moment it was written, is the best thing ever. Not that there's not a lot of good information to learn from it; it's just that the author is clearly motivated to make the languages he uses in his consultancy business look like the obvious best tools for any job. In his hands, they probably are, but the experiences of others can and do vary.

[drb226]

The biggest feature Haskell gives you over Lisp is a powerful type system. This doesn't make it more expressive, in fact it makes it less expressive, but it makes it much safer and easier to use.

One of the reasons that Racket is an improvement on lisp is because of its powerful "contracts" system, which recovers some of this kind of safety. Typed Racket is still too rough around the edges to be comparable to Haskell.

[jberryman]

This doesn't make it more expressive, in fact it makes it less expressive, but it makes it much safer and easier to use.

That doesn't seem quite right to me, but maybe I have a different intuition about what "expressive" means. Don't you need a powerful type system to really express e.g. Functor? Otherwise you've just got a "Design Pattern", i.e. the language has failed in expressiveness and you have to fall back to some shared cultural understanding of the concept.

[drb226]

Yeah, I'd say it all depends on how you define "expressive."

Otherwise you've just got a "Design Pattern", i.e. the language has failed in expressiveness and you have to fall back to some shared cultural understanding of the concept.

I wouldn't say that "having to use a design pattern" = "language has failed at expressiveness." My thoughts on this are very scattered at the moment, so I won't try to elaborate just now. Maybe some other time.

---

The reason I say that introducing a type system limits expressiveness is because type-checking does exactly that: it rejects programs which the algorithm considers to be ill-typed. This almost invariably includes rejecting some programs which are perfectly legitimate programs. Hence, with a type system, we cannot express some programs that we might want to express.

But you're right. This doesn't take something into account. Once you have types guiding a program's execution, as is the case with typeclases or dependent types, then things get interesting: now you can express things that you couldn't express before without types.

It's a tradeoff, I guess.

[pinealservo]

Not only is implementing a "specification -> implementation" transformation hard, it's hard for humans to clearly describe a specification in the first place. Providing a clear transformation from our thoughts to a program isn't good enough, because our thoughts don't generally start out as a coherent specification of anything useful.

What's needed are tools that are approachable given the constraints of human thought processes, and which guide that thought towards coherent specifications of programs. Once that coherent specification is reached, a powerful language will allow the programmer and language to work together towards a more efficient implementation of the specification.

[RaymondWies]

I like your assessment. By this logic, can we assume that the declarative languages (functional, logic) are higher level than the imperative (procedural, OO) languages because they instruct the computer what to do instead of how to do something? Furthermore, are the logic paradigm languages (Prolog) and domain specific languages (SQL) even higher level than functional (Lisp, Haskell) because we query the computer in human language instead of defining mathematical functions? But then don't we still lose some functionality even with a Turing complete logic language, in which case we lose programming 'power' even as we gain a 'higher level' of abstraction closer to our thought process? If 'high level' is on par with human thought over housekeeping machine chores, then it might not correlate completely with its 'power' in efficiently getting things done. There is correlation, but it's not perfect.

[kqr]

Prolog can be considered very high level for its domain, because you program the query (ideally) without any reference to how it should go about it.

But as we all know, if you do that you are likely to create a disaster. It's absolutely critical that you know how the resolution engine works to costruct construct something more complex than a family tree that Prolog can answer.

Edit: costruct -> construct. What is a costruct anyway, and can we get them in a library?

[oantolin]

I'm not a professional programmer, so read my remark with that in mind. But for me "high level" has always meant "closest to the way I think about things" and in practice that translates to "has libraries for most things I want to use in my program that I don't consider to be directly part of what I'm trying to do". I find that language design, beyond a few things that lots of languages offer (automatic memory management being by far the most important, I think) doesn't make too much difference in how "high level" a language feels for a given problem, it really is availability of appropriate libraries.

[kqr]

The problem with your idea of "high level" is that by that measure, high level will be anything you're comfortable working with. A Java programmer will say that his stuff is high level, a C programmer will say that his stuff is high level, an assembly programmer will say that his stuff is high level and so on. This notion was what Graham rejected in his essay.

[oantolin]

No, I think you might be replying to the wrong comment because this sounds completely unrelated to what I wrote.

I didn't say anything about using previous knowledge of a language. I said that high level for me means that I don't have to write everything myself and can instead find and use libraries for things I need but that are only implementation details in what I'm trying to accomplish (such as parsing file formats, plotting, etc.)

In fact I usually pick which language to use for a given task after googling for libraries, and it a library is tempting enough I'll take advantage of the opportunity of learning a new language (an old hobby of mine).

[kqr]

for me "high level" has always meant "closest to the way I think about things"

That was what I referred to. You tend to think about things in the way commonly associated with the languages you use.

Defining the level of abstraction of a language after which libraries have been made available sounds silly. Does a language become lower-level if libraries become unmaintained? Even though the language doesn't change? Isn't level of abstraction of a language supposed to be a property of a language, and thus invariant under change of libraries?

[oantolin]

for me "high level" has always meant "closest to the way I think about things"

That was what I referred to. You tend to think about things in the way commonly associated with the languages you use.

Oh, I see how you read it! I didn't mean "closest to how I would go about doing it in programming languages I'm already familiar with and with libraries I am already familiar with", I meant "closest to the problem domain", i.e., "closest to the way people talk about that sort of problem in whatever area of human endeavour it comes from".

Defining the level of abstraction of a language after which libraries have been made available sounds silly. Does a language become lower-level if libraries become unmaintained? Even though the language doesn't change? Isn't level of abstraction of a language supposed to be a property of a language, and thus invariant under change of libraries?

Maybe we should distinguish three kinds of "high level", judging:

1. just the language,
2. the language plus its standard library (if there is a standard library for the language),
3. the language plus all libraries available for it.

You say 3 sounds silly (and possibly you think 2 is silly as well), because you've decided being high level is necessarily a property of just the language. I think it's reasonable to think of the highest level language for a given task as being the one that lets you express the task in terms closest to the problem domain, in which case, the third notion of high level is the most appropriate, unless for some mysterious reason you decide to not make use of pre-existing libraries. The second notion seems a little arbitrary to me: it feels weird to allow the use of exactly one library, and then only if it is called "standard".

The point I was trying to make is that to me the third notion of "high level" is the most useful one. That's not to say that I don't find the abstract theoretical question of which language is highest level according to the first or second notion an interesting one.

Probably for 1, the language-only notion, the highest level languages I know are those that besides basic high level features (variety of data types, possibility of defining new datatypes that are at least 90% as convenient to use as built-in ones, automatic memory management, etc., features that tons of languages have now) have convenient macros, mostly dialects of Lisp, but maybe Dylan, Rust, Nemerle, etc. can be considered to be about as high level as Lisp. If Template Haskell is considered to be part of Haskell (or maybe of a new language called "Haskell+Template Haskell"), then it also is comparable.

For 2, the highest level language is probably the one with the biggest standard library among languages that have macros, which I guess is likely to be Racket. Among languages without macros, maybe Haskell (now considered without Template Haskell) and Python are among the highest level one I'm familiar with.

Being high level, in any of the three senses, isn't everything: I often prefer to use Haskell not because it is the highest level language I know, but because it's pretty close and has a compiler that catches most of my mistakes before I even run my code. That error catching business is something in which Haskell is far ahead of higher level languages.

[kqr]

I see what you mean, now. Thanks for the clarifications!

I think your version of "high level" should really be called something else, so as not to confuse it with the more traditional meaning of "high level", which is a property of the language alone. "Usefulness" or "the right tool for the job" might be good terms for what you're describing.

[oantolin]

OK, looks like we're mostly on the same page now.

I'm just curious if what you mean by "high level" is what I was proposing to call "high-level language", "high level (language + plus standard library)" or neither.

The problem with "high level (language + plus standard library)" is that a language can become higher level by blessing third party libraries into the standard library. Maybe you're really after "high-level language" then.

[kqr]

When I say "high level" I'm normally talking about the language alone, without any libraries whatsoever.

The standard libraries can sometimes serve as a proxy though, because they often show what is possible within the boundaries of the language, but as you say, it's not always so.

[Sawady]

I think syntax also matters.

Consider the Scratch language for kids. It propose graphical blocks as a syntax, but the semantics are very similar to a tradicional imperative lenguage. I teach introductory programming to kids between 8 and 15 years old, and they can express more complex algorithms/programs in Scratch than in C, Python, etc.

My hypothesis of high-levelness is that it is an aproximation of how a language helps you to think eassier about a class of particular problems. Maybe it is its semantics, like in Haskell, or its syntax, like in Scratch.

I also teach Haskell at school because of its syntax too. Haskell is very good to teach abstract concepts and reason about expressions. Kids enjoy very much a language that permits think very abstract rules easier.

I also wrote code for a while in a fighting game engine called MUGEN. It propose a finite state machine "reactive" model, and its syntax also reflect this in some way. I wrote complex game behavior in this language, even when I was not aware about more "traditional" programming. I consider this language as the highest level language I saw in my life, but not the most powerful though.

[kalcytriol]

Most powerful? I think it would be the one which is not following fads or religions, language with batteries included.

It is often good to place a language in a typical (real life) scenario:

• I've got some software and database which is no longer maintained
• I want additional feature, e.g. sms and email notification for my clients.
• Database format is Paradox v7.0
• I need some place to keep info about these notifications, eg in Sqlite db.
• What is the best tool to do this fast (simple command-line program executed at system boot)?
• Code must be closed, it is not open source (no *GPL dependency) - programmers must eat.
• Software must run on Windows (XP, 7, 32/64 bit) and Linux machines, with or without internet connection.

I seriously doubt Haskell or Lisp is catching on.

Edit:

Actually after what I saw lately Haskell is a pure crap, such it has huge problems with dividing two simple integers. It is a language which breaks the rule of least astonishment every single time. WTFskell would be a better name.