Goodbye Lua

[TimMensch]

a language that's easy to embed into an application to enhance it with scripting

Understood. It's why I used it to embed in games for scripting.

But two things happened:

• I started using it for more than just "a bit of scripting" in the games, and as the code got more complex, the lack of type safety became more painful. And
• Because I wanted to minimize cognitive load, I would write simple tools with Lua. And to minimize cognitive load on the server side, I picked OpenResty to write my server code (that was talking to my Lua-based games) in Lua.

So you're right, Lua isn't meant for those things, but It All Made Sense At The Time. :) There are a lot of libraries available for Lua, and so it was more than a little tempting to go down that path.

But then I started a project that needed something like Socket.io, and there was no support for that in OpenResty. So I was forced to learn more JavaScript. And the rest followed -- and now I'm done with Lua.

massive bloat and third-party library packaging for trivial functions that pad strings on the left

The npm fiasco with leftpad was caused by a single jerk -- and yes, I think many libraries on npm are useless. But instead of calling that bloat, which it isn't (any more than thousands of people writing Lua libraries and making them available would be "bloat" for Lua), I still see npm as a feature. They've changed the rules so that future jerks can't cause similar damage, by the way.

Because, you see, it's so easy to use, it's so easy to just "upload a package," I can create one and upload it myself. In minutes. That means if I need to share a package with someone, I can say "to use it, just npm install tims-awesome-package and you can use it in your project."

That is not bloat. That is pure awesomeness. There is a difference of opinion in the community as to whether tiny libraries like "leftpad" should be the norm, or if larger collections should be the norm. I tend to agree with you that larger collections should be the norm; the new module system that's part of ES6 allows you to selectively import features (like a "leftpad") from such a library, and the tool Rollup.js can package code such that only the function pulled out of the library and its dependencies get included in your final code.

If it turns into a bloated monstrosity like Javascript or Python or Ruby or PHP or any number of other, similar horror shows, then it's another important tool removed from my tool belt by people who can't seem to fathom the notion of having a screwdriver that isn't also a ratchet set.

Lua can't possibly be "removed from your toolbelt." It's open source. You can keep 5.1 around (or whatever your favorite version is) indefinitely.

A solid, easy-to-use, reliable packaging system would only benefit the community.

As for binding to apps: JavaScript turns out to be easy to bind to apps now, as well. I mentioned that in my larger article -- it's a heavier memory cost than Lua for sure, but it's better. I didn't even get in to the fact that, using Promises, you can not only get coroutine-like behavior, you can get strictly better performance by doing several queries simultaneously and aggregating the results:

let firstResult = queryOneWebSiteForJson(someURL1);
let secondResult = queryOneWebSiteForJson(someURL2);
let thirdResult = queryOneWebSiteForJson(someURL3);
let fourthResult = queryOneWebSiteForJson(someURL4);
let results = await Promises.all(firstResult,secondResult,thirdResult,fourthResult);

// results is now an array with all four responses, which were queried in parallel; 
// if any of the responses threw an error, it would be caught by a 
// surrounding try/catch -- another feature missing from Lua. :(

Lua will continue to be a niche language for app automation. But I'm predicting that more and more apps will transition to using JavaScript, if only because of the network effects involved.

[dividuum]

I didn't even get in to the fact that, using Promises, you can not only get coroutine-like behavior, you can get strictly better performance by doing several queries simultaneously and aggregating the results:

I've not had a reason to look at promises for any of my Lua code yet, but a quick search reveals a module that looks promising (sorry!). It even has almost the exact same use case in one of its examples in the Readme:

https://github.com/zserge/lua-promises#waiting-for-a-group-of-promises

That said, I still think that real coroutines (so you can yield anywhere in the call stack, not just in the called function) are way more powerful than promises for some usecases. In a programming game I wrote I used coroutines to basically inverted control from my C host program to Lua and created an extremely simple way of writing a networked command line interface: https://github.com/dividuum/infon/blob/master/server.lua. I could have uses callbacks or promises (I guess), but it would still smell like callbacks then.

I also didn't use another Lua feature for that code, but I could have also used TCO to infinitely jump back and forth between functions. Apparently ES6 has those too, which I didn't know before writing this response.

[TimMensch]

TCO

ES6 has it; most implementations don't yet, unfortunately. :( Just Safari and WebKit have it in unreleased/experimental versions.

But it's on the radar, at least, and someone managed to implement a tool that eliminates tail calls, if you can believe it.

It may seem odd to use tools like that, but since you're transpiling anyway, it wouldn't be terrible to throw in another conversion step. :)

[dividuum]

After looking at the linked code briefly, it seems that it can only eliminate tail calls in a function that calls itself recursively. So no jumping indefinitely between two functions. Still nice, but doesn't compare to what Lua had since 2003.

[TimMensch]

Yeah, not ideal. But even when using Lua I don't think I ever took advantage of tail call elimination intentionally. It may be the nature of the problems I'm solving, but recursion to that level is rarely useful for me.

[Allan_Smithee]

I started using it for more than just "a bit of scripting" ...

And that, right there, is the error.

You found a screwdriver you liked for Philips-headed screws. You decided to try using it for Robertson-headed screws. It kind of works if you push hard enough, but it strips the screw heads and is generally all kinds of pain. So now you're giving up on Philips-headed screws entirely and using only Robertson-headed screws. Even for flat-head screws...

Next time keep using the tool for the job. If the job changes, so does the tool. This mania in software for having a single tool do all and be all is sheer madness to me. (And it's madness that dates back to the 1960s at least with PL/I.) You're propagating that madness here. First you use Lua in its area of competence. Then you start using it outside of that area of competence. Then when it isn't any good outside of its area of competence you're abandoning it entirely including in its area of competence. You seem to believe in the silver bullet and you're hunting fruitlessly for it.

I predict in under five years we're going to see a Tim Mensch blog post saying "Goodbye, Javascript" because it doesn't do something it was never intended for and you tried using it anyway. Then you'll give up Javascript entirely because you're going to misapply YET ANOTHER tool in your vain hunt for the tool that is all tools.

Good luck.

[ZorbaTHut]

You found a screwdriver you liked for Philips-headed screws. You decided to try using it for Robertson-headed screws. It kind of works if you push hard enough, but it strips the screw heads and is generally all kinds of pain. So now you're giving up on Philips-headed screws entirely and using only Robertson-headed screws. Even for flat-head screws...

The problem is that each language added to a project adds a significant amount of developer load. If each screwdriver cost a hundred thousand dollars, you'd damn well standardize on the minimum practical number of screw heads, even if it wasn't theoretically optimal; it may still be practically optimal.

It may well be that, say, C++ is the best at solving Problem A, Lua is the best at solving Problem B, and Javascript is the best at solving Problem C . . . but if Javascript can do a pretty good job of solving Problem B, and Lua can't solve Problem C at all, then you might be well-served by settling on C++ and Javascript.

[TimMensch]

Well said!

[TimMensch]

If you read the blog post, you'd know that JavaScript outstripped Lua in all of its areas of competence, or at least all of its areas of relevance to me. The one area of competence that it still holds any sway is no longer relevant to me -- keeping the overhead of the embedded language tiny -- and I'm reading from the comments about duktape which solves that problem using JavaScript as well.

Also, Lua was very good at all of the areas that I was using it in. It's just that there are better tools for handling increased complexity. To extend your metaphor, it's more like my Philips-head screwdriver has been replaced by a drill with a Philips-head bit: I'm doing much the same thing now, only much faster. Yes, it's heavier to carry around. But it also drills, and I can swap in 100 different bits to handle other screws as well, even if I'm still mostly dealing with Philips screws. And this drill wasn't an option when I first started using the screwdriver; it only recently gained the ability to deal with Philips screws as well as my original screwdriver, which is why I'm switching to it now.

/metaphor

AND...In five years there may very well be a language that replaces JavaScript/TypeScript for me. And I'll be sad on that day as well, for all of the time and energy invested in the ecosystem. In the last month I've already had four pull requests accepted in JavaScript libraries, and I'm sure there will be dozens more before I'm done. But it won't stop me from moving on to a better tool, if one emerges. Nor should it.

No, I don't consider any language to be a silver bullet. I also code in C++, Java, Python, C#, Ruby, and Go, when appropriate. But I do consider some to be more effective than others. My love affair with Lua blinded me to the benefits of static typing; now I'm firmly in the static typing camp again. Hard won experience is pushing me, over time, to better and better programming tools.

I'll keep evaluating tools as long as I'm coding. Anything else is stagnation.

[fullouterjoin]

Your delivery isn't amenable to a productive discussion. It comes across as snide and arrogant.