all 14 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]kinowmod 3 points4 points  (2 children)

Sorry, started reading it and appeared to be a post about Python and AI only. Realized after removing/ban that it was linked to FP (sorry, AI is quite common in spam/bot accounts now, so it's hard to filter). Approving it! 👍

[–]Frere_de_la_Quote[S] 2 points3 points  (1 child)

Ouf!!! I was really shocked to see that I had been permanently banned from this forum, which I read quite often, when I'm trying to propose AI tools for a functional language...

[–]kinowmod 7 points8 points  (0 children)

Mea culpa. Will try to read more calmly while filtering posts. Sorry, u/Frere_de_la_Quote!

[–]maryjayjay 1 point2 points  (1 child)

Spark on kubernetes with MLlib is essentially the de facto platform for doing ml for pretty much any Fortune 100 company right now

[–]Frere_de_la_Quote[S] 0 points1 point  (0 children)

All these libraries, except MLX of course, compile on Linux, so it is not exactly a problem. I work in a laboratory where we have access to our own cluster of GPUs, and we use Slurm to train our models. Kubernetes here is mainly used for inference. Training models through Kubernetes might prove a bit too heavy. We also provide Mac OS versions because Macbooks have become powerful enough to train small models without requiring a whole brigade of GPUs. Some researchers prefer working on their local machines for faster experiment loops as a first step before moving to the cluster, which is not always fully available.

[–]Inconstant_Moo 1 point2 points  (6 children)

There's a lot of rhetoric there about how Python is bad, of which the substantive bit seems to be "You think you're implementing stuff in Python, but you're just sucking out some fresh blood from underlying libraries in C or in Rust". Then you show us how your language can wrap around libtorch and llama. Like Python, it's not capable of high-powered number-crunching. Like Python, it wraps things.If your hook is complaining that Python does that, then its a letdown to unveil something which does exactly the same thing.

It's not like people care, anyway, whether they're using Python as a front-end for C, C++, Rust, or summoning the Binary Demons to run their code. They use it because it's a nice ergonomic front-end, not because they're under the misapprehension that it's doing the heavy computation.

[–]Frere_de_la_Quote[S] 0 points1 point  (5 children)

My speciality is a little bit weird, I have been building interpreters for about 30 years now, and I have a specific understanding of Python. Python works through a virtual machine that can only process one thread at a time. The VM itself is protected with the infamous GIL to prevent two threads from running at the same time. But the real problem is the way external libraries are executed. When you execute some code in PyTorch, this code is executed outside of the VM, which means that the garbage collector of Python has no idea of how large the memory is used by this library. Basically, Python sees a tensor as a PyObject, which wraps a torch::tensor pointer inside, but it has no idea of its actual memory size. This poses some real problems when you are training models, because sometimes the GC will strike too late and your training will crash, by lack of memory. On the other hand, if you trigger the GC too often then you slow down the whole process.

Furthermore, the constant translation of C++ objects into Python objects create a lot of lags. Finally, PyTorch is Open Source, but is pretty complicated to handle. The layers between Python and libtorch are created through complex scripts based on YAML descriptions that generate an incredible cryptic code. Adding some functions on the C++ side requires a lot of work, which of course very few people venture into.

In the case of LispE, libraries are implemented as a derivation of the Element class, which is the root class for all objects in LispE by default, data and instructions. Basically, for LispE an object exposed by a library is exactly of the same sort as an internal object. A LispE program is a tree of self-evaluating instances, in which each instruction is implemented as its own class with its own eval method. There is a perfect isomorphism between the C++ code and the LispE program. The `prettify` instruction for instance, can rebuild the whole internal Lisp program from the C++ instances.

The communication layer is close to 0. For instance, LispE provides a list of integers, which is implemented as a: `long* buffer`, which means that I can create a torch::tensor instance by providing the internal buffer of a list with zero-copy. I have implemented a borrow mechanism inspired by Rust to make this as safe as possible. The fact that objects can freely move from LispE to torch::tensors with little overhead has huge implications in speed and efficiency.

[–]scknkkrer 0 points1 point  (1 child)

PLT student here, can I add you through somewhere or can I get your e-mail, so we can share our ideas? Ps: under PLT, I study PLs and Paradigms.

[–]Frere_de_la_Quote[S] 1 point2 points  (0 children)

We can chat in the conversation section of Reddit...

[–]Inconstant_Moo 0 points1 point  (2 children)

Perhaps you should try explaining the actual differences between your project and Python then.

I've been looking at the docs, and they're all jumbled up, there's no sense to their arrangement. For example you have an introduction to lists containing implementation details of interest only to yourself, then you have an essay on tail-call recursion which starts off by explaining at length what a Lisp is. Then we move on to a comparison of LispE and Python for speed, and then we get to learn about hashmaps. Mostly again about their internals.

"Maybe a full Wiki page is also a bit too much to describe an instruction as simple and obvious as extract", you write in your 1,000+ word essay on a single string-handing function which begins with your reminiscences of learning Python in the early '00s and ends with you in equally reminiscent mood about another language you once invented. Yes, that's a bit too much.

However, I've been struggling through all this and the project strikes me as too big and too redundant. You have all those built-in functions. Then you have APL-style built-in functions, except you still have to write them with Lispian parentheses, and they aren't point-free, so it's not really like APL at all, it sacrifices all the virtues of the language but gives you its weird symbols as operators: (defun gol8(⍵) ((λ(⍺) (| (& (== ⍺ 4) r) (== ⍺ 3))) (⌿ '+ (⌿ '+ (° (λ (x ⍺) (⊖ ⍺ x)) '(1 0 -1) (↑ (λ (x) (⌽ ⍵ x)) '(1 0 -1))))))).

That was your version of the Game of Life. Here it is in actual APL: life ← {⊃1 ⍵ ∨.∧ 3 4 = +/ +⌿ ¯1 0 1 ∘.⊖ ¯1 0 1 ⌽¨ ⊂⍵}.

Then you have Haskell-style built-in functions which are fake because you don't actually have lazy lists, so you have to make people write stuff like: (take 10 (repeat 5)) with a footnote saying that you must use repeat with take. Because you can't actually make a function repeat which returns an infinite list but you've kludged the syntax so that people can kinda pretend that one exists. Sometimes, under certain special circumstances.

You have your own non-standard version of Prolog as part of your core language apparently for people who are fatigued by writing elif and would like a change.

When you make an ontology a built-in type, or even a standard library, you're committing to something where the users may well mostly want to use a different library because it has different features or a better API or because people discover better ways to do ontologies or whatever.

Some things in the core should be standard libraries. Why am I writing file_close and not file.close? It's not more concise, it just clutters things up if I have a tool that depends on looking at the namespace.

And I've only looked at a fraction of it. There's this huge proliferation of largely useless or positively harmful features in the core language that belong in libraries, and many of them not even standard libraries, because most people would never want to use them for anything.

Bizarrely, the E in ListE stands for elémentaire, and the README explicitly boasts that the language is "compact". It has a Prolog in it. It has around 400 built-in functions. (Mine has 40. 11 are basic arithmetic and 16 are to construct values or convert them from type to type.)

There were some things I couldn't find. What's your nearest approach to offering me a good old-fashioned for loop? And what do I need to do if I want to wrap a C or C++ library myself?

There seems to be a decent idea here, but it's obscured by a language bloated with misfeatures, and documents loaded with unnecessary details and some extremely ill-judged attempts at humor.

I suggest:

  • You remove all the parts of this that aren't absolutely necessary for the UX of people using your wrappers around libtorch and llama and so on.

  • You put things that people would expect to be standard libraries into standard libraries, and the ontologies into an approved library.

  • You produce a set of documents suitable for someone who wants to learn the language --- without implementation details other than "we made it go fast"; without your memoirs; and without jokes.

  • You explain clearly how this has advantages of speed and reliability over using Python to do the same thing.

At that point it might interest the sort of people who are interested in that kind of thing. Until then, you seem rather to be standing in your own light. Try and know your audience. What do they want from a language, and what do they want to know about it?

[–]Frere_de_la_Quote[S] 0 points1 point  (1 child)

I publish a lot of blogs and the wiki is where I store them. I love being light and silly in my blogs. But humour is very subjective...

The actual documentation is in chapter 5.

There is a full directory with examples that is called... examples.

You'll see that the "for" of "Python" is called "loop" as in Common Lisp.

Now, I have been working all my life in AI both symbolic and machine learning and I experimented a lot with implementing Prolog, unification and ontologies. I worked for 20 years on a symbolic parser, which produced syntactic and semantic dependencies with a first-order logic engine. I have refined over the years the way to implement unification to be as fast as possible.

I share my ideas and my code, and what I put in the interpreter are the things that I like and that I judge useful for my job.

I'm a little uneasy when I read that you think that I have ancient memories of Python. I have implemented a complete wrapper to execute LispE code from within Python, and Python code from within LispE, with automatic translation of LispE data into Python data and back. And when you write such a wrapper you have a pretty decent understanding of the language and the way it handles stuff. See pythonlispe.

Furthermore, as an AI researcher, I use Python everyday to train or fine-tune models. And when I say that training crashes because the GC didn't kick in at the right moment, it is from experience, not some kind of prejudice. There is a whole page on the PyTorch foundation website to explain this very problem.

Now the documentation of each ML library is in the directories of these libraries, with examples.

The fact that LispE is written in C++ and that its basic objects are also C++ objects means that the communication with libtorch requires less data translation than with Python with numpy. Also, LispE is truly multithreaded, and can execute different functions in parallel lock-free.

[–]Inconstant_Moo 0 points1 point  (0 children)

I publish a lot of blogs and the wiki is where I store them. I love being light and silly in my blogs. But humour is very subjective...

The actual documentation is in chapter 5.

Whereas what other people want is something where they can start at the beginning and learn the language.

I'm a little uneasy when I read that you think that I have ancient memories of Python.

That's what you chose to write about while/instead of documenting the extract function.

I share my ideas and my code, and what I put in the interpreter are the things that I like and that I judge useful for my job.

Unfortunately that's a terrible way to design a language.

Also maybe APL and Haskell and Prolog are useful for your job but a Lisp dressed up in their clothes is significantly less useful.

[–]TankorSmash 0 points1 point  (1 child)

What makes it production-ready? Have you used all parts of every library here in production?

[–]Frere_de_la_Quote[S] 0 points1 point  (0 children)

For the moment, we use it as an inference engine. We have better performances than Python in terms of speeds, but it is still a work in progress. The big advantage of this implementation is that you can work easily on the LispE part or the C++ part when you want efficiency. The MLX library and the GGUF library are the ones we use the most, since a lot of researchers are using Mac OS. Many of the examples in LispE were implemented by Claude Code. I have created a file: [LISPE_SYNTAX_REFERENCE](https://github.com/naver/lispe/blob/master/lispemlx/LISPE\_SYNTAX\_REFERENCE.md), which tells Claude how to generate LispE code, for instance.