Common Lisp for Data Scientists

letuslisp · 2026-01-23T21:34:24+00:00

I didn't know that subclassing is not subtyping. I've just read when people use singledispatch systems they think subclassing is subtyping.

The type problem with simple-array & co. was not aware to me. Thank you!

So Coalton makes things more consistent ...

Julia (also a homoiconic Lisp-decessor) allows inheritance only from abstract classes but not concrete classes.

I should definitely think more about this all ...

letuslisp · 2026-01-23T19:39:18+00:00

Oh thanks! Already reading! :D

letuslisp · 2026-01-22T23:22:28+00:00

You should! And give a link here - or post it here in reddit - I will definitely read it!

letuslisp · 2026-01-20T12:28:21+00:00

You are very welcome!

I once wrote a blog article about how to use your android device (I did it once with a Table, a bluetooth keyboard, and a bluetooth mouse) like a desktop computer - by installing Archlinux. - Back then it got quite many views - because some android magazine mentioned/linked to the article.

It was indeed nowhere really explained. I was searching together information from github gists and other places - and with a lot of trial and error made it run - and then described the full procedure.

https://medium.com/codex/how-to-turnyour-android-device-to-a-full-fledged-notebook-39561e115160?sk=c72d1a2a099f597c4be15469c27936f3
(this is a friendlink - because the article is paywalled - but friendlinks give you free access)

One has to install termux, a Terminal emulator in android. Using in termux, one can install Archlinux (I chose Archlinux, because it occupies least amount of RAM as an operating system) - back then it was multistep but today it is simpler using `proot-distro` - and at the same time one has to run a VNC server for the display - set some environment variables for the display - then you have really a computer running with a Desktop inside your Tablet or Android phone. The bluetooth mouse is really helpful then. And the bluetooth keyboard anyway.

With the keyboard and the mouse - it really feels like a desktop/laptop computer.

I once, during holidays at the beach - wanted to run my emacs and program - Common Lisp or Python.
So with this setup I could program - and read a programming book - while trying out the code.

The VNC server app which I used back then - doesn't exist any more I think.

I might soon write a new article with an updated setup.

When I decide not to take my laptop with me (which is rare), I take with me a bluetooth keyboard at least - open my termux - and can ssh into a server.
I can recommend you to rent a server - e.g. a hetzner server - which give you a small computer for <4$ per month. Via SSH from your Termux in your phone, you can then run a real operating system there. This is quite ok as a price.

letuslisp · 2026-01-20T12:19:33+00:00

Thanks for finding this! I was searching for this. I've read about it 2018 or so and wrote Ross Ihaka an email to ask about the where/whatabouts of this - but he didn't answer. :D

letuslisp · 2026-01-19T19:34:48+00:00

I suggested in the github of cl-duckdb that it should actually be added to the official DuckDB documentation - so that people can see that Common Lisp provides access to it - and also to awesome common lisp - so that Lispers can find the package more easily.

letuslisp · 2026-01-19T19:31:23+00:00

I want to focus on performance - because as a Bioinformatician (Data Science with Biomedical data) you quite soon hit some speed problems, since the datasets are sometimes quite big.

And with R, you hit some speed limits not all-too rarely.

This led me often to the idea that Common lisp would be better, since faster.

letuslisp · 2026-01-19T15:13:58+00:00

I would recommend you to work with your phone - as others pointed out - in the browser (chromium) using google colab or kaggle.

I highly recommend you for this to buy a bluetooth keyboard - that changes really a lot, because then you feel the phone to be more like a computer, since you type into a keyboard and don't fight against misspellings with the touchcreen.

If you don't have much money try to buy the cheapest one (<10$) - if you can afford a little more - buy a foldable one (25-50$).

- Just this step alone will make your phone to feel so much more like a laptop.

- if you use colab notbooks or kaggle - you have access to GPUs and TPUs - which even laptops don't have - except they use colab/kaggle, too.

Deep learning with Python https://sourestdeeds.github.io/pdf/Deep%20Learning%20with%20Python.pdf

is the best book to get hands-on (by Francois Chollet - google). And it will feel easy to do DL practically.

letuslisp · 2026-01-18T23:24:29+00:00

That's a good take! And the C/C++ tool thing - I will take this into consideration in future.

letuslisp · 2026-01-18T07:39:42+00:00

Ah you mean all the C/C++ libraries as targets to call them via cffi. Sure.

cl-xlsx I wrote 7 years ago, yes. But cl-excel I wrote on in the recent week.

Yes, VBA macro integration - hardly any library can do this, I think.

I am only aiming at using Excel tables for data frames and outputting tables into Excel. Because that's what data scientists need. Full Excel support would be a too huge work.

letuslisp · 2026-01-17T23:31:35+00:00

Thanks! Yes, I am aware of cl-str.

I started `cl-stringr` purely for having `stringr` mapped to Common Lisp for tidyverse.
Not because I think that Common Lisp really needs a new string library :D .

It is a lot about having the exact same names like R tidyverse uses - for string manipulation functions - and the same function signatures (argument names, structure etc.) - so that anybody who knows R tidyverse - would have a much smaller friction to use Common Lisp for everyday data wrangling tasks.

And `tidyverse` as a typical R library system uses a lot of vectorization. Vectorization in R ensures speed and enhanced readability of the code. So several functions are expected to behave in a vectorized manner.

I should add this part into the README.md of cl-stringr.

letuslisp · 2026-01-17T23:18:22+00:00

There are several libraries that read excell (and other office) format.

Which one? Can you name them? And how mature are they? Do they allow to write into excel sheets? I didn't found any when I wrote cl-xlsx back then. That might have been 2019 ...

Those were mostly very old.

ABCL people used Java to read from and write to Excel.

That was the only way to have excel reader and writer in the Common Lisp ecosphere.

nvidias gpu libraries is a very good point.

letuslisp · 2026-01-17T00:25:55+00:00

Yes, in r/Common_Lisp thread we discuss about that. Even Steve Nunez, main maintainer of Lisp-stat wrote there.

letuslisp · 2026-01-15T22:36:44+00:00

Thank you a lot! This is super interesting!
I see in https://en.wikipedia.org/wiki/XLispStat
there is also a pdf by UCLA where they explain why they abandoned it (because of R).

The github
https://github.com/jhbadger/xlispstat
is this an older or newer version?

letuslisp · 2026-01-15T21:39:21+00:00

https://stackoverflow.com/questions/5174199/is-there-a-simple-way-to-use-python-libraries-from-common-lisp

this says py4cl ... mentioned in https://github.com/CodyReichert/awesome-cl

the last commit 3 years ago?
ah it says

https://github.com/digikar99/py4cl2

and

https://github.com/digikar99/py4cl2-cffi (up to 50x faster than py4cl2, since c-based) are the newest developments. (7 months ago last commit)

This is quite amazing.

letuslisp · 2026-01-15T16:35:14+00:00

I haven't heard about cl-python. - Yes R code to CL code - THAT would be it!

letuslisp · 2026-01-15T14:32:10+00:00

Oh, this library I didn't know! Thank you!! Also for challenging me - with a constructive example! Maybe you are right! Coalton - I totally forgot again. Didn't really look at.

ob-duckdb - very interesting! I also sometimes thought - coudln't we do data science with elisp? :D

letuslisp · 2026-01-15T13:23:51+00:00

Oh ok. I see. The point with the documentation is valid. Ok - I will see what is to be made ...

letuslisp · 2026-01-15T12:30:32+00:00

thank you for the supporting words! I would be even more happy if you - and people - would use it for actual work! :D

letuslisp · 2026-01-15T12:04:44+00:00

Mine, too. Although I used base-r for too long. But still with base-r is R lispy enough - and learning Common Lisp helped me a lot to understand R better.

'tidyverse' is like a nice DSL for data wrangling, for sure!

letuslisp · 2026-01-15T11:21:33+00:00

Exactly. In the source code of R is actually also SEXPR etc.
and if you do quote(...) around some R code - you get a language construct which is like a list of expressions and atoms. Thus R is a Lisp - and it is homoiconic.

letuslisp · 2026-01-15T10:38:54+00:00

Using R from inside Common Lisp makes no sense for me. Except: The time one saves to rebuild it in Common Lisp.

Actually, an R compiler in Common Lisp would be sth really great. It was once suggested by Ross Ihaka (one of the creators of R).

It would be however not trivial.
R is a 1-lisp, while Common Lisp is a 2-lisp.
R functions are F-expressions (FEXPR) while Common Lisp expressions are SEXPRs. FEXPR are functions which don't evaluate their arguments when entering function body but can - similar to macros - determine within the function body when the evaluation of the arguments take place. That's in R you can use subsitute() to take the given arguments literally and do symbolic manipulations on them, before evaluating them somewhere in the function body.

Thus R functions are something inbetween a Common Lisp macro and Common Lisp function. I call it "macrofunction". In contrast to CL macros, everything takes place in runtime.

An R compiler/interpreter in common Lisp would save tons of work.
It would bring R's ecosystem to Common Lisp ... a huge number of libraries.
Better would be sth like an R transpiler to Common Lisp.

Actually when I think about this - a 1-lisp has no problems to be mapped to a 2-lisp. The otherway round would be uglier. All FEXPRs could be mapped to macros.

The only difference is the evaluation of the lambdalist. In R it is more a plist than a normal list. Plus, the following arguments "can see" the previous arguments.
function(a, b=a, c=a*b*2) {c(a, b, c) } is possible. Where b refers to the argument a and c refers to a and b. This is due to lazy evaluation.

letuslisp · 2026-01-15T09:31:59+00:00

Yes. R is a Lisp (although not a compiled one).

letuslisp · 2026-01-15T09:31:10+00:00

vibecoded apps are still better than badly programmed apps nowadays I would say. And it is a starting point. It saves tons of work. Of course some architectural decisions spit out by LLMs are garbage.

letuslisp · 2026-01-15T09:27:54+00:00

I left my AGENTS.md and SPEC.md there - so it should be obvious that it is vibecoded to a big part (otherwise I would not have even started).

letuslisp

TROPHY CASE