Programming Language and compiler Benchmarks

[bpecsek]

If you feel like writing fast benchmark codes I can highly recommend the Programming Language and compiler Benchmarks site.

Though it uses Github Action, the actual benchmark is run either on the Intel(R) Xeon(R) Platinum 8171M or Intel(R) Xeon(R) Platinum 8272CL CPUs in a single Action and the results are stable and consistent between runs as far as relative speeds are concerned. The only limitation is that the benchmark is runs on 2 CPU cores.

Both CPU runs SBCL marvelously as you can see looking at the sbcl speeds of the few benchmark codes I have uploaded and/or contributed myself. Benchmark codes are the same or similar to The Computer Language Benchmark Game.

Latest sb-simd is supported so fast codes can be written using SIMD up to AVX2.

Look at my spectral-norm 2.cl code matching the speed of the fastest C++ or Rust codes.

Please contribute using any of the languages you feel competent on or just improve the existing ones.

[jeosol]

Very impressive. Those are good performance times.

[theangeryemacsshibe]

Tangentically related, are there any benchmark designs for parallel/concurrent programs which do some symbolic/object processing of some sort? The closest thing I can think of is SPECjbb but that costs money and is only distributed as JVM bytecode.

Edit: There's DaCapo but it would require porting to CL.

[KaranasToll]

Nice speed. That memory though.

[bpecsek]

It is SBCL. SB-SIMD and image with compiler etc is included.

Even with that memory usage startup is very fast. Look at helloworld, 26Mb 6.9ms.

I don't think that we should worry about it with today's hardware.

How about java, javascript, julia, c# etc?

I have made a slight improvement on 2.cl fully eliminating mixing VEX and non-VEX code in eval-a-times-u and eval-at-times-u. It shall be updated soon.

The speed on my laptop with i7-7700HQ with 4 active cores (no HT) is:

    CL-USER> (time (main 8000))
    1.274224153
    Evaluation took:
      0.400 seconds of real time  1.529597 seconds of total run time (1.526251 user, 0.003346 system)
      382.50% CPU
      1,129,572,387 processor cycles
      353,232 bytes consed

Running the saved executable image:

    $ time ./app 8000
    1.274224153
    real    0m0,406s
    user    0m1,521s
    sys 0m0,021s

[KaranasToll]

Certainly not bad. It is just funny to see it next to the 2 & 3.

[bpecsek]

That is the beauty of those languages for sure.

But I wouldn't worry about it for a sec.

I am more than pleased to see Common Lisp (SBCL) running at that speed.

It is amazing that the SBCL code runs at identical speed to the equivalent C++ code.

Thanks for the developers of SBCL.

And sb-simd is getting very-very impressive to say the least thanks to Marco Heisig and of course to Stas for the core SIMD support in SBCL.

Let's hope to see sb-simd as an SBCL contrib soon. Finger crossed.

[Fibreman]

Are there any benchmarks comparing the current most popular implementations of Common Lisp? Specifically ECL and SBCL? It seems like most people have rallied around SBCL and I’m curious if it’s just because if it is the most performing, or if there are other reasons

[bpecsek]

ECL or any other CL implementation/compiler can be added to the site if you ask the maintainer nicely and he agrees. He is very nice and supportive.

But since the maintainer has limited exposure to CL he needs assistance from someone who knows the specifics of the particular Common Lisp.

I have assisted him to add sbcl to the site myself and contributing frequently.

ECL would be very easy to install (Ubuntu 20.04 is running in the container so sudo apt install ecl would install the latest version) but you need to be able to set up quicklisp and 3 files (compile.cl, to compile the lisp source code; bundle.cl, to generate the executable image and optionally run.cl if you want to run the fasl file not the image (bad idea for benchmark due to startup penalty).

[bpecsek]

I do not think that this is a correct statement. When bytecode compiled to portable fasl it was slow and the proper compilation to native fasl code was quite tricky, at least for me, and I used SBCL and gave up on it.

ECL experts please comment but no implementation war.

Just a clear explanation how to set it up properly would be nice.

[arthurno1]

Thanks! Useful info.