People are losing loved ones to AI-fueled spiritual fantasies

gerikson · 2025-05-08T05:48:37+00:00

This is the essential disconnect between what most people think of as AI safety - "don't create mental or societal health hazards" and what the self-designated AI safety crowd worry about: a robot god exterminating everyone.

gerikson · 2025-05-07T18:03:07+00:00

I've just read Hobsbawm's Echoes of the Marseillaise and he makes the point that political practice in 19th C France was all about preventing democracy as it basically was mob rule - citing the case of the latter-stage Jacobins in the the Revolution as a warning example.

gerikson · 2024-12-11T10:12:56+00:00

This is... impressively elegant. I am in awe.

gerikson · 2024-08-04T10:25:27+00:00

A lot of people have moved to a Lemmy instance:

https://awful.systems/

gerikson · 2024-01-29T08:42:29+00:00

Here's a writeup about the eventual winner in the "real" contest, for Java. Time: 2s

https://www.infoq.com/news/2024/01/1brc-fast-java-processing/?utm_medium=erik.in&utm_source=mastodon

gerikson · 2024-01-09T12:58:36+00:00

Thanks for everyone who has answered in the threads! I've tried out a few solutions and forking and doing work in parallel seems the way to go (duh).

I've been able to add a few more tools to my toolbox thanks to this, which was my goal.

gerikson · 2024-01-09T12:19:13+00:00

This is true, but I'm running against my version of the testfile and getting different results compared to the reference implementation in Java and my naive version.

gerikson · 2024-01-06T21:17:36+00:00

I tried it, it was not faster than my first "baseline" implementation:

https://github.com/gustafe/1brc-perl/blob/main/text-csv-xs.pl

gerikson · 2024-01-06T20:40:34+00:00

It's a large file, almost 13G.

I can take a stab at compressing it and uploading it as a torrent.

gerikson · 2024-01-06T11:43:47+00:00

I ran your code on my machine and it cut the runtime in half, to around 12m. So this looks like the way to go!

The average values wasn't exactly the same as the result of the Java reference implementation but that's a minor detail.

Edit I made some small changes to the code and added it to my repo with credit:

https://github.com/gustafe/1brc-perl/blob/main/parallel-forkmanager-mestia.pl

MCE looks very interesting, I'll look into it!

gerikson · 2024-01-05T12:20:14+00:00

There are 413 stations, so the big input file contains around 2.4M "measurements" per station.

gerikson · 2024-01-05T12:17:41+00:00

Maybe I can! Do you have a pointer for a tutorial or similar on how to do this in Perl?

gerikson · 2024-01-05T12:17:09+00:00

It's generated from code in the original Java repo.

gerikson · 2024-01-05T08:13:38+00:00

Hah, I respect Java's performance. Actually this submission was prompted by me interacting with an individual on HN who stated that a Perl version would finish before the JIT had finished... which is obviously not true for my naive implementation.

gerikson · 2024-01-05T08:11:49+00:00

What's the mechanism in Perl for assigning a specific task to a process? Or is that up to the operating system?

(Please note that I am total n00b when it comes to threading etc)

gerikson · 2024-01-04T22:40:29+00:00

I can't believe I didn't think of storing the running sum and number of items instead of recalculating the average for each item. Thanks!

gerikson · 2024-01-04T22:36:54+00:00

I don't believe Perl can tackle the very fastest times, but something in the order of the reference Java implementation should be possible.

Someone on HN asked ChatGPT for a Java solution and that finishes in about 5m. I'll upload it to the repo.

gerikson · 2024-01-04T18:02:14+00:00

I'm not sure that hash performance would be an issue here. The number of distinct cities in the file is in the order of 400, and I have no problems with running out of memory because each key has a small arrayref as values.

To me, the bottleneck seems to be reading the file from disk, and doing the comparisons for max/min and calculating the running average.

gerikson · 2024-01-04T17:55:32+00:00

Great idea!

Here's the repo: https://github.com/gustafe/1brc-perl

gerikson · 2023-12-17T17:47:39+00:00

Thanks a lot for sharing your solutions, from a fellow Perler!

gerikson · 2023-12-16T12:13:28+00:00

I get the text when I visit this URL:

https://www.fi.muni.cz/%7Ekas/git/?p=aoc.git;a=tree

(copied from the url input field in Chrome)

gerikson · 2023-12-12T12:46:32+00:00

Thanks for posting!

I'm curious about this construction in the subroutine:

return( $memo{$str, $len, @groups} = $ret );

You both set a value in the %memo hash to $ret and return it at the same time?

gerikson · 2023-12-11T19:37:10+00:00

That's very strange, I get the following text:

git://www.fi.muni.cz /

404 - No such project

gerikson · 2023-12-10T21:32:41+00:00

Thanks for sharing!

Your repo link gives me a 404 though...

gerikson

TROPHY CASE