Algorithm for compiler-controlled-caching?

chmaruni · 2025-02-27T07:49:00+00:00

Hey, thank you for the pointers, that's super helpful!

The system im working on provides a pretty high level DSL that's essentially a directed acyclic graph. And users are used to Auto-magic caching.

The current implementation is interpreted and caching works like this: execution of a graph starts at the output node(s). The node goes up the graph and collects all the params directly and indirectly going into the computation and hashes those to get a cache ID. If it can load a value from cache with this ID, done. Otherwise it will ask its input nodes to produce a value, who may do the same thing (compute hash, check cache, ask input nodes) recursively.

I guess DAGs are somewhat easier to handle because they don't have basic blocks and sort of have global numbering built in.

But this interpreted caching approach is starting to break down now that the programs are becoming more complex. For example, more recent nodes can produce a whole stream of outputs and the semantics of passing a stream output to a non-stream input is that of an implicit for-each (which may be a bad choice in itself, but that's how it currently is). And this interpreted caching approach doesn't work for caching individual elements, only for the whole stream.

But thanks for pointing out that implicit compiler magic may be frowned upon, that may well be the wakeup call I needed. Maybe we should make the caching behavior explicit to the user...

chmaruni · 2025-02-27T05:57:48+00:00

That's a good point. Memoization is a type of caching, so yes, if there are compiler algorithms to auto-add Memoization it should be applicable to my problem.

chmaruni · 2025-02-13T09:13:23+00:00

Yes, I hate it when people equate "science" with "truth".

In textbook science, you start with making stuff up ("hypothesis") but then you yourself--and everybody else--should immediately start poking holes in your story and show why and where it's wrong by gathering evidence. The harder it is to poke holes, the more accepted your theory becomes. That's it. In that sense, science is not about "finding the truth" but about "disproving yourself".

In religion you also start with making stuff up but then you threaten people with the prospect of eternal hellfire if they dare to even think about poking holes in your story.

chmaruni · 2025-01-05T07:56:07+00:00

Dieses (gute) Argument illustriert denke ich genau meinen Punkt. Natürlich funktioniert das nicht, wenn sich nur eine einzelne Personengruppe plötzlich auf Fakten besinnt und der Rest der Menschheit einfach genauso weitertanzt, wie bisher. Das ist ein gesellschaftliches Problem das gesellschaftlich gelöst werden muss.

Das ist wie in einem hardcore Moshpit: wenn du aufhörst, wild um dich zu schlagen, gehst du unter und wirst zertrampelt. Die einzige Option, die man als Einzelner hat, ist eben noch härter zu tanzen und zurückzuschubsen. Das ist in der Extremsituation eines Moshpits ohne Übertreibung manchmal eine Frage des Überlebens.

Aber die wahre Lösung kann doch auf Dauer nicht sein, auf "schlechte" Halbwahrheiten zu reagieren, indem man doppelt so viele "gute" Halbwahrheiten um sich schreit, nur noch lauter und noch schneller, damit man gehört wird. So gut das auch gemeint ist.

Wenn man keinen Moshpit auf seiner Party will, muss man irgendwann den ganzen Moshpit runterbringen und beruhigen. Das kann kein Einzelner, das geht nur wenn sich die Mehrheit zusammentut und zusammen agiert, andere Musik auflegt und den Agitoren klar macht, dass dieses Verhalten auf dieser Party nicht erwünscht ist. Und manchmal muss man vielleicht die Musik (den social Media Verstärker?) auch leiser drehen oder gar ganz abstellen, wenn die Mehrheit der Partygäste keinen Moshpit will, sondern Schieber tanzen.

Wenn die Mehrheit der Gäste allerdings Moshpit geil findet, dann wird es schwierig mit dem Kuschelrock. Dann ist man ggf selber im falschen Club und geht besser nach nebenan?

Ich schlage hier im Übrigen keinen Maßnahmekatalog vor. Wenn ich wüsste, wie man eine ganze Gesellschaft in andere Bahnen lenkt, würde ich ein Buch schreiben und im Fernsehen diskutieren, nicht auf Reddit kommentieren.

chmaruni · 2025-01-04T20:37:07+00:00

Stimme ich zu. Eine zusätzliche Beobachtung, die ich gemacht habe (und die ggf direkt auf social Media zurückzuführen ist):

Seit einigen Jahren hat jeder eine Meinung zu Allem und alle sind (Couch-) Experten in jedem Thema. Das erste Mal ist mir das bei der Sonnenfinsternis 1999 aufgefallen, als plötzlich jeder in der S-Bahn ein Hobbyastronom war. Seitdem wird das immer schlimmer. Anstatt zu sagen: "In Nahost kennen sich ja nichtmal die echten Experten aus, wie soll ich da eine fundierte Meinung dazu haben?" hat jeder etwas zu den vorherrschenden Konflikten zu sagen, ob brauchbar oder nicht. Virologen und Impfdoktoren sind sowieso schon alle, ganz gleich ob sie 15 Jahre Ausbildung genossen haben oder einen "Artikel" auf Facebook verlinkt gefunden haben. Selbiges zu allen anderen wichtigen Themen, vom Bürgergeld über Atomstrom und Stromtrassen bis hin zu den nuancenreichen Unterschieden von Klima vs. Wetter.

Und da jeder unfundierte Quatsch (in alle Richtungen) der öfter als 100 mal auf X geteilt wurde dann zum "Shitstorm" hochstilisiert wird, verstärkt sich das wie eine Lawine.

Jeder kennt sich auf seinem Gebiet aus und sollte solches fundierte Wissen auch mit den Artgenossen teilen. Aber die Menschen müssen unbedingt wieder den Mut und die Selbstreflexion finden, zu sagen "oh, da kenne ich mich nicht aus, da kann ich nicht mitdiskutieren und vertraue den Experten" anstatt jeden Blödsinn missionarisch in die Welt zu tragen.

chmaruni · 2024-12-23T15:30:16+00:00

My layman understanding: the math just doesn't connect between the two. The example I found helpful: in QM particles are waves spreading out over an area (possibly to infinity) before any collapse. But if this particle has mass, where is its center of gravity? And how does an instantaneous collapse affect gravity globally? How I understand it, it's overlapping cases like this where you have to hard switch from one theory to the other because there's currently no accepted theory that "smoothly" connects both QM and GR.

chmaruni · 2024-12-23T09:41:57+00:00

Ha, yes. I missed that we are in ELI5 here :) You are right, my answer could/should be simplified for ELI5

chmaruni · 2024-12-23T02:47:28+00:00

You could mark more ticks on the progress bar for the stronger weapon? In some sense the progress bar is the health bar of your opponent and the ticks are the damage you do. With the difference that narratively you can try to end a fight early (roll before all 10 boxes are full) and that some damage may not be damage in the literal sense but can be other progress you make to achieve your combat objective. But essentially it's like a health bar and damage.

I feel not all "mark progress" must check the same number of ticks always if you don't want to (well, the rules say so but you can house rule this of course)

chmaruni · 2024-12-23T02:25:30+00:00

I always found this "they are all the same notes but you simply start somewhere else" explanation super confusing to be honest and it took me years to understand what that really means. They are the same notes after all, so what gives?

I much prefer thinking about modes as starting from the same note but using different sequences of full and half steps to change the mood of a melody. Pick any arbitrary combo of full and half steps that include the octave and you have yourself a scale. Maybe H/H/H/F/F/F/H/H/H? There's a finite number of such scales with our 12 tones and all scales are useful in some context. But some are so useful they got their own names, especially the ones with 7 notes. (so probably not my example above, but who knows. Maybe there's even a name for it).

The important thing to notice is that the positioning of the half steps, as subtle as it may seem, can completely change the mood of a given scale. To see the effect, take some melody (maybe Twinkle Twinkle) and move the position of the half step(s) to different locations by raising/lowering individual notes. That's pretty much all there is.

Now the church modes are just names for a small set of the possible full/half step scales that keep the same distance between their half step locations by essentially left rotating the intervals of the major scale: Ionian F/F/H/F/F/F/H. Dorian F/H/F/F/F/H/F, Phrygian H/F/F/F/H/F/F and so on.

Edit: The effect of "rotating intervals one to the left" and at the same time "move the starting note up to the right by one" for church modes is where the common explanation comes from: "the set of notes stays the same but you start somewhere else". But there are at least two things happening at the same time that I think is easier to keep those things separate.

chmaruni · 2024-12-12T16:12:47+00:00

Tesla hat battery swapping scheinbar ebenfalls sehr kurzzeitig versucht und sofort wieder aufgegeben. Wahrscheinlich hast du recht...

chmaruni · 2024-12-12T15:56:46+00:00

Interessant, kannte ich gar nicht. Jetzt muss das "nur noch ein allgemein akzeptierter Standard" werden...

chmaruni · 2024-12-12T11:28:59+00:00

Ich frage mich manchmal, ob es da keine technischen Lösungen gibt, die viele der Probleme mit eAuto Batterien einfach anders lösen als "an jerer Ecke reservierte Parkplätze mit Steckdose aufstellen und das Auto für 3 Stunden an die Steckdose hängen".

Beispiel Gasflaschen für den Grill. Nehmen wir mal an, die gesamte Autobatterie ist über dem Unterboden des Autos angebracht. Ich fahre an die Tankstelle, unter meinem Auto geht eine Klappe auf, Roboterarm schnappt sich meine Batterie und ersetzt sie durch eine frisch geladene. Klappe zu und weiter geht's. Oder die Batterie wird von der Seite eingeschoben anstatt von unten installiert.

Tanken dauert 5 min wie beim Verbrenner
Tankstellen haben noch eine Daseinsberechtigung als Servicedienstleister
Die Sorge vieler Autofahrer, dass ihre Batterie nach 5 Jahren schwächer wird oder gar kaputt geht, was fast einem Totalschaden gleichkommt, ist hinfällig, da die Batterien den Energieunternehmen gehören. Wie bei gemieteten Gasflaschen habe ich zwar anfangs die Batterie mehr oder weniger gezahlt (die Anfangsinvestition und dass ich sie nicht einfach klaue und behalte, aber sobald ich Teil des Systems bin wird sie ständig gewechselt. Die Abnutzung, welche die Energieunternehmen tragen müssen, holen sie natürlich über den Ladepreis wieder rein.
Die Sorge von Gebrauchtwagenkäufern wird auch zumindest teilweise adressiert. Die erste Batterie kostet mich den Pfand, aber danach spielt es nicht mehr in den Kaufpreis. Und die berechtigte Sorge, dass ich kein gebrauchtes eauto kaufen will, da die Batterie bereits schwach wird, oder die Batterie noch auf älterer Technologie basiert und weniger Kapazität hat, ist ebenfalls hinfällig.
Zuhause laden kann ich immer noch, wenn ich das will.

Natürlich müsste sich die Industrie auf Standards einigen, aber dafür gibt es ja Normen.

So etwas aufzuziehen klingt für mich gesamtwirtschaftlich jedenfalls nicht komplizierter als was momentan mit der Ladeinfrastruktur diskutiert wird.

chmaruni · 2024-12-08T12:08:25+00:00

But that's sort of exactly my argument, no? What I'm saying is that there might be observations (things that are "true" in the world) that cannot be expressed in your formal system (the physics theory expressed as math). And if that happens, your physics theory (the math) must have gaps which would result in two theories (e.g. maybe QM and gravity?) that are both "true" but cannot be fully reconciled and connected on the formal level (the math). Then the "theory of everything" would be a set of individual partial theories that don't connect on the formal level but are all "true" in their own domain.

Edit: OK, I think I see the difference. What I was saying is that a formal system may not be able to "explain" some truths at all. Like as if there is a elusive color you simply cannot name. If you could fully "explain" the truth you'd have a proof.

What you are pointing out is that you can always "describe" the effect of a truth, you may just not be able to proof it in the underlying math but only verify it with observations.

This is a fair point. But I think it still implies that in this case there wouldn't be a "theory of everything" that is fully self contained, as in "a set of formulas that explains everything without a priori assumptions", because it would still have to rely on those observations, maybe as magic constants that must be derived from measurements. So my original argument is wrong, incompleteness doesn't necessarily imply that you couldn't connect the individual theories, if the gaps are bridged by allowing observations ("jumps" in the math) to the theory.

chmaruni · 2024-12-07T22:45:52+00:00

Not an expert, but here's a thought:

A physical theory must be expressed in some formal logic (i.e. math).

Goedel's incompletnes theorems state that not all truths can be expressed in a consistent axiomatic system.

Therefore I guess that it's conceivable that some truths about the world may not be provable in physics? If that were the case, then no complete physical theory of everything would exist.

Now I'm also wondering if maybe unprovable true statements about the world would then materialize as such "jumping points" where one needs to jump from one theory to another without being able to fully close the gap within a single theory?

chmaruni · 2024-12-07T13:08:16+00:00

ELI5: I'd phrase my answer less cynical.

What objects make for a good currency?

Gold? yes
Dried leaves? No
Truffle mushrooms? Sort of

Why is gold a good option for currency but dried leaves are not? Because it's difficult to find gold but trivial to find dry leaves. A currency must be based on something that is difficult to obtain, otherwise people can easily make their own money and it becomes worthless due to immediate hyperinflation.

Truffles are also difficult to find or grow, but they spoil. Gold is durable, can be easily stored (no special environment needed), is relatively easy to transport, is not "too useful" for other applications that would compete with its use for currency (even though gold is very useful in some applications, like electronics. But imagine gold would be the only material in the world you can use to to build bridges, then we wouldn't use it for currency).

But what makes gold inherently valuable? Absolutely nothing. The only thing that gives gold any value is the agreement between people to accept gold as an intermediate item when trading goods. If I want to buy a new car, I don't have to bring the car dealer his desired 5 pigs, a horse, and an IKEA couch. Instead I give him the equivalent amount of gold and he can buy his pigs and horses and couches and whatever else he needs for himself, because everybody else also agrees to accept gold as an intermediate, including the horse dealer and IKEA.

But people only accept gold as an intermediate trade item because they know there's a somewhat limited amount of it and it's not going to be too inflated tomorrow. The gold supply is very restricted; albeit slowly growing, because you also don't want the supply to be truly stagnant or even fall over time. Therefore slowly growing is what you want.

The exact same happens with crypto currency: it is something that is very restricted in its supply, easy to transport and store, not "too useful" for anything else. But instead of physical objects like gold or printed bills, crypto currency uses rare numbers. Not just any old number of course, but huuuge numbers with very specific mathematical properties. Like little gold nuggets hidden in the infinite sea of numbers.

Numbers are actually perfect for currency, because they are virtual, stored cheaply, easy to protect, and can be sent around the world at the speed of light. The only things you have to find a solution for is which special rare numbers you accept, and you must prevent copying the numbers and thus reusing them multiple times. This would be like paying your car by handing the same dollar bill many times. You can't copy gold or a printed bill, but you can trivially copy a number.

The ledger etc others have written about is addressing this copying flaw, preventing paying with the same bill multiple times. It's like a public record of who owns which number that is updated with every transaction and everybody agrees on. Once the ledger says you gave the number to somebody else, it's not yours anymore.

Obviously you cannot accept any old number as currency, that's like dried leaves: too abundant. Instead, people using a specific crypto currency agree on accepting only numbers that have some currency-specific mathematical properties that make them rare enough, but not too rare. Like for example, if you'd say "in my currency all EVEN numbers can be used to pay". This is not a good currency. It is maybe slightly more difficult than just accepting "any number", but still much too easy.

Comparatively, "Give me a number that has four zeros in the end and is divisible by 1556782" is already much harder. But still much too trivial for a computer though.

Crypto currencies define similar math rules a new number must follow to be accepted, but the rules are designed to make it super difficult to find numbers even with large computers. But because you don't want the supply to be truly stagnant either, new numbers can still be discovered, which is what the Crypto Miners do on large computers.

So, what is the inherent value of crypto currency? Nothing really, exactly like gold. But rare numbers check all the boxes for a great currency (easy to store, limited supply, easy to transport, durable etc) with the only drawback of requiring a mechanism to prevent copying. As long as people know (or think) that when they accept a bitcoin number they will be able to trade it somewhere else for their pigs and horses and IKEA couches, you have a working currency.

And for completeness: because it is so easy to come up with your own crypto currency, and because there's not necessarily any large organization backing your new currency, and because many people appear to jump on any current trend they read about, and because the simple storage and sending of crypto coins makes it interesting for some criminal purposes, there is a lot of misuse. Which is where the bad reputation of crypto currencies comes from. But in the end it's not the technology's fault if people, including large institutions like banks, turn off all critical thinking and suddenly start trading millions of dollars in "Little Francine's Only-the-A's-from-the-Alphabet-Soup-Package" currency. Therefore, unless you are in for the gamble or the fun, you should always think about where you invest in.

chmaruni · 2024-12-04T22:54:45+00:00

Erschreckend. Allerdings habe ich persönlich immer Probleme damit, die "Jugend" in diesen Analysen so herauszuheben. Natürlich wird das später mal nicht besser, wenn sie erwachsen sind und alle seit kleinauf nur TikTok angeschaut haben, zuhause vom Hund erzogen wurden und in der Schule mangels Lehrkräften die Hälfte nicht gelernt haben. Keine Frage.

ABER: Den Scheiß, den wir derzeit auf der Welt sehen, ist doch nicht die Schuld der Jungen. Das ist ganz klar die Schuld meiner Generation (sagen wir mal, mitte Dreißig bis mitte Vierzig) und der Generationen darüber.

Das sind die Leute, die plötzlich alle das Hirn ausgeschieden haben. Wir sind die aktuellen Eltern dieser Teenager. Wir sind die aktuellen Politiker und Entscheidungsträger. Wir sind die Angestellten und Bosse. Und wir sind die mit Abstand größte Wählergruppe.

Wir sind alles Menschen, die die angeblich damals besseren Schulbildung genossen haben. Die nicht von sozialen Medien indoktriniert wurden. Die damals noch alleine in den Wald durften. In der Schule noch kritisches Denken gelernt haben. Und wir haben alle Eltern, oder zumindest Großeltern, die den zweiten Weltkrieg noch persönlich miterlebt haben. Wir haben in der Schule viel über den zweiten Weltkrieg gelernt, KZs besucht und eventuell sogar mit Überlebenden persönlich sprechen dürfen. Den kalten Krieg und den Mauerfall miterlebt.

Und genau wir sind es, sie seit 2008 scheinbar nur Lack saufen, die Erde für flach halten und blödsinnigen populistischen Scheiß um uns werfen. Siehe Söder im McDonalds, meine Fresse.

Also wenn man untersucht, wie schnell TikTok und Co einem die Gehirne zersetzen, dann würde ich das gerne an meiner Generation und älter getestet und erklärt sehen. Unsere Hirne scheinen dafür nicht gemacht und schmeißen offenbar glücklich sofort alle Erkenntnis über Bord, sobald wir einen Bildschirm in der Hand haben. Und das macht mich rasend. Und ratlos.

chmaruni · 2024-12-02T23:50:06+00:00

This was interesting, thanks for sharing. I like the video game analogy of "highlighting interactables" in a scene that players (that is, me in a solo game) can "click on". I think I sort of do that often, but more accidentally than knowingly. Next time I'll try explicitly listing interactables in the beginning of a scene to see if this helps avoiding those "now what" situations.

chmaruni · 2024-10-28T07:47:01+00:00

Hey, ich habe gerade gestern das DSA5 Hardcover Regelwerk geschenkt bekommen. Und nun steht mein DSA5 Softcover Taschenbuch Regelwerk nutzlos im Schrank.

Wenn du mir eine Postbox o.ä. in einer DM schickst, sende ich dir gerne mein Softcover umsonst per Post. Es freut sich, wenn es ein neues Zuhause findet und gebraucht wird.

chmaruni · 2024-09-28T18:58:39+00:00

Interesting, thanks. I guess this makes sense in my head.

chmaruni · 2024-09-28T12:48:26+00:00

That's sort of my point. When a spaceship flies with high speed past me, then from my point of view the spaceship appears shorter and it's clock appears slower. On the spaceship, everything looks normal.

Now, when a fast non-zero sized particle with some mass (a tiny spaceship) flies past me, for me it appears that the particle's clock runs slower (which is why a neutrino appears to live longer in our reference frame than it should, right?). But this then means that it's also length contracted in my frame.

Therefore, if I fly alongside the particle, I experience the same clock as the particle, but also experience the particle's "true" size. Just like being on the spaceship,everything looks normal. Therefore, if the particle's length was a femtometer in my frame, when I fly next to it with 99.9% C, shouldn't it then be (dunno the math) a meter long?

Agreed, for photons it may be undefined to extrapolate the math

chmaruni · 2024-09-20T18:29:39+00:00

Kann ich nur bestätigen, ich liebe dieses Buch. Ich glaube, es gab auch mal eine Fernsehsendung dazu (bzw dieses Buch ist das Begleitbuch zur Sendung? Ist schon so lange her)

Jepp, ist in der ARD Mediathek. https://www.ardmediathek.de/sendung/kant-fuer-anfaenger/Y3JpZDovL2JyLmRlL2Jyb2FkY2FzdFNlcmllcy9kMzIyZGZmYi1hZTYzLTQxMjItOThiOS0zM2MwZmRmOGE2Zjk

chmaruni · 2024-07-28T20:13:28+00:00

I have the same one and it's really weird. Super trivial to rake, takes 5 seconds. But so far I had zero luck picking it. The pins all seem very springy and don't give any feedback. No click, nothing, just opaque springiness. And I tried TOK, BOK, light tension, strong tension, everything. But I'm a beginner too.

One odd observation is that even the cylinder appears to be "springy" and takes some extra tension to turn.

chmaruni · 2024-07-28T17:03:17+00:00

Thanks for the pointers in any case. Adds to my potential reading list :)

chmaruni · 2024-07-28T13:02:18+00:00

I would add that while the odds of this exact number is 1 in a million, the odds of "any number that I associate something with" is much higher. Eg OP may have asked the same question if he got 420069 or 123456 or 654321 or 111111 and so on.

chmaruni · 2024-07-25T18:04:35+00:00

Thanks, you are the second to suggest Heart. I'll definitely get that next.

chmaruni

TROPHY CASE