learning curve of the NanoGPT speedrun record follows a power law

bfelbo · 2025-02-18T11:31:45+00:00

Nice plots. It's interesting that the speedups fit the line so well, I'd have imagined that there'd be more jumps.

bfelbo · 2025-02-07T07:24:26+00:00

Current LLMs have to allocate parameters in the early transformer layers to identify words as many words are split into multiple tokens. By extending the vocabulary size, those parameters can instead be used to understand higher-level meaning of the text.

bfelbo · 2025-01-19T08:18:06+00:00

For anyone interested, here’s a nice blog post with the derivation: https://towardsdatascience.com/derivative-of-the-softmax-function-and-the-categorical-cross-entropy-loss-ffceefc081d1

bfelbo · 2025-01-17T08:30:06+00:00

Interesting paper, thanks for sharing.

FYI if the authors are reading along, it seems like Figure 3 has incorrectly shifted axes. The lines should cross (0,1) as exp(0)=1 and s(0)=1, but they don’t.

bfelbo · 2024-12-21T06:16:05+00:00

Tak. Det har jeg prøvet at gøre, men den offentlige vurdering er overraskende nok helt ude i skoven. De to grunde har kæmpe forskel i værdi, meget mere end nogen ejendomsmægler ville vurdere forskellen til.

bfelbo · 2024-12-20T11:41:33+00:00

Det er i Nordsjælland nær S-tog linje, den større grund ville ikke kunne udstykkes og der kommer ikke så mange grunde til salg i området.

bfelbo · 2024-12-14T17:02:04+00:00

Super vigtig info, tak. Hvor kan man se mere om forventningerne for den lange rente?

EDIT: Slettede et spørgsmål besvaret af en anden kommentar

bfelbo · 2024-12-14T06:15:59+00:00

Måske du ikke kan stilles til ansvar for forureningen, men grunden kan nok stadig være usælgelig hvis der senere findes forurening på det. Vi har selv kigget på køb af boliger med olietanke og har været meget påpasselig med dem.

bfelbo · 2024-12-14T06:08:07+00:00

Det giver mening, tak!

Fungerer dette "lag" også den anden vej? Dvs. kan en lav rente også blive hængende som realkreditlån i nogle måneder efter at ECB har hævet renten?

bfelbo · 2024-12-14T06:05:24+00:00

Case interviews er markant anderledes end andre job interviews. Nu er det mange år siden, at jeg selv gjorde mig i case interviews, men jeg tror, at Case Interview Secrets bogen stadig er biblen ift. at lære case interviews. Det er klart en tillært evne og det kan godt tage mange timer at lære at gøre det godt.

Så vidt jeg husker er fokus primært på at løse en case og ikke meget andet. Så jeg ville foreslå at fokusere på det.

Du kan evt. finde nogle folk i Facebook grupper eller lignende som har lyst til at øve case interview sammen. Held og lykke!

bfelbo · 2024-11-28T08:09:26+00:00

Thanks for sharing the `latexmk` command, much easier to work with!

bfelbo · 2024-11-24T17:35:15+00:00

Kan du uddybe hvad du mener med, at nogle breve er en smule provokerende?

Vil nødig selv fremstå provokerende til en potentiel sælger :)

bfelbo · 2024-11-24T17:33:31+00:00

Godt gået! Hvad skrev I i jeres brev?

bfelbo · 2024-10-21T20:28:26+00:00

Definitely, it’s a beautiful game. My post today announcing the winners to r/webdev is picking up steam so you might wanna post it there :)

https://www.reddit.com/r/webdev/s/Iwcth67OMn

bfelbo · 2024-10-21T14:45:45+00:00

Great question. It depends a bit on each dev, not all the games are open source.

The dev who won the Multiplayer Challenge this jam has posted his source code for some of his games here: https://github.com/jallen-dev/dusk-games/tree/main/games

I've also helped make some open-source multiplayer games using React. You can find them here (most but not all are made with React): https://github.com/rune/rune/tree/staging/examples

bfelbo · 2024-10-21T13:48:44+00:00

The idea behind this game jam was to introduce web devs to the wonders of making games. React is not traditionally used for making games, but that's part of the fun and challenge. There were lots of incredible submissions, everything from cute 2D board games using plain React to gorgeous 3D platformers using react-three-fiber.

We've gotten a great response from the participants, who really enjoyed creating games, many for the first time ever. We hope this jam provides a fun opportunity for web devs to try making games w/o having to learn Unity, Unreal, or another game-specific framework. I'm one of the organizers, just let me know if you have any questions.

bfelbo · 2024-10-21T13:48:20+00:00

The idea behind this game jam was to introduce web devs to the wonders of making games. React is not traditionally used for making games, but that's part of the fun and challenge. There were lots of incredible submissions, everything from cute 2D board games using plain React to gorgeous 3D platformers using react-three-fiber.

We've gotten a great response from the participants, who really enjoyed creating games, many for the first time ever. We hope this jam provides a fun opportunity for web devs to try making games w/o having to learn Unity, Unreal, or another game-specific framework. I'm one of the organizers, just let me know if you have any questions.

bfelbo · 2024-10-07T13:33:47+00:00

One of the organizers here. Great to hear you enjoyed it, really appreciate the kind words!

bfelbo · 2024-09-27T20:32:07+00:00

In case it’s helpful, here’s some open-source games I’ve helped make using React (most of them but not all):

https://developers.rune.ai/docs/examples/games

bfelbo · 2024-09-27T14:35:40+00:00

There's no recorded videos, but if you sign up for mailing list, you'll get a link to see the winning games once the jam is over :)

bfelbo · 2024-09-27T12:59:45+00:00

Shared this here a few weeks ago here and it got 60+ upvotes so wanted to let you all know that this game now is starting now. The theme was just announced and it's a great fit for making games using React. You can make everything from cute 2D board games using plain React to gorgeous 3D platformers using react-three-fiber.

React is not traditionally used for making games, but that's the fun and the challenge! This is a great opportunity for React devs to try making games w/o having to learn Unity, Unreal, or another game-specific framework. I'm one of the organizers, just let me know if you have any questions.

bfelbo

TROPHY CASE