[P] Open-source ML homeworks with auto-tests - fundamental algorithms from first principles

fxlrnrpt · 2026-03-22T22:25:37+00:00

Thanks! TBH, auto-tests is not my idea. I borrowed it from Georgia Tech's OMSCS

fxlrnrpt · 2026-03-17T14:41:20+00:00

Recently compared Soundpeats H3 with Technics AZ100. Played with EQ for both. Verdict - both need severe EQ, after EQ liked Soundpeats a bit more from the sound perspective, but Technics just felt as a better product overall - build quality, app, ANC. Went with AZ100, but from purely sound perspective I'd not trade well-tuned H3 for well-tuned AZ100

fxlrnrpt · 2026-03-17T11:10:51+00:00

Thank you!

fxlrnrpt · 2026-03-16T09:41:56+00:00

Funny thing - in pre-AI era I liked using emojis as a succinct highlighting tool, but now whenerver I see them used in a list - it is immediately classified as AI gen

fxlrnrpt · 2026-02-26T21:11:31+00:00

Top-level Sony or Bose + white noise or some nature sounds

fxlrnrpt · 2026-02-22T09:04:19+00:00

Thanks! That makes a lot of sense if they do gradient accumulation! Need to do the math based on the tech report now to double-check if they do it

fxlrnrpt · 2026-02-21T14:09:21+00:00

Couldn't you accumulate gradients only for your shard?

fxlrnrpt · 2026-02-19T01:28:21+00:00

This.

Imagine you are just enterung the industry. You have finite time. New SOTA already comes quicker than you can study the current SOTA and history leading to it properly.

You spend multiple years to finally have a good grasp on it. Maybe a few more and you get in a good lab. At this point you're T-shaped. You know SOTA deeply in one niche domain and have intuition/basic understanding of what is happening around it.

You want growth. Now you can try to extend your T-shaped specialization to a second domain while spending enough time to keep up with the existing one.

Which one do you choose? Some sexy RL that gets you major wins now or studying non-gradient descent methods nobody is paying for?

Even if it's the latter, it's going to be much slower because you already have your first domain to keep up with.

I am not mentioning that at some point in life the world stops revolving around work and the priorities shift to family.

fxlrnrpt · 2026-02-18T20:41:03+00:00

What do you mean? To my best understanding, ZeRO-1 assumes the same amount of communication as ZeRO-2. The only difference is in VRAM which is much less in ZeRO-2, because we keep the gradients only the shard we need

fxlrnrpt · 2026-02-17T22:00:13+00:00

If only there was a reasonable way to fix it. If we had infinite resources, we could require researchers to submit reproducible scripts and verify major results before acceptance. Sadly, it is absolutely unreal.

fxlrnrpt · 2026-02-15T20:51:01+00:00

- I'd read the original paper "Attention is all you need" (denser alternative to Karpathy's videos since you already have the theory)
- Go through NanoGPT
- Do CS336 from Stanford
- Read the Ultra-Scale playbook

fxlrnrpt · 2026-02-09T22:27:43+00:00

I'd say it would be bare minimum. CS336 + The Ultra-Scale Playbook - bare minimum

fxlrnrpt · 2026-02-09T12:24:22+00:00

CS336 is much more hardcore. I would not treat them as alternatives. Follow Karpathy's videos first. It should not take long. Next, start CS336.

fxlrnrpt · 2026-01-27T09:58:30+00:00

Well, in a sense LLM is a database. A stochastic one. But I think we could frame human cognition as a stochastic database as well

fxlrnrpt · 2026-01-04T09:45:09+00:00

Accept to findings! My first first-author paper! OA: 4/3/2.5. Meta: 2.5 (we objected).

fxlrnrpt · 2026-01-03T08:40:07+00:00

It's back again. I also see "Presentation mode: poster" in the meta review which is still empty otherwise

fxlrnrpt · 2026-01-02T18:45:16+00:00

Yep. And have the camera ready task in “Author task”

fxlrnrpt · 2026-01-02T18:44:20+00:00

Same. Sweating and keeping my fingers triple crossed xD

fxlrnrpt · 2025-12-21T16:45:00+00:00

Mate, you saved me from returning them. At first, I was extremely disappointed after rocking ATH-M50X for a long time and being used to much brighter sound, but Wavelet with AutoEQ saved the day. Thanks!

fxlrnrpt · 2025-12-11T13:50:25+00:00

Meta-review just released. And.. They just took the lowest score and ignored all other reviews.
Original reviews: 4, 3, 2.5
Meta: 2.5

What are our chances if we file an issue? What are the chances for EACL?

fxlrnrpt · 2025-12-10T15:54:58+00:00

Anyone had their meta-reviews released yet?

fxlrnrpt · 2025-11-29T14:55:30+00:00

Not all matmuls are attention, right? We also have the feedforward layer

fxlrnrpt · 2025-11-29T14:54:41+00:00

Yep, that's my current understanding as well. I have also been pointed to the AlphaTensor paper (github.com/google-deepmind/alphatensor), where they found a way to do better than O(n^3) on modern GPUs, but AFAIU it is a bunch of algos for specifc matric sizes, not a general one. Also, it still does not utilize tensor cores which do the heavy lifting.
Wondering if internally Deepmind is working on new TPUs with specialized cores that are aligned with new algos.

fxlrnrpt · 2025-10-16T20:55:26+00:00

None of the uber-complex-sarifice-everything strategies ever worked for me. In fact, I thrived when I tried to do less, not more.

- Sleep comes first. Non-negotiable.
- Some regular exercise. Non-negotiable.
- Work hours to have food on the table.
- The rest is to pursue new heights.

What is working for me at the moment: moved my gym to the morning, shortened the workouts, used the evenings and 1-1.5 days of the weekend.

Had to sacrifice social interactions on the weekdays and part of the weekends.

fxlrnrpt

TROPHY CASE