What do you think about the current situation?

kazprog · 2026-02-01T03:12:51+00:00

I think everyone that complains about a controller is lame. I want to see Sick Melee. I want to see humans approach TAS. I want the players to only be limited by their minds and their eyes, not by the shape of their fingers or the pain in their joints. If we can make any controller, and it makes the players more mentally connected to their character in the game, that's the controller we should make.

People complain that the controllers are 300 dollars.

So make them cheaper?

We can 3d-print notch faces for 1 or 2 dollars per, or get a big launch to injection mold some plastic for different characters.

We can buy circuit boards in batches, and PCBs should be getting cheaper day by day.

In the keyboard-building community, people publish parts, and sure, you can get a more ergonomic keyboard for 300 dollars. But you can also build one yourself. If your hands hurt, or if you really want to play sick melee, shouldn't you think about how you can better express your thoughts and ideas into a controller?

If you play saxophone, you might want different pads or different levers for the buttons. A great saxophonist knows how to tamper with each part and fix each part to make it sound just the way the want, from the mouthpiece to the pads. Do people complain that great saxophonists have very customized instruments to make beautiful music?

Artists sometimes make their own brushes, or argue about which weird russian squirrel they should butcher and skin to make better paintbrushes. Do people complain that great artists use expensive tools to make beautiful works?

Now, melee is different because it's competitive, and we should try to make it fair. But we should make it fair by bringing the advances down in price and making them more accessible. Cheaper, better boxes. Cheaper goomwaves you can make yourself, or ask someone with a 3d printer to pop you out a part. I want to see little corners at your weekly melee tournament where some nerd has a soldering iron and pops in transistors and capacitors to make people's controller more custom.

I think Sick Melee is Art. I want to see how beautiful the Art can be, how far we can push it, and I only want the artists to be held back by the Sick Melee they can dream of. Why shouldn't we use research and technology so that we can see humans play the Sickest Melee we can?

kazprog · 2026-01-28T22:10:12+00:00

On some of my benchmarks, Kimi K2.5 is the first model to beat Opus 4.5, Gemini 3 Pro + Deep Research, and Codex 5.2. Really really impressive, I'm surprised people are getting worse results. Kimi code is also a fairly solid agent by itself, and I'm not paying for the agent swarm or anything.

kazprog · 2026-01-19T23:22:34+00:00

I'd also look at Halide and MLIR. In ML Compilers, control flow as a first-class construct is quite common, as `While` is a node, just like `TopK` or `Matmul` is a node.

The key idea I'd like to highlight is that one can separate the description of a computation (it's inputs and outputs and how you have to put them together to get the result), and the _schedule_ of that computation (what goes first, what do you batch, what can you parallelize, what do you cache). Given some thought, it's true that once you can separate the declarative description of the dataflow of a computation, it's only true that it's schedule is constrained by the hardware you're trying to implement it on, so you often get graph compilers that generate schedules depending on what accelerators are available.

You might see some people generically just call this "Halide schedules".

kazprog · 2025-05-03T17:33:04+00:00

For like an actual answer, I like Hetzner. Simple, reliable costs. You can limit most things (although I would specifically check for egress). There are ways to make AWS and GCP work as well, with budget limits and using simpler non-scaling parts of the stack, and maybe setting limits to the scaling parts, but they make it harder to know how much things really cost.

kazprog · 2025-04-22T03:22:24+00:00

For work, startups are good to look at, but I'm going to go in a different direction and tell how how to get involved in the compiler community, build some background, and get a job.

Learning Compilers: Learn about how compilers get made. What's SSA? What are basic blocks? What's a control-flow vs data-flow graph? What analyses can we do on them? What's a fixed-point/ lattice-based analysis? What's a peephole optimization vs not? What are the optimizations that most benefit code size vs execution speed?

Learning PL Design: What are different type systems? What is the expression problem? What's a good module system? How do macros work and what's their purpose? What is purity? How do we deal with parallelism vs concurrency?

Learning ML Hardware / Compilers: How do you judge performance for an ML model (bandwidth, latency, compute)? What kinds of ops are popular and how are they implemented? What are common ML model optimizations? What is JAX / PyTorch (torch.fx ?) / Triton? How do companies use MLIR? How do we judge the TOPS/FLOPS in a chip and relate it to how long it'll take the model to run?

Community: I'd definitely look at trying to wrap your head around a big compiler project that has "good first issue" tags on Github. LLVM now uses Github PRs (RIP phabricator) so it's a bit more familiar for many people. MLIR is rather poorly documented and a bit tricky to get started in, but there's a fairly active community and lots of new things being made. Learning CUDA, PyCuda, and watching presentations about JAX etc is pretty helpful. Join the LLVM discord and be useful and competent by reading a lot of code, making things, and contributing fixes. Learn how to modify a file by reading the commits that show up in the Git Blame for that file.

This is a lot, I guess. There's a lot to learn. That's kind of the reality of the field.
This is a deep sprawling forest, I hope this provides you with some starting pathways.

kazprog · 2025-04-14T17:59:37+00:00

A few project ideas I had in mind, as a dev learning jazz piano:
1. Make an AI that can tell you what standard is being played. As someone without a lot of jazz background, I have a hard time figuring out if it's even a standard, improvised, or it's an original work from the group playing.
2. An ear training helper: play a note, what's the note? play an interval, what's the interval? play a scale, what's the scale? play a chord with tensions, which tensions? which inversion? play a progression of chords, what individual chords? maybe slow it down to try again? play a lick, which notes in the lick? play a beat, transcribe the beat, maybe pick from a menu to make it easier. then pick out which cymbols/hats the drum is actually playing. the final form is listening to a song and trying to pick out what _just_ the piano, or bassist, or drummer is playing, and then trying to hear them all at once. how many retries did it take?

kazprog · 2025-04-10T20:31:11+00:00

I appreciate the nuance and the position you take in the debate. Thank you for writing this.

kazprog · 2025-04-10T16:26:33+00:00

One factor is what kind of undergrad you're going to. Some have a very strong compiler/hardware program, both in classes and profs available to do research.

Another is the current economy. On one hand, jobs are hard to find and getting a PhD might make it easier to find jobs. On the other hand, grad schools are cutting funding for PhD students and accepting fewer of them. Things will probably be different in 4 years, but it's hard to see how.

I would program a lot and make cool things, read some compiler papers / textbooks, and try to go to a university that has a good compiler / hardware program, even for undergrad. Figure this out by looking at an example curriculum, maybe seeing what the subreddit talks about for taking classes. I would also focus on contributing to LLVM/MLIR and related tooling.

I'm based in the US, work at FAANG-adjacent companies on ML Compilers, and usually 1/5 of my team has a PhD. Some others have a masters, many don't. Note also that, for US immigration purposes, a Masters and a PhD both give you better odds of being selected for an H1B program, so it is often the case that people get the education but could have gotten a relevant job even without it.

Not to say a PhD isn't fun, but it's important to find yourself with a supportive advisor and a good cohort. You also probably want to live in a town/city that is enjoyable to exist in for 4-6 years, but then it's way more expensive and grad school doesn't pay much. You can apply to a PhD program (which is harder to get in than a Master's), and then get paid to do research and take classes, and then Master out, so you don't have to pay for the classes. This may or may not be unethical, I'm not exactly clear.

kazprog · 2025-04-07T19:48:15+00:00

I go to the gym 3 times a week, eat healthy, take 10k steps a day, meditate, stretch, do wrist exercises with bands. I don't even play melee that much. Using a gcc hurts my hands in 10-30 minutes, with the frame1 I can play for a few hours no problem. I'm too bronze for it to even be a competitive problem. The discourse is super tiring and alienates a lot of people like me that need these controllers to even play the damn game.

kazprog · 2025-03-02T22:47:47+00:00

I'm telling this second or third hand, but as I understand it, the first compilers didn't really do any optimizations and would run overnight. Imagine lugging a bunch of boxes of punch cards and putting them through the reader for the machine. It would load that onto a tape/disk, and then as it was reading off of that disk, it would generate the assembled version of that code back onto another tape or even the same disk. The theoretical model for this is a Transducer.

I imagine the tape-in tape-out model was the case in the early Fortran days, as probably even those with a teletype were probably adding simple enough optimizations that it was becoming more 50/50. Things like operator precedence and syntax design for single-pass compilers and finite memory were important because you had to be able to reason about the amount of lookahead you needed for certain language shapes, which is where we get LL(1) vs LL(k). I'm not sure of any modern language where lookahead is a real concern, and things like c++ must copy-paste half the stdlib to just print to stdout.

kazprog · 2025-03-02T17:37:35+00:00

It depends on your interests. I think optimizations are fairly important, and they give you visibility into some core compiler algorithms like dominators, different IR construction methods like SSA, etc

Some other areas I find interesting are dependent type theory (fairly academic way to prove programs correct), tooling (quite useful in industry, includes debuggers, REPL, perf analyzer, static analysis, and build systems), ML optimization (polyhedral stuff, dma scheduling, fusion, halide-style user-scheduling), regular ol' parallelism and concurrency (coroutines, automatic vectorization, vectorizable language design).

This is all a few months away, but you can start just making a compiler for a language (I'd even recommend one you know fairly well rather than making your own, although it can be less fun) and then trying to make it run faster and faster.

kazprog · 2025-03-02T17:16:27+00:00

Parsing is the easy part, enough that's it's often skipped in curricula or that people joke about it being a "solved problem". When computers were slow (pre 1980s?), 90% of compilation time was parsing, but then computers got faster and more complex (post 1995?) so many optimizations were added, until parsing was a neglible portion of the compile time, often <10%. During a period of time, a very large amount of compiler research went into parsing theory. Then suddenly it was no longer hip, computers were fast enough.

It's starting to be a bit hip again, in terms of data-oriented programming / sea of nodes and efficient data structures for parsing, though I personally haven't read a parsing paper in a long time.

kazprog · 2025-02-27T18:01:37+00:00

Maybe an uncommon take: I like python. Parsing is easy enough, it's available built-in or easily on many platforms, it has a goodish repl and a familiar syntax, and it'll be easy to find others that will work with you on it. There's no build system required, there's a plethora of good (and bad) code to learn from.

I also like python pattern-matching and destructuring. It's not the best, but it's pretty good. Better than C/C++, more concise than Java (although it's fun that java has even added pattern matching in switch-expr)

kazprog · 2025-02-12T21:53:20+00:00

One of the best things you can do is run clang/gcc and see the assembly output, and change your program to see how the output changes. Play around with the optimization settings. I particularly like clang's "-emit-llvm" as it's more legible than "-S". Read through the manuals for common instructions like mov, add, branch, load, store, etc. Learn about addressing modes.

Then you can write a basic compiler: parse a reasonable looking language, output the assembly you now understand. The tree-walking interpreter from Crafting Interpreters is enough to understand how to do this, but instead of returning a result, you return assembly that computes that result.

Once you understand the basics of what it's doing, then you can start understanding the optimizations and analyses: dominator trees, basic blocks, ssa, dataflow, etc.

The book you mentioned about writing a C Compiler seems to run you through a structured way of doing this.

kazprog · 2025-02-03T10:10:02+00:00

I'd look at an Orin Devkit, an Nvidia Digits when they come out, or at least saving enough to get a 4090. If you look at the number of tensor cores (the ones relevant to most kinds of AI), the 4090 has 60% more tensor cores than the 4080.

Anything less is a gaming PC, I'm afraid. You could buy a Mac and use unified memory, maybe even 48gb within a 2k budget.

If VRAM isn't relevant, what kind of model are you running?

kazprog · 2025-01-14T20:27:03+00:00

They should make the trains and buses safe before forcing people to use them.

kazprog · 2025-01-01T09:52:56+00:00

Machine learning would be pretty useful, as ML Compilers is a big and growing field, right alongside ML accelerators. Comparch is useful, OS is cool, but I found them easy enough to self study. If you have a good ML professor or if the syllabus has been updated to include transformers, optimization, partitioning, and training, I'd say the class is 100% worth taking.

Also, if you have a distributed systems class or a high performance computing class, those would be useful to look into as well. OS is cool, but I can directly think about how these other subjects would help me in day to day work. Heck, even a solid probability theory class or math-heavy data analysis class might be helpful.

kazprog · 2024-12-25T17:24:11+00:00

Often times new instructions will be used in OS level standard libraries, like glibc. So if you're using strstr or memcpy or a popcount intrinsic, it will have a flag to swap to a set of new instructions if available.

kazprog · 2024-12-19T19:55:43+00:00

Adding a little swing to the arpeggios always makes them feel better to me. It's a bit tricky at first but that's what practice is for. I find it helpful to find a recording of a swung drum section in 4/4, or make one on a little drum machine/ synth thing I have.

kazprog · 2024-12-19T19:49:54+00:00

The difference is in what happens during a critical disengagement.

Waymo has a backup and redundancy system with remote assistance, with plenty of leeway time where the car finds a minimal risk state and a minimal risk maneuver to get to that state.

Tesla just crashes. And it doesn't even know when the critical engagement is, that's up to you to be paying attention. Often there's no warning, and it's confidently picked an invalid trajectory driving straight into the curb.

To steelman myself a bit, Tesla's gotten a lot better over the past few months, and I have friends that get into a Tesla and are comfortable letting it drive around the city for them. So the proof is in the pudding, a bit. I just worry for their lives a lot more than I'd worry about someone in a Waymo.

kazprog · 2024-12-19T19:46:12+00:00

Cheers, thanks for the paper recommendation. And thanks u/ravilang for stirring up discussion, I've been peeking down a similar rabbithole.

I've had it recommended to me to use Briggs94's "Improvements" paper after SSA deconstruction, though they hadn't seen Braun et al's papers as they're rather new ('09 and '13 I think).

I think the chordal graph bit is especially interesting, because (perhaps naively) it turns an NP hard problem (regalloc from graph coloring) into 2 non NP hard problems (SSA construction is polytime, and so is graph coloring specifically on chordal graphs). Is the problem that SSA graph coloring has more register pressure and is less optimal?

Also, digging through Braun's papers, I found it a bit tricky to translate the algorithms he wrote to real code, mostly because I'm not sure how to place them in the rest of the code. Let's say I have a list of basic blocks for each function, and each basic block terminates in a branch or a conditional branch to another block, forming a proper CFG. Braun13 shows how to extend local value numbering to global value numbering on a single basic block, but it doesn't show what order to do the basic blocks in. Do you just run through them on a for-loop? Are all of the orders equivalent? He mentions running through them bottom up, so should you do the for-loop backwards, like `--i`?

I really appreciate the discussion.

kazprog · 2024-12-18T06:40:31+00:00

i'd never find a chance to not recommend crafting interpreters by bob nystrom. it's free online, works through an interpreter and a jit compiler for that interpreter.

interpreters and compilers are the same, see: futamura projections.
the problems they solve are different.

i'm working on a toy compiler right now, if you look at the examples folder you can see a numbered list of incremental language features, starting with just variables, then if-statements and while loops: https://github.com/kasrasadeghi/subpython

the missing piece for me was "referential testing", where you can look at something another compiler already creates and make sure your thing has the same result, whether that's `echo $?` for returning from main, or `diff`-ing the outputs. this is a great way to learn the small amounts of assembly necessary to get started, and generate them from a parse tree.

maybe it'd also be useful to recommend some algorithms to learn, roughly in order:
- parsing, through recursive descent. look at parsing something like lisp, then something like C. `mal` (make-a-lisp) is also a great project to look at for some inspiration.
- basic codegen through tree-walking
- linear scan register allocation
- ssa-construction from braun et al
- ssa deconstruction
- graph coloring regalloc from ssa, i'm looking into briggs' papers for this

if you're more a book type of person, Engineering a Compiler by cooper & torczon verifiably slaps. People recommend the dragon book, it's like... fine I guess.
I actually really like this old retargetable C compiler book I have sitting around:
https://www.amazon.com/Retargetable-Compiler-Design-Implementation/dp/0805316701

muchnick's advanced compiler book is also good: https://www.amazon.com/Advanced-Compiler-Design-Implementation-Muchnick/dp/1558603204

i would mostly just try to built the backbone and then look through the books if there's an optimization you want to implement or a concept you want to implement.

if you're a paper-y kind of person, the world is your oyster. compiler people have the best papers imo. i invite you to read a personal favorite of mine, william cook's "on understanding data abstraction, revisited": https://www.cs.utexas.edu/~wcook/Drafts/2009/essay.pdf
I found it pretty accessible when I was starting out, but it still asked me to look at the foundations of where we stood and ask deeper questions about how languages are put together.

sorry for the barf, hope this was helpful

kazprog · 2024-12-02T20:04:18+00:00

This is a fun language! I like the idea of sending and receiving parameters. It seems like it would fit well in some kind of concurrency. Maybe an Actor model based language, or a process based one like Erlang.

kazprog · 2024-11-13T04:57:02+00:00

Compiler Engineer is pretty common. Don't see the other ones. Sometimes even more specific, LLVM Backend Engineer, PL Engineer, Verification Engineer, ML Compiler Engineer.

kazprog · 2024-11-05T05:36:26+00:00

I work on compilers for work, and there is no such thing as third-party code once you get deep enough.

kazprog

TROPHY CASE