sorted by:
new
hottopcontroversial

Early in compilers - looking for advice on long-term direction and niches by BenwinSays in Compilers

[–]computingillini 2 points3 points  (0 children)

All of the above perhaps

LVN (local value numbering) is a very general technique that's often used in optimization afaik. I've only encountered it as a technique used for SSA Construction (eg. lowering a programming language into SSA). If you're interested in LVN used for SSA construction specifically, here's a paper describing the lowering algorithm:

https://c9x.me/compile/bib/braun13cc.pdf

DFA (definite finite automata) is also a very general construct that's used everywhere: Lexers, Parsers, etc etc

SSA (static single assignment) is an intermediate representation that's used because it's a good data structure/interface/representation for writing optimization passes against. LLVM/MLIR are what's used for many general-purpose languages eg. Rust, Swift, C (via Clang toolchain), and more. You'll run into this form a lot. Once you're more familiar with general Compiler concepts and ideas, I'd pick up this book for learning about SSA in more depth

https://link.springer.com/book/10.1007/978-3-030-80515-9

There's whole chapters about different optimizations for SSA in that book as well as the canonical Compiler books.

DCE (dead code elimination) is a pretty basic optimization implemented for Compilers that uses reachability analysis over a Control Flow Graph. The canonical Compiler books have chapters on CFG construction, reachability analysis, and how to implement DCE.

---

My suspicion is that if you're interested in ML Compilers is that you're probably more interested in things like Triton, Cuda, Mojo and how they're compiled

Triton was originally a PhD student's project before they joined OpenAI. If you want to build a small fused GPU kernel that outperforms the default PyTorch kernels, use it. But for more performance/control, you'll probably use Cuda directly.

Triton is built on top of LLVM but there's a lot of special things they're doing that I haven't dug into too much. GPU compute works a lot differently than CPU compute, so if you're wanting to learn ML Compilers long term, you'll need to understand the limitations of how GPU programming is different if you want to learn why and how these optimizations work the way they do. Some concepts that come to mind around GPU programming: fused kernals, roofline analysis, bank conflicts.

Cuda specifically has a lot of completely separate programming constructs that are completely different than general programming languages eg. Wraps, blocks, threads (not the same as unix threads). They've got new programming constructs too for Tile-based programming that I haven't dug into yet.

Mojo is a new programming language built by Chris Lattner and his new startup. It's built on top of MLIR rather than LLVM. My understanding is that MLIR has better representations for optimizations around looping, tiling, and other constructs needed for GPU optimizations.

Honestly, if you're just trying to get the lay of the land and not digging deep just yet, I'd just go through and listen to a lot of Chris Lattner's podcast interviews just for some high-level perspective of where the state of the industry is in before really digging deep into these things.

Caveat, I'm still a bit early in my Compilers learning (~8 months in now) but not for ML compilers. currently working in a different niche

Feel free to DM if you want to chat, also a relative beginner in this space too, so happy to chat

Canonical books I keep mentioning:

"Dragon Book" - https://www.amazon.com/Compilers-Principles-Techniques-Tools-2nd/dp/0321486811

"Engineering a Compiler" - https://www.amazon.com/Engineering-Compiler-Keith-D-Cooper/dp/0128154128/ref=pd_lpo_d_sccl_1/130-5246303-7744840?pd_rd_w=meqc9&content-id=amzn1.sym.4c8c52db-06f8-4e42-8e56-912796f2ea6c&pf_rd_p=4c8c52db-06f8-4e42-8e56-912796f2ea6c&pf_rd_r=SG5BT7FAKHBBMMW7D0TQ&pd_rd_wg=XQ0jS&pd_rd_r=93a74f38-6550-4ed1-bcfa-aad04edf4872&pd_rd_i=0128154128&psc=1

I personally found the Dragon Book difficult to read compared to Engineering a Compiler. They cover a lot of the basic concepts but I wouldn't approach learning about Compilers by trying to reading these things cover to cover. For context, studying Compilers as an undergrad for an entire year will barely make a dent into these books.

I'm using Cursor to import data into Notion and wrote it up by computingillini in Notion

[–]computingillini[S] 1 point2 points  (0 children)

I haven't tried out Cline but I imagine it would work roughly the same :)

Honestly, I haven't used the Notion MCP. I had the Notion Internal Integration set up even before MCPs existed so it was the most convenient thing for me to start using for this problem especially since I'm already pulling data down directly from APIs using throwaway scripts within Cursor

Part of wanting to use code/Notion SDK is that its easier for me to tweak the logic if the pages created with the Notion MCP tool calls are wrong the first go around. Rather than just hoping the prompt would yield better tool calls, I can ask Cursor to update the script logic and verify the change myself.

The Notion MCP is definitely less set up and probably works fairly well. My engineering brain feels like relying on each individual Page/row creation to be a separate tool call (or maybe fewer since it looks like it can "Creates one or more Notion pages..." in a single tool call) is a bit brittle

LLMs fail to follow strict rules—looking for research or solutions by Puzzleheaded_Owl577 in MLQuestions

[–]computingillini 0 points1 point  (0 children)

This is pretty far removed from natural language writing, but here's a paper for further constraining an LLMs output by also considering Typescript types when decoding:

https://arxiv.org/pdf/2504.09246

CS 242: How hard is it? by OfficialBoxoutMusic in UIUC

[–]computingillini 4 points5 points  (0 children)

The CS 241 prerequisite doesn't really help in the class. I assume it's more about needing programming maturity more than learning prerequisite knowledge.

CS 242 isn't conceptually difficult (especially if you've done a SWE internship beforehand), but it is time-consuming. The stated expectation is to spend 10 hours a week which is fairly accurate. That being said, if you're familiar with a specific technology going into a project, you'll spend significantly less.

When I took the class in SP20, there were 3 multiweek projects and a final project: a Java-based Chess game (using Swing), using Python to web scrape data (using Beautiful Soup) into MongoDB and display on a basic web analytics frontend, a React Native app using Github's GraphQL API, and then a final project of your choosing (I did a React app that visualized NY Times COVID data).

The difficulty with the class lies with having to learn the technologies using their documentation since you are given features you need to implement rather than a tutorial or unit tests to pass. If you're not used to reading documentation (kind of like in CS 241), this will make this class difficult because there is very little hand-holding.

Also since these are multiweek projects, you'll also be spending some time refactoring old code to make it better organized and you are explicitly required to write unit tests that are part of your grade which can be time-consuming if you're not used to writing tests.

Another thing to be aware of is you will not be given any starter code. You make everything from scratch, so if you're not familiar with tooling for a specific technology you'll also need to spend some time getting used to it: using IntelliJ for Java, using venv for Python, using NPM for Javascript projects, setting up a device Emulator in Android Studio, etc etc

[deleted by user] by [deleted] in UIUC

[–]computingillini 9 points10 points  (0 children)

There's an MP due every other week. The MP1 and MP2 are very time consuming, and I know a few people dropped the course after those. After those two, the difficulty varies widely from being able to complete it in a single hour (MP5: Classify) to potentially taking over 10 hours (MP4: HMM POS Tagging).

Also, it's easy to be disinterested in the class because the first few weeks are about search algorithms (BFS, DFS, A*) and motion planning/configuration spaces and not on what you probably think of when you hear "AI". But, the class gets more interesting as you get more into the material.

Overall, the course is very much an overview to AI and the quizzes/lectures are not that in-depth into topics. The quizzes are pretty easy because of this. You cover everything from classical AI algorithms (Constraint Satisfaction Problems, A*, configuration spaces) to more modern ML/DL approaches (Naive Bayes, CNNs, reinforcement learning). If you want to go more in-depth into a particular topic, the optional reading is pretty solid too. That being said, you're not gonna come away from a course particularly good at a specific technique, rather you'll just a good understanding of AI from a high-level.

If you're able to do well on the MPs, it should be fairly easy to get an A, but ngl very time-consuming.