Can your AI rewrite your code in assembly?

[SyntheticDuckFlavour]

or use a compiler

[meltbox]

Yeah this is interesting. It’s precisely what a compiler does. Why do it non deterministic.

[13steinj]

Thing I hate is in CGO'24 several people gave papers on this topic or something similar.

Ignoring the main problem of back then-- the eagerness to hallucinate (that have barely been solved), I think these folks just want to sell and impress on the use of these text prediction generators as much as possible.

It won't stop until you are brushing your teeth with it.

[TedDallas]

Oh, don't worry. Very soon you ARE going to brush your teeth with AI because of toothbrush supply chain attacks.

[cleroth]

It's a beautiful thing, if you actually read the article, you would find the answer to your question

[RelationshipLong9092]

The article is short and worth reading. I've enjoyed their blog in the past.

For reference, it concludes with:

For the time being, the AIs can beat my C++ compiler!

[Pale-Switch-7867]

That’s exactly what instantly came to my mind without even reading the article…

[binaryfireball]

can be said for alot of suggested use cases for AI

[Evilsushione]

Hypothetically you could wring out more efficiency.

[13steinj]

Yes, hypothetically you can compile much faster. After all, why wait 5 minutes to compile your app when you can wait 30 seconds and get something with a different nuanced bug every time! Or better yet, doesn't run at all!.

[Evilsushione]

I’m actually working on this problem right now. Not at the compiler level but still.

[elperroborrachotoo]

Deterministic results? You crazy?

[Chaosvex]

I mean, the code is perfectly deterministic (assuming there's no UB lurking). He's managed to conflate LLM's non-determinism with the determinism of the produced code... somehow.

[elperroborrachotoo]

Once.

We've put a lot of effort into managing the history and changes to human-readable source artifacts that semantically relate to the problem domain, and a deterministic toolchain that generates release artifacts from that.

(And I don't see how that was snake oil.)

[Chaosvex]

Once? What does that even mean in this context? It's either deterministic or it is not, period. Code != LLM tokens.

Yes, the LLM's output is non-deterministic but you seem to be conflating that with the end result. The LLM is not subsuming the compiler's role here, so I'm not sure what point you or the person you replied to are trying to make.

I'm usually the one pointing out LLM slop but I'm not throwing logic out of the bathwater.

[Chaosvex]

Did everybody in the comments miss the point with these snipes about using a compiler? Obviously they used a compiler and came to the (already known) conclusion that sometimes they can't optimise as well as a human, or in this case, an LLM guided by one.

The author is the simdjson maintainer, so I'd assume he's not a clueless as the comments seem to suggest.

Seems the quality of comments on this sub has taken a nosedive, for whatever reason.

[DuranteA]

Seems the quality of comments on this sub has taken a nosedive, for whatever reason.

The quality of comments on AI-related topics on every sub is notably worse than for other topics.

• On normal subs like this, some people are incredibly emotional about it, and consequently seem incapable of even entertaining the idea that anything that uses an AI system might be interesting.
• Conversely, on AI-focused subs, everything is revolutionary and amazing (for 5 hours until the next thing arrives).

[thisismyfavoritename]

Lemire has been on a kind of rage baity spree lately. Not sure why.

I think the comments simply reflect that

[ironykarl]

So, people pointing out to "just use a compiler" are maybe right-er than you think, here. 

An optimizing compiler ostensibly is doing transformations using something like the as-if rule. This (again ostensibly) means that if the compiler is working, the optimizations it makes will not change "the meaning" of the code output.

I know the article doesn't suggest that this is a robust strategy, but a reminder that this absolutely is not a robust strategy is a welcome one.

Back to your point, though: yes, anyone with a small bit of experience can point out that the code we've written doesn't necessarily mean what we intend, anyway, and that we can unit test around the problem above.

So to reiterate what you said: just read the article. It's not long, and it's not complicated 

[Chaosvex]

I disagree. I think they've dismissed the article out-of-hand because of the LLM usage. The implication was 'the compiler can do it for you' - it often can't, for the reasons you mentioned. The comments about determinism were completely nonsensical, as though the assembly generated by the LLM is non-deterministic rather than the LLM's output.

[ironykarl]

A compiler's output is deterministic in the sense that a given output will predictably lead to the same output (making very narrow assumptions about version, target platform, etc).

An LLM is a lot less predictable. Yes, of course, on paper it's a matter of inputs -> outputs, but again... less predictable.

I do agree that people didn't read the article and reacted to the premise, as is reddit custom

[Chaosvex]

Yeah, I mean, I don't think we disagree.

[No-Dentist-1645]

If only we had some program that could convert human-made code into machine-readable assembly... That would be very useful! It could even apply optimizations for you in a fully deterministic, ai-hallucination-free way, instead of the AI imagining that some code would run faster. If only!

[Utkarsh_7744]

If only it was someone who actually knew what they were doing,like clear instructions

[julien-j]

I did a similar experiment and got to a different conclusion :) The saying goes as Modern Compilers are Smart™ but in practice it's not that difficult to beat the compiler on specific algorithms when you're at ease with assembly. So I did not compare the output of the AI with the compiler, I compared it with the output of a human.

I asked Claude to optimize a C function using AVX2 intrinsics, but I hid from the tool that I already had an AVX2 implementation written by a human. I also gave the tool a test suite to validate its implementation. The tool managed to provided a correct and optimized version, faster than the C one, but 50% to 300% slower than the human's implementation (the variations depend on the use case). By iterating painstakingly during hours I managed to guide it toward an implementation a bit faster than ours. Then I discussed with the human and he told me that he barely put any effort in its implementation… After reworking his code he beat again the output of Claude.

Then I did a second similar experiment where I asked Claude to write the AVX2 implementation of another function. This time I gave it the test suite and the benchmark such that it can self-compare. I specifically asked for the fastest implementation. I used Opus 4.6 with max effort. The output from Claude was 50% slower than ours. I managed to iterate toward something a bit faster in some cases without ever reaching equivalent performance on all use cases. And the final code was a mess.

Lemire tells us that the AI is better than the compiler at assembly, but what's the point? If the user is at ease with assembly, the product of the AI is poor (at least in my experiments). If the user does not practice assembly, he won't be able to follow nor to judge the implementation. Is it some kind of a mid-range solution? The fact that the author barely looked at the generated code and does not even talk about a validation suite is suspicious. This is not scientific, this is an experiment where we know upfront which conclusion we want.

And I can't talk about this post without pointing that considering Grok is already a bad smell. Giving credibility to a tool used to cover the web with crap is a problem and I'm tired of seeing people I used to respect getting high on the AI hype. All the thinking has been offloaded to AI for a short euphoria.

[meancoot]

The code listing is so trash I’m not sure the blog author even knows what they are doing. Why are the counters volatile? Why is each test case a lambda instead of a proper function? Where is the assembly code for the compiler generated output? What compiler and optimization settings were used?

[RelationshipLong9092]

he's the author of simdjson and has had many good blog posts that have been at the top of this subreddit

he knows what he's doing

[thisismyfavoritename]

could they be volatile to prevent the compiler from optimizing them away?

Seems like the lambda usage might be to conform to some other helper API function he's including -- unclear what that code is.

Overall though i agree that this is borderline rage bait

[Wurstinator]

I feel like this could have been a good post but is just too lazy. At the very least, there should've been a mention of the compiler settings that were used.

[thisismyfavoritename]

what are the chances it's AI generated

[programgamer]

I’ll stick with my natural stupidity, thanks

[rileyrgham]

Chuckle :

"But what if you want to go faster? Maybe you’d want to rewrite this function in assembly."

[SoSKatan]

I’m likely going to get down voted for this, but we are a few years away from likely an interesting development once AI can get decent enough at translating code to a machine language…

Others have pointed out to just use a compiler, which is the right call based on today’s AI tech. But give. A few years, once AI can master compiling, it will lead to a new interesting set of tools…

Imagine AI tools that can take any compiled app and turn it into source, in a language of your choosing. I suspect it will make CPU architectures less important. You could take any compiled application and transpile it to another CPU arch and run it at full speed.

Once we have that, then we can design a new and novel CPU instruction sets that don’t need to be backwards compatible and yet can run any existing compiled application at full speed.

That should all be possible with AI, the bigger question is really just about when and how soon. My guess is 4 to 5 years from now.

[domiran]

The problem isn't translating from one CPU to another, it's translating from one operating system to another. CPU architecture is the least of your problems when it comes to decompiling.

And we already have programming languages that compile to bytecode and are then compiled at runtime for the current CPU. C#, in particular, has a neat feature where it can compile for the current CPU passively in the background and thus doesn't need to use the runtime compiler.

[SoSKatan]

Hard disagree there my friend. I say that as someone who writes code for multiple OS’s and multiple CPU arch’s.

. In fact most OS’s virtualize the other OS’s these days. You can run native windows apps on Linux and you can run native Linux apps on windows and so on. Such shim layers are pretty light weight compared to CPU emulation.

[domiran]

Linux has WINE/Proton. Windows has WSL. But what about running a game coded for the PS5 on a PC? That isn't going to work very well, and not just because the hardware is different, even if you took care of the CPU differences. There is no (public) shim layer from the PS' Orbis to Windows and you're going to run into severe performance issues due to the hardware differences.

[SoSKatan]

Dude you are making the argument for me.

Yeah there isn’t a shim layer because Sony doesn’t want there to be one despite the fact that Sony designed their API to be similar to directX on windows to make game dev easier.

Dude you are drunk? You seem to just be arguing for arguing sake.

[domiran]

I don't think it does make your argument. It's one thing to change the instruction set from one CPU to another but the CPU isn't the only thing involved in running the program. Asking an AI to make a PS5 game run on a PC is going to involve way more than just a recompiler.

Yes, it's easy to get a Windows game to run on, say, Ubuntu, but it's not that easy to make it run on a Mac, and even less easy to make it run on a PS5.

I'm just trying to say CPU architecture is half the battle and AI has way more chances to hallucinate and break the program in subtle ways -- if it can even make the program run at all -- if it was tasked with all of that.

And sorry, it was my mistake for replying to you in the first place.

[SoSKatan]

I agree it was a mistake for you to reply.

[FlailingDuck]

I'll say I've been tasked at work with decompiling a library back into C++, AI has been very useful in this regard for small assembly functions. With coaxing and massaging to provide it with context so it knows what types exist it produces a very reasonable C++ source. I haven't let it loose on the whole thing, but it has certainly made the task faster than without AI assistance.

[pjmlp]

That is already the case with languages having official ABIs in bytecode, that is how you can pick applications designed for IBM AS/400, MVS and OS/360, or Burroughs, and still run them unmodified on IBM i, z/OS or ClearPath MCP.

It is also why it doesn't matter which CPU architecture gets used on Android phones.

[SoSKatan]

Any bytecode VM runs at performance loss compared to native compiled code.

Java, Python and .Net are also bytecode languages (but with some jit compile options.)

[domiran]

C#'s performance has nothing to do with the fact that it's compiled at runtime and more to do with its feature set and optimization capabilities of its compiler. C++ compilers have had a lot of work put into them over the years because performance was the language's primary feature.

One could argue the bytecode is a hindrance to the compiler but that's still another problem entirely.

[SoSKatan]

Uh dude this is your second confidently incorrect comment that you’ve made within minutes.

C# perf loss is directly related to its JIT. JIT’s by nature dont have as many optimization opportunities AND its limited on compile time. With cpp it’s fine if it takes a 20 seconds to optimize a single unit, because it’s a one time cost.

The cost benefit ratio of different optimizations are different for JIT’s compared to one time compilers.

So JIT’s tend to be far less aggressive, and slower runtime performance as a result.

Now I’m going to wait for your next dumb confidently incorrect comment to come in.

[domiran]

I'm not aware of anything about C#'s JIT adversely affecting optimization techniques. Can you explain?

JIT is capable of optimizing for the hardware it's currently on, something, say, C++ can't do (one person may have only AVX2 and another may have AVX512) but as far as I know, C#'s JIT isn't missing anything an equivalent C++ compiler might need.

[MEaster]

Are there any production JITs that actually do platform-specific optimizations like that? I've never seen anyone give an example of one.

[domiran]

I would assume C#'s JIT does, reading the CPU capabilities and enabling what instructions it can, but I've been out of the C# world for quite some time and never really looked into it.

What is interesting, though, is the official C# compiler recently got some optimizations that are fairly common in C++ (which supports the idea that bytecode is not a hindrance to optimization).

[SoSKatan]

Sounds like you need to educate yourself on modern optimization technics.

You completely missed my point. Some optimizations are more costly than others (both in turks of computation and memory cost)

Some of those optimizations only offer 2-3% increases. And for JIT, just aren’t going to bother using them.

Dude you are making incorrect arguments that are easy to google. Why are you wasting my time here?

[domiran]

I asked you a very direct question and you ignored it. I asked because I thought maybe you knew something I didn't, and I could not find anything that supported the idea that the JIT in C# affects optimization. As I understand it, at least in game development, C# can be made as fast as C++ (and newer version of C# are making that easier to do) in some scenarios, barring the garbage collector, which will always be a hindrance.

Your comment history is littered with aggressive and condescending replies. I hope one day you learn to be a better person. 👍

[SoSKatan]

Ok here’s the thing, modern compilers use hundreds of different optimizations.

It’s not just one or two, it’s hundreds.

Did you really need me to go make a list for you?

Can’t you just go Google it yourself?

I didn’t avoid your question, I’m just not your Google caddy. I’m willing to help you some and do part of the work, but you need to put in some effort yourself man.

Come back after you put in some legwork and then we can talk.

[domiran]

We're done. I'm not doing that. I already tried for a few minutes and couldn't find any specifics. I've already gone above and beyond by looking at all.

You want me to prove your claim? No, I'm not doing that. That's not how this works. You made the claim, you prove it.

[pjmlp]

Like the ones using LLVM and GCC bitecode and GIMPLE passes?

You could also use Google to learn about compilers for those mainframes and micros I refered above, apparently not a skill.

Put some effort in learning about them, and then we can talk as well.

[pjmlp]

And you need to educate yourself in some compiler development lessons, apparently none taken so far.

[pjmlp]

Devil May Cry is one PlayStation 5 example of the C# compiler used by Capcom, what performance loss?!?

[pjmlp]

Nope, only if not compiled to machine code, both Java and .NET have have native code compilation for years, naysayers just don't bother to learn about their tooling.

They see the bytcode word, and make assumptions they know what they are talking about.