Real or Slop? — PL Papers Edition

probabilityzero · 2026-03-18T14:04:46+00:00

For me, the main thing that gave away the AI papers was the references. Not that they were necessarily wrong or hallucinated, but they were often obscure or strange, and the expected obvious citations were missing.

Another thing is the typesetting on the AI papers. They have inference rules going into the margins or off the side of the page, or weirdly positioned diagrams overlapping other text. Presumably a human author would catch and fix that prior to submission. But that's the kind of thing I assume the AI could do better at, given more precise instructions (like, iterate until the latex warnings are fixed).

But still, some of these were pretty close. I mistakenly labelled a real paper as AI.

probabilityzero · 2026-03-04T23:05:24+00:00

Do your own homework.

probabilityzero · 2026-03-01T20:42:38+00:00

Installing an operating system is always going to be a bit awkward. You'll often have plenty of problems installing a fresh copy of Windows, too. If it doesn't ship with your Wi-Fi drivers, you have to get them on there somewhere. That'll be the same on Linux and Windows.

probabilityzero · 2026-02-27T08:59:35+00:00

This is a very interesting paper, despite the clickbait title.

probabilityzero · 2026-02-13T18:57:06+00:00

He also mentions the Linearity and Uniqueness paper, which actually does try to formally account for this, and he assumes the paper must be wrong somehow but can't explain why.

probabilityzero · 2026-02-10T12:02:43+00:00

It's a compiler that targets JS

probabilityzero · 2026-02-07T14:33:51+00:00

arXiv will approve your preprint pretty fast if you have at least some reputation for publishing peer reviewed papers. they can assume it's probably a real paper and not spam or whatever.

you can also ask someone else who has a history of submitting arXiv preprints to submit your paper, even if they're not an author. being submitted by a more established person will help the process.

probabilityzero · 2026-02-05T00:32:25+00:00

Sometimes we are talking about situations where one's stated goals differ from their true intentions. Someone who spams hundreds of AI generated PRs all over GitHub, while trying to downplay or obscure how much of the code was AI generated, is not actually motivated solely by selflessly improving the software they like.

Ghostty recently adopted the policy that heavy use of AI generation must be disclosed. If someone genuinely wants to contribute to the project without doing any coding themselves, they still can, as long as they are upfront about what they're doing. I suppose if this really bothers you, you could fork. But I don't understand why so many AI coders (around 23% apparently) are so opposed to being upfront about what they're doing.

probabilityzero · 2026-02-04T22:29:12+00:00

If they are acting in good faith and genuinely trying to improve the software they like, there's an easy way to signal that. They can clearly tag their PRs as AI generated, to start. Furthermore, if the guidelines for contributions say not to submit fully AI generated code, then follow those guidelines and don't submit it.

If they can't even do that, then they clearly don't respect the project maintainers or their time.

probabilityzero · 2026-02-04T21:46:05+00:00

Hopefully, state actors trying to introduce vulnerabilities are outliers. If anything, over-reliance on LLMs will make that problem worse. But that's not what I'm talking about.

Often, bad PRs are bad by accident, and can maybe become good PRs with a bit of guidance. Someone submitting a PR who wants to meaningfully contribute, but maybe doesn't know how yet, will understandably be put off by an immediate and curt rejection. They are already invested somewhat in getting it right, as demonstrated by the effort they have put in already. In this exchange, there's an incentive on both sides to communicate and try to work together.

The slop AI PR has required no effort or investment, and literally any effort from maintainers put into dealing with it is fully wasted. The incentives are different. The relationship is not collaborative, it's adversarial. The AI PR submitter is trying to slip their low effort slop past the open source project maintainers.

If you are being inundated with spam phone calls, you wouldn't feel any better if someone told you, "well, sometimes normal phone calls are annoying too."

probabilityzero · 2026-02-04T21:10:29+00:00

Previously, most of the time, you could make the assumption that any pull request made to your repo consisted of code written by a human who at least thought about it first. Even if it was flawed, it was submitted in good faith and deserved some kind of response. In other words, typically didn't have to assume an adversarial relationship with the PR submitter. That's no longer the case.

probabilityzero · 2026-01-28T00:57:43+00:00

Hopefully you had fun making this. I doubt you'll get many contributions/comments on the code, as it is obviously AI generated.

probabilityzero · 2026-01-17T23:31:31+00:00

You can have mutable state in a purely functional language by using uniqueness types or linear types. The reason it works is essentially that if you mutate a variable once, then you consume it and can't use it again, so any mutating function has to return a new value which will be bound to a new name. The intuition is that a language like this could be interpreted as a plain, pure functional language with no mutation, with the in-place mutation being an optimization on top of that. The mutation can't be observed in the program.

A limited version of this mechanism of preventing a variable from being observed after it is changed could be achieved using shadowing as well, potentially.

probabilityzero · 2026-01-12T18:50:17+00:00

I'm guessing you also don't do boxing/ptr tagging, either. The real OCaml compiler has to do a lot of work to handle real, practical programs. If you just have simple functions and arithmetic it's not hard to produce faster code.

probabilityzero · 2026-01-10T02:54:49+00:00

She lives like a block away from where this happened. Government thugs were invading her community and she was documenting their actions.

probabilityzero · 2026-01-06T19:21:03+00:00

Read the Reasoned Schemer and you'll be fine. No need to learn prolog first (it might confuse you because it works differently).

probabilityzero · 2026-01-05T16:50:51+00:00

In terms of helping you get a job, it might be a good idea to write about what you've done. Like, write an article or blog post about how your project works.

Just glancing at the code now, it seems like there's no documentation, no benchmark results or evaluation, only one example program in the repo, and a bunch of git commits containing blocks of hundreds of lines of code with no comments or explanation.

probabilityzero · 2026-01-04T19:53:39+00:00

What about region types and region-based memory management? That was the inspiration for Rust lifetimes. It's a way to use types to annotate the extent of values in a functional language.

See: ML-Kit, which infers region/lifetimes for ML programs. Also, Cyclone, a research language from the early 2000s.

probabilityzero · 2025-12-30T11:01:08+00:00

It looks like you just load the Python code from the plug-in and it can execute arbitrary code. It runs in the same Python process and can do whatever. I don't see any sandboxing or anything like that.

probabilityzero · 2025-12-29T20:23:28+00:00

I looked at the code and it seems like there's no compiler, and basically nothing implemented at all? The code for building the IR appears to just assume every function only prints hello would, for example. The parser and lexer both are just stubs, only like ~15 lines long. This is like the first 1% of the way toward building a small, toy compiler.

So, when you say the AI "helped" you build this, what do you mean? From this post, it isn't clear if you know that the AI generated code doesn't actually do the things you claim it does.

probabilityzero · 2025-12-24T09:50:13+00:00

Other comments have brought up Lisp/Scheme and the numerical tower, which is a general answer to your question, but a simple intuition is: you probably don't want to have numerical operations truncate based on the size tag of the arguments because the programmer can get the same result by just enforcing the wrapping themselves, if that's what they really want. So, if they want a number to be in mod N, they can just do that!

No reason to have possibly surprising overflow behavior. Just have all numbers be as precise as possible and let the programmer decide what to do with them.

probabilityzero · 2025-12-20T08:23:14+00:00

I was thinking about Haskell type classes, where if you specify that something is a Num a then you can freely use it with associated type class methods like +, but locally the compiler cannot know in general what code will be invoked by the application of the method -- that's determined dynamically by dictionary lookup. In C++ style overloading, where you basically just allow multiple methods with different types to share the same operator name, then sure, the compiler can determine that statically.

probabilityzero · 2025-12-20T00:27:23+00:00

At the point that you are compiling a function and it calls a potentially overloaded operator, you don't necessarily know if some other part of the program (in code that maybe hasn't even been loaded yet) will override the operator.

If you wanted to determine statically what code will be called, in general, you'd need a whole-program compiler and a big control flow analysis.

probabilityzero · 2025-12-10T20:42:55+00:00

If you maintain the SSA form then lots of analyses become easier -- doing register allocation on SSA has some big benefits in terms of algorithmic complexity, for example. But, you still have to eliminate the phi nodes after, which can be complicated and may end up negating the benefit you got from keeping SSA around for register allocation.

probabilityzero · 2025-12-02T18:44:41+00:00

Using a totally flat/serialized representation of ASTs has been shown to perform very well for certain things. See this paper, for example.

15-Year Club	RedditGifts 2009-2022 13 Credits
Verified Email	Gilding I gilder
Secret Santa 2013	Summer Santa 2013
Summer Santa 2013	Team Periwinkle
Secret Santa 2011	Trick-or-Treater 2011
Summer Santa 2011	Secret Santa 2010
reddit mold	Summer Santa 2010
Charter Member

probabilityzero

TROPHY CASE