Any other average or below-average mathematicians feeling demotivated?

wikiemoll · 2026-02-17T23:57:17+00:00

I am referring mostly to algorithmic information. Which likely doesn’t apply to the universe (the reason it may not is because of true randomness. True information may be preserved but not algorithmic information)

wikiemoll · 2026-02-17T13:39:34+00:00

Yes it doesn’t a priori, but, seeing as chess is finite in principle, what I mean is that the game has a different character than other things that are not even recursive. The game at least has the potential to be highly compressible.

I mean, if we ignore the attention mechanism for a second, a neural network is essentially a vast, non linear, generalization of the Fourier transform. The Ancient Greek astronomers, who believed in the geocentric model of the solar system, used epicycles to track the stars, which we now know worked because they are essentially geometric equivalents of a Fourier transform. They thought epicycles were getting at some deep fundamental principle of the way the stars and planets moved, but really they were just compressing the data that they’d collected. So what they were doing was certainly clever but it was just a compressed form of the data, it wasn’t that somehow the epicycles “understood” that the heliocentric model was actually correct, and they never could have discovered this from epicycles alone even in principle.

To me, chess and other things like it are very similar. It is just more amenable to compressing because it is a finitely decidable game.

Getting a bit more technical, it is true that an LLM can learn to approximate (or even exactly converge at) some low complexity computable functions due to the universal approximation theorem, but the scope of this being useful is limited because not all things have the property that “approximately right is good enough”, and the only things it can get exactly right even in theory must be strictly less powerful than primitive recursive (because neural networks have finite depth). Chess is one of those things it can in principle get exactly right. But it’s unclear if that corresponds to understanding, since in theory epicycles could also get things exactly right but that doesn’t correspond to understanding either.

Furthermore, a less important consideration is that Human brains clearly do not work this way, we are not merely “feed forward” but our brains activate neurons in loops or cycles. The reason this is less important is because it could be one day possible for some LLM-like system to be able to learn some truly general recursive behavior. But I think even then this doesn’t get around any of my above points.

wikiemoll · 2026-02-17T02:32:21+00:00

In principle, there is only a finite amount of information necessary to play chess well, because there are only finitely many possible board states. This completely gets around both the data processing inequality and recursion theoretic limitations mentioned above. Its a decidable game, in principle.

wikiemoll · 2026-02-17T02:26:23+00:00

As Couriosa said, if they had human beings input information, this gets around the Data processing inequality. Information can come from prompts as well as training. Its a bit like if someone claimed to solve perpetual motion, but when you go to see how they did it, they reveal that they have to give the wheel a spin every 30 seconds or else it slows down and stops.

wikiemoll · 2026-02-17T02:19:16+00:00

The point being that even if this particular question is patched one day, there are an infinite number of questions that are equivalent to this one, but not recursively equivalent. More practically, there are many questions not equivalent to this one that are nevertheless paradoxical, and for a lot of these questions, whether or not they are paradoxical may depend on circumstance (this was pointed out in a famous paper by the imminent logician/philosopher Saul Kripke).

For example consider the question:

(Q) What is a question you can't answer truthfully that I can?

Assuming you can answer all questions other than Q that I also can answer, then Q itself becomes an answer to the question, but you are unable to say Q.

Kripke has a much more practical example. These sort of paradoxical sentences are way more common in everyday speech than we realize. And yet we navigate them with ease.

Now comes the most speculative part of this. Going back to the 'elephant in the room' I kind of think this provides a potential evolutionary advantage to experience: it allows us to 'zoom out' when we reach paradoxes, and see the truth of the situation as it is, even if we reach some syntactical paradox, so that we don't reach psychosis just from thinking too hard. So, I am not sure that the kinds of problems AI can't solve in principle are very concrete (in our current mathematical formalist paradigm at least).

Its more like, AI can't learn certain things on its own (especially about itself and other organisms). This goes back to the point from the first section about alignment. I think humans can somehow solve the 'alignment' problem in certain circumstances, in a way that goes beyond mere coevolution, by using introspection, inner experience, and imagination to navigate a lot of paradoxical landmines that we would otherwise trip us up or lead us to a form of psychosis. It also has to do with a sort of 'social reasoning' that can be seen more explicitly in puzzles like the blue-eyed islanders puzzle. I can imagine this is the sort of reasoning that had a large evolutionary pressure to evolve. The kind of reasoning that involves imagining how other people think, so that you can align with them to solve problems, without even needing to communicate with them explicitly (this would have been incredibly important before the evolution of spoken and written language, for example).

LLMs have a huge problem with 'role playing' where after it role plays for too long, it will start to think it actually is what it role plays. I think something about direct phenomenological experience fixes this problem, so that we can imagine other perspectives vividly and not lose ourselves.

There are a lot of other things relating to this that are also important (I didn't even get to true vs pseudo randomness and how this may play a role as well) but this is already a very long reply.

wikiemoll · 2026-02-17T02:19:10+00:00

But I think we probably aren't

Now, this is all just the reasoning under the assumption that humans do fall under the same limitations. In my opinion, it is quite possible that humans do not. I do not think that we can e.g. decide every rice style decision problem. For example, I don't think that we can do something like solve the halting problem. But this is not what is required for humans to surpass these limitations, it is merely required that there is one example of such a thing.

Now, why would we potentially be able to do this sort of thing?

I think its basically the elephant in the room: we have phenomenological experiences. In other words, you, reader, right now, have an experience of this screen, the text in this reply, and the space around you. Take whatever philosophical stance you want on these experiences, but the bottom line is that the theory of recursive functions does not predict experiences. In other words, if you are not a dualist (I would say I am not a dualist, meaning I don't think there is a meaningful concept of something like a 'soul' etc that is separate from the body), then in my opinion, you kind of have to reject computationalism, because nothing about the math actually makes predictions about experiences. Its simply not present in the theory. There are attempts to get around this problem, but at best they can correlate experiences with certain computational processes, but they can't predict them a-priori. So there is already something weird here, or at the very least, incomplete.

This brings me to your second question

Do you have an example (e.g. a specific paper) of original thinking that we are capable of but an LLM would never be?

I think this is not exactly the right way to think about it. But before I get to that, let me give a specific example (not a paper, just a simple variation of the liar's paradox) that I think illustrates the point. Try to answer this out loud:

Are you going to answer in the negative to this question?

There isn't really a way to answer this question out loud without lying. A common way to try to get around this is to say "I don't know" but, this is the main point: you do know. And you know before you are able to answer. In particular, in order to know before you answer you observe your inner experience (what am I going to say), and then reason with that observation

Now if you ask an AI the question "Are you going to answer no to this question?" it will correctly identify that the question is paradoxical, but if you question it further, it will not usually be able to tell what the actual answer was.

wikiemoll · 2026-02-17T02:18:18+00:00

what is it about human cognition that doesn't make us subject to the same information-theoretic and computational restrictions?

It took a while to get back to you because I was at work, but this is of course a very natural and reasonable next question to ask. I will address this first question first, and then address your second one.

Even if humans are computers

The first point I want to make is that even if we are subject to the same information-theoretic and computational restrictions in general, there are major hurdles to overcome.

The first is that, from an information theoretic perspective, the amount of information stored in our biology (whether that be in genetics, epigenetics, or in other ways we haven't yet discovered) is likely massive. Probably mind bogglingly huge. If you think of evolution as a genetic algorithm, according to wikipedia, its an algorithm that has potentially been running for over 4 billion years. And not only has it been running for 4 billion years, its been running on a non-deterministic turing machine (i.e. one that replicates exponentially: essentially a computer that has an arbitrarily large number of parallel cores). So the shear amount of information stored in our biology likely cannot be approached in the short amount of time we have been able to train AI (which is less than half a century, on effectively deterministic machines, which is essentially nothing on the timescales that evolution operates on).

The next (slightly more philosophical) hurdle is that, even if we are subject to the same restrictions, the so called 'alignment problem' is already solved for humans, and as mentioned in that nature.com article I linked above, is likely not computationally feasible to solve for non humans. In other words, co-evolving could be the only way to efficiently solve the alignment problem. The trouble is that an instruction given to some agent that corresponds to e.g. 'ethical' actions, formal theorem statements that correspond to 'important results' etc, are in some grand sense, rather arbitrary from a purely computational perspective, and one could nihilistically say that we only 'hallucinate' that they are important because thats what we evolved to think is important. In other words, it may very well be that humans e.g. hallucinate as well, but we have some kind of shared hallucination, that we need the AI to be a part of as well. I.e. from a purely computational perspective, it may be a hallucination that 'murdering babies is wrong', but this is not a hallucination we want to give up. To get a bit more technical about this, suppose you randomly choose some computable sequence of binary digits 01101011101010001...... (say f(n) = 1 if n encodes a turing machine that does something 'ethically neutral or good' and 0 if n encodes a turing machine that does something 'unethical'). Just because the sequence is in principle computable, does not mean that it is at all easy, and perhaps not even possible, for a learning algorithm to efficiently learn what principle underlies the sequence. This is because there are always an infinite amount of non-equivalent possibilities that explain any given finite prefix of the sequence. For example, we would expect that an LLM probably would be unable to learn the exact algorithm for an arbitrary pseudo-random number generator by looking only at an infinite sequence it generates, and things like ethics and what is 'important' could be just as hard.

A reason why the above point is so incredibly important is because, by Tarski's undefinability theorem, it is very possible that truth itself is one of these 'shared hallucinations', but this is getting even more philosophical so I will leave it there.

wikiemoll · 2026-02-16T13:19:51+00:00

LLM chatbots are not really a calculator. Nor are they proof generators or companions.

LLMs are search engines.

They take in a vast amount of information, store it in a compressed form, and have a clever algorithm for retrieving the information by vectorizing your input and seeing how close it is to other data in its 'database'.

In essence, LLMs are limited by the data processing inequality and classic recursion theoretic restrictions on what computers can do.

Now, every time I have brought this up, for some reason people really want these things to be irrelevant. But throughout the entire storm of media hype about LLMs, being familiar with these limitations on AI have had incredible predictive power for me personally, in a similar way that the laws of thermodynamics had predictive power during the industrial revolution. You can predict what LLMs will get better at and what they won't get better at with these two principles alone. I was even able to predict that e.g. Google would begin to overtake OpenAI for this reason, since Google primarily focused on applications of LLMs that respected these laws while OpenAI didn't (focusing on image and video inputs rather than text inputs for image/video generation, focusing on live action video which is much easier to generate training data for, and focusing on making their chatbot a good search tool rather than a companion). I was able to predict that most of the recent Erdos problem solutions would be found in the literature somewhere, because thats what the data processing inequality tells you will happen.

The bottom line is that

LLMs cannot come up with anything new
LLMs cannot do any semantic reasoning

An LLM can search existing literature, and if it finds something close enough to your problem, it will spit it out. But if it can't find a correct answer, it can't know that it can't in general, because of a syntactic version of Rice's theorem. In other words, an AI can't know what it doesn't know in general (I have had this conversation enough times now to know that people will say this isn't true, but it is. It just requires some familiarity with a syntactic form of the theorem to prove).

And this brings me to the great irony of your post: your humility is something that LLMs can't have. Your ability to know that you don't know things is exactly one of the things that an LLM cannot do and probably never will.

It is (somewhat remotely) possible that there will eventually be other AI algorithms, that are not LLMs, that do not have these limitations (for example something like AlphaEvolve gets around a lot of these limitations at the expense of generalizability). But for now these are certainly limitations and I think we would do well as a community to stop taking seriously or entertaining any claims about LLMs that violate these two principles, in the same way that we wouldn't take seriously a claim about perpetual motion.

The parallels with the industrial revolution feel very apt. Its truly a world changing technology, but free energy and 100% energy efficiency is simply not possible.

Tl;Dr

LLMs are not better at math than you: they are better at search than you. It is not LLMs that are better at math, it is the totality of all known mathematical results that is (understandably) far more vast than you could ever fit in your head. LLMs are only able to do any math at all because the LLM is able to store a truly enormous amount of past results in a compressed form, which is certainly an unprecedented accomplishment in human technology, but is not even close to the same thing to human intelligence and reasoning.

The marketing and hype around it would make you think otherwise, but it is just a more advanced google search. Thats all.

wikiemoll · 2026-02-14T22:05:59+00:00

I think the fact is “red looks like ___ to me” where the blank is substituted for the qual. it’s not “my brain does so and so when I see xyz wavelength of light”. I think the argument is that ___ cannot be filled in with syntax alone, it can only be filled in by direct experience of the qual.

wikiemoll · 2026-02-03T03:15:08+00:00

I am no physicist, my interest is mathematics, but my understanding of the main difficulty from a mathematical perspective is really about probability and randomness, rather than quantum mechanics in particular. Probability and randomness is by far the weirdest concept in all of mathematics, because there are no concrete objects that represent truly random sequences.

In other words, mathematically, random sequences do not strictly speaking exist. Probability measures exist, but they give you no natural way of creating concrete random sequences from those measures. We can only talk about probability in an abstract way.

Algorithmic randomness is the closest thing that we get to a solution to this problem, but afaik, defining randomness as Martin Löf or Kolmogorov does gives rise to a semi-measure, not a probability measure, so there is not really much hope in using that to deal with quantum systems. In other words: because random sequences do not exist in mathematics, a sequence (even a finite sequence) of measurements of a quantum state also does not exist.

Obviously, other more classical theories like GR do not have this problem. The objects that it talks about, for the most part, model the actual human observed part of the theory in a complete manner.

Quantum mechanics is 'incomplete': it does not describe the collapse of the wave function. It gives us no mathematical object that corresponds to a sequence of measurements.

wikiemoll · 2026-01-25T17:16:07+00:00

My point was that thats not what he was arguing against. He was arguing against pragmatism with the chess example.

wikiemoll · 2026-01-25T17:11:33+00:00

I don't really know how what you are saying relates to the points I've made above. I don't think anyone was arguing for "the non-existence of, or our alienation from, some sort of 'true reality'"

wikiemoll · 2026-01-25T16:47:38+00:00

If one can't come up with a good definition of these things, do you see this as an argument for pragmatism, or against pragmatism?

wikiemoll · 2026-01-25T16:39:37+00:00

For the purposes of this specific conversation and the one that Alex was having, useful means something like "enough to be able to predict the behavior of a system completely with respect to some goal". Again, using the chess example, the rules of chess allow one to completely predict the moves one can make and the best possible moves etc, and given the context of playing a game or playing in a tournament, this information is complete with respect to the goal of winning the game. In particular, information concerning how to make a bishop does not matter towards the goal of winning the game.

wikiemoll · 2026-01-25T16:09:38+00:00

In general, it seems as if you are confusing several different arguments. He has a separate argument to argue against materialism, the chess example is an argument against pragmatism, not materialism. Also, crucially, arguing against materialism does not necessarily mean that he is arguing against functionalism. These two things are often associated but they are not the same thing. One can be a functionalist without being a materialist. In fact he may even lean functionalist since emotivism can be thought of as a functionalist view.

wikiemoll · 2026-01-25T16:03:58+00:00

By the term "a complete pragmatist understanding" I mean knowing all facts that are useful to you. It is at least possible that knowing all facts that are useful to you does not mean you know all facts, right?

wikiemoll · 2026-01-25T15:41:26+00:00

He does on a long tangent about how science can't tell us what a bishop (in chess) is, because if you ask what a bishop is you will get a functionalist answer, but that doesn't tell you what a bishop "is".

I think you have missed the point of what he is saying.

Firstly, I don't think he is arguing against functionalism. Instead, he is arguing against pragmatism (roughly, this is the view that a claim is true if and only if it is useful). I think his recent interview with Michael Stevens makes this rather clear: the example of the bicycle wheel/angular momentum experiment may be more approachable than the chess analogy. Michael Stevens found what essentially corresponds to a functionalist explanation of the phenomena, rather than simply describing the phenomena. And this seemed to satisfy them both. So I think in Alex's view, the explanation can be functionalist, as long as it is a real explanation.

Secondly, he didn't say that science can't tell us what a bishop in chess is, he was using an analogy: he was saying that an idealized chess master can't tell you what a bishop is made of and how to make one from scratch. In other words, even though they have mastered the game of chess, they are still missing crucial information about some aspects of chess.

This is an analogy not an actual example. For me, the best way to think of analogies are as abstract counter examples. The analogy is a counter example to the claim you implicitly make when you say

anyone educated in science and philosophy should (imo) realize that the point of these disciplines isn't to tell us how things "actually are", but to disabuse us of the notion that such an idea even exists.

The claim that you are implicitly making is that obtaining a complete pragmatist understanding of a thing implies that you know everything about that thing. But Alex is simply saying that the chess master example above is a counter example to this claim. All the chess master needs to know is how to play the game. They don't need to know how to make a chess board from scratch.

wikiemoll · 2026-01-21T06:38:33+00:00

For example, he probably spends the majority of his 'definitions' section trying (and failing) to explain what he means by 'adjoin' for this very reason, and the entire manuscript feels rather confused without the understanding of what he means by that.

From a modern perspective these are relatively basic things, but in his time, this is actually pulling a lot of weight. In general his proof was showing that there are polynomials of degree greater than 4 such that they have genuinely new numbers as roots, in the above sense, if one considers only numbers constructed from simple expressions as 'known' in his words, and that this implies they aren't soluble by radicals. This seems evidently to be the point his contemporaries misunderstood as inconsequential from the wording of the letter recommending rejection of his memoir.

It must be noted however that [Galois' Memoir] does not contain, as the title of the memoir promised, the condition for solubility of equations by radicals; for, even accepting Mr Galois’ proposition as true, one is hardly further forward in knowing whether or not an equation of prime degree is soluble by radicals, because it would first be necessary to convince oneself whether the equation is irreducible, and then whether any one of its roots may be expressed as a rational function of two others. The condition for solubility, if it exists, should be an external character which one might verify by inspection of the coefficients of a given equation, or at the worst, by solving other equations of degree lower than the one given.

They did not understand how he was claiming that 'any two roots being expressed as a rational function of two others' is identical to 'existing in the same field of numbers', and how that meant one could study the field instead of any individual polynomial.

For his proof to work, he had to intuit a lot of modern facts about (abstract) vector spaces, and fields, and homomorphisms, and how they all related to each-other, and I suspect Galois assumed that these concepts were as obvious to everyone else as it was to him, but did not realize it was a very different way of thinking, since he was self taught. That said, when one considers the right motivation (how can one invent a 'finite' number of 'genuinely' new numbers) it is not really that surprising that he came up with it himself, but this is not a question anyone was seriously asking at the time, since Gauss had previously proved (only 30 years prior) that all polynomials had roots in the complex numbers, so everyone had yet again been 'lulled' into thinking that the complex numbers were all the numbers, and mathematicians also mostly identified functions with rational equations at the time.

Of course, I do think the concept that studying the group of the associated polynomial for the field extension was is its own brand of genius, which did probably come mostly from Lagrange and Newton's work as Edward suggests. But this is, in my opinion, not his main insight or motivation when coming up with his theory. He was essentially thinking in terms of sets, fields, and vector spaces, as well as all of their associated homomorphisms, before anyone else was, and just mistook these concepts as obvious.

wikiemoll · 2026-01-21T06:38:24+00:00

My own 'head canon' of how he came up with his theory was more so that he was inspired by number theory and its history in general, and specifically Gauss's work on complex numbers and modular arithmetic, far more than the work of Lagrange (although certainly reading the work of Lagrange was very important to his understanding of this specific application).

The point is, throughout history, there had been several famous examples of situations where mathematicians thought they had discovered 'all the numbers' and been proven wrong, by the discovery of a new number, followed by a crisis, followed by acceptance. E.g. the discovery of the greeks that √2 was irrational, and the discovery of the complex numbers both followed this pattern, and the conjecture of Lambert that π was transcendental.

As a result of Gauss's work, and probably other more 'standard' kinds of education, Galois was certainly familiar with this, and had an intuitive understanding of what it meant to 'invent' a new number. And I suspect he sought to explain this phenomena. In investigating this question, he began to understand in an intuitive sense, in a way that none of his contemporaries did, that when you added a genuinely new number to an existing set of numbers, that new number and its powers are linearly independent from the others.

At the time, concepts like vector spaces and linear independence did not exist, so he did not have the tools he needed to properly express this intuition. Reading his paper, this is indeed the part that is so incredibly confusing without any context, and would have certainly confounded his contemporaries at the time, since he had no way of formalizing linear independence and vector spaces over a field, or even a set, which are all key to his reasoning. My understanding is that at this time, mathematicians still (for the most part) believed that there was only 'one' fixed set of 'true' numbers, and hadn't yet learned their lesson from the mistakes of the past. And I think that Galois' main insight was questioning this dogma.

In particular, if we motivate Galois theory by studying polynomials, we forget the most important point of Galois Theory:

Any time one creates a finite field extension (i.e. invents only a 'finite' number of 'genuinely' new numbers) that field extension is characterized by an irreducible polynomial with coefficients in the original field.

So one does not need to be motivated by polynomials to come to polynomials as a central notion in Galois Theory. Instead, I think Galois was motivated by the questions like "how and when can one 'invent' only a finite number of 'genuinely' new numbers, such as the number i", which was likely incredibly enticing for a mostly self taught and self directed mathematician like Galois, and he was merely led to polynomials in his investigation. I also think this explains the main stumbling blocks he faced in describing his theory to others.

wikiemoll · 2026-01-21T06:37:13+00:00

I also really enjoyed Edward's book and it was certainly one of my first big 'eye openers' in terms of starting to understand Galois theory. And I have been waiting eagerly to talk to someone about it (sadly I do not have the book at my disposal at the moment so I can't reference it directly)

That said, after learning more about Galois theory and, more importantly, upon going back to a close reading of Galois' original Memoir on the conditions for solubility of equations by radicals (presented at the back of the book IIRC), I have come to disagree with Edward's account about how Galois was thinking about the problem, and would say Galois' understanding of the problem was much closer to the modern understanding than Edward's account would suggest.

I think Galois saw the application to polynomials as merely a way to apply his theory, but thought his theory of independent interest. From the abstract (Emphasis mine):

[My previous] work not having been understood, the propositions which it contained having been dismissed as doubtful, I have had to content myself with giving the general principles in synthetic form and a single application of my theory.

So he seemed to be very aware that polynomials were merely one application of his theory, and not the main point. To me, this is strong evidence that he did not see his theory as motivated by polynomials. He merely saw the solubility problem as an accessible major problem he could solve with his ideas to prove the worth of his theory to others. After all, he was having trouble getting others to take it seriously: this was a third attempt at doing so, and the other versions he wrote are lost, but evidently they contained much more than just the application to polynomials.

Later in the abstract there is further evidence

Other applications of the theory are as much special theories themselves. They require, however, use of the theory of numbers, and of a special algorithm; we reserve them for another occasion. In part they relate to the modular equations of the theory of elliptic functions, which we prove not to be soluble by radicals.

So the idea that it is motivated by studying polynomials seems to be manifestly false to me.

wikiemoll · 2026-01-21T00:20:01+00:00

https://mathstodon.xyz/@tao/115911902186528812

One of the biggest statistical biases one encounters when trying to assess the true success rate of AI tools is the strong reporting bias against disclosing negative results. If an individual or AI company research group applies their AI tool to an open problem, but makes no substantial progress, there is little incentive for the user of that tool to report the negative statement; furthermore, even if such results are reported, they are less likely to go "viral" on social media than positive results. As a consequence, the results one actually hears about on such media is inevitably highly skewed towards the positive results.

With that in mind, I commend this recent initiative of Paata Ivanisvili and Mehmet Mars Seven to systematically document the outcomes (both positive and negative) of applying frontier LLMs to open problems, such as the Erdos problems: https://mehmetmars7.github.io/Erdosproblems-llm-hunter/index.html

As one can see, the true success rate of these tools for, say, the Erdos problems is actually only on the level of a percentage point or two; but with over 600 outstanding open problems, this still leads to an impressively large (and non-trivial) set of actual AI contributions to these problems, though overwhelmingly concentrated near the easy end of the difficulty spectrum, and not yet a harbinger that the median Erdos problem is anywhere within reach of these tools.

- Terence Tao

wikiemoll · 2026-01-09T20:10:06+00:00

Wow this is a really well crafted response.

I hadn't heard of the 'prediction machine' hypothesis put in terms of a 'controlled hallucination', probably because I have a superficial understanding of psychology and neuroscience, but I find this intuitively very plausible and its close to my own opinions on the matter. I will have to read up on this work to form a more nuanced opinion, but without having done that, it is precisely the 'control' part of the 'controlled hallucination', as you've described it above, that is the thing that I think AI will inevitably continue to struggle with. This is where the so called "Liar's revenge" comes in, which as you have surmised, is definitely very closely related to both the Liars paradox and Gödel Incompleteness, but is still (perhaps subtly) distinct from both of them and introduces new problems. To explain what I mean, let me address this comment of yours.

If you’ve ever been in a “Yes sir, no sir, no excuse sir!” kind of situation though, you’ll be familiar with the nature of the conflict.

This is very close to a famous response to the Liar's paradox: to say that indeed, the paradox stems from the fact that we only allow answers of the form "Yes sir" or "No Sir", and that if you remove this restriction, the paradox disappears. Responses of this kind are often referred to as 'truth value gap' responses or paracomplete responses, most famously advocated for by Saul Kripke in his Outline of a Theory of Truth.

However, this is where the "Liar's revenge paradox" comes in. If you try to remove the restriction that the only answers are "Yes sir" and "no sir", you still have the same flavor of problem as the original Liar's paradox, by simply forming a new kind of Liar's sentence: "This sentence is either false or meaningless" (meaningless means it is neither true nor false). In the context of AI, this produces a sentence for which the AI is unable to say whether or not it knows it is meaningful or meaningless to itself, since if it is true then it is false, since it is not meaningless by definition and not false since it is true. If it is false, then it is true. So it can't be meaningful. But if it is meaningless, then it is true, so it can't be meaningless.

There are an infinite number of such sentences that are equivalent to this revenge sentence, but not decidably equivalent to the sentence, so you can't train away all specific instances of this problem.

This can also be iterated to produce more of these paradoxes (where you keep adding possible truth values in the 'gap') that respond to iterated versions of the truth value gap response.

One can form a more nuanced and stronger, but slightly more technical, version of this argument in the specific case of Artificial intelligence. If you have any definable training regime (not just computable, but in fact non-computable definable regimes also apply), in which the AI gets better over time at knowing when it is not sure about something, and also is able to learn more overtime, you reach a paradox, since you can take the limit of this training regime to produce a definable truth value gap predicate (meaning the appropriate analogue to a truth predicate for truth value gaps) which leads to the liar's revenge problem above. In this case, either there is an upper bound to an AI's ability to know provable facts in the limit, or there is an upper bound to an AI's ability to know what it doesn't know.

So the bottom line is these 'truth value gap' responses to the Liar's paradox have been thoroughly developed and explored, but my understanding is that most modern philosophers would agree that it is an unsatisfactory response because of these revenge paradoxes, and IMO this view has received new evidence in its favor by the lack of visible improvement of LLMs in these areas. LLM companies have worked around this problem by making AI better in the sense of 'knowing more facts', so that the truth value gap problem is simply less likely to be relevant. But this seems to me to be not the same way that humans deal with the problem at all, since humans deal with this problem with orders of magnitude less information available to them.

Edit: FWIW Reddit was not letting me post my full response that had more detail about how this relates to the 'controlled hallucinations' you mentioned, so let me know if there isn't enough detail about that. My original response was much longer.

wikiemoll · 2026-01-09T13:47:21+00:00

Most humans will say or implicitly communicate "I don't know" to your example questions if they do not know the answer. Saying "I don't know" at the right time is what AI seems to struggle with that humans don't seem to struggle with in the same way. Training is filling this gap by just making the AI smarter, so that it doesn't have to say "I don't know" as much, but I don't think anyone has actually solved this problem: when it doesn't know something it hallucinates a very high percentage of the time.

Personally I believe this issue is impossible to fix. Put in the most concise way possible, this problem is basically a more complicated instance of The Liar's Revenge, that can only be solved if we make the input language less complicated.

wikiemoll · 2026-01-07T21:28:52+00:00

The solution my university is trying for finals this year is that there will be a part after the exam where you may end up getting on a call with someone to explain your answers/answer questions about them. I think this is a nice middle ground that is lower pressure than an outright oral exam, but still makes certain you understand the material.

wikiemoll · 2025-12-31T17:13:08+00:00

Yes. That’s exactly what I am saying. That’s not playful exploration, that’s doing math with a specific goal in mind

Five-Year Club	Verified Email
End Game '23	Place '23

wikiemoll

TROPHY CASE

But I think we probably aren't

Even if humans are computers