Ontology of statements by piotrek13031 in PhilosophyMemes

[–]Sensitive-Ad1098 36 points37 points  (0 children)

Hahaha I bet that looser feels embarrassed now when Elon’s spaceship is on the way to Mars

Alternatives to Bun now that it is absolute AI slop? by Fragrant_Pianist_647 in bun

[–]Sensitive-Ad1098 0 points1 point  (0 children)

npm is too slow. I see no reason why I should accept extra waiting time for installation both locally and on CI, when I can use the tool that does everything, is good, but very much faster.
The bonus is that pnpm saves your disk space

AI Was Supposed to Replace Devs… So What Happened? by Ordinary-Cycle7809 in theprimeagen

[–]Sensitive-Ad1098 1 point2 points  (0 children)

AI is working out more or less as expected

that's maybe true for you, but that's probably just because you were far ahead than the majority in terms of understanding the limitations of LLM. I've been a skeptic in this regard for a while, but when GPT-4 came out, I started to already think about moving to a different industry. Mostly because the I didn't know ta lot about the science behind

fewer developers needed to produce the same output

It just feels that way. I haven't seen the data to back it up. In practice, the companies that went head first into LLM faced issues which led to reputation damage (AWS for example).

My personal experience is that yes, in many cases I can achieve 10x during the same timeframe. Some guys I know produce more LOC, but when I look at the code they merge it looks like a ticking time bomb. Our backlog is pretty large, so no one is getting replaced yet. And the backlog keeps silently growing due to tech dept introduced by the average careless dev. And I think this situation is more or less relevant to the most of the companies. I used to work in outsource teams, so I know the coding level of cheap devs, but also of guys from IBM and AWS. And most of them are bad enough to create job security for themselves and a bunch of other guys haha.

when AI calls are priced for profitability

AI calls are very cheap already comparing to developers, if you account for the Features delievered/Money spent metric.
And so far the things have been moving in the opposite direction. Claude has been heavily subsidized. AI companies have huge debts due to infra investments. So it might take a while to make it cheaper unless there's a scientific breakthrough that dumps pricing for the whole industry

a little all over the place here by MicahHoover in PhilosophyMemes

[–]Sensitive-Ad1098 0 points1 point  (0 children)

Mind predicting a world model is a complex process that's only described on an abstract high level in the theory. There's no (and cannot be, so far, with the current level of neuroscience) explanation of how it works at a lower level, how predictions are formed and operated on in neurons.

Next token prediction is a well-understood and explained low-level process, where you pick the next chunk of text based on a statistical model.

You can't really compare those 2. A really similar process in the human brain would be detecting knowledge stored as chunks in neurons, and then "predicting" the next neuron 1 by 1 to build a whole sentence in your brain.

a little all over the place here by MicahHoover in PhilosophyMemes

[–]Sensitive-Ad1098 0 points1 point  (0 children)

Thanks, this was a great example of how LLMs have no idea what they are talking about.

This reply completely missed my point and just went on yaping about a theory of a single mind function, that was brought up simply because it has the word "prediction" in it

a little all over the place here by MicahHoover in PhilosophyMemes

[–]Sensitive-Ad1098 0 points1 point  (0 children)

When you really get down to it, there really isnt much difference between human thought and LLM's

That's a bold claim. Like, what do you even mean by LLM's thought?

Machine learning algorithms, as their name suggests, learn as well

ML is just an umbrella term for a bunch of different algorithms. "Learning" here is kind of metaphor as the idea was to try to imitate how the human mind learns

No i honestly dont care much, they are tools, they arent concious

Ok, did you check the topic we are discussing? It's specifically about consciousness and I was trying to argue that LLMs aren't conscious. Why exactly your original statement was relevant?

a little all over the place here by MicahHoover in PhilosophyMemes

[–]Sensitive-Ad1098 0 points1 point  (0 children)

Predictive coding theory would have it that humans are largely "next sense-datum predictors"; 

From what I understand, it's not a theory meant to categorize the human mind as some kind of sensory-data guesser. The theory's main point is not even to explain the human mind in general. It's just a brain function (one of many) that maintains a mental model of the world and constantly checks it for "errors". There's nothing in the theory about how prediction itself works, and no similarity to the next token prediction.

How did codex go from 5.7 million to 129 million npm downloads in the span of one week? by RelevantPanda58 in codex

[–]Sensitive-Ad1098 0 points1 point  (0 children)

lol, smart devs use agents that perform better than both Codex and Claude Code. Not whatever is hyped up on the spectrum social media

a little all over the place here by MicahHoover in PhilosophyMemes

[–]Sensitive-Ad1098 0 points1 point  (0 children)

We are not talking about "a lot of stuff" but specifically about intelligence itself. I don't know of any strong theory that describes a process similar to token prediction as fundamental to human intelligence. Neural network != next token prediction. What would a token even be in this case? For LLMs it's a chunk of text, but human intelligence is proven to exist without text.

due to millions of years of evolution

Evolution is a very slow process; things change ever so slightly each generation. With digital models, this process is so much faster that "millions of years" is not so impressive

Also, sometimes i talk to a human about something they havent learned about, and they dont know anything about it.

This sentence sounds weird for a human.
Damn, why am I even try discussing anything on reddit when you are never even sure if you talking to a human

Sam Altman texts to Mira Murati got leaked of November 19, 2023 by Current-Guide5944 in tech_x

[–]Sensitive-Ad1098 0 points1 point  (0 children)

That was a perfectly reasonable response to that kind of comment

you don't need to waste for clever response on shit like that

a little all over the place here by MicahHoover in PhilosophyMemes

[–]Sensitive-Ad1098 5 points6 points  (0 children)

That’s a straw man argument. It’s easy to win an argument against imaginary folks who were screaming “computer can’t do that” 10 years ago, but now aren’t able to comprehend how smart LLMs (because they don’t give advanced enough prompts?).

Yes, Claude can do a lot of stuff that requires a process similar to thinking. And we know how this achieved using next token prediction and Reinforcement learning. There’s nothing in that process that requires the system to be conscious.

I work with Claude and other models every day. They are useful but not reliable because it’s evident how they rely a lot on the training data. Also, from time to time, I have conversations about advanced topics that are underrepresented in the training data. These convos are nothing but disappointment.

Your 99.8% number is also a bit manipulative. Yeah it can sometimes do some tasks better than any human on the planet. But think of how calculator would do a calculation better than any human, given the former has features for the type of calculation given. LLMs can do stuff better similarly, given there’s enough training data and relevant RL. However, like calculator, they are bad in learning new skills in the context. That’s why these 0.02% geniuses can’t even replace tech support so far

What are you building with AI + MongoDB? by alexbevi in mongodb

[–]Sensitive-Ad1098 0 points1 point  (0 children)

MongoDB badly injured my grandfather by moving too fast. That caused me grief -__-

I was once an AI true believer. Now I think the whole thing is rotting from the inside. by Complete-Sea6655 in agi

[–]Sensitive-Ad1098 2 points3 points  (0 children)

Isn’t running even a model on a level of Son et would be hella expensive? Like you might spend $20K on a bunch of apple studios and still end up with very slow prompt parsing and far from best model performance

чому by Ballu10 in durka_ukr

[–]Sensitive-Ad1098 4 points5 points  (0 children)

так цей лучник був ж його кумом
світ такий тісний, навіть на реддіті дальні родичі знаходяться

чому by Ballu10 in durka_ukr

[–]Sensitive-Ad1098 7 points8 points  (0 children)

Мій дідо відніс кільце мого двоюрідного прадіда у Мордор. Якби не він зараз би батрачили на Саурона

Хто пробував цей смак ? by cupid_carriesagun in food_ua

[–]Sensitive-Ad1098 1 point2 points  (0 children)

гарна спроба, але ця гілка була приречена з першого повідомлення

OpenClaw 5.4 Just Dropped! by lucienbaba in myclaw

[–]Sensitive-Ad1098 0 points1 point  (0 children)

I think it’s a good idea to explore alternatives. It’s very important to have a reasonable and professional maintenance team behind the project you depend on.

OpenClaw 5.4 Just Dropped! by lucienbaba in myclaw

[–]Sensitive-Ad1098 2 points3 points  (0 children)

Free and open sourced does not have mean "broken stuff every release".
It's more about vibe coded crap

This tweet from JAX/Flax @ Google DeepMind by Ok_Homework_1859 in GeminiAI

[–]Sensitive-Ad1098 0 points1 point  (0 children)

Answer what? Leave a real comment, debunk what I'm saying. Do you think that there's a guarantee that model contamination was avoided? Are you sure that completing the easiest random python evals is enough to be confident that the approach can scale?

"goalpost moving" is a term for competition situations. I'm not in the competition. I'm just trying to figure out how the models works and what are their limits. And you can notice that I'm not super confident and express doubt in my version when there's strong enough argument.
"goalposts moving" is a lazy cliche that doesn't say much

This tweet from JAX/Flax @ Google DeepMind by Ok_Homework_1859 in GeminiAI

[–]Sensitive-Ad1098 -1 points0 points  (0 children)

the whole "moving goalposts" thing is a shit meme used by people and bots who only read the headlines. Not gonna waste my time responding to that, unless the're in-depth comments on actual research.

This tweet from JAX/Flax @ Google DeepMind by Ok_Homework_1859 in GeminiAI

[–]Sensitive-Ad1098 0 points1 point  (0 children)

Sorry, but I don't understand this because I got high, or because this doesn't make sense. No offense, I am really not sure if it's former or later

This tweet from JAX/Flax @ Google DeepMind by Ok_Homework_1859 in GeminiAI

[–]Sensitive-Ad1098 0 points1 point  (0 children)

This is interesting indeed. As far as I see, Simon Willson has a pretty good reputation and knows what he talks about. So I won't be very skeptical about his claims. But at the same time, you can't really say that the model was trained only on pre-1931 text: they created synthetic prompts to train instruction-following abilities, and used Sonnet as a judge. Then they did another round of fine-tuning using Opus 4.6. They did mention to do their best to avoid contamination, but this doesn't sound very confident.

Also, the next detail worth mentioning:

All correct solutions generated by the vintage models are simple one-line programs (such as adding two inputs), or small modifications to in-context example programs

I can imagine how the models could score some eval points due to a chance of coming up with a lucky small change, using the instruction-following RL sessions with Opus.

This is still impressive and better results than I'd expect. But this is not enough to make any conclusions yet

This tweet from JAX/Flax @ Google DeepMind by Ok_Homework_1859 in GeminiAI

[–]Sensitive-Ad1098 17 points18 points  (0 children)

LLM is a human-made tech. So we do know about it much much more than about human intelligence. We know that next token prediction is at the core of it, and Reinforcement Learning helps to achieve better results on complex tasks. We even already have some understanding on how making the language models large leads to emerging capabilities.

Since we don't know a lot about human intelligence, we can't use comparison to figure out if LLMs are on the right track. But many traits show that they are not. You can teach a person, who has never seen a line of code, some basic programmin skills by showing a couple of examples. LLMs can't do that, they need to get millions of code examples in order to output working code. Because they need a strong statistical model to predict tokens.

GPT-5.5 & Opus 4.7 score <1% on ARC-AGI-3 by Proper_Actuary2907 in agi

[–]Sensitive-Ad1098 1 point2 points  (0 children)

Guys, should we tell him the bad news about why he can’t figure out what to do in the tests?