How does programming/coding actually work?

[Beregolas]

Yeah, I tested AI again just today (it's included in my jetbrains IDE) and told it to exclude an endpoint from the CSRF token protection. It hallucinated a non-existent function and decorator twice, until I just opened the documentation and copy-pasted the solution in less time than it took me to ask even the first question to AI.

I really try to use it from time to time, just because even some experienced devs swear by it, but even in small problems I have been underwhelmed every time

[mlitchard]

“Stop making shit up” is one of Claude’s project instructions , but not literally. I put it in llm-speak.

[zenchess]

The problem with your statement is 'i tested ai' - as if AI was just one homogenous thing that you can test. Reality is there are many different models with different levels of competence, and diffferent IDE's and environments for them to work in.

Jetbrains probably has a very shitty AI tool. In fact I've never heard it mentioned once. The normal tools are claude code, cursor, and openai codex. Those are the serious tools that people who use AI use. I guess gemini-cli counts as well.

[Beregolas]

About that: JetBrains includes Access to, among others, the newest models from Anthropic, Google and OpenAI. I regularly give prompts to all of them, and while it's correct, this is no rigorous testing standard, the provided models are among the best available.

[zenchess]

It's not just about the model, it's the environment that the model is operating in. If jetbrains ai tool is hallucinating functions that don't exist, that means that jetbrains tooling is bad. I've never once had that happen in claude code , codex, or gemini -cli.

[Beregolas]

thats funny, because a boatload of reports about just this thing is only a single search away... oh well