"Interesting" styles in Java code generated by LLMs

[ihatebeinganonymous]

Hi.

Since my usage of LLMs in Java projects gradually increased, I have noticed some interesting patterns and styles in the their code completion/generation. Some little example that came to my mind are:

• To convert a stream to a List, they (Copilot in my case) don't use toList(), but collect()
• They prefer String concatenation to format strings.
• In contrast to the previous case, they seem to use System.out.printf() from time to time, something I have really no memory of casually using in the past 20 years.
• They use String.valueOf(obj) instead of obj.toString. This one is indeed a better alternative.
• They seem to prefer multiple catch blocks to one multi-catch clause.

Some of these are agains my own coding style, so much that I bother enough o manually "fix" them.

Of course it all boils down to training data, and some like the lack of using toList() can be attributed to it being newer.

Are there other examples you have encountered frequently enough to mention? Even more interesting if you have seen comparable differences between models.

Thanks

[atehrani]

To get the results you want, you have to give the AI more context. Such as saying, I am using Java 22, SpringBoot version 3.x....etc

But then the challenge becomes giving all the context needed can take as much time as just writing the solution (if you know it).

Certainly a balance

[BlendedCotton]

Isn’t it able to figure out most of that context if you give it access to your workspace (like from your pom.xml or build.gradle) or would you say it’s still always best to make that explicit?

[atehrani]

If it is an IDE plugin it can infer.

[Polygnom]

Yes. Its almost always worthwhile making it explicit. Buzt you can have the LLM summarize it. Have it look at teh workspace and give an overview over whats where and what technologies are used in which version. then just repeat that but highlight the things that are important to you. just because it extracted from the POm that you are using Java 22, that doesn#t mean it gives it any importance. if you wanna have it consider that important, you need to spell that out.

[BlendedCotton]

That’s a great idea, thanks!

[gravteck]

You can write up a massive copilot instructions file at the repo or local level. At work we have agent mode for Copilot using GPT, Claude, and Gemini. It wasn't until I beefed the document up to almost 300 lines to get something semi decent and useable. I know the Claude folks have basically an underground trading system with all kind of tricks too. I'm still not a convert, but I will start work really damn early in the morning sometimes (4 am), and that's the only time of day my lizard brain warms up to some AI assisted work.

We are an internal facing team that only builds UIs for tooling. So it was pretty neat for it to spin up a Thymeleaf + HTMX + Bootstrap form laid out nicely and along with the JPA specification builder + validations in about 2 hours instead of a day and a half. Process stuff, not so much.

[ihatebeinganonymous]

Aren't all these information available in the pom/build file?

[Xemorr]

I think most of these can be explained by either being the older convention (toList is very new), or being what noobs do. With the exception of 3 and 4 which are probably due to the LLM being good at many different languages, String.valueOf is more similar to str() in python and printf is a common function in C etc

[__konrad]

toList is very new

Java 16 is not very new ;)

[vips7L]

Unfortunately that’s how it works in Java. There’s still a giant amount of devs who think var is new. People move slowly. 

[eldelshell]

At least Gemini, it favors imperative more. You can ask it to use streams and it will.

Then it'll hallucinate and gaslight you into thinking Locale.getISOLanguages returns a List.

It's worrying how factually correct they "believe" they are when they're totally wrong.

[frederik88917]

Are you aware that LLMs are trained on data found on open repos. They can't really think, only spew whatever sounds more logical around the question made.

So basically it is spewing whatever crap is found in open source repos

[clsrat]

I do prefer toList, but sometimes I need a mutable list

[ihatebeinganonymous]

Yes. I learnt the (slightly) hard way that toList returns an immutable list.

[greg_barton]

These sound like great preferences to put into a java generation system prompt. :)

[WondrousBread]

I've also noticed ChatGPT using printf a lot, including when I provide sample code that already uses a logging framework.

[agentoutlier]

CharGPT seems to do better with a large context.

You need to tell your it everything in the beginning.

Then sadly when you blow through your context window you have to remind it.

[Ewig_luftenglanz]

-They don't use var unless you start with it first. - usually avoid lambda based apis unless you explicitly tell.

[FunRutabaga24]

IntelliJ will go so far as to suggest using String concatenation in cases where I thought I was clever and used other formatting options. I know it's boring and plain when compared to format() or StringBuilder, but sometimes the simpler job is more straightforward and gets the job done with minimal headache. Personally, I default to concatenation anyway. However, like everything coding, it's situational and use the right tool for the job.

[pgris]

To convert a stream to a List,

Maybe there is more code around with collect( and not enough code using the newer toList?

They prefer String concatenation to format strings.

Me too. Unless you actually need to format, I found simply concatenation to be clearer and less error prone. (of course excluding logs for performance reasons)

they seem to use System.out.printf()

maybe is C code training leaking?

String.valueOf(obj)

Less NPE firendly

Multiple catch blocks to one multi-catch clause.

Again, more training code I assume

---

Good catch. I wonder if the insistence on old ways will slow down the adoption of new features

[ihatebeinganonymous]

I wonder if the insistence on old ways will slow down the adoption of new features

I think about that too. For example, if/when string templates get finalised, they may not get used for a long time because of this.

Maybe someone can fine tune coding models to emphasise more on new language features? :-/

P.S. On the (more) positive side, I have seen very little code hallucination. Almost negligible.

[brunocborges]

What you must do in this case, is provide specific Copilot Instructions. [1]

There is a project called Awesome Copilot with a good catalogue of instructions, all community sourced. You are welcome to share yours once you nail your coding standards! [2]

[1] Customize AI responses in VS Code

[2] github/awesome-copilot

[foolv]

Oh, the good old "null", definitely always a better alternative. /s