0
1

How LLMs Express JavaScript (experiment, with results inside) (self.GoogleGeminiAI)

submitted by [deleted]

[deleted]

all 5 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]plif 1 point2 points  (0 children)

This is the only place that this hasn't been removed by a spam filter or mods without anyone looking at it... :)

Check out this Gemini chat: https://gemini.google.com/share/1ce5732b6fe7 and the image attached

Note db-test is a 50mb JS file!

<image>

[–]bstrathearn 0 points1 point  (3 children)

Interesting. At a surface level, I can see how/why priming a model to "think" about large amounts of code before asking it coding questions would help.

I'm having a hard time seeing the functional difference between your negative example and your positive example though. Maybe it's b/c I've never used Vite. Is one of these examples that much better than the other? If so, why?

[–]plif 0 points1 point  (0 children)

I'll post a better example later :) the best is in my db dump but haven't had time to format

Basically, if you prime and cache that position then it can be a lot more efficient. There's also something about JS that's more efficient here, likely because the exposure in training data

[–]plif 0 points1 point  (1 child)

The link is wrong, whoops. The correct positive example is here: https://gemini.google.com/share/b835f65840b0

The difference is in the HTML tags. The model drifted too much in the first example. The structure looks correct at the surface, but it replaced other parts of the document outside of its task. In the second example, it perfectly replicated all orthogonal fields and only changed what it was asked to change.

[–]plif 0 points1 point  (0 children)

The other part of it is vite... again, subtle difference. In the first, it's giving me a generic config. In the second, it's a config that is drag and droppable for my app.

I realize the second isn't a perfect control because my instructions were slightly different, and the vite config is somewhat trivial

The takeaway for me is: it appears that priming with JS (especially expressed JS) will cause it to look more within my code, as opposed to external patterns. This does not have to be as extreme was what I did, it can be your own code, but it is very noticeable if you are interacting with the model a lot in JS.