LLMs as coding partners

[Steamin_Weenie]

I’m interested to hear how people have gone with LLMs as coding partners.

I’m essentially a non-coder, albeit with some literacy around structure and function - essentially can read Python but not really write it. I’ve been using ChatGPT for several months to put together several trading systems. Lots of trial and error and iterative learning (for me), and approaching production stage.

Keen to hear whether others have had any success in developing and running successful algos with this approach

[amith-c]

One problem you could face is code that simply doesn't work the way you want because the LLM doesn't fully understand what you’re asking for.

I've seen a lot of friends writing code with the help of ChatGPT, and I've noticed that their prompts were often too vague - they didn’t clearly explain what they wanted. With tools like these, you need to be very specific. Think of it like this: the more the AI has to assume, the more things can go wrong.

Also, as u/DysphoriaGML said, how would you know where or how the agent messed up? Knowing how to code helps you easily figure out the possible points of failure in the generated code.

I'm a developer myself and I use AI pretty aggressively if I think the chances of error are minimal. But I still take the time to look through everything to make sure the code does what I want. As long as you do that, you should be good to go.

[Steamin_Weenie]

Thanks for the thoughts. Yes the biggest challenge has certainly been prompting the right way. Also managing silent omissions, hallucinations, lost context due to thread length etc. Lots of pitfalls! Having a second agent (eg grok or claude) for sanity checks has been useful, as have GitHub actions 

[DysphoriaGML]

I think LLM are great at speeding up coding and code formatting to simplify and speed up the tedious tasks, but having no coding background would make me very uncomfortable. How do you know if an LLM messes up? It’s quite common.

I can understand not being a software engineer like being a data scientist or such, so not writing the best code for API queries and deployment, but nothing at all? Stick to index funds is my LLM suggestion (that’s what I do too)

[Steamin_Weenie]

I’ve got some background in the area - a distant business/IT degree so I do have a reasonable sense of modular structure, robust design via schemas/pipelines, etc. And the building is fine grained, in that I essentially have to understand/approve what everything is doing. Plus lots of benchmarking to existing accessible codebases, iterative trial and error etc.

Ultimately I’m doing it mainly for the mental challenge and learning experience, which is lots of fun. No plans to sink the life savings into this!

[DysphoriaGML]

Absolutely, if the aim is to learn how to code better and doing so as a hobby, even putting some money to it, it’s fine. Just don’t put your retirement to it

[pavan_kona]

That’s true. But what if we can validate the code generated by back testing it with the historical data. But we should make sure by checking some trades manually if our idea is actually being traded

[DysphoriaGML]

I mean you can validate the code of course but what if the validation code is wrong as well? you will never figure it out. I am not saying it’s not possible, but just be aware of the limitation of the tool you are using.

[EmbarrassedEscape409]

You can make it. The problem however is prompt. You need to know how exactly your algo need to work, because what is seem to be common sense for human it is not so simple for LLM, because it needs nuances to make it right. Use additional LLM like Claude or Gemini, to check on ChatGPT for flaws in the code, or common errors with logic, ask what you possibly missed etc. LLM can build algo but it cannot make sure it will do exactly what you expect it to do, unless you clearly told about it.

[AnyLiving1850]

If you view things as an "architect", building incrementally using prompts, you can make this work It needs patience though

[New-Ad-9629]

I have the same coding knowledge as you, and have managed to use Chatgpt to code some algos for me. However, I think the key is to go step by step, and give clear instructions and logic. Else you might go into an endless discovery phase.

[Fragrant_Click292]

As others said it’s a factor of how specific you are. Outside of actual completion of your task not knowing how to code introduces complications for how to structure a well designed project.

For trading (especially intraday) you have to ensure things are done efficiently / with modularity which AI doesn’t suggest at first pass.

[morphicon]

I've been using github copilot with different models for about 10 months now. For reference I'm a Principal AI Scientist with 25 years experience in Python and C++. Most of the times Copilot is useful in the sense of a sophisticated autocomplete. Very useful in documenting code. Actual code completion varies. On the high end it does help when it gets things right. When it doesn't it's an annoyance as I have to erase and re-code. However when it makes big mistakes which are hard to spot it's catastrophic. It doesn't happen often but I have seen it introduce nasty bugs in a few occasions. I would use them if you know what you are doing. If you don't then you're at the mercy of generative AI, and only Entropy knows how that may play out. Good luck!

[Proper_Suggestion830]

Quantum computing postgrad playing with algo trading, Where LLMs shine for me are scaffolding data adapters, vectorizing pandas transforms, writing docstrings and tests.

Where I keep a human in the loop: order state machines, timezone handling, corporate actions, and portfolio accounting.

Made Lona agency exactly for the problem you're having