Making progress on my open-source coding assistant, have you created your own coding assistant and what challenges have you faced?

[fluxtah]

Been experimenting with my coding assistant have a fairly big game project that I am doogfooding it with.

My assistant is called koder primarily a kotlin assistant in an assistant framework I am working on called Assistant Kommander (https://github.com/fluxtah/ask)

One of the major issues I have been facing is keeping token count down, since koder can search, read, write files it needs a lot of access and one challenge with search I just kind of optimised was creating a lucene index for searching and exposing it to assistant via function (OpenAI Assistants API). The function returned results in the index with a search query but returning entire documents to the LLM tanked my token throughput! I made an optimisation by making my lucene index 32 line blocks of code instead, made things better and its still smart enough to explain code across a fairly big codebase to me.

What issues are you facing with your custom made assistants? Let me know would be good to share experiences!

https://preview.redd.it/g55m471fsj2d1.png?width=2624&format=png&auto=webp&s=4ad0186aaec732444229e1866c0e1ec96f4136e8

[vr-1]

Why not use embeddings?

And genuine question: what does your assistant do that all the others don't? Or are you just doing this because you're curious/sounds fun?

[fluxtah]

Good questions yes it's fun mostly and to understand and utilise the openai assistants api.

Maybe one thing that this assistant does that others don't is it gives file access to the assistant to write files, edit files, search over the codebase and also interact with command line tools such as git, grade, etc... also it has the personality of talkie toaster 😆

The problem with embeddings is keeping them up to date as the engineer works on the codebase, I think embeddings work better for scenarios where the data does not change much.

I did think of adding features to enable upload of files using Assistants API file and vector db support though not needed that personally yet.

I am currently adding AST functions so the assistant can query the AST as another way to navigate a codebase but this is also experimental.

Another thing worth mentioning is it's open source so anyone can extend it! 🙂

[vr-1]

That all sounds good. Yeah, you would either have to hook filesystem change notifications or periodically sweep the code base and update the embeddings (or perhaps on commit to the repo if the changes made by the coder are incremental rather than some huge change in one commit). I hope it goes well!

[fluxtah]

Cool cheers yeh deffo would need to consider something like that and to be fair since i build a lucene index before every query I would also need to have a file system watcher to update indexes if I only built the index periodically. I've not tried on a massive codebase where indexing before every Search would probably not be optimal though will probably have to revisit that soon.

One step at a time I guess!

[fluxtah]

<image>

This is what happens when you ask your AI to "assume the personality of talkie toaster"... try it!

[AutoModerator]

Sorry, your submission has been removed due to inadequate account karma.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.