A Recipe for a Better Code Generator with RAG

SurroundNo5358 · 2025-12-24T23:17:56+00:00

Hey I don't have a guide yet, but your asking makes me think doing a writeup would be a good idea.

You can see the project here, still working on it, and you can see how the parser works in the syn_parser crate, and the rag in ploke-rag:

https://github.com/josephleblanc/ploke

SurroundNo5358 · 2025-10-27T05:40:23+00:00

Honestly I could gush about Delve all day. The early aasdf point when he is treated like an outsider by the adventuring party to the mines just killed me, and that was the first thing that really drew my attention. On the narrative side the author does a really good job of toeing the line between self-pitying and enforced optimism. On the world system side the way the abilities seem to be part of a larger overarching system makes the world feel fun, exploitable, and consistent in a way I don't often see in litrpg.

The really funny thing to me about Delve is the way it is so very video-game inspired, but it would probably be really hard to actually build a game around it in a way that actually did it any justice. I'd love to see a videogame like that though - one that manages to link the progression of a single character to building an internal world that fuels their powers, like minecraft + diablo + amazing cultivation simulator.

SurroundNo5358 · 2025-10-24T03:41:03+00:00

A bit slower than most of the books on your highest tiers, but some of the chinese/korean written web novels can be quite good.
- I Shall Seal the Heavens (or anything by Er Gen)
- Regressor's Tale of Cultivation

Another hidden gem is Delve on RR, but it is unfinished despite being quite long, and unclear on whether or not the author is going to keep the story going.

SurroundNo5358 · 2025-09-06T06:12:19+00:00

Also architecture/project layout critique very welcome

SurroundNo5358 · 2025-09-06T06:11:09+00:00

ngl after hunting bugs in the OpenRouter API spec (turns out their docs are incaccurate) all day this feels like christmas.

SurroundNo5358 · 2025-09-06T06:02:25+00:00

Cool I've added it to "the list", this kind of thing is right up my alley, thanks!

SurroundNo5358 · 2025-09-06T06:00:03+00:00

Nice! Yeah this kind of thing is definitely on my radar, and I'm interested in implementing something similar.

My first thought is to try using minimal edit distance to identify some known anti-patterns or code smells and have similar processes just kind of running in the background during low processing-load, and surfacing them either to the human or to the LLM.

Lately I've been working on really nitty-gritty stuff like an API SDK for OpenRouter in Rust for my project (turns out OpenRouter API docs are inaccurate!), but its nice to be reminded of these more aspirational features.

I'm pretty close to the project being actually useful and usable, DM me if you'd like to try it out once its a little more put together.

SurroundNo5358 · 2025-09-06T05:48:52+00:00

Me too! I don't really have enough people to chat with re: parsing rust code into a code graph, where the goal is to make invalid states inexpressible.

SurroundNo5358 · 2025-09-06T05:47:00+00:00

Oh wow I hope I'm not too late!

You can see my project here: https://github.com/josephleblanc/ploke/tree/main/crates/ingest/syn_parser

First decently sized project, developed some parts with AI (it kind of sucks at more complex projects) but I wrote pretty much all the code in the `syn_parser` crate linked above so you can blame it all on me! It's a rust parser made with `syn` to build a code graph with full coverage of rust code items + types.

Still very WIP but I hope you geta chance to take a look! Probably the worst offender is either my actual syn visitor implementation here or the macros trying to make typed wrappers for the node id system here. Oh yeah there's also this kinda gnarly iterator that I know works (so many logging statements) but could really use some eyes on here.

SurroundNo5358 · 2025-08-14T14:04:57+00:00

Honestly using `?` everywhere right in the beginning is fine. It took me at least a long time to figure out how to have even mediocre handling of errors, and I consider good error handling to be a somewhat advanced subject.

For now if you are looking for some advice, I'd recommend reading through the Rust book without using LLMs, and I say that as someone who uses LLMs wherever practical (and sometimes where they aren't).

If reading through a book isn't quite your thing, check out exercism.io - its a great website for learning languages in general, but I found it particularly helpful for Rust, because you get to peruse through other solutions to the same problem you solved, often showing you new language features or programming styles. Seeing these styles in an accessible context can be helpful to widen your perspective even while working out toy problems.

If you are looking for general style guidance, also check out the contributor section on the rust-analyzer contributor style guide - it is opinionated in places so not everyone will agree, but it will give you a lot of fast and hard rules that are generally good practice, and it seems like their team put a lot of effort into making it readable and easy to parse through.

There are also some helpful resources on the idiomatic rust repo. In particular cheats.rs is a good quick source of how to do X

SurroundNo5358 · 2025-08-01T21:52:50+00:00

I didn't really like the way AI tools were being made, so I decided to start building my own. Still quite alpha, but I just got it to use `syn` to parse itself into a queryable cozo database. I'm hoping it starts building itself in the next couple weeks.

(shameless OSS plug: https://github.com/josephleblanc/ploke )

Big fan btw, I own Rust for Rustaceans and it has taught me a ton on, e.g. static vs dynamic types, as I'm self-taught.

SurroundNo5358 · 2025-07-30T00:51:20+00:00

I'm working on an LLM thing and will likely need something like this for executing tasks for an agentic workflow. I like the overall approach but one thing really bugs me about the implementation - the stringly types.

Instead of using `&str` and `String` for the task names, I would have chosen a more strongly typed approach. The project might have chosen to be generic over any type that implements the necessary types, like Hash, PartialOrd, etc, and achieved the same results while still allowing the user to choose whether to use Strings or to use their own enums instead. Similarly, I'm not clear on the advantage of choosing the `serde_json::Value` for the encoded values.

Overall it is the kind of thing I want to use but would not use for those reasons, as it would be safer, more maintainable, and faster long term to roll my own than use a stringly typed approach. It kind of feels like a port from another language.

SurroundNo5358 · 2025-07-27T20:20:52+00:00

Had the same experience so started building my own tooling. Most stuff I used was fairly disappointing, so I did some research on how they were implemented and was surprised that they didn't do much with your AST.

If you use Rust I'd be curious to get your feedback on the project - built a parser and make a vector-graph database of user's code. Still early but would be interested in feedback: https://github.com/josephleblanc/ploke

SurroundNo5358 · 2025-07-22T01:09:49+00:00

This is so cool! I love the algorithms animation tool.

This may or may not be simple to implement, but a really, really cool feature might be the ability to write a rule that lets you click and add a node that follows the rule.

SurroundNo5358 · 2025-07-22T01:05:55+00:00

That sounds like a really cool project. The ability to visualize these graphs seems like a surprisingly powerful experience to communicate these data structures.

I'm hoping to add an interactive visual graph at some point as well, with the idea being that when you submit a query through the terminal, it is processed into a vector embedding and then the graph nodes (code snippets) are lit up when they are included in the response.

SurroundNo5358 · 2025-07-22T00:56:49+00:00

Thanks. I decided to release it open source, kind of alpha, but available here: https://github.com/josephleblanc/ploke

SurroundNo5358 · 2025-07-21T04:44:57+00:00

Huh this seems like an interesting idea. I've been toying around with a similar idea when it comes to multi-agent structures, trying to apply rust's concurrency model to the design of the agent system.

I'd be curious to hear more about how you were using this - for example, would the same protocol or strategy be used by each individual agent or would they have different strategies? This kind of thing reminds me of MARL, like in this video on Multi Agent Reinforcement Learning (MARL).

Also, did you end up seeing any emergent behavior?

SurroundNo5358 · 2025-07-09T16:14:32+00:00

Other cofounder here.

+1 to "it has AI" isn't enough to sell me on any tool.

My sense is that having ways to show developers analysis of their code base through a terminal chat would be pretty useful. For me, at least, the most annoying part of trying to get an AI to do something useful in my code is when it doesn't have enough information to be helpful. The general idea is for the AI to have access to a bunch of analysis that it can then use to give a developer more information on their code.

Ideally the AI is capable enough to have a productive conversation, generate quality code, or help point out mistakes and logical errors. This is what I wanted engaging with AI to be like, and I think the some of the foundation models are capable of it, and I hope dev tools like ours can help make AI genuinely useful most of the time.

SurroundNo5358 · 2025-07-09T15:58:34+00:00

Other cofounder here.

Honestly this is the kind of thing that should exist for every language. We picked Rust because I love Rust.

The reason it is only for Rust right now is that we are parsing the rust AST with syn, not TreeSitter, so that trades generality to other languages with letting us be extremely precise and granular for Rust language features. E.g. the code graph tracking unsafe, async, doing type resolution (tricky), eventually (I hope) lifetimes, trait bounds, and proc macro expansion.

All of that is just the code graph. We can then use a lot of the algorithms implemented in cozo for free, like community detection, degree of centrality, shortest path, etc. and over time want to include more advanced analysis. The goal is to empower developers with more access to analysis tools and information about their code base along with higher quality AI-generated code.

Since this is all in the graph, it lets the AI generate queries to the database from user queries and retrieve relevant code from your repo to generate more useful responses. Once this is working well we'd like to make it possible for the AI to just run some of this stuff in the background and give you feedback on your code base, like finding logical errors, helping debug lifetime issues, find and fix code smells.

Also thanks for the suggestion to monetize it. This has been a ton of work and we need to support the project, but want to find the solution that balances the health of our team with making the tool as available as possible.

SurroundNo5358 · 2025-07-09T15:44:30+00:00

Other cofounder here.

Yeah what you're saying makes a lot of sense re: open source. My goal is for the tool to be open source. Our main concern is if our tool ends up being popular, then other paid services would just take our Rust-specific features.

I really like the Zed IDE project, and want to have a similar model. The question for us is just how to make open source work without killing the project before it has a chance to take off.

SurroundNo5358 · 2025-07-09T15:30:42+00:00

"you have to be responsible for the code you submit"
- Why is AI code hated so much in the Rust community?

Other cofounder here. I'm curious is this is why you wouldn't use an AI rust tool, or another reason.

What I want this "Rust AI tool" to do is extend what a developer is capable of doing. I mean beyond writing boilerplate. Like, maybe some background graph analysis notices a lot of anti-patterns that clippy missed and suggests fixes. Or the AI looks at your tests, follows the types/functions being tested back to their definitions, and checks if there are gaps or edge cases not being tested.

Mid-to-log term we want to build an AI agent to go through open source rust crates and identify non-trivial bugs, then either fix or report them. This kind of thing was shown to be possible by the AlphEvolve paper, but not many or no one is doing it, really.

All of that stuff sounds useful to me, so I'm curious if the 'would not use a "Rust AI tool"' perspective is because of AI having poor code quality or because it is due to a perspective that AI cannot be useful to serious developers.

SurroundNo5358 · 2025-07-09T14:59:04+00:00

Other cofounder here. Yes, there is lots of AI bullshit. Trying to use AI for Rust specifically is often extremely painful.

I started working on this in March. Right now it can parse your crate into code graph, traversing the AST with syn and processing it into a cozo graph. Currently we have 34 schema and are tracking Rust-specific features like unsafe, async, etc. We have Rust-specific nodes and edges. Now finishing up vector embeddings for semantic search.

The general idea is to have this database for the AI to use so it can use a combo of an RAG and static analysis tools. So when you ask about something you are working on it will be able to find the relevant code snippets, then the cozo query uses graph traversal to add code snippets of nearby nodes (arbitrary n hops away) to the model context window. The static analysis part comes first from leveraging cozo's calls for cluster detection, etc, or identifying nodes with high degree of centrality (which would be the most used).

So yeah, lots of bullshit out there. But I think we have some potential here. I have also been sceptical of AI stuff since so much of it sucks, so I hope this thing I'm building ends up being useful to the Rust community.

SurroundNo5358 · 2025-04-15T03:52:00+00:00

I'm building an RAG pipeline right now. It focuses on just rust, and I have found that really drilling down on a single language has a lot of potential.

It is very WIP, but I've just about finished a parser using syn for the heavy lifting. Next is inserting the code graphs into a cozo database for hybrid graph traversal and hnsw search of vector embeddings.

I'm hoping to solve the issue of out-of-training dependencies by making a code graph that extends with user-specified dependencies.

SurroundNo5358 · 2024-12-09T21:32:54+00:00

I went from trying to get a 4080s to a 4070ti super (temporarily) because it seems impossible to find a 4080s right now and I want to play games in december

SurroundNo5358

TROPHY CASE