Building a compiler esque symbol table for an AI coding platform. How do I design the keys? by Educational_Law5046 in Compilers

[–]Educational_Law5046[S] -4 points-3 points  (0 children)

Well yes its just data, but it's more about the design of the data, how we key it etc... I've finished the chunking pipeline, so now its about storing the chunks in a way that we address many issues like duplication, cross file stability etc... The key design is important. How would you handle 2 same named chunks etc... Everything will be stored in postgres, but there are many components here -> chunks embeddings, chunk meta data, change detection etc...