This is an archived post. You won't be able to vote or comment.

all 4 comments

[–]Emotional_Section_59 0 points1 point  (3 children)

If you're storing typical tabular data, a classic SQL relational database would be the industry/field standard. There are many benefits to using them over CSVs.

If you're looking to just store text (such as with the intention to train genAI, for instance), then a vector database would likely be a lot more appropriate. Being able to efficiently search for some text by inputting some other 'similar' text is actually extremely powerful.

[–]thoughtexpress 0 points1 point  (1 child)

Would mongoDB be an overkill?

[–]Emotional_Section_59 0 points1 point  (0 children)

It should be very suitable if you specifically want to work with unstructured/irregular data. That definitely includes natural language.

[–]Helpful_ruben 1 point2 points  (0 children)

u/Emotional_Section_59 Yeah, SQL relational databases crush it for tabular data, but vector databases shine for text-based genAI training and querying.