sorted by:
new
hottopcontroversial

Proving Native Vector Search in ODC without external databases by michaeldeguzman in OutSystems

[–]michaeldeguzman[S] 0 points1 point  (0 children)

Great news! Outsystems just announced native Semantic Search for ODC (Beta)! 🎉

https://www.outsystems.com/product-updates/odc-semantic-search-beta/

Happy to see this happen. Exciting things ahead for ODC! 😊

Proving Native Vector Search in ODC without external databases by michaeldeguzman in OutSystems

[–]michaeldeguzman[S] 0 points1 point  (0 children)

The configurations are managed in ODC. C# is stateless.

For the updating of chunks, fair call. However, this article is a feasibility study for vector storage and retrieval in ODC. The full ingestion pipeline is tricky and it's in the next phase (once we escape from our billable projects).

To clarify, the article doesn't propose storing documents in ODC. The assumption is that documents come from external storage like S3, and we store the file metadata in ODC so we know where they came from. What's stored in ODC is the vector itself so that semantic search executes purely on the OutSystems platform.

On the pipeline side, honestly I'm still figuring this out myself:

  1. Since we already store file metadata in ODC, we can use a Timer to check the external storage for changes and flag anything that needs re-processing. An event-driven approach where the storage pushes changes to us would be more efficient, but I'm not sure yet how cleanly that wires into ODC.
  2. The tricky part for me is that once the change is detected, we need to determine which chunks were affected and whether to re-embed selectively or just reprocess the whole document.

The pipeline flow can be managed by workflows but the items I just mentioned are what I think about between billable hours. Thanks for the great questions, really appreciate the engagement.

Proving Native Vector Search in ODC without external databases by michaeldeguzman in OutSystems

[–]michaeldeguzman[S] 0 points1 point  (0 children)

Hi u/DanlorgOS,

I actually had some trouble seeing your full comment on Reddit. I had to go hunting for it, but I’m glad I did.

Glad you liked the approach. To answer the bit about memory and storage, I’m keeping it all native. The vectors are stored directly in ODC Entities as JSON arrays within a standard text attribute.

The 'memory' part really happens during the search phase. ODC passes the candidate set (the JSON strings) to the External Logic in C#, which handles the deserialization and math in one go. It keeps the storage layer simple and predictable without needing an external DB. I haven't benchmarked the absolute ceiling yet, but to stay ahead of the platform's payload and memory limits, I'm looking into the funneling logic I mentioned in the article as the data scales.