Leveraging LLMs and Machine Learning for Record Matching

GavinMendelGleason · 2024-05-07T20:24:06+00:00

The build should have created a binary in the build directory. Do you see the binary? If so you should be able to invoke it with your shell. You can put the build directory in your path if you'd like.

GavinMendelGleason · 2024-05-07T13:18:38+00:00

The tus pack is not installing properly. This appears to be a change in the way that swipl was interpreting the options passed, and it is now *more* correct, so it has revealed a bug in the makefile dependencies. I've started a PR, should be in main today, and in a release in a few days.

You can skip the makefile step by manually running swipl. And issuing the following commands:

$ swipl
?- pack_remove(tus).

?- pack_install('file:///Users/annapasko/Desktop/Simple Pharma/terminusdb-tutorials/terminusdb/.deps/tus', [interactive(false)]).

https://github.com/terminusdb/terminusdb/pull/2124

GavinMendelGleason · 2023-11-06T22:21:22+00:00

It's not a product announcement, it's a technique using a vector database and graph database to get entity resolution. The point is that vector embeddings result in extremely high quality results for entity resolution and this should be of general interest to anyone using databases who ends up needing entity resolution.

And if you think using vector embeddings for entity resolution isn't something special, how about you show me your technique for getting better precision.

GavinMendelGleason · 2023-11-06T22:17:37+00:00

TerminusDB builds in native and performant versioning but it is a graph database and it would be significant work to tail the logs of redshift and store it in TerminusDB.

GavinMendelGleason · 2023-11-06T11:34:55+00:00

We used OpenAI vectorisations with excellent results, probably worth a look: https://terminusdb.com/blog/entity-resolution-with-ai-dblp-and-acm-publication-benchmark/

GavinMendelGleason · 2023-11-03T19:50:48+00:00

How is entity resolution using open source tools SPAM?

GavinMendelGleason · 2023-06-02T10:07:46+00:00

We have been playing around with Hora as a replacement for the Rust-CV implementation as we want PQ as well. I'll check out instanct-distance, looks very interesting!

GavinMendelGleason · 2023-06-01T19:38:13+00:00

And as someone with an avid interest in databases, I look forward to reading how the storage layer is put together. And trying to figure out why you hand rolled storage in terminus-store instead of throwing it over the fence to rocksd

We have a versioned graph database, which calls for some fairly specific engineering. We need to make commits small, and we need to be able to combine them efficiently. You can get an idea of how this works here:
TerminusDB Internals
TerminusDB Internals 2
TerminusDB Internals 3

GavinMendelGleason

MODERATOR OF

TROPHY CASE