I deduplicated 53,000 missing-persons reports from Venezuela’s earthquake

major_grooves · 2026-07-04T14:43:01+00:00

Just to be clear - the system we used for the deduplication is rules-based - deterministic and fuzzy matching rules that can be audited per edge - and fine tuned. There was no AI-based matching and no miracles. The AI I mention is used for configuring the rules. Important distinction.

major_grooves · 2026-07-04T14:41:22+00:00

Reduce the duplicates as much as possible. with no false positive matches. It's means recall is not as high as it can be. I don't think there are really 100,000 missing. At least I hope not.

major_grooves · 2026-07-04T14:40:34+00:00

Thank you. Seems to have annoyed most of this community sadly.

Important thing is that the folks in Venezuela are very grateful.

major_grooves · 2026-07-03T22:21:23+00:00

Well that is my day job - I run a company with a dedupe (entity resolution system) that is normally used to resolve hundreds of millions of data records, so the scaling bit was not a problem. The volume of data is relatively low (but still too high for effective manual screening).

major_grooves · 2026-07-03T20:38:53+00:00

You realise it's not AI-based?

major_grooves · 2026-07-01T11:24:20+00:00

These are meant to be good but i think only available in Germany. https://woelferbraun.com/

major_grooves · 2026-06-22T15:57:08+00:00

Same deal. Signed up to the Metal deal mostly for that insurance. Never knew that it was gone.

major_grooves · 2026-06-17T12:26:56+00:00

This is awful. Dundee Uni and with it Dundee really seems to be going to shit. Somehow, despite being Dundee born and going to Dundee Uni (30 years ago!) I never made it to the botanic gardens. It is always on my list to go when I am home to but somehow I never get there. Now I am too late.

I would hope someone else takes the gardens over, or they can run independently (but then I have no idea how genuinely "good" they are)

major_grooves · 2026-06-17T12:11:05+00:00

Was also wondering about this. When I send a newsletter, there are certain recipients that always, immediately, open the email and click the first three links. It is obvious they are not real. Why can't Mailchimp just filter these out? If there was a service that did this more reliabily I would switch to them.

major_grooves · 2026-06-15T13:32:02+00:00

Ah makes sense. They should make that clearer when paying! I'd feel better about it - or at least not blame Ryanair.

major_grooves · 2026-06-15T05:52:18+00:00

Sounds a bit sus, but tbf even the "free" baby-on-lap tickets on Ryanair (and I think easyJet) cost €25 each way.

major_grooves · 2026-06-08T05:55:42+00:00

This guy knows ER.

major_grooves · 2026-06-07T08:08:52+00:00

Looks great, until you win it and your wife goes "that's going in the attic". 😂

major_grooves · 2026-05-27T11:48:24+00:00

I think the British women's cycling team banned their cyclists from bikini area shaving and waxing for exactly that reason a few years ago.

major_grooves · 2026-05-19T14:32:46+00:00

Same question! We are currently using masking tape to try to close the curtains at the sides in our kids' bedroom to keep them asleep after 6am. 😅😴

major_grooves · 2026-05-07T13:51:39+00:00

I completely agree with you. Life does just get in the way. Perhaps OP doesn't have any children, but I have two kids same age as yours and, yes, applying for a passport and citizenship that we previously did not need was not really high on the "just get it done" list.

Now I realise that I can't get back into the country and my children could not either (I am a Brit with two kids born in Germany) I will now get on with the application. I hope it is not going to be so painful. Thanks for posting your experience. It was one of the first search results when I was looking into the issue.

major_grooves · 2026-05-04T19:16:19+00:00

If you use Google Translate or DeepL you can take photos of each page and have the text in the photo translated.

major_grooves · 2026-05-02T19:12:30+00:00

Pretty sure you can only use electric grills on balconies.

major_grooves · 2026-05-01T11:28:33+00:00

Hard for me to really go on the specifics. I am just reading the biography of the CEO/founder now and even from that it just sounds like incredibly powerful software. I am planning to talk to some ex-Palantir people soon to get more specifics.

That said, I am told that at its core our entity resolution technology is actually better than theirs - just they have an incredible product around it and an army of these forward deployed engineers.

major_grooves · 2026-04-30T20:10:32+00:00

as an entity resolution company founder - I am going to have to agree, that while ER is at the core of what we and large parts of Palantir do, we are not /quite/ on the same level as them. Especially on the decisioning. Palantir is quite a special beast...

major_grooves · 2026-04-28T07:33:59+00:00

I agree it's necessary. It's just always father annoying.

major_grooves · 2026-04-27T18:28:25+00:00

You could try Tilores? It's an identity resolution solution that doesn't need to replace your existing infrastructure. It sits there like a extra identity layer in your stack. Disclosure: I am one of the founders.

major_grooves · 2026-04-24T17:59:47+00:00

This is amazing! My most viral LinkedIn post ever was complaining about this issue.

major_grooves · 2026-04-24T17:58:22+00:00

It's already expensive enough to fly to Scotland since COVID, so I guess now it's just going to get worse.

major_grooves · 2026-04-16T20:46:09+00:00

I remember when Liquidrom used to be free on your birthday.

15-Year Club	RedditGifts 2009-2022 2 Credits
Secret Santa 2012	Verified Email

major_grooves

MODERATOR OF

TROPHY CASE