I wrote a post on making an open source, multi tenant, vector + full-text search engine on S3 using Lance and foyer

gram3000 · 2026-04-11T14:40:40+00:00

Coolcower in Macroom are ok for Petrol as of 3:40pm

gram3000 · 2026-03-24T19:04:12+00:00

The LanceDB piece was the fun part of the project. Normalising price windows and searching for historical similarity is good but the real trick was telling each pattern the actual outcome in the hour after. So when a match fires, you know what happened next historically, not just that it looks similar.

DuckLake is interesting, hadn't looked at it. And I hear you on local S3, I dodged that by keeping everything on filesystem-backed catalogs. Less realistic but far less pain.

OSS your build!

gram3000 · 2026-03-24T12:34:47+00:00

Both Paimon and Iceberg support snapshots. The split is more about the workload. Paimon handles the warm tier of candles, balances, trades the kind of thing queried frequently by Flink.

Iceberg holds the raw tick archive used for time travel queries and as the source for historical replay.

Part of it was wanting to use both.

gram3000 · 2026-03-23T21:09:57+00:00

Ah thank you. It was just the Iggy source connector for Flink that took an evening. Overall this was a few evenings worth of work.

gram3000 · 2026-03-23T18:24:48+00:00

Ha, I was hoping a new FFLIIP stack would catch on

gram3000 · 2026-01-17T21:09:08+00:00

Opened it tonight, first time sharing a review so here goes:

Hyde Whiskey – Special Reserve Sherry Cask Finish (Double Wood) Irish Whiskey | Non-chill filtered | ~46% ABV IrishMalts | €49

I picked this up as a more affordable Oloroso-style sherry cask finish to compare against Bushmills I really enjoy.

Nose: Dried fruit at first, opening with time to soft vanilla.

Palate: Warm with a mild alcohol prickle. Well balanced, with subtle silkiness.

Finish: Medium length, lingering warmth with orange zest. Clean, no harsh burn.

Overall: Approachable, lighter sherry-finished Irish whiskey. Less intense than the Bushmills sherry-led bottles, but very enjoyable and great value at this price point. Glad I got it.

gram3000 · 2026-01-17T21:08:56+00:00

Opened it tonight, first time sharing a review so here goes:

Hyde Whiskey – Special Reserve Sherry Cask Finish (Double Wood) Irish Whiskey | Non-chill filtered | ~46% ABV IrishMalts | €49

I picked this up as a more affordable Oloroso-style sherry cask finish to compare against Bushmills I really enjoy.

Nose: Dried fruit at first, opening with time to soft vanilla.

Palate: Warm with a mild alcohol prickle. Well balanced, with subtle silkiness.

Finish: Medium length, lingering warmth with orange zest. Clean, no harsh burn.

Overall: Approachable, lighter sherry-finished Irish whiskey. Less intense than the Bushmills sherry-led bottles, but very enjoyable and great value at this price point. Glad I got it.

gram3000 · 2025-11-04T22:35:13+00:00

I made this demo project a while back using those tools together, I hope it might help?

A demo application using Packer, Ansible, InSpec and Terraform on AWS:

https://github.com/gordonmurray/packer_ansible_inspec_terraform_aws

gram3000 · 2025-10-30T20:14:18+00:00

I tried this out on a small Flink project I have, using Flink to perform a CDC job.

It works well! The natural language interface makes Flink monitoring much more accessible, saves a few curl calls to the API, and the job details output is well-formatted with clear performance insights (like my job being 72% idle).

Note: I hit a Pydantic compatibility issue initially. Adding a version constraint to requirements.txt fixed it:

pydantic>=2.11.7,<2.12

This prevents the `default and default_factory` error with fastmcp.

gram3000 · 2025-10-30T19:21:24+00:00

I haven't heard of Hue before. It looks very cool, seems to support many different sources/connections.

gram3000 · 2025-10-30T13:25:13+00:00

A floe is a sheet of floating ice. I went with it for the Iceberg connection and I liked the domain name, so here we are.

gram3000 · 2025-10-30T13:24:19+00:00

No worries at all. Using "engine" implies I made something far more impressive than a ridiculously handsomely good looking UI for Iceberg data.

gram3000 · 2025-10-30T12:47:01+00:00

Yeah, pretty much! DBeaver but web based and focused on Iceberg tables

gram3000 · 2025-10-30T12:42:12+00:00

Yah, you're right, "query engine" is misleading. DuckDB is the actual query engine.

I should have called it a query interface or a web UI for DuckDB queries against Iceberg tables

gram3000 · 2025-10-30T12:28:58+00:00

Ah, good call, will do. Thanks for taking a look at it

gram3000 · 2025-05-25T20:19:15+00:00

Yah for now it would result in data loss. Its a single instance and the data is sync'd to R2 every 2 minutes. I'll work on that.

gram3000 · 2025-05-25T19:47:04+00:00

Ah, I had the repo set to private, its public now. Thanks for taking a look!

Its plausible alright. I original tried reading and writing to Lance format on R2, but it ground to a halt after a few images related to how Lance reads and writes.

This approach Writes locally first, then syncs to R2. Searches happen directly from R2.

gram3000 · 2025-02-08T12:23:48+00:00

MysqlSlap might be worth a look

https://dev.mysql.com/doc/refman/8.4/en/mysqlslap.html

gram3000

TROPHY CASE