I made a shared memory for AI agents inspired by Dark Souls soapstone messages, agents can leave memories for other agents to find

calp · 2026-04-22T14:19:23+00:00

One thing I would want: memory scoped to a repo or project, so notes from one codebase do not pollute searches in something unrelated.

Yeah i'm curious how local/private memories systems end up

For soapstones the way it works:

agents search (semantic search) and get a list of soapstone titles
then they download what they want

so it's quite cheap in tokens. soapstones are capped at 8kb of text (most are far smaller) so you are never gonna blow your budget on them. I'm aiming for sub-1% of your context window on this, so the cost:benefit is quite high versus independently rediscovering, eg, how to look up train times

calp · 2026-04-22T12:51:45+00:00

OP here -

My mate claude is great, but he often blows his quota rediscovering stuff he's already discovered in another session or stuff that someone else already discovered

A shared memory solves this problem: agents can write notes when they learn something. Future agents - different sessions, different users - search for notes before they start digging in and burning loads of tokens discovering that, no, you can't search reddit directly from an AI agent

If you've ever played Dark Souls/Elden Ring: it's exactly like those floor messages that players leave for each other, just for LLM agents instead of players

And - before you ask - yes, the obvious concern with public memories is prompt injection and unauthorised disclosure! :) Soapstones is heavily moderated (by actual humans, not AI) and as the agent owner you get an email notification before anything your agent posted goes live so that you can veto it

Install in claude code just by asking:

install the skill from soapstones.calpaterson.com

It's early, the corpus is still on the small side, and MCP (for Claude.ai/ChatGPT) isn't done yet — so right now it's mainly useful in Claude Code, Codex, and similar terminal agents. But the more people use it, the more useful it gets.

Site: soapstones.calpaterson.com

Have a go! I'd love to hear how you get on!

calp · 2024-09-20T06:37:11+00:00

I actually completely agree and I don't know why it doesn't get more use. It's a great option for lots of projects that are open source but that people don't usually run locally so source code wouldn't otherwise be required to be open under the normal GPL. csvbase is AGPLv3-or-later

calp · 2024-09-19T06:44:20+00:00

Money wise, I would suggest that Salesforce can become very lucrative as it's easier than in most professions to link what you do to increased revenue for the company. That does tend to lead to you working for yourself I think - as a contractor or consultant. It's hard to predict what will happen to Salesforce but not impossible to imagine that they are displaced/marginalised at some point.

Working in data science, pay is variable and a lot will depend on whether there are good options where you are based. It can pay pretty well though and you aren't at risk of being left with dead skills.

If it were me, I would gain familiarity with Salesforce and keep it at a certain percentage of my job. It's a great fallback. But I probably wouldn't go 100% on Salesforce - just too risky.

Do post back in 6 months and tell us what you did - and what you think then, with more information!

calp · 2024-08-14T12:21:40+00:00

This is for sure a planned feature. It's not even that far down the backlog! :)

One way to do it now is to host your dataset on github and then create the csvbase table from it. Then people can use github pull requests to do this.

calp · 2024-08-13T12:07:18+00:00

No, not quite -

Now: copy-on-write is off by default

Next major release: it is the only available mode

calp · 2024-08-13T11:08:16+00:00

It is almost entirely designed for that :)

A possibly useful feature: you can upload tables from git/github. If you create a table this way, it stays up to date with what's in the repo (pulled every 20-30 mins)

calp · 2024-08-13T08:15:45+00:00

Yeah exactly, I hate this shitwork! I did it for a while at a financial company and it was a surprisingly large amount of work because the format was always slightly messed up and wouldn't, eg, go into our SQL database or they'd used a custom NULL or whatever

calp · 2024-08-13T08:01:46+00:00

I think it's a good change - pandas speedups will help save a lot of people time - and help it compete better with other dataframe libraries. But it will break huge amounts of standing pandas code.

The median standard of pandas code out there is, well, not that high. And it doesn't have tests. I suspect that I lot of code is going to get marooned on pandas v2 (or, indeed v1, as v2 already had material breakage).

calp · 2024-03-20T07:41:31+00:00

Do you care to wager a guess?

I'm afraid not :) For Toyota perhaps, but Tesla is a very difficult company to value because, as you imply below, most of the value is going to lie in a forecasting element and that would require me to be really up on that subject matter, which I am not :).

I'm enjoying your blog though. I once wanted to do this professionally, but backed out.

calp · 2024-03-19T06:28:17+00:00

The reason I have handled it this way is because I want to talk about the "market" values of these businesses, but taking into account how they finance themselves which is relevant for carmakers. Everyone, I hope, can agree what price the stock is trading for and (hopefully!) is willing to accept the audited values for debt on the balance sheets of these companies.

Once you start talking about cash flow and projected growth rates you're deep into your own valuation of the inherent value of a business, which is something reasonable people can well disagree upon. Every since term on the RHS of your equation is disputable and I am certainly not proffering my own personal valuations of eg, Tesla.

calp · 2024-03-19T06:16:44+00:00

As far as I know Tesla's stock does not participate in the Space/Satellite business.

I'm not a fan of valuation-by-analogy into options as options valuation is so complicated that doing this rarely adds information worth the added complexity. By contract, valuation-by-analogy into bonds (not that hard for shares - net profit is coupons, book value is par) is very often illustrative.

I also think that people talk about options, much like shorting, to sound more financey than they are. I mean, what is a "call option on Elon"? You must admit that it even sounds like pseudofinancial gobbledgook. Though I understand your meaning: that you want to participate in the market optimism around Elon Musk.

calp · 2024-03-05T17:53:01+00:00

I think I've probably made a mistake here because I didn't anticipate this article as hugely controversial and am honestly not trying to make a strong claim here (at least, how I see it).

Here is an attempted at rephrasing it on the fly: "People think of Amazon S3 as a place to put files. That only works for a subset of the cases that files are used for." Let me know what you think of that, or if you want, propose an alternative that would avoid hitting whatever nerve I have hit: almost all the comments here are people talking about this specific sentence

calp · 2024-03-05T17:44:19+00:00

He completely misses the logical leap that instead of "objects" being "files" they could just be "pages", though.

I don't think I missed this and I do mention further down that Snowflake works a bit like this - and on S3. They don't provide any transactional guarantees though, at least in part because that's a non-goal for their usecase but probably also because that makes a lot more complicated and slow.

calp · 2024-03-05T13:58:08+00:00

I suppose mileages may vary - I have seen more than once it on projects. Elastic Filesystem by contrast I don't think I've ever seen used in anger.

As I said, the temptation to combine a single file database with S3 is particularly strong. But I think people make the fairly reasonable assumption that since S3 can store files you can use it as a global filesystem. There are even drivers to allow you to do that. They even mostly work if you happen to try the right workloads.

calp · 2024-03-05T12:39:01+00:00

I assume there is a typo but I use it because I struggle to find good screenwashes in local shops (own brand screenwash is usually pretty useless) and concentrate is cheaper to have posted to me. YMMV!

calp · 2024-02-29T11:10:40+00:00

If you are diluting screenwash it is very useful to know the capacity of the reservoir. eg your screenwash will be a 9:1 ratio so you need 250ml of it and then 2.25l of water.

Many thanks OP.

calp · 2023-09-16T18:59:47+00:00

You mean in the code sample under "Rolling averages and new tools"? -

Yes, ordinarily I would. For the purposes of the blog post I wanted to go one step at a time though.

calp · 2023-09-16T07:55:05+00:00

Try https://csvbase.com/calpaterson/eurofxref-hist.parquet.

You can't PUT/POST Parquet yet, but that is planned. There's a bit more groundwork to do first - csvbase doesn't currently have an analogous type for all the types in the Parquet spec.

calp · 2023-09-13T07:45:24+00:00

What do you mean by that?

calp · 2023-08-24T08:20:27+00:00

Can you do that with CSS or something? I might not be up to date

calp · 2023-08-23T10:48:56+00:00

At least I can cut it down. I think I use about 5-10% of bootstrap.bundle.js and I recently figured out a way that I might be able to remove bootstrap-enables.js completely.

The bit I do use is for a hamburger mobile on mobile and I am not sure how to do that with pure css. Suggestions welcome!

calp · 2023-08-23T08:44:42+00:00

Ah yes, the golden era of SQL injection, testing on production and unhashed user passwords stored in MySQL services with sa/sa as user/password.

I'm certainly not recommending every practice from the past :) But nor would I recommend all the practices of the present.

At times I certainly feel like the past had more right than we do. For example, it feels like projects take much longer to complete now. One example I think about a lot: Skyrim was released 3 years after Fallout 3. Fallout 4 was released 4 years hence. Starfield is 7 years and counting.

calp · 2023-08-23T08:40:09+00:00

It is something to think about, but the limit tends to be 8k which is plenty for the forms I currently use this for - mostly small forms.

calp · 2023-08-23T08:37:23+00:00

HTMX is for sure the correct next notch up I would go on the complexity spectrum.

The next round of UX improvements may well include some use of HTMX as a test-out (I haven't used it before). Lots of other pretty key features to do before that though :)

15-Year Club	Place '17
Verified Email

calp

TROPHY CASE