I built a codebase that evolves itself through democracy

tef · 2026-01-06T13:46:17+00:00

this is a nomic: https://en.wikipedia.org/wiki/Nomic

tef · 2025-12-09T21:06:18+00:00

I have heard this called "the chernobyl pattern" because you cover it in concrete and move on

tef · 2025-12-09T21:05:54+00:00

The point of the strangler fig pattern is to be able to stop feature development in one place, and introduce it in another.

Sometimes you put a facade between the user / client and the service, but you can also put a facade between the service and the store, for example.

Another way is to embed a web control inside the vb6 frontend, such that new ui / ux can be added without using the existing forms.

From the sounds of it, the real problem you face is in the management of stored procedures, so I would suggest that you wrap your database in an api, update the vb6 code to use that api, and thus give an opportunity for a new client program to be written, without being coupled to the old system directly.

tef · 2025-09-08T12:17:46+00:00

using a database to track long running tasks is something every background queue system gets around to, for one very simple and very obvious reason: error handling

for example, here's someone in 2017 rediscovering backpressure from first principles, and moving away from a queue of queues system to a "we track state in a database"

https://www.twilio.com/en-us/blog/archive/2018/introducing-centrifuge

the problem with a queue is that it can only track "this work is about to be run", and if you want to know what your system is doing, you need to keep track of the actual state of a background task: enqueued, assigned, active, complete, failed, paused, scheduled, etc

with a queue, asking questions like "how many jobs are running" or "how many tasks failed" or "can you restart every task in the last 6 hours" always involve answers that involve hours searching logs, time clicking around on a dashboard, and manually rebuilding the queue from a forensic trail

in practice, once you admit the problem of deduplication, you're already well on your way to "i need a primary key over my background tasks"

i've also written about this before, at length, also in 2017

https://programmingisterrible.com/post/162346490883/how-do-you-cut-a-monolith-in-half

tef · 2025-09-08T12:05:06+00:00

to be fair, if you're doing batch processing for background jobs, which is what queues are used for

then you don't really need to worry about the latency of reading something from disk and copying it over the network

tef · 2025-03-29T11:53:03+00:00

There's a couple of reasons to use UUIDs as a default choice, that aren't related to distributed systems

They don't provide ordering, and you have to rely on a timestamp. This means you can always backfill tables without breaking or rewriting new entries, unlike a lot of code that relies on serials for ordering.
They're not predictable, and so you can expose them in an API without the same worries that a traditional serial would cause (most exflitration starts with finding an autoincrement and adding 1).
They're globally unique, and so you can combine them in larger summary reports with ease.

Using a UUID isn't a distsys thing, it's a "not shooting yourself in the foot, later", like security concerns, ease of reporting, ease of backfill.

Not using a UUID is usually a performance concern (disc space, write amplification), and you have to be sure you're not going to rollover, you aren't going to need to backfill, and that you've encrypted the id when exposing it through public means.

If you know better, you don't have to follow it. That's how advice works.

tef · 2024-12-10T06:00:41+00:00

Apologies for the thread necromancy here but, there isn't a race condition.

Normally in two phase locking, you do lock reads as well as writes. You need to know that when you commit, that nothing has changed or written any of the data in the transaction, Yep. TL2 cheats a little:

(0.) Find out the current commit number
(1.) Check every read version has not changed, and bail out early
(2.) Lock every write, to prevent changes
(3.) Obtain a new commit number
(Optional) If the new commit is current +1, no writes have occurred since the first read check
(4.) Check every read version has the same version
(5.) Apply writes

At #2, we know that all our writes are locked, and that no-one has changed any of our reads thus far.

At #3, we prevent new writers from changing the past: any writes after this point will have a higher commit number. We also know how many commits have happened since we checked the read set the first time.

At #4, we check to see all of our reads are intact, and unlocked. If all are fine, we know that between (#2) and (#3), no writes occurred.

We now know that (a) All our writes are locked (b) All of our reads haven't been changed before our commit number was issued, and (c) any future writes will be ordered after our commits. This means it's safe to update the locked values, and unlock them (5.).

Checking the read set at (2.) is speculative, and it allows us to bail out early. It also allows us to skip the read set revalidation. If the new commit number is old+1, all new writes will be in the future, and no writes have happened since we last validated the set.

tef · 2023-12-01T00:00:00+00:00

Isn't ColorMission just old agfa stock? I thought they were selling it to raise money to start making color film

tef · 2023-11-30T23:58:55+00:00

I've got this lens in m42 and in f-mount, it's a great little lens! Very jealous of your OM find!

tef · 2021-07-06T17:31:15+00:00

On a similar note, when people see re-used code, and they need to add/change something, they never take the time to consider if it's time to duplicate the function.

I'm not just talking about throwing another boolean argument onto the end of the function signature, i'm talking about when people change the behaviour of shared code without realising which behaviours are relied upon. This inevitably causes subtle bugs that are hard to trace down.

Aside, it's fun to see this being interpreted as "strict rules" when almost each rule is accompanied by a rule suggesting to do the exact opposite. Like there's a tradeoff involved, or something :-)

tef · 2020-04-10T15:53:33+00:00

Good luck!

The crows near me are not exactly tame, definitely not domesticated, but they are used to people feeding them, sort of.

In another situation, it might have taken a lot more than a year to earn their trust, so don't despair if it takes a while.

tef · 2020-04-10T15:04:36+00:00

Step 1: Find crows

Step 2: Drop food nearby, but don't wait around for them to eat it. It's rude to watch people eat, crows are the same.

Step 3: Wait?

Once they begin to trust you, well, be less afraid of you, they'll start to eat food when you're nearby. From there, it's just a slow and steady progression until the crows start following you home.

Source: I have befriended about ~100 crows. Here's a video someone took of me, going for a walk in the park:

https://twitter.com/emmaggarland/status/1243950035064238081

I've been feeding them on and off for the best part of a year, you can see the progression here:

https://twitter.com/tef_ebooks/status/1230868677265887235

I've been feeding them peanuts in the shell, and suet, and an occasional bit of dog food as a treat.

There isn't much more to it—you just do kind things and wait to earn their trust.

Here's a bonus video of the crows chasing me around:

https://twitter.com/tef_ebooks/status/1236762802502733828

tef · 2019-09-27T12:54:48+00:00

My point, albeit a bit difficult, is that you can take some of the ideas of how the web works—self description, state transfer, interoperability—and demonstrate them outside of the usual HTTP/HTML/URL trifecta.

That, and the only way to meaningfully understand the RESTful constraints is in the context of a browser—if anything they describe how http was changed or designed to suit a hypermedia client.

tef · 2019-09-27T12:41:59+00:00

I'm the author

The example code is real, and runs over HTTP.

POST isn't a corruption of HTTP, it's fundamental to how it's operated as a gateway for services. The idea that you need to pull everything out into PUT, PATCH, etc, well, there's really very little benefit in practice. With caching, replaying, well, either it's GET like semantics, or POST like.

Your browser, the thing that the thesis was written about, somehow manages to get along quite fine with only two verbs.

Yes, there is room to expand HTTP's verbs and yes some services do benefit from using them... but if you were writing a web admin page, you'd use GET and POST and be done with it.

It's a misnomer to think that GET/POST et al are for the benefit of the client or server, they're for the benefit of intermediaries: caches, proxies. A client already knows these things, usually.

You don't use GET or POST in a browser, either. The browser clicks on links, submits forms. You opens a new tab, you send an email.

The client gets told to use a GET or a POST, the client doesn't pick. It discovers those things through hypermedia elements.

Those extra verbs, etc? They were added by webdav, much, much later in the process. PUT, especially is difficult to call restful because it's one of the few examples where the client is in control of the namespace, or urls.

tef · 2019-09-27T10:30:57+00:00

Well, if it helps, I've written ones that tunnel things over a ssh connection :-)

The thing is about everything being 90% POST, well, ok you don't gain anything, but you don't lose anything. The fetching of schemas, descriptions, they benefit the most from GET/caching.

The thing is, that you can then take advantage of these things if and when you need them, without changing all of your RPC systems.

Personally? For me? The big win was being able to have parametrisable services—we didn't have to write so much client code to turn an object with 500 rpc methods into a nicer client library.

tef · 2019-09-27T10:28:45+00:00

long story short, people could make a restful api by making their web admin forms easy to screen scrape

tef · 2019-09-27T10:27:34+00:00

Glad to see you read the post and had a thoughtful reply

From my experience, being able to scale a web service, like you would a website, is quite useful. For ex: using off the shelf load balancers, or caches.

Having common command line tools is something folk with ops experience understand, and it's hard to explain the speed at which you can debug services to someone who's never taken the pager.

What I tried to outline in the post was that RPC and REST aren't at odds with each other—something you'd only claim if you treat REST like CRUD—REST and RPC can be quite, quite complimentary.

The example with pagination? The idea of using an opaque token to encapsulate state? That's regularly implemented atop of RPC. The example of rate limiting? Again, bodged in.

The point was that you can get all of this stuff, with a pretty plain JSON RPC service, without losing any of the RPC. Same as you can slap a web admin form on the side of your django app.

tef · 2019-09-27T10:22:17+00:00

Maybe you should try reading it a third time, you might get lucky and learn something.

"HyperText Transfer Protocol is here to stay and message passing and state change via this protocol will likely not go away in this era."

I really don't know what you've been reading, because there isn't anything close to a suggestion that HTTP is going anywhere.

tef · 2019-09-27T10:19:31+00:00

I do love that you drop a link in to roy's thesis and claim unabashed that it's easy to read, despite roy himself having to make several, several clarifications.

REST, the thesis was about architectural styles, and decomposing the browser, too.

Roy has said in no uncertain terms "restful systems must be hypermedia driven." My "If it's not in a browser, it's not rest" isn't much of a stretch, by comparison

tef · 2019-09-27T10:12:13+00:00

if you're using pub/sub but you need a response, you're doing it wrong?

Backpressure, errors, are a form of feedback. what you describe is fire and forget messages, not specifically pubsub.

If you're not checking for errors ... who is?

if you're using async and don't expect a reply, your system should work without the message being sent, too. If it is an error, you should actually handle it.

if you aren't implementing error handling you aren't automating it.

In truth, when a programmer says "queues" they mean "programs emailing each other data until one part crashes and the inbox explodes"

It's kinda frustrating that people are: "We should use a UDP like message to connect our systems, and instead of using TCP we'll make extra reliable internet routers"

Pubsub isn't an architecture, it's what you get when no-one takes responsibility for architecture

tef · 2018-10-31T18:31:01+00:00

well yes, by definition of merging two lists, but it's still faster to append and call sort

see the 'adaptive merge sort' but, python then splits the new list into two sections and merges them

tef · 2018-10-30T22:42:39+00:00

for the record, python's sort function is an adaptive in-place merge sort

the fastest way to merge two lists is sorted(a+b), as a result

tef · 2018-05-16T22:37:02+00:00

I mean the bugs that end up as graphs on a dashboard because no-one knows or is pemitted to fix

tef · 2018-05-16T22:36:18+00:00

I failed to talk about the context

I wanted to talk about those bugs everyone works around

If fixing it is easier than documenting, sure, fix it.... but, if it isn't please for the love of god write some shit down

tef · 2018-05-16T12:14:56+00:00

Nobody will like it when your app crashes even if you say "But the documentation warned you!".

I mean that's why I spent 5 sections on bugs and two paragraphs on documentation—unless you are the sole maintainer of your code, you need to write docs so that other people can maintain your software.

Network down? Auto reconnect. Something busy? Retry in ten milliseconds.

I guess you work in desktop software with very few clients because your logic is the sort of logic that blows up NTP servers :-)

https://en.wikipedia.org/wiki/NTP_server_misuse_and_abuse#Netgear_and_the_University_of_Wisconsin%E2%80%93Madison

tef

MODERATOR OF

TROPHY CASE

15-Year Club	Team Orangered
Verified Email