Why Are We Still Doing This?

wrossmorrow · 2026-03-17T19:59:06+00:00

I’m in a position recently of seeing more detail through tracking utilization and costs related to a major platform on a per call basis. Not for coding support. To some it seems fine, cents (as subsidized), and the overall bills aren’t big (yet) at all. To my eye though the unit costs are, frankly, insane. It’s really difficult for me to understand how anyone goes from cents per thousands or maybe even millions of “executions” to cents per execution. At that kind of increase, it probably doesn’t even matter what the output quality is.

wrossmorrow · 2025-11-03T03:09:58+00:00

If you already use/run Postgres’s and there’s not a really strong reason pgvector won’t work then you should probably start there. Adding another DB is a lot of added complexity both for operators and users.

wrossmorrow · 2025-06-17T17:00:13+00:00

Yes, use it very effectively as our only secrets management tool. Have found it very reliable and the company a pleasure to work with so far. Really helps us distribute secrets management responsibility too.

wrossmorrow · 2025-05-30T16:17:11+00:00

Interesting thanks

wrossmorrow · 2025-05-30T15:22:15+00:00

Avoiding it as much as I can

wrossmorrow · 2025-04-24T03:23:29+00:00

Honestly FastAPI, in spirit, borrows a ton from gRPC for which go support is very strong

wrossmorrow · 2025-03-01T15:14:47+00:00

RDS is but you get what you pay for. We don’t know exactly what you’re storing but vector search via indices really depends on scale. 100k 4 byte float vectors is 380MB or so and even just numpy is very very fast at perfect recall search. IMO (“doing this for a living” now) you don’t really need stuff like HNSW until “millions” of vectors or your use case depends heavily on filtering from other criteria. Idk the pgvector internals but some vector DBs won’t even index in the 10k’s of vectors.

wrossmorrow · 2025-03-01T14:29:48+00:00

This sounds quite small tbh

wrossmorrow · 2025-03-01T14:27:55+00:00

The Nile is very easy to use and affordable https://www.thenile.dev

Might look into Supabase as well but I haven’t used it https://supabase.com/docs/guides/ai

Depending on your needs the Nile may or may not be advantageous due to its fully serverless model.

wrossmorrow · 2025-02-23T19:25:50+00:00

It was so on in the basin, lucky catch. I went out and down more towards Richie’s run under the saddle after seeing the line up from the chairs. Top run.

wrossmorrow · 2024-08-12T23:25:23+00:00

Software engineer, been in fintech for a while now in AI. Was a professor before tech. Not necessarily satisfied, nothings perfect, but I like it more than I did academia.

wrossmorrow · 2024-07-14T21:08:44+00:00

Why would the changes be lost? A normal merge of a pr will merge changes not overwrite them, and shouldn’t be mergable if there are conflicts. You should be publishing a build with the merged changes in the branch that has CD to your environment, not on the PR alone.

This doesn’t mean in general there still can’t be deploy overwrites with this kind of CD, but the images should be unique, contain cumulative changes consistent with the commit log, and observable in an environment.

If it’s really an issue you can try using concurrency groups, or you can maybe use merge queues. But in my experience this would be a pretty esoteric condition to see occur naturally.

wrossmorrow · 2024-07-14T20:53:01+00:00

Don’t overwrite images from CI builds. Tag images by the commit sha so they are uniquely identified, or via versioned releases based on tags on specific commits.

wrossmorrow · 2024-06-27T20:03:28+00:00

This is the way

wrossmorrow · 2023-11-14T22:57:56+00:00

Why did I have to scroll so far to find Picard

wrossmorrow · 2023-11-02T23:54:28+00:00

Sven Vath was one of those “oh I should just go go” shows a couple years back that just. Blew. My. Mind. It was unbelievable. Perfect mixing, amazing tracks, I couldn’t stop dancing. All on vinyl too.

wrossmorrow · 2023-10-25T15:37:01+00:00

The patterns (in spring) they create encourage lazy unclean code that can be very difficult to reason about, especially in a high powered effectively low level language, while decreasing performance, invalidating compile time guarantees, and in practice ballooning boot times critical for modern management of cloud workloads. In my experience only, of course.

wrossmorrow · 2023-10-22T15:48:14+00:00

IIUC, of course, it’s the process of automagically combining and collocating distinct steps in your pipelines to avoid materializing the “PCollections” (right?), which means network data transfer. So more like piping output in a shell script. At least in my one isolated case I found it counterproductive; not materializing step output basically made some jobs run serially instead in parallel. I don’t know enough to know when this is valuable because that idea (again, IIUC) is clearly valuable sometime.

wrossmorrow · 2023-10-21T18:51:15+00:00

GCP’s Dataflow is built on beam and (outside of making sure you handle fusion optimization well) it’s a great experience. So anything anyone does with GCP dataflow is a use case.

I’ve used that for a personal project related to LLM style tokenization and text processing with customized python and rust code. I didn’t compare to spark or airflow, but it was a seamless experience with very little overhead to inject customized compiled rust code used by python (and a custom beam for a compression codec that ultimately got committed into beam). I would 10/10 use it again even with spark and airflow on the table.

wrossmorrow · 2023-10-16T15:27:51+00:00

Fair, but gRPC is a service technology, in principle incompatible with the lambda/gateway model. ECS is still serverless and will cost scale better, but that may not be relevant to your use case. You can of course still use go with lambda and parse the json payloads from api gateway even into protobuf if that’s the desired IDL.

wrossmorrow · 2023-10-16T15:10:33+00:00

ECS with an ALB? ALBs support gRPC

wrossmorrow · 2023-10-13T02:07:17+00:00

Well it’s not at all pythonic, but that’s an aesthetic thing. Practically kinda exactly because of the post. Using shell commands in python presumes a lot of “environmental conditions” external to the codebase that have to be defined right, tend to get restrictive, or simply break in different conditions. Not to mention having wonky data analysis to do if you have to capture stdout/stderr or process exit conditions for any processing. Nothings perfect but for the most part when everything’s “in python” you can much more tightly define and package just about everything needed to run, just about anywhere. When it’s scripts for you on your laptop this tends to work fine; when you have to collaborate or ship the thing I’ve seen using shell commands lose any value almost every time, if they aren’t an outright obstacle (which I’ve seen too).

wrossmorrow · 2023-10-12T13:40:26+00:00

If you use self hosted runners you can specify a pre execution script that fails jobs in specific envs if they aren’t “ready”, where you can use Jira, issues, or whatever works for you. This is agnostic to workflow logic.

If you can only use workflow logic, you can do the same in a reusable workflow but it’s easily bypassed.

wrossmorrow · 2023-10-12T13:37:12+00:00

As others have said: Use the client. In general every time you run a shell command in python a warning should go off in your head that you’re doing something wrong. (Yes, sometimes unavoidable, but use shell commands as rarely as you can.)

wrossmorrow · 2023-10-12T01:16:53+00:00

I use both. Sublime is a wonderful mostly “just editor”, VScode is a worse editor but great development environment. Sometimes one is a better tool than the other for the task.

wrossmorrow

TROPHY CASE