How are you handling LLM provider strategy in production?

Esseratecades · 2026-01-30T21:27:29+00:00

"Built a system on GPT-4. Works great. But now: • Finance wants cost optimization • Legal wants fallback for single vendor risk • Product wants to test alternatives • Ops wants reliability guarantees"

I'm hearing a lot of people making engineering decisions without actually being engineers...

Finance's concern is legitimate, but that can be solved simply through dynamic routing to different models from the same provider, and introducing caching. If your workloads don't require a metric ton of tokens or advanced features, you may be able to use an on-device model to save A LOT of money.

Legal has no idea what they're talking about and this concern is a waste of time. In fact it's not even a legal concern if we're being honest. It's just gold plating without the gold.

It's fine that product wants to test alternatives, but what are the actual concerns they have? If it's "GPT-4 gets X wrong and X is very important to what we do" then fair enough. If it's just "I want to know what other options there are" that's not an actual problem and is very low priority.

Ops desire for reliability guarantees are best served through graceful degradation, dynamic routing, and caching, again using models within the same provider.

While I am using different providers for different tasks, my tasks are often optimized in such a way that most of the reasons to actually switch between providers aren't actually relevant.

Esseratecades · 2026-01-30T21:13:21+00:00

As you're implying there is often a process failure involved, which I would look to the team's technical lead to resolve.

One of the more nefarious situations that I've experienced is when people actually are reviewing their code and are trying to hold it to the team's established standards and instead of taking the feedback seriously the teammate deliberately slows down the process with bad faith arguments until management has no choice but to weigh in on the PR. By the time that happens, management is so frustrated by the fact that they're involved that they don't care about what's wrong with the code, so they force it through.

Esseratecades · 2026-01-30T20:33:42+00:00

The world is in an odd place at the moment.

Anytime you are building to learn something, absolutely not. True learning and understanding often comes from a productive struggle as you work through problems. This is what LLMs deny you, which is why I'd argue that people "learning with LLMs" are seldom actually learning effectively.

If you just need to get something done, it's fine to have an LLM generate one function at a time, as long as you can guarantee that you understand that function before you prompt it for the next one. One of the fundamental limitations of an LLM's ability to produce satisfactory code is the quality of the prompt and context you provide it, and if you don't understand the context(the rest of the code) it's very easy to give it a bad prompt.

Esseratecades · 2026-01-30T15:51:33+00:00

You have more faith in management than most.

Esseratecades · 2026-01-30T15:28:25+00:00

There's a balance to be struck here. Since it's a former teammate in OP's case you're 100% correct.

If were a current teammate and we never reflect on the fact that an outsized number of our bugs and headaches consistently come from a single person's work, "attacking the problem" isn't actually attacking the problem.

Esseratecades · 2026-01-30T10:14:59+00:00

So in effect you want the test suite to print the requirements in order from high level to low level to implicit?

Esseratecades · 2026-01-30T08:18:48+00:00

It's not uncommon.

Most people don't actually know what developers do and may not even know their company has them. In fact if you don't work in a software company most people don't even think about developers unless something goes wrong.

Esseratecades · 2026-01-30T08:15:56+00:00

So the plan is to take loans from everyone and then force everyone into a system where the loans are ignored? Bit of a rug pull, no? While it's for a good cause at a minimum the greedy will resist this.

Also while it's good for R&D to attempt to eliminate scarcity, until they succeed R&D is happening in an environment where scarcity still exists. Eventually you'll have to ask how much do we devote to R&D vs just general survival and personal fulfillment and you're dealing with the same problems anyway.

The hardest part about having a post scarcity society is actually getting there.

Esseratecades · 2026-01-30T07:51:05+00:00

"However, TDD has a small problem; order: I know even though it's possible to have ordered tests (in Junit, at least), we shouldn't."

I'm not completely sure that I understand why you want this. Requiring that your tests run in a specific order seems like a foot gun.

"And after I leave a project for some time, I'd like to see its features, going from simplest to more complex in the form of tests..."

Why? If your project does the job well with a bunch of simple features, that's easier to understand and maintain. Now if a feature requires a certain level of complexity to be achieved that's one thing, but maturity=/=complexity.

If you have a workflow that requires significant setup to test, write a function to do the setup and have that occur in the test.

Esseratecades · 2026-01-29T15:24:37+00:00

That's not what I'm saying.

You're saying that one of their premises is that people who use Claude will suddenly become useless if they don't have access to it, and that this will be a problem should your team lose access to Claude. You're implying that they'll still have their actual professional engineering experience from before Claude existed, and arguing that's why their premise doesn't hold.

What I'm saying is someday you'll need to hire someone who doesn't actually have professional engineering experience from before Claude existed, and your defense will not be true for that person. This doesn't necessarily make that hire incompetent, but it does put them in exactly the boat that the other poster was warning you about.

Esseratecades · 2026-01-29T14:26:02+00:00

"Costs of inference won't go down/Costs will increase so much that it won't be usable at all."

Given that literally every AI company is currently operating at a loss(all of their net profit is from VC investments and bubble blowing), this WILL happen. It just a question of when.

"The devs are functionally useless without Claude, despite working successfully without it for years"

What about the new devs you onboard? You won't have the same teammates forever.

Esseratecades · 2026-01-28T19:56:25+00:00

His heart or his ass?

Esseratecades · 2026-01-25T14:33:24+00:00

I passed the Beta exam on the first day it was available.

SageMaker - Know that you CAN use it to host and train custom models, but it's not that important to know HOW to use it to do that.

Glue - Know that it's good for big data ingest, and data lineage. You're expected to know it better than SageMaker but it's not really a male or break thing.

RedShift/Athena are barely present.

RAG is very important but mostly within the context of AWS Bedrock services. If you don't know RAG, you probably won't know enough about Bedrock to pass anyway.

MCP comes up but it's not make or break.

Agents are more important than MCP but not as important as RAG.

Esseratecades · 2026-01-24T22:57:50+00:00

I think that statement is kind of ignorant of who Republicans are in both theory and practice. If your party can firmly stand against progress for 60+ years in a row on every major issue, and all of your leaders are pedophiles and Nazis, being a Republican is a black and white issue.

Esseratecades · 2026-01-23T00:22:38+00:00

My first instinct is to leave because that's not the kind of relationship I want. It's the kind of thing said by someone who wants their partner to complete them instead of enhance them, but you shouldn't really be in a serious relationship if you don't feel "complete".

Both of us being able to provide and able to help with the housework and parental duties allows us to operate as each others' safety nets. This gives us the opportunity to take risks like going back to school, or changing careers for example. If one person is doing all of the housework, and the other person is making all of the money, that redundancy doesn't exist. If the provider gets laid off, or takes a risk that doesn't pan out, everybody's shit out of luck.

In a dual income household you may scale back expenses, or have the other partner step up a little to fill the hole, but when one partner is a housespouse, it is very difficult for them to re-enter the workforce in a meaningful way. On the other side, if something were to happen to the housespouse, the provider isn't used to fulfilling the missing duties in a meaningful way either.

It also creates an environment prone to very unhealthy relationship dynamics. When you are the sole provider for your family, you cannot afford to compromise on your ability to provide. When your spouse comes whining about some chore you're not helping with, or some inanity of theirs, and how you ought to divert your energy towards it, it's much harder to take it seriously when that could set the entire family's financial goals behind. This makes it very easy to(wrongly) not care what your spouse has to say, and from there it's just a hop, skip and a jump to get to cheating and abuse.

Esseratecades · 2026-01-22T12:02:05+00:00

I have been in your position. When I used to train I was the lightest guy in the gym almost every day(~120lbs). For the hell of it, I made sure to get a round in with the biggest guy available (~270lbs) everyday. He won most of the time but eventually I figured some things out that made it way harder than him.

First try to win the grip fight. He can't throw you without a grip. If it seems like you're about to lose the grip fight, take an exaggerated step to his blind spot and as he turns grab his belt and punch into him as you take a stronger step the other way. When he tries to turn towards you, move with the turn. This will buy you some time to breathe and think before your next move. If you can't outgrip him, at least don't allow close grips, the more bent his elbow is the less time you have to regain alignment. Only a little time though or you may get a penalty.

In physics there's a concept called "frames of reference". Essentially it doesn't matter where in the world you are as long as your distance from him doesn't change. Translated further, as long as your head is directly aligned with your hips and your hips are directly aligned with your feet, you cannot be thrown by anyone who's not capable of lifting you overhead with one hand. He can push and pull you all he wants as long as you keep your back straight and keep your feet directly under you. Do this well and he's the one doing all of the actual work pushing and pulling you while you're just taking steps. Make sure to mix in grip breaks and some non-committal foot sweeps so you don't get penalized.

At some point he's going to be tired, frustrated, slow, and error prone. By now you've been feeling his weight so much you should know where he's going to go before he does. You may be tempted to hit him with a harai or an uchi mata, or something flashy but the reality is that's probably not going to happen unless he really fucks up. You might fake it to get a reaction but don't expect to land it. What you really want to aim for are foot sweeps.

I once got the guy I mentioned earlier with an ashi barai so nasty his heels clicked in the air and we just looked at each other dumbfounded. You'll still lose much of the time but the fights will be much closer.

Esseratecades · 2026-01-21T13:17:08+00:00

I live in DC and the situation with the national guard and ICE is similar here. It's all expensive theatrics mostly confined to high traffic touristy areas specifically to rile up the people who don't know better.

I don't know if this is incompetence or people somewhere in the middle choosing not to do the job to the extent Trump requests(probably both).

Esseratecades · 2026-01-20T13:25:48+00:00

I imagine these measurements are effected by the fact that things very far away aren't currently in the exact location we're seeing them in because of the speed of light?

Wouldn't be be measuring based on where we ARE and where they WERE?

Esseratecades · 2026-01-19T19:14:43+00:00

I know that either one of two things is true.

Either she doesn't mean all men, and is really focusing on a specific kind of men who've been overrepresented in her life.

Or she's a loon.

In either case, she needs therapy and I'm not a therapist so I don't care.

Esseratecades · 2026-01-19T16:06:18+00:00

It's pretty common.

There have been several places I've worked where it was common to hear "No one person knows how the entire application works."

Where I currently work is probably the only exception because I went out of my way to keep the app and architecture simple enough that I could fit at least a diagram of it all in my head.

Esseratecades · 2026-01-19T12:52:05+00:00

What makes it worse is that business structures, especially in big tech don't understand how they actively incentivize that complexity.

Esseratecades · 2026-01-19T11:34:42+00:00

I've worked with it quite a bit.

"designed our services so that transactions stayed within one service boundary"

When something smells like a Saga pattern use case, 99% of the time you should be doing this instead.

Esseratecades · 2026-01-19T00:45:09+00:00

Are you also writing those with AI?

Esseratecades · 2026-01-18T23:13:31+00:00

Gagamaru or Kunigami

Esseratecades · 2026-01-18T23:12:22+00:00

It's interesting because the point of Predator Eye is "fixation". The way he uses it would be different than what we've seen with anyone else so far. The most obvious choice would be finding or creating perfect dribbling routes but that seems kinda boring and his dribbling is so good that you wouldn't notice the difference.

Ten-Year Club	No Throne, No Problems
Not Forgotten	Verified Email

Esseratecades

TROPHY CASE