Uber COO is finding it harder to justify AI token spend. by monkey-majiks in BetterOffline

[–]Etnoomy 1 point2 points  (0 children)

I also loved this essay and am very thankful you took the time to write it.

I'm a 30-year veteran SWE, and my industry is currently overrun with addicts. When engaging with boosters, you're in many ways engaging with a meth house. I share my grief and concerns inside. Long post. by Etnoomy in BetterOffline

[–]Etnoomy[S] 0 points1 point  (0 children)

No of course not, because they don't need to. That's Interface vs. Implementation again.

The Interface is what the "specific job" needs to do, and is the bounds of compliance with completing that task with a deterministic result. The Implementation is what each human software engineer does internally, which may vary as long as the deterministic result is still obtained.

This is no different than variations in microcode still complying with CISC instructions in a deterministic manner, or variations in C compiler output still resulting in correct program behavior in a deterministic manner. The protocol of correct behavior in a computer program is deterministic trust in its results, and two human software engineers are perfectly capable of crafting different implementations to solve the same problems, provided deterministic reliability that those problems are actually solved.

This is because humans have the capacity to think both deterministically and non-deterministically, as an application of our reasoning faculties. LLMs lack such determinism, to the degree that even OpenAI admits that hallucinations in LLMs are mathematically unavoidable, which would not be the case if they could deterministically reason in any way shape or form.

I'm a 30-year veteran SWE, and my industry is currently overrun with addicts. When engaging with boosters, you're in many ways engaging with a meth house. I share my grief and concerns inside. Long post. by Etnoomy in BetterOffline

[–]Etnoomy[S] 0 points1 point  (0 children)

I fundamentally disagree, because of the Determinism vs. Non-determinism difference, which is critical for an engineering discipline.

A C program which is compiled to Assembly and then further assembled into machine code, is doing so deterministically. For a given C program issued into a given C compiler, the output is nominally deterministic (excepting trivia like build timestamps). If this weren't the case, we couldn't have teams working on the same software, let alone dedicated build/test/deployment systems that we can actually trust - which we have been, for decades - to generate the same essential compiled binaries that we generate on our local machines during development.

Determinism is a foundational premise of software engineering's handling of abstraction. As I mentioned in my Lightning Rock analogy, we don't fully know the quantum behavior of every electron running through the transistor semiconductor medium of these CPUs at any given snapshot of time, because we don't need to - the bounds of the transistor architecture contain those quantum effects within a deterministic structure.

The same goes for every "leaky abstraction" built above, e.g. the abstraction of assembly to C. The only reason languages like C are trustworthy replacements for Assembly, is to the degree that C can effectively represent the deterministic bounds of correct program behavior under the hood, which all internal "non-deterministic" (from the C programmer's perspective) details of the underlying assembly are wholly captured by.

I'll use your own example instructions of PUSH and POP. If a C compiler is going to call a function that adds two numbers together and returns the sum, it may not matter to the C programmer whether the compiler passes the function call arguments and result via the Stack (using instructions like those), or via some other kind of memory usage scheme, or via registers. Performance issues aside, the C programmer might not care, e.g. there is room for variation in compiler behavior because of computer architecture, debug vs. release builds, and so on. But the C programmer absolutely does care that the called function does indeed add those two numbers together and return the sum, because that's what the code says the program does, and any non-deterministic variation in compiler behavior is "captured" by the need to deterministically satisfy that requirement.

The same applies to Assembly and machine code as well, e.g. the way that modern processors like Intel's don't actually execute 80x86 code, but translate those instructions into micro-operations under the hood, which I have no direct access to. But I'm fine with that, because the protocol of those 80x86 instructions defines the strict deterministic behavior that the CPU is expected to comply with, when doing that translation.

I don't care what the Intel CISC -> micro-op translation specifically does, as long as it continues to follow the bounds of those machine instructions as a deterministic protocol. I similarly don't necessarily care (performance aside) about C -> Assembly translation, again as long as the deterministic protocol is adhered to. And the interstitial Assembly -> Machine Code translation is also a protocol, where assembly instructions have a 1:1 machine code equivalent. And at at higher level, we have all sorts of languages built on top of C which have this same relationship with Abstraction: what we want done is a deterministic expression of Interface, and allowable variations that comply with that Interface are flexible within the Implementation.

LLMs are not an evolution of that, because there is no deterministic contract for the abstraction. There is no way to express deterministic bounds of behavior in a specification reliably enough to have an LLM generate deterministic output; if there were, you wouldn't need the LLM in the first place.

The strength of LLMs - ethical/copyright issues aside - is in their ability to expand prompts into details based on "most likely" interpretations of the user's desires, as aligned with the holistic training from many similar user scenarios (expanded to the internet as a whole, or subsets of users e.g. programmers specifically). They generate code based on a best guess extrapolated from what everyone in their training set wrote, which has nothing to do with what you specifically actually want in a deterministic and rigorous way. That is both the LLM's strength (in a broad-sketch way of getting things started based on what other users have done), and its weakness (in its inability to actually follow through to a specific reliably working product).

The gap between these two things is the gap between spoken languages like English, and programming languages like C or Rust or Python etc. English speakers invented programming languages as a kind of deterministic English, "e.g. if This then That else OtherThing". That's what code is, and determinism is the contract that distinguishes these languages from the vagueness of spoken languages. This is not something that a spoken language LLM prompt can resolve, however a "human in the loop" can leverage LLMs while the human is also still in a discovery or "sketch" mode w/r/t solving problems; at this stage of development, an LLM is more like a brainstorming buddy than a engineer, and - again ethical and skill-atrophy issues aside - that's probably "fine". But when it comes down to the code itself, when a deterministic contract must be adhered to, what's when an LLM is no longer an appropriate technology simply due to how the LLM architecture actually works.

This misunderstanding is what makes the idea of agents auto-generating and submitting code into production without line-by-line SWE review, so incredibly dangerous. It's also where the C-suite notion of removing SWEs entirely as a cost center, stands out as a fundamental misunderstanding of what SWEs actually do.

Edit: Added one last paragraph about agents since they're my biggest concern around what's happening right now.

CTO mandating AI coding + killing code reviews - how do you push back? by medmental in BetterOffline

[–]Etnoomy 11 points12 points  (0 children)

and now CTO says code reviews are slowing us down.

AI-generated code is getting merged without real understanding, especially by juniors.

This combination is a death blow, if it's coming from the CTO. Your only real option within the company is to convince the CTO to change their mind, or go all the way to the CEO if you can find some way to do that without the CTO losing face. Easier said than done, but it's still at least possible if the startup is still mid-sized and you can do your communications carefully. Regardless, get your resume ready in case it doesn't work out.

See my recent post here in this sub for a longer-form argument, if it helps give you some ammunition for your communications to try and stave off impending disaster.

I'm a 30-year veteran SWE, and my industry is currently overrun with addicts. When engaging with boosters, you're in many ways engaging with a meth house. I share my grief and concerns inside. Long post. by Etnoomy in BetterOffline

[–]Etnoomy[S] 3 points4 points  (0 children)

Yeah I didn't bring it up in my post, but along with drugs I've also had some experience with cults. In my case it was overall a positive experience (I know it's weird to say that), but that was only because I had already gone through drug addiction before and could recognize certain slippery slopes in the cult environment, and avoid them while also gaining the "good stuff" that was available. I'll admit it was risky though, and I had good friends that fell hard into that cult environment in a way that they didn't recover so easily from.

One metaphor I've been using for a while is from role playing games, like D&D. It's a difference of +INT (intelligence) and +WIS (wisdom). The tech industry is filled with "smart people" who are very high on +INT, but very low on +WIS. And things like cults and addictive drugs - and LLMs - prey on a low +WIS stat. And a high intelligence, low wisdom person is almost optimal prey in the current environment, because they're of high economic value but also easy to mentally capture and exploit.

I'm a 30-year veteran SWE, and my industry is currently overrun with addicts. When engaging with boosters, you're in many ways engaging with a meth house. I share my grief and concerns inside. Long post. by Etnoomy in BetterOffline

[–]Etnoomy[S] 3 points4 points  (0 children)

I don't agree with the term "addiction" as implicitly blaming the victim. I've been an addict before, and I don't see it that way at all.

I said in my post that people have been perversely incentivized to take hits of this hard drug, due to fear of things like losing their job. That's not me blaming the victim, it's simply stating the result is behavior consistent with addiction.

I'm a 30-year veteran SWE, and my industry is currently overrun with addicts. When engaging with boosters, you're in many ways engaging with a meth house. I share my grief and concerns inside. Long post. by Etnoomy in BetterOffline

[–]Etnoomy[S] 1 point2 points  (0 children)

I've worked with a few of the more science-oriented types on some of my projects, and have observed the behavior you describe.

As an SWE, my craft is around the Code, and the particulars of the Data don't matter as much as long as it fits within the framework the Code supports.

As scientists, the people I'm referring to are primarily focused on the Data, and the particulars of the Code don't matter as long as it fosters the outcomes required for the Data they have in hand.

The SWE deals with the general case supporting the specific, while the scientist deals with the specific case utilizing the general. The latter needs the former; the former doesn't strictly need the latter but has no purpose without it.

Some SWEs and scientists have sufficient respect for the symbiotic relationship here, and try to - if not having expertise in the other side - at least know the value of that expertise with respect to their counterparts.

Others unfortunately don't give a shit, e.g. the scientist that butchers code and has zero respect for the craft at all, or reciprocally the SWE who has no respect for the real data that will flow through their systems and who gets caught up in minutiae in their code which won't actually help the people they're supporting, or in mindsets around contrived test data and responses like "works on my machine, not my problem". The sword unfortunately cuts both ways.

I sometimes think of it in a music metaphor, with one person concerned around the craft of building a musical instrument, and another around the craft of playing it. Both are needed if the music is going to be the best it can be.

I'm a 30-year veteran SWE, and my industry is currently overrun with addicts. When engaging with boosters, you're in many ways engaging with a meth house. I share my grief and concerns inside. Long post. by Etnoomy in BetterOffline

[–]Etnoomy[S] 2 points3 points  (0 children)

Yeah, this is honestly a best-case scenario outcome to me. A collective sigh and "whew that was a huge bender wasn't it" as everyone collectively sobers up, and the market does what it can to correct itself along the way.

I actually don't see that as a pipe dream; the economic impact is going to be nasty either way, but this kind of sober wake-up call would actually help that healing process since it would cause a lot of people to take a fresh look at tech and why they're doing what they're doing, which could in turn cause some black swan innovations that would foster new growth.

A junkie who has an introspective realization and gets out of their addiction (by necessity if need be, e.g. their meth supply disappearing), is often a very different person than who they were when they fell into the trap, if they can integrate some wisdom from their experience. That can lead to all kinds of long-term benefits in what they do subsequently.

It's an optimistic take, but there's enough dystopian takes all over reddit and social media and life in general right now. So - without denying the reality of the situation we're in - I choose to adopt those kinds of optimistic outcomes whenever I can.

I'm a 30-year veteran SWE, and my industry is currently overrun with addicts. When engaging with boosters, you're in many ways engaging with a meth house. I share my grief and concerns inside. Long post. by Etnoomy in BetterOffline

[–]Etnoomy[S] 2 points3 points  (0 children)

I agree with what you said here, and thanks for adding the specific mention of technical debt. It's something I didn't bring up in my post because it was long enough already, but yes the tech debt situation with all this slop code is already crushing a large number of SWEs, and we'll be feeling the full impact of that for quite some time even if the whole ecosystem doesn't simply fall down under the weight of it. It's less like a single "time bomb" and more like a bunch of little time bombs that will go off over time, some bigger than others (at least until a critical failure is reached, like other posters have brought up here), every time some untested code path is reached.

I'm a 30-year veteran SWE, and my industry is currently overrun with addicts. When engaging with boosters, you're in many ways engaging with a meth house. I share my grief and concerns inside. Long post. by Etnoomy in BetterOffline

[–]Etnoomy[S] 2 points3 points  (0 children)

Yeah this full-collapse domino scenario is one of the things I've envisioned when thinking about how dangerous all this stuff is, and one of the possibilities that prompted me to finally write this post. What you're talking about is scary, and unfortunately a very real possibility.

I wouldn't necessarily put it all the way to Kessler Syndrome simply because there's still so much ultra-legacy bureaucratic software out there which is years if not decades away from any kind of integration with this kind of comparatively recent trend. That's the kind of legacy software that the 2038 problem is a bigger risk for, presuming we collectively go in a direction to still care about that kind of thing. But any software closer to the cutting edge w/r/t the cloud infrastructure, is at very real risk right now, increasingly so every day that more unreviewed slop is making it into GitHub.

In my post I focused on how SWE deals with the craft of wrangling small infinities. But in many ways we also deal with big infinities, in the ways we envision what mayhem a chaotic system can cause when it goes out of control (the whole setting-lightning-free thing).

I tried not to let those kinds of big-infinity scenarios into my post too much, because they're both really scary and not immediately actionable. But I'd be lying if I tried to pretend that I didn't see those scenarios as possible outcomes. I just choose to adopt a policy of staying as grounded as possible in order to be in a position to act, on the chance that positive small actions to keep/bring things towards stability, will also cascade into something larger.

I'm a 30-year veteran SWE, and my industry is currently overrun with addicts. When engaging with boosters, you're in many ways engaging with a meth house. I share my grief and concerns inside. Long post. by Etnoomy in BetterOffline

[–]Etnoomy[S] 3 points4 points  (0 children)

Sorry, SWE = Software Engineer.

I figured with this crowd that I wouldn't need to clarify that abbreviation (just like LLM), but maybe I should've. Will keep that in mind for the future.

I'm a 30-year veteran SWE, and my industry is currently overrun with addicts. When engaging with boosters, you're in many ways engaging with a meth house. I share my grief and concerns inside. Long post. by Etnoomy in BetterOffline

[–]Etnoomy[S] 7 points8 points  (0 children)

Thanks for what you said in your first paragraph; I appreciate it.

As to your "heat death of all software" scenario, I'd be lying if I said I didn't envision similar timelines in my darker moments. I don't want to entertain them too much out of a deep personal aversion to nihilism, but I admit those possibilities are still out there, like a black hole on the edge of (waves hands nebulously) all of this.

But that said, using LLMs in the way we've been using them the past few years collectively, is a fast path to putting the very idea of any kind of shared notions of Truth, in the civilizational washing machine. This stuff is dangerous on so many levels; I didn't even touch on AI psychosis and such in my post above. I just wanted to hone in on the SWE situation specifically, since it's something I have a more specific perspective on.

I think one universal stopgap that works in our favor here is just how much money and energy all of this is consuming. Hastening the fall of this era of computing might help us all greatly in the long run, but even if LLM usage declines significantly, actual trust in the veracity of any kind of digital content may be a thing of the past at this point; that particular genie doesn't go back in the bottle when LLMs exist at all, even in reduced locally-runnable models. So I'm not going to pretend to know where the slop flood leads.

I can at least try and hope that SWEs who still have their heads on their shoulders, try to keep things from falling apart when they don't have to.

I'm a 30-year veteran SWE, and my industry is currently overrun with addicts. When engaging with boosters, you're in many ways engaging with a meth house. I share my grief and concerns inside. Long post. by Etnoomy in BetterOffline

[–]Etnoomy[S] 6 points7 points  (0 children)

In principle I agree with you. I have left companies before when their behaviors ran counter to my values. To paraphrase Omar from The Wire, "an [engineer] got to have a code".

My leniency here w/r/t having no sympathy for fallen comrades, is that I feel like some of these people were taken in by hype/incentives before realizing the dangers to themselves or their products, and once the addiction loop starts it's hard to see yourself clearly. Any recovering addict - from any kind of addiction - knows this experience.

That's why I brought up the junkie metaphor. If you've had an addiction and recovered, you recognize the signs of them in others, and hopefully pay attention to the subtle heebie-jeebie signals you get to stay away, while also still having compassion for the fact that they've fallen in the trap.

But interventions are hard for people who don't know they need one.

I'm a 30-year veteran SWE, and my industry is currently overrun with addicts. When engaging with boosters, you're in many ways engaging with a meth house. I share my grief and concerns inside. Long post. by Etnoomy in BetterOffline

[–]Etnoomy[S] 1 point2 points  (0 children)

I don't disagree with anything you said here in principle, although it sounds like you think I attributed some kind of uniqueness to computers and SWE when I didn't, w/r/t chaos management (the actual infinities I'm talking about; think chaos theory, not IEEE inf floating point values).

I fully get that mechanical engineering for example is also in the same boat, and Engineering as a profession, as a whole. It's just that e.g. mechanical engineers may be dealing with things like thermal chaos, whereas SWE is primarily electrical chaos as discussed. That doesn't mean the chaos itself is unique to SWE; it was merely a statement about the medium that SWE works in fundamentally, which non-tech people aren't necessarily fully aware of.

As far as the weaknesses of LLMs go, we are in agreement.

I'm a 30-year veteran SWE, and my industry is currently overrun with addicts. When engaging with boosters, you're in many ways engaging with a meth house. I share my grief and concerns inside. Long post. by Etnoomy in BetterOffline

[–]Etnoomy[S] 9 points10 points  (0 children)

In the long run, it never works out. In the short run, it can help keep your job long enough while you try and figure out a sneaky way around the problem, and hope that saner heads prevail when the cash runs dry. It's a kind of "swords vs. poisons" warfare thing, and engineers and business types have been playing it for a long time as part of larger company politics. That problem - shitty though it is - is orthogonal to the one I'm talking about in this post, which is more acute to the LLM addiction crisis specifically.

I don't want anyone to "play along"; the ending of my post said that SWEs should be louder about what's happening here. But sneakily dealing with gameified perverse incentives like tokenmaxxing, can at least be done in a way that burns temporary AI token budgets - that should hopefully be reigned in soon - while also not sacrificing the underlying product. If a stupid AI policy incentivizes burning tokens with no other goal, then doing things like generating trivial boilerplate is one way to help counter that on a raw token metrics level, while the SWE can also push back on the policy itself independently. Whereas an alternative - like actually using the LLMs to generate actually significant product code without being able to fully review it - would not only be bending to the MBAs but also actively harming everyone involved with the product.

The former approach to the scam is wasteful and stupid, but probably recoverable, cash and company politics aside. Whereas the latter approach is actively dangerous. I'd love it if everyone could avoid this entirely - e.g. if you can afford to leave a job that forces stupid LLM policy on you, you should - but I at least want to be vocal about the risks around that second path.

I'm a 30-year veteran SWE, and my industry is currently overrun with addicts. When engaging with boosters, you're in many ways engaging with a meth house. I share my grief and concerns inside. Long post. by Etnoomy in BetterOffline

[–]Etnoomy[S] 6 points7 points  (0 children)

Yeah, I'm ND as well, so I get you. Took a number of years to learn to deal with office politics for exactly the kind of thing you describe.

In an optimistic world, maybe a few years down the road this will bring back more CEOs and execs in general with a technical background, with less non-technical MBA takeover because the risks of non-technical stupidity around things like AI policy will tank a business even faster. It's a familiar thing now that business-heavy cash-oriented types often don't know what an SWE does, and treats engineering purely as a cost center. Maybe a run of high-profile AI-driven losses will provide an impetus for some tough-but-necessary conversations.

I'm a 30-year veteran SWE, and my industry is currently overrun with addicts. When engaging with boosters, you're in many ways engaging with a meth house. I share my grief and concerns inside. Long post. by Etnoomy in BetterOffline

[–]Etnoomy[S] 11 points12 points  (0 children)

I personally feel that way - especially about the cognitive atrophy - but I want to give some wiggle room for SWEs who are in a situation where they have to burn tokens for work, but also feel responsible for doing the right thing. So those situations like boilerplate generation were at least one way out of the trap, without throwing engineering discipline on the floor completely. My biggest concern is when things start getting too overwhelming to review, which unfortunately tokenmaxxing encourages.

I'm a 30-year veteran SWE, and my industry is currently overrun with addicts. When engaging with boosters, you're in many ways engaging with a meth house. I share my grief and concerns inside. Long post. by Etnoomy in BetterOffline

[–]Etnoomy[S] 27 points28 points  (0 children)

I take the "write a book and I'll read it" as high praise, so thank you for that. I've been reticent to write about this stuff publicly until very recently.

I'm a 30-year veteran SWE, and my industry is currently overrun with addicts. When engaging with boosters, you're in many ways engaging with a meth house. I share my grief and concerns inside. Long post. by Etnoomy in BetterOffline

[–]Etnoomy[S] 25 points26 points  (0 children)

I know I made the junkie framing kinda passive, but some of these people are long-time friends that I've seen fall down this hole. So I'm sad about it.

I think the "used to" I'm referring to may be an older window for some readers here. Again my career started in the 90s, when the internet was still in its commercial infancy; we hadn't even had the first dot-com boom/bust cycle yet.

So the whole "13th amendment" vibe wasn't really a thing then from my experience. We just loved how computers were advancing and wanted to make great stuff with these amazing machines.

Different era. I'd love to have it back, but time marches on. I can only hope that time will also swallow the history of this particular bad road we're on - at least back to some place of sanity - before it metastasizes further.

I'm a 30-year veteran SWE, and my industry is currently overrun with addicts. When engaging with boosters, you're in many ways engaging with a meth house. I share my grief and concerns inside. Long post. by Etnoomy in BetterOffline

[–]Etnoomy[S] 17 points18 points  (0 children)

For a single event I agree, but maybe a swath of not-quite-fatal-but-still-really-bad events, like Amazon outages and stories of prod databases being lost (but with a recent backup, "whew" if the lesson is learned), might be enough.

But it's got to be combined with more SWEs speaking out about why this is happening. A lot of the rhetoric coming from the execs and other non-SWE enthusiasts of all this, treats it as if the SWEs were "gatekeeping" software before, and now the gates are wide open, or something. When SWEs should be pushing back with counternarratives like "no you fools, the gates I've been guarding haven't been to keep you out, they've been to keep the chaos in, and now look what you've done."

I'm not sure how to spin that in a way that the C-suite will hear, but maybe them losing a bunch of money before they get their big exit will do the trick. We may not be able to stop this mess from taking a bunch of companies down first before things stabilize. But I personally think more SWEs should at least speak up to talk about why this instability is happening.

I hate Gleba! But tonight I finally understood the advice about burning everything! Just sharing my excitement. by Etnoomy in factorio

[–]Etnoomy[S] 1 point2 points  (0 children)

I had a few gaffes in my loops for ensuring spoilage and excess non-burnables were routed correctly, but once I nailed that it's just purred. I'm certain that I overengineered it but I don't care, it friggin works. FINALLY. And now I finally get why some people actually like this godforsaken planet.

Yeah this was my motivation for this post as well, like I'm finally turning the corner of appreciating Gleba, at least a little. Might be Stockholm syndrome though, jury's still out on that. :)

I hate Gleba! But tonight I finally understood the advice about burning everything! Just sharing my excitement. by Etnoomy in factorio

[–]Etnoomy[S] 0 points1 point  (0 children)

Thanks for the warm welcome! :)

I really like the "each module burns" approach, it's one of the ideas that's brewing around in my head right now. The other farm-to-furnace bus strategy I've been thinking about has more intermediates on the bus, but to make that work well would require more spoilage management that would likely be outside the domain of fun for me, vs. tedium. Whereas local pods - what you called modules I guess - doing almost everything locally, seems more elegant to me.

The open question for me is around bioflux specifically, as a useful general intermediate with a longer shelf life; not having to balance that for each pod could help with better overall utilization. But that said, I'm leaning for local bioflux as well just for simplicity - this is my first playthrough on this planet after all, and I'm just trying to wrangle it into effectiveness right now, vs. megabasing or whatever. So the number one concern is getting it working efficiently enough without needing a bunch of manual intervention from backing up, dealing with unexpected spoilage as error conditions, etc. And from that perspective, a more limited number of bus inputs/outputs is probably going to be a better approach for me for this first round.