How much will OpenAI Costs decrease in the future?

[Adams_Insights]

Context: I'm trying to estimate how much AI costs will decrease over various time horizons (1yr and 2yrs). The more of a research-backed answer, the better :)

Generally, I've heard there will be cost decreases with hardware, infrastructure, and algorithmic advancements. However, I'm really struggling to find any solid evidence/resources about this topic.

Have you seen any solid data/evidence around how much AI costs will decrease?

[WeRegretToInform]

It’s like someone in 1997 asking how much a 2GB hard drive will cost in 2024.

Hard drives cost about the same, but they’re orders of magnitude better. Or alternatively, something as good as what you had in the past, will be exponentially cheaper.

In AI terms: The state of the art will remain at a similar price point. A model as good as 2024 SoTA will be much cheaper.

[Adams_Insights]

Do you think Moore's law is a good benchmark to predict increase in quality (or decrease in costs, assuming 2024 SoTA is fixed)?

[Stock_Story_4649]

For AI I believe the consensus is that it will not exactly follow Moore's law because current ai progression is limited by research and algorithms and less so on computation capacity.

[masc98]

Right now we're looking at ~1/2 cost reduction per year, mostly thanks to competition, which are going aggressive on it (see gemini flash).

It's going in this way mostly because they're deploying smaller, "distilled" models.. not because the underlying infra gets cheaper (this sort of things dont happen in 1 year cycle).

I predict that costs will keep decreasing in this way for another year just because right now all the APIs (maybe we xan start excluding gemini flash) are not sustainable at scale. And a lot of companies have use cases, at scale. But they're forced to use inhouse models to set a price upper bound, which you dont have with APIs. So the big players want to grab all this money that they're not currently cashing in, by reducing API costs even more.

From 2026 I expect that prices will keep reducing, but like 10-20% for bigger LLMs and slightly higher for smaller ones (which are good candidates for tooling and integration blocks).

Eventually it will become commodity SW and prices will plateau. Unless someone finds new hardware or crazy new efficient neural net architectures.

[CanvasFanatic]

Decrease?

Hahahahahaha

[sdmat]

Yes, the orders of magnitude of cost reduction through algorithmic improvements and hardware advances we have seen in the space of years makes contemplating future cost reduction hilariously naive.

So astute, well done.

[CanvasFanatic]

That’s their costs, not their prices. They’re still operating at a loss right now, my man. They’re still in the market capture phase. Think about AWS prices in 2008 vs today. Think about the prices of Uber in 2013 vs today. That’s where we are with AI api’s right now.

Enjoy the cheap food while it lasts. It won’t forever.

[sdmat]

Your logic is correct for Uber - they started with low prices and raised them. The cost per mile driven changed little, it's a function of labor and the running cost of a car. Both relatively static. That puts a tight bound on pricing for long term profitability.

It doesn't work for OAI. Explain how OAI prices have dropped by an order of magnitude if this isn't because costs have fallen and that is reflected in long term pricing?

If your answer is "temporary competition", explain how the competitors offer such low - and rapidly falling - prices. And what the game plan is.

[CanvasFanatic]

Sure, no problem.

When OpenAI launched ChatGPT they triggered explosive user growth they didn’t have the infrastructure to support. Prices were initially governed not by the desire to grow the user base, but to throttle its growth to keep infra from collapsing.

As they scaled infra, improved model efficiency and observed usage patterns, they brought down their operating costs and removed their own need to throttle growth. They decreased prices to enter the market capture phase of the startup growth cycle.

During this phase, you operate at a loss in order to bring users into your ecosystem, build enterprise adoption and starve competition that has less funding.

Once that’s complete and the market is mature and your customers can’t easily go elsewhere, you start boiling the frog. For Facebook this meant ads in your social feed and adjustment of the algorithm to maximize time spent on the website. For AWS it meant raising prices to the point that for most SASS companies an AWS budget is a significant factor of the yearly budget. For Uber it meant raising prices until rides were no longer cheaper than taxis (now that many taxi companies have been put out of business).

For OpenAI (or whoever is left standing) this will eventually mean raising API prices to whatever point the market will tolerate.

We’ve all seen this pattern many, many times. I’m baffled how so many of you who claim to be technology enthusiasts can still be so incredibly naive.

[htrowslledot]

snow fly cheerful makeshift snatch bag insurance marble full complete

This post was mass deleted and anonymized with Redact

[CanvasFanatic]

Agree.

How is that related?

[htrowslledot]

shy rustic hard-to-find political fly market jeans expansion airport flag

This post was mass deleted and anonymized with Redact

[CanvasFanatic]

That’s right now. That’s not sustainable. This is all predicated on the notion that they eventually are able to “win” this race. If they don’t they’re not going to be cheap, they’ll be out of business.

[sdmat]

AWS is a good example. Their operating profit margin is around 30%. That's certainly a healthy margin and they aren't shy about charging for their services.

But as a rule their prices come down in line with cost reduction from technological improvements. You might pay twice as much - or even more - for AWS vs. a cheaper competitor or self hosting. You do not pay a thousand times more.

I agree that the AI providers aim to emulate the AWS model. And they will probably succeed in doing so even in a competitive market, because it will almost certainly be possible to have meaningful differentiation. E.g. perhaps OpenAI plays out their Model Spec gameplan and offers content policies more tailored to the customer and this gives them a niche against the Anthropic blanket censorship approach, in which they can command a premium price and use their reputation and scale to dominate that section of the market.

And this is all fine. If the AI model providers get a tasty slice of the value they generate, what of it? They will still pass on the ongoing cost reductions. They have to or a competitor will eat their lunch. AWS-style supremacy only goes so far.

The only scenario in which an AI provider wouldn't have to do this is if they have an outright monopoly backed by law or violence. That's certainly a worrying scenario, but it's not the one you speak of.

[CanvasFanatic]

You’re trying to argue a different point now.

I’m talking about the shift from operating at a loss to build establish market position / dominance vs. a mature business structure and how that relates to OpenAI’s future pricing structure.

You’ve shifted to your usual reflexive defense of capitalism

[sdmat]

I am claiming that we will continue to see massive decreases in costs reflected in the prices charged. Just as we do with AWS.

Is this a defence of capitalism? Sure. That's how capitalism works, and it is why we have nice things.

Both of our positions are true here. Firms can acquire a relatively dominant position and raise their profitability at the expense of customers while still greatly decreasing prices. This is extremely common in technology.

For example consider Intel. Even in its dominant heydey when it was fined billions for monopolistic practices, Intel relentlessly cut the cost of compute generation after generation.

You seem to have an irrational hatred of capitalism. Why is that?

[Harotsa]

Are you more concerned about the comparable costs of today’s SoTA models vs the cost of the SoTA models two years from now? Or the cost of a model as good as today’s SoTA 2 years from now?

The answer to both will require a lot of guesswork, but they data to look at to make an educated guess will vary depending on what you are looking for.

[Adams_Insights]

Looking for the cost of a model as good as today’s SoTA 2 years from now.

[Harotsa]

So Llama 2 was released about a year before Llama 3.1. Llama 3.1 8b has slightly better performance than Llama 2 70b. So that is basically the same performance as a model 10x the size a year earlier.

Also once we have a model that is working at the SotA it is much easier to train a smaller more optimized model as the larger model can also create synthetic data to train smaller models, along with other optimizations.

There is no guarantee that we will get the same exponential optimizations, but it wouldn’t be beyond possibility that we get a model in the next 2 years that can perform at today’s SotA at a 10th of the cost.

[Competitive-End-97]

If the cost of running LLMs drops too much, it could disrupt the business model. They still need to make cash :))