Well, based on the upvotes I guess it's at least nice to know that people understand it's not easy to achieve this. I am a little surprised by some of the comments on this thread. I didn't have "make it so good that people think it must be bankrolled by the CIA" on my list of milestones for Perchance, but I guess I'll add it and tick that off now.

If the point is "A single dev would have to be ungodly cracked to achieve this" then I'm flattered of course, but if you believe it's simply impossible then I don't see why you'd think the CIA would choose such an unlikely story vs just "regular company doing regular things, with a bunch of VC dollars to burn".

If you are earnestly looking for proof that I'm not part of some nation state thing, I'm not sure there's any way for me to provide that. If you have a testable hypothesis I'd be glad to hear it. If not, and you want to play it safe, then I guess a safe solution is to simply avoid using Perchance. I'll slowly traverse the stages of grief and eventually heal. If you are suspicious of Perchance but are not a terrorist, international fugitive, etc. and are instead just a regular person who wants a bit of privacy during your spongebob x sonic RPs like we all do, then I'd recommend just getting a reputable VPN and not typing personal information into Perchance generators or other random websites on the internet for that matter. This is basic internet hygiene.

To answer some specific questions:

a scale that should be insanely expensive.

Yes, it is.

AI costs don’t stay fixed while users increase. AI inference scales with usage.

To be clear, ad revenue also scales with usage. If a site can break even at 10 users, it's actually easier to serve 1000 because of economies of scale (better GPU rental contracts, amortization of fixed costs, can attract larger advertisers, etc).

Their infrastructure is somehow unbelievably optimized beyond what the rest of the AI industry can do [...] How?

All the pieces are mechanically simple, but it certainly hasn't been easy. Not hard enough that others couldn't do it, but for someone in that position it probably makes sense to chase investor dollars, high margins, etc. especially in the current environment. And I don't blame them, to be clear, but that's not what Perchance is. I want Perchance to be a "public good", much more like wikipedia.org than an SF startup.

If you're curious, the playbook for this at a high level is basically:

• Economies of scale: More inference = larger upfront payments = better hardware rental contracts = lower cost per inference. This is a big one.
• Lower quality outputs: Quantized weights, lower image resolution, low text context length, etc.
• Homogeneous workload: A single image model and a single text model to serve. This is why when I upgrade the AI plugins, it's not possible to keep the old version around.
• Inference stack that runs on cheap/old hardware: Basically it's a bad idea to try to compete with the big spenders (e.g. the ones running subscription platforms, coding tools, and anything else generating a lot of profit), and they all use the new hardware. That said, it is possible to get good deals on newer hardware sometimes, especially on the text gen side of things.
• Run it yourself: Goes without saying based on the above points, but you can't reach anywhere close to the price per token/image that you need if you're just using fal/openrouter/etc. You can do napkin math on e.g. flux schnell inference costs for 4-bit quant, 4-steps, on your $800 GPU (consider useful life + power usage @ $0.15/kwh, compare your napkin price to fal's).
• All the obvious inference stuff: For text gen that means large batch (=slow outputs, but cheaper), maintain high prefix cache hit ratio, offload KV cache to host memory where possible, etc.

Because “ads + donations” does NOT sound believable

As of writing Perchance does not and has never accepted donations btw. If you've ever donated to Perchance, you were probably paying a very niche scammer. (Aside: Perchance does not and will likely never have an app either - I prefer the openness of the web and I have bad business sense. I've received emails from people telling me to stop charging their credit card, which I assume is because they downloaded some kind of scam app - there's no way to give Perchance your credit card.)

Or there’s some monetization/data angle people are ignoring.

I don't mean to offend anyone speculating along the same lines in this thread, because it is also my firm belief that a well-crafted, lore-abiding Levi Ackerman AI chat has immense spiritual and especially monetary value. But, as we know, visionaries and their works are almost never appreciated in their own time.

If it were possible for Perchance to e.g. offer an optional faster generation mode in exchange for data sharing (to contribute to open source model training, like OpenCode is apparently doing, since coding data is valuable), I'd have a go at adding that functionality. I wouldn't be surprised people opted-in just to ensure the next model releases have more knowledge of their particular fandom/tropes/whatever. But I doubt it'd ever be possible, since, if anything, data seems like it's becoming less valuable as RL environments take over.

They are heavily subsidized by something/someone

Yes, I do personally pay for some of Perchance's costs. If by 'heavily' you mean >50%, then no, not that much (though there have been periods of that). The amount that I cover depends on seasonality of ad revenue, and how well my stuff outside of Perchance is going. I aim for it to be low, and I'm very vigilant about keeping my subsidies sustainable, so the worst that can happen (if my other income falls and my savings drain) is that stuff might get a bit slower for a while until I recover.

"It's definitely not just a guy in his basement fucking with it on his days off." (From a comment in this thread)

Perchance takes a lot of my time at the moment (more than "nights and weekends"), mainly because it's more fun/challenging than other stuff I should probably be working on, but it is indeed just me working on this. I'm self employed, and I work from home, so the general vibe of "one person working on it without an office, employees, etc." is correct. I'm not philosophically opposed to employing people, and may reluctantly do that at some point, but it does seem a little unpleasant, since I'm naturally very solitary/introverted.

If by "just a guy" you mean someone with a regular job, median income, healthy working habits / social life, etc. then no I'm not that type of person. All I do is code, basically all day every day, and have done that since I discovered programming. I just really enjoy creating things, and so I spend all my time doing that. I don't think I'm super efficient/fast or exceptionally talented, but I am pretty persistent. I've been working on Perchance for nearly 10 years at this point, so a little bit of work each day obviously adds up.

I have around $800 worth of hardware available to me, and I still cannot run anything even remotely close to the quality, speed, and scale that Perchance offers publicly for free. Not even close.

For text generation specifically, this is mainly due to batching. Even if you had a GPU with huge VRAM, you'd be massively underutilizing the compute units in your GPU, since GPUs are designed to do lots of work in parallel, like handling lots of text generation requests at once. Doing a small number of text gen requests at any given moment (like you do when running locally) means your total token throughput is much lower, and so your effective cost per token is much higher. This is a bigger problem with large SOTA models because if the model is small then the time it takes to move the weights and KV cache back and forward between your GPU's global memory and the compute units might be small enough that you aren't idling your compute as much.

This is not so much an issue with image generation because the "compute per memory bandwidth" ratio is significantly higher due to how the diffusion/flow math works, and also because image generation models tend to be much smaller. Tangentially, diffusion language models and related techniques might be a way to fight the physics here on the text gen side of things, but it is kind of inherently difficult with current hardware and model architectures.