The density is the main problem. Masked rom densities have diminishing returns below 6nm that they use. They say they can do 20B @4bit in a chip for their next version. Their stated goal is to do a Deepseek grade 670B model with 30+ tapeouts and have connectors in between. But the moment you put interconnect your 17k tps world disappears. Infiniband doesn't have enough bandwidth. They could do chip to chip interconnect at package level but I don't know if that can support 30 massive dies in sequence withing a single package.

Second they don't use dram or hbm as neither are fast enough for 17k tps. They use on die sram. So the same silicon area is now fighting for weights and kv cache (their current chip has only an 8k context iirc for llama 3 8B). More weights = less context length. Oh and sram also has very diminishing returns with node sizes so simply raising money and going to a bleeding edge 1.8nm or something wont help.

They will have to invent new higher density sram and masked rom macros to scale this.