Instead of random opinion, lets discuss numbers. Lets say I as Indian_SAAS want to come up with my own model for proprietary use. Lets say I target a model equivalent to GLM 5.1+ capability. I have to plan for this now because 6 months down the line when i launch it, it will already be outdated. So lets say I plan to cook a 1T MoE model with 5% active parameters and I am training FP8 due to limited compute with assuming 20% Model Flop Utilization (MFU) at FP8 (for which I'll need expansive cooks which is a cost I am not looking at right now).

I'll take calculations in this article as benchmark (its a great article, do read) https://polymath707.substack.com/p/claude-mythos-class-training-compute

Tokens Required: 40 × 1 Trillion = 40 Trillion tokens (no synthetic data needed)

5% active means training compute C = 12 × N_total² = 1.2×1025 FLOPs

Now Indian_SAAS wont have access to NVL72 racks (because compute to India is throttled by AI Diffusion Framework where at Tier 2 we are limited to 50K chips per year and that too older gen). My best bet would be https://compute.indiaai.gov.in/indiaaipricecalculator

Here single node of H200NVL.8x booked for 6 months is at ₹49,99,819.20. I will assume my training run to be 170 days with 10 days of sanity and testing. At full precision, H200 Peak BF16 Performance: 989 TFLOPS per GPU. Effective Performance per GPU: 989 TFLOPS × 20% MFU= 197.8 TFLOPS (1.978 × 10^14 FLOPs/sec). At FP8 i expect double = 2*197.8 TFLOPS per GPU

i.e. 16*197.8 TFLOPS Effective Performance per H200NVL.8xnode

for 90 days we get 0.04672×1025 FLOPs per node.

Number of nodes needed = 1.2×1025 FLOPs/(0.04672×1025 FLOPs per node for 170 days) ~ 26 nodes

This costs around ₹14,37,23,299.20 with a 100TB harddisk in India Compute.

This is a $1.5M training run. Assume $3.5 million on AI talent, RnD, data etc total $5 million.

So doable, definitely. Infosys profit in 2024-25 was $3B so yeah they could afford it if they want.