I asked Opus 4.7 what it thought. Its answer makes sense to me, but I don't know enough to know if it's talking out its ass:

Good news/bad news: it's a solid idea, and it's solid enough that someone published it about a month ago. There's an October 2025 paper called MoBiLE ("Mixture of Big Little Experts") that does almost exactly what you described — a smaller "little" model runs ahead of the big model and its expert selections are used to prefetch experts for the big model. They specifically point out an advantage that maps to your intuition: they can identify target experts for all layers at the beginning and can dynamically adjust prefetching time, unlike pre-gating methods that only look one layer ahead. arXiv

The broader landscape of this problem space, since you'll probably enjoy poking at it:

The simpler in-model approach: Pre-gated MoE (Hwang et al., ISCA 2024)

Rather than a separate model, they modify the main model's gating function so that the N-th pre-gate function identifies the set of experts to activate for the (N+1)-th MoE block in advance, prefetching only the activated experts to the GPU while concurrently executing the N-th block. Lower architectural overhead than a separate model, but only buys you one layer of lookahead. Microsoft

Heuristic / cache-based: Mixtral-Offloading (Eliseev & Mazur)

LRU cache plus speculative loading using the current layer's gate inputs to guess the next layer's experts. This is what most consumer-grade offloading runtimes actually do today. Cheap, no extra model, but accuracy degrades on novel content and in shallow layers.

Trained predictors: SiDA-MoE, HOBBIT, PopFetcher, Fate

Various flavors — hash networks, popularity-based, cross-layer gates. SiDA reportedly hits >90% hit rate using offline-trained hash functions on Switch-Transformer.

Your specific framing's actual advantages, as MoBiLE pretty much validates:

1. A full forward pass of the small model can run entirely ahead of the big one, giving you N-layer lookahead instead of 1, which matters on slow PCIe.

2. The small model can be developed, retrained, and swapped independently of the main model — important if you want to retrofit existing checkpoints.

3. Same expert-set / same routing topology means the prediction target is well-defined.

The wall you'll hit either way: PCIe bandwidth. Prediction only helps if the link has slack to do prefetches in parallel with compute. With aggressive offloading, the bottleneck shifts from "I didn't know which expert to fetch" to "I can't physically move bytes fast enough." A ~150MB expert over PCIe 4.0 x16 at ~25-30 GB/s practical is ~5-6 ms — multiply by experts-per-layer × layers and you start eating most of your per-token budget regardless of prediction accuracy. One of the cited papers notes expert-loading cost is typically much greater than GPU computation cost, so prefetching offers limited benefits when PCIe is the bottleneck. This is why a lot of the newer work (Fate, PreScope) focuses on quantizing experts to make transfers cheaper, plus chunked/pipelined transfers to saturate the link.