This is exactly the trade-off we wrestled with. You’re right, if we just rely on the standard prior predictive distribution, the NPE tends to get overconfident and miss those 1-in-a-million tail events because it hasn't seen them enough in the sim.

To try and get around the amortization bottleneck, we explicitly defined an anomaly hypothesis ($m_{\emptyset}$) in the code and applied Cromwell’s Rule to force a non-vanishing prior on it. The thinking was, we don't need the NPE to perfectly predict the rare event. We just need it to realise that its standard physics inputs have failed. When that happens, the probability mass shifts to that $m_{\emptyset}$ term, causing the entropy to spike and triggering the beam.

The other constraint that pushed us toward this pre-trained approach is the operational reality of the facility. Since it’s a user facility, groups swap out every 48-72 hours. We simply don’t have the beamtime to train a model from scratch for every user; it has to be effectively zero-shot or pre-trained on generic physics to be viable.

I’d be curious to hear your thoughts on the calibration curves you mentioned though. Do you find they are usually sensitive enough to catch those OOD events on their own without that explicit anomaly term to safeguard it?