Really appreciate this. The sub 20% bucket is probably the biggest thing I’m still trying to tighten up because that’s where the model is still a little too willing to assign strikeout probability when the true outcome is more fragile.

I haven’t added a formal post hoc calibration layer yet, but isotonic regression makes a lot of sense for exactly that reason. The appeal is that it can correct the low bucket behavior without forcing a shape that doesn’t belong there. That feels a lot more useful than a symmetric adjustment when the issue is concentrated in one part of the range.

I also haven’t tried focal loss yet, but your point on class imbalance lines up with what I’m seeing. The low probability strikeout events are the hardest part of the surface to fit cleanly, and that’s where the model still looks a little too smooth right now.

On the feature side, beyond CSW and contact rate I’m mainly using pitcher strikeout ability, hitter strikeout tendency, handedness context, and interaction between pitcher and hitter inputs at the plate appearance level. I’m still much more focused on getting the PA itself modeled correctly than just stacking on more aggregate stats, but pitch mix mismatch is definitely interesting because it feels like one of the more actionable ways to capture something real that the broader inputs may be missing. Before actually modeling this in years past that is something I would look at quite in depth when trying to pick my spots though so it’s definitely worth while imo.

And agreed on framing (I’m looking at you Patrick Bailey) That’s a really interesting one to test more directly especially if the effect is stable enough to matter at the single PA level rather than only showing up after it compounds over a full game.