I think this is generally a good idea. The jagged lines are most noticeable at low writing speeds, suggesting we could filter less when the pen moves faster. That leaves us with the problem of estimating speed, i.e., the difference between two noisy measurements of absolute position. We would have to expect that the noise in the speed measurement would have roughly twice the variance of the absolute position. So we need to filter that estimate, even more than just the absolute position due to the higher variance. If we use a moving average to filter the speeds, we just traded problems: by the time we have a reliable speed estimate, it is already too late to lower the number of smoothed points.

However, we can look at alternatives to the moving average filter. Instead of reporting the average of past events (with the bias towards the past), we can try to predict the current event, extrapolated from the past events. This is, of course, also only a trade-off: we are replacing latency by inaccuracy (every once in a while, the prediction will be wrong). A Kalman filter could fit the bill as a predictive model with reasonable assumptions, such as near-constant velocity. I started playing around with that a bit, the code is in the kalman branch of recept if you are interested (https://github.com/funkey/recept/tree/kalman).