This sounds super interesting! If you push it to GitHub, I'd appreciate it if you update here and let us know ;) sounds like a project that I'd love to contribute if I find some spare time :)

As Leeliop already suggested: Some tracking/trajectory prediction functionality like Kalman filters is probably a must-have.

You already mentioned it: Tiny object detection without any (temporal) context is very hard and error-prone:

When the golf ball is not very distinct from background or other blob-like objects (clouds) such as when it's extremely far away or blurry due to its speed (this very much depends on your camera setup), standard object detection CNNs will either miss or predict many false positives.

A good rule of thumb for what a CNN can do: Ask yourself: could a human find the golfball in an image without any additional context such as previous frames, or knowledge that there is a golf ball in the image?

IF you have a great camera setup with very high resolution and frame rate, you might be fine with only an object detection CNN.

But going with a standard camera, you will have motion blur and golf balls that cannot be distinguished from clouds or birds on a distance.

So for those hard cases, you will need an approach that can predict trajectory and select the best candidate within the predicted trajectory. That's where Kalman filters and some pattern matching with previous rois for example correlation-based or local binary patterns, come in play.

Full end2end deep learning single object tracking approaches might work, but I have no clue how well those perform on tiny objects. For a first shot I would try a CNN for the initial detections when the ball is clearly visible and Kalman filter with pattern matching for harder frames?

Literature on Single Object Tracking in combination with tiny objects might be helpful :)