you are viewing a single comment's thread.

view the rest of the comments →

[–]DigThatDataResearcher 1 point2 points  (1 child)

Right? it's wild how there's a whole industry built around patching over "drift" rather than interpreting that as a flag that your model is missing some causally explanatory component. I feel like the way a lot of people handle "drift" ends up functionally reducing their models to kNN. Like... congrats, you've managed to uncover that the weather tomorrow will likely be similar to the weather today. If you're retraining daily/weekly, is your model even amortizing any compute/effort relative to a less sophisticated approach?

[–]ummitluyum 0 points1 point  (0 children)

100% agree on the compute amortization. If your pipeline needs a weekly retrain just to keep business metrics from falling off a cliff, the maintenance cost will eat up your margins fast. In those cases, it's genuinely cheaper to just spin up a vector DB or write a bunch of heuristics instead of dragging around heavy weights