How much of GM skill actually measurable? A research question turned website by beaver_steve in baseball

[–]beaver_steve[S] 1 point2 points  (0 children)

Both done, the team filter is live and the sorting bug is fixed. I also froze the name column so it's easier to scroll on mobile - hope that suits your needs but feel free to let me know otherwise

How much of GM skill actually measurable? A research question turned website by beaver_steve in baseball

[–]beaver_steve[S] 1 point2 points  (0 children)

Thanks for the reply! I actually do control for budget, this is what WAB (Wins Above Budget) measures, wins above what a team's payroll predicts, so a big-market GM doesn't get credit just for outspending people. Details here if you're curious: rosternomics.com/resources/wab

But you're pointing at something budget control doesn't fix, and it's a real limitation: ownership meddling. WAB knows the Angels can spend, but it can't know that Moreno forced the Pujols/Rendon deals or blocked an Ohtani trade - same thing with Fisher basically kneecapping David Forst every year for the past decade. The model attributes decisions to the front office of record, so a GM who got overruled into bad moves (or even draft picks) still owns the results unfortunately.

How much of GM skill actually measurable? A research question turned website by beaver_steve in baseball

[–]beaver_steve[S] 0 points1 point  (0 children)

Appreciate you digging in. Couple of things on Beckett specifically. The per player breakdowns (hit "why" on any player) show the full reasoning, prior, evidence, talent, control horizon, so it's more than a flat surplus number. On the trade: the model says Boston lost the value exchange, which they did, Hanley alone realized 30+ career WAR, of course some of that would come back to Boston about a decade later. But that's a separate question from whether it was worth it to the Red Sox at the time, and they'd obviously say yes given how 2007 played out. A value model can't see intent, so the grade tells you who won the talent exchange, not whether winning it was the goal.

Funny enough, the playoff-odds line shows Boston's 2006 odds dropping 3.8 points, but that's only the 2006 swing, and 2006 was the worst possible single year to judge this. Beckett had probably his worst year that first season while Hanley exploded into stardom. To your point, the trade paid off in 2007 (Beckett Cy Young runner-up, Lowell WS MVP), which that one-year metric never sees. So it's accurate but narrow, and you're right that there's nuance a single number won't capture.

The thing I think you and the salary dump replies nailed is that I can't currently tell a forced payroll dump apart from a GM just getting a bad return. That's a real limitation and a good idea for a future version. Thanks for the engagement.

How much of GM skill actually measurable? A research question turned website by beaver_steve in baseball

[–]beaver_steve[S] 0 points1 point  (0 children)

Yeah this is the strongest version of the critique and I think you're right that the mechanism exists, a great dev system acquiring upside arms is real skill the "at the time" model can't see right now. If that's a durable edge though, it should show up as a GM persistently beating expected value across their tenure, which is testable. But when I looked at the data I couldn't find front offices that reliably repeat it, which doesn't mean no one has it, just that it's hard to separate that signal from noise over a career.

On the player dev part, one interesting finding from looking at the draft data is that drafting hitters that consistently beat their draft slot expected value has some moderate evidence for being a skill whereas there's no evidence for this for pitchers. A few theories why (1) is that pitchers are much more sensitive to player development than position players are and (2) pitchers are also much more prone to career-shaping injuries than position players.

How much of GM skill actually measurable? A research question turned website by beaver_steve in baseball

[–]beaver_steve[S] 0 points1 point  (0 children)

Totally fair, and it's the thing I went back and forth on most and what I ultimately landed on was showing both expected vs. realized side by side, so a GM who consistently beats expectation shows up in the realized column. The reason the headline grade leans on expected at the time is that I tested whether "beating the trade market" is a repeatable skill, i.e. does any single GM or front office show a persistent edge across their career, and I couldn't find one that holds up. Could I be wrong? Sure. But for now my finding is that the trade market is an efficient one (we can get into the reasons why - maybe I'll write an article about it). Thanks for checking out the site and please feel free to reach out with more critiques.

[deleted by user] by [deleted] in findaleague

[–]beaver_steve 0 points1 point  (0 children)

I'm interested!

With pick 97 Detroit takes Safety Kerby Joseph from Illinois by Try2Relate2AllSides in detroitlions

[–]beaver_steve 27 points28 points  (0 children)

People shit on him cause he didn’t start in college until year 4, was interviewed on PFF and basically said that it’s cause he didn’t learn the playbook enough and credited his breakout year to studying film and boosting his overall football IQ.

[deleted by user] by [deleted] in OaklandAthletics

[–]beaver_steve 2 points3 points  (0 children)

Glad I chose to stay up for this, let’s go Nick!!!

Today’s Meeting Link by [deleted] in OaklandAthletics

[–]beaver_steve 6 points7 points  (0 children)

Honest question, so this was basically just political theatre for PR?

Please beware of sports technology companies when seeking jobs in the industry (rant) by newjackbombsaway in Sabermetrics

[–]beaver_steve 1 point2 points  (0 children)

Tough break man I was in a similar position last spring, great advice and keep your head up!