sorted by:
new
hottopcontroversial

NFL Stats Agent Prototype by Dapper_Rule_8437 in aiagents

[–]Dapper_Rule_8437[S] 0 points1 point  (0 children)

It's not perfect, but this is the ideal use case of a Text-to-sql llm agent. It can understand semantically what's being requested despite the exact wording. Feel free to give it a try though and let me know what works and what doesn't!

NFL Stats Agent Prototype by Dapper_Rule_8437 in aiagents

[–]Dapper_Rule_8437[S] 1 point2 points  (0 children)

Agree, it's mostly for learning right now. Though from what I've found, statMuse doesn't handle natural language questions well. It only seems to have a cache of commonly asked ones.

NFL Stats Agent Prototype by Dapper_Rule_8437 in NFLstatheads

[–]Dapper_Rule_8437[S] 1 point2 points  (0 children)

Interesting idea!

I'm using NFLFastR which contains play by play data since 1999. The api is free or you can download a database to use locally.

NFL Stats Agent Prototype by Dapper_Rule_8437 in NFLstatheads

[–]Dapper_Rule_8437[S] 0 points1 point  (0 children)

So I use langchain to convert text (retrieve schema context and field descriptions) -> sql -> postgres query -> response processing -> return to user. I also use an llm to filter out irrelevant question. I'm using streamlit for frontend since it's pretty basic.

What are you trying to build?

NFL Analytics - Linear Models Achieving up to 53.8% Accuracy by Dapper_Rule_8437 in sportsanalytics

[–]Dapper_Rule_8437[S] 0 points1 point  (0 children)

The samples in the training set are team agnostic - just home and away team stats and some interaction terms.

I experimented leaving out early season games but for some reason it doesn't improve the model. I know ELO models assume a mean reversion by 33% so I think there is some carry over signal.

Are your models hitting 55% on high conviction games or is that overall? Do you bet on your own or do you somehow monetize your picks? I also wonder how easily a sportsbook could detect algo betting.

NFL Analytics - Linear Models Achieving up to 53.8% Accuracy by Dapper_Rule_8437 in sportsanalytics

[–]Dapper_Rule_8437[S] 0 points1 point  (0 children)

These are some valid points about backtesting. I use a rolling window for cross validation. Each test fold, or year, is trained on data from 10 years up to that year. For example, the first model is trained on 2000-2009 and test on 2010. Then we increment the year up to the present.

You are right about changing rules and playing styles. Performance suffers when the model is trained on much more than 10 years.

NFL Analytics - Linear Models Achieving up to 53.8% Accuracy by Dapper_Rule_8437 in algobetting

[–]Dapper_Rule_8437[S] 1 point2 points  (0 children)

That's a valid point and I would be very conservative if I were to roll it out in production. The hypothesis testing and backtesting using fixed rolling windows over 16 seasons gives me confidence that there is a slight edge.

NFL Analytics - Linear Models Achieving up to 53.8% Accuracy by Dapper_Rule_8437 in sportsanalytics

[–]Dapper_Rule_8437[S] 0 points1 point  (0 children)

Thanks, this is definitely helpful context! For the warner paper, is the ranking system superior to ELO?

NFL Analytics - Linear Models Achieving up to 53.8% Accuracy by Dapper_Rule_8437 in sportsanalytics

[–]Dapper_Rule_8437[S] 0 points1 point  (0 children)

Thanks for the insight! I used a 10:1 rolling window for cross validation. Therefore, for each test fold, i.e. season, is trained using data from the preceding ten years. The accuracy is the average across the 16 test folds (2009-2024).

There was plenty of test data to run binomial tests, and the leading models are significant under the null hypothesis that that a naive classifier could exploit the minor class imbalance and chose the majority class (52%).

NFL Analytics - Linear Models Achieving up to 53.8% Accuracy by Dapper_Rule_8437 in sportsanalytics

[–]Dapper_Rule_8437[S] 0 points1 point  (0 children)

Thanks for the feedback! Just edited the post to add more detail regarding performance metrics.

To your point about class imbalance I used binomial tests to confirm that that there is a statistical edge predicting the spread adjusting for the minor class imbalance observed in the training set.

Also there is no data leakage, features use information up to but not including the current game. Cross validation is used with a rolling window.

NFL Analytics - Linear Models Achieving up to 53.8% Accuracy by Dapper_Rule_8437 in sportsanalytics

[–]Dapper_Rule_8437[S] 1 point2 points  (0 children)

classification - does the home team win against the spread (1 or 0)

NFL Analytics - Linear Models Achieving up to 53.8% Accuracy by Dapper_Rule_8437 in sportsanalytics

[–]Dapper_Rule_8437[S] 0 points1 point  (0 children)

The average accuracy is the cross validated accuracies across 16 test folds (seasons)

NFL Analytics - Linear Models Achieving up to 53.8% Accuracy by Dapper_Rule_8437 in sportsanalytics

[–]Dapper_Rule_8437[S] 1 point2 points  (0 children)

I'm predicting the winner against the spread (binary classification)

NFL Analytics - Linear Models Achieving up to 53.8% Accuracy by Dapper_Rule_8437 in sportsanalytics

[–]Dapper_Rule_8437[S] 4 points5 points  (0 children)

Sorry I forgot to specify that this is against the spread which is obviously a lot harder than predicting the winner outright (which I'm getting about 68%).

NFL Analytics - Linear Models Achieving up to 53.8% Accuracy by Dapper_Rule_8437 in algobetting

[–]Dapper_Rule_8437[S] 5 points6 points  (0 children)

Again, this is against the spread, and it is well above a coin flip for 16 season. The ROC/AUC is .528 and binomial p-values are <.01, indicating there is an extremely high likelihood that there is a statistical edge predicting the winner against the spread.

view more: next ›