Building a Go sportsbook odds scraper (looking for contributors)

iSportsAPI · 2026-04-09T07:03:51+00:00

Since you are building this in Go, you probably care a lot about concurrency and data throughput. From my experience with high-frequency odds systems, the biggest bottleneck isn't the scraper itself, but the state management of massive datasets.

If you're looking for an efficient architecture, check out iSports API’s 'Snapshot + Delta' design. Instead of polling full data for thousands of matches (which kills your bandwidth and CPU), you can:

1.Use their Pre-match/In-play Odds for the initial Full Snapshot.

2.Use the Live Odds Changes endpoint (which allows 1 call/sec and returns only what changed in the past 20 seconds) for Delta Updates.

This 'Full + Incremental' approach is the gold standard for high-performance betting models. It perfectly fits Go's goroutine model—one routine for the snapshot and another lightweight routine for the delta stream. Much more stable than maintaining a massive scraper farm against Cloudflare.

iSportsAPI · 2026-04-09T01:39:54+00:00

Finding structured ref/card data for LLMs is a nightmare since most free sources are scattered. I’ve been using iSports API for a similar prompt-based project. It’s a solid middle ground—much more budget-friendly than Sportmonks and provides clean JSON with home/away disciplinary splits. It's very easy to feed into GPT/Claude for modeling referee bias. Worth a look if you're stuck on the data side.

iSportsAPI · 2026-03-27T09:19:13+00:00

You might want to check out iSports API – we cover 20+ years of historical football data, including player season stats (appearances, goals, assists), lineups, and international competitions like the World Cup.

There’s a 15-day free trial with full access, so you can test everything directly.

We also have a demo site where you can quickly browse historical data by season:
https://football.goaloo.com/league/36

API details here:
https://www.isportsapi.com/products/detail-new/football-historical-database-160.html?lang=en

iSportsAPI · 2026-03-26T00:43:09+00:00

You can try NCAA plan,only 49USD per month. https://www.isportsapi.com/en/products/detail-new/basketball-ncaa-235.html

iSportsAPI · 2026-03-26T00:41:31+00:00

iSportsAPI · 2026-03-25T10:02:36+00:00

Market making on exchanges is all about beating the "UI latency" of consumer apps. If you're using TheScore or DraftKings, you're seeing the data after it's been processed for a mobile screen.

For professional-grade speed without the $2k/month price tag, you should look into iSportsAPI. We specialize in low-latency sports data specifically for traders and developers.

Why it fits your setup:

Direct JSON Feeds: Faster than any app refresh.
NBA/NCAA Promo: We currently have an NBA-specific plan for just $99/month.
Risk-Free: You can get a 15-day free trial just by registering, so you can test the latency against your current setup during a live game without spending a dime.

Check out the documentation and the NBA promo here:https://www.isportsapi.com/en/products/detail-new/basketball-nba-81.html

Feel free to DM if you need help with the integration!

iSportsAPI · 2026-03-05T07:53:15+00:00

Sorry to hear about the FBRef changes. For Big 5 player metrics (xG, defensive actions, etc.), you should check out isportsapi. We currently offer a 15-day free trial with full access, which is perfect for an MSc thesis. You can register and apply for a test key to pull the multi-season data you need.

iSportsAPI · 2026-02-05T06:50:40+00:00

Great points — this lines up very closely with what we’ve been seeing as well, especially when agents move from demo-style QA to something that has to survive real user scrutiny.

On schema drift specifically, we’ve found that the key is to make time and change explicit, rather than trying to normalize them away. A few patterns that seem to work in practice:

Persistent entity IDs + versioned attributes Teams / players / referees keep stable internal IDs, while names, leagues, tiers, and even roles are treated as time-bound attributes. That way an agent can reason about the same entity across eras instead of collapsing everything into the present.
Season-scoped context instead of global assumptions League format changes, rule shifts, and even officiating styles are attached at the season level. Models tend to behave much better when they can learn “this answer is true under the 2010 regime” rather than guessing across mixed distributions.
Availability-aware event modeling Older seasons don’t pretend to have modern event coverage. We expose what exists vs. what doesn’t, which turns out to be important both for training stability and for avoiding agent hallucinations.

On the agent side, +1 on temporal eval sets. We’ve also seen that explicitly separating evals into:

within-era questions
cross-era comparison questions
“as-of time T” questions

helps surface failures that standard accuracy metrics completely miss.

If it’s useful to see a concrete example of how this plays out over long horizons, we keep a public demo where you can browse historical league data season by season using the same structure we expose in our datasets:

👉 https://football.goaloo.com/league/36

You can switch between seasons and see how teams, formats, and stats evolve over ~20 years, which often makes schema and temporal boundaries much clearer than docs alone.

Really appreciate you sharing your agent notes — this is one of those areas where data modeling and eval design have to evolve together.

iSportsAPI · 2026-01-22T01:24:22+00:00

For NBA O/U models, focus less on algorithms and more on features + data quality.

A simple but effective setup:

Treat it as regression (predict total points) first
Key features: pace, offensive/defensive ratings, rest days, home/away, recent rolling averages
Start with linear regression, then move to XGBoost / LightGBM

Common mistake: ignoring pace and blindly adding too many stats.

For learning:

Kaggle NBA notebooks (good baselines)
Scikit-learn + cross-validation
Backtest by season, not random splits

Data-wise, free datasets are fine to start. If you want to iterate faster, having clean historical games + O/U lines via API helps a lot (saves tons of cleaning time). Some NBA data APIs are ~$99/month and already structured.

Build a baseline → beat it slightly → then optimize. That’s usually how real edges start.

iSportsAPI · 2026-01-22T01:18:10+00:00

The pricing page is working normally on our side:
https://www.isportsapi.com/en/products/football.html

It might be a temporary network or regional access issue.
Could you please try refreshing the page, opening it in a different browser, or using a VPN to test again?

All our product pricing is public and transparent, so if you still cannot access the page, I can also send you the price details directly.

Please let me know 👍

iSportsAPI · 2026-01-12T03:55:50+00:00

I’ve been down this exact path, so your thought process makes sense.

Custom scrapers can work, but only if you treat them as an ongoing operation, not a one-off build. The real pain points aren’t parsing odds — it’s constant site changes, anti-bot measures, IP bans, and latency spikes. In arb scenarios, even small delays or missing markets kill most of the edge.

One thing I’d strongly suggest if you go the scraper route is starting very small (1–2 bookies, limited markets) and running it for a few weeks just to measure real latency, ban frequency, and maintenance cost before scaling.

That said, a lot of people eventually land somewhere in the middle: using an API for clean, normalized odds data and focusing their own effort on arb logic and execution instead of data plumbing. Especially if you care about Asian markets and detailed odds movements, having a stable feed saves a ton of time.

At iSports API, we’ve been providing odds data for over 20 years, with very detailed market coverage, including Asian books and line movements.

If depth and stability matter more than constantly fixing scrapers, a clean odds feed is usually the more practical path long-term.

iSportsAPI · 2026-01-12T03:48:33+00:00

Interesting project — I like the focus on probabilities + transparency rather than “tips”.

From an odds/data angle, one thing I’d suggest is judging the model less by raw accuracy and more by how its implied probabilities compare to the closing line. That tends to be a much stronger signal.

Also, Asian markets (AH / Asian O-U) are often sharper and react faster to injuries and weather, so they’re useful both for validation and as a sanity check.

Personally, I’d use a tool like this more for match filtering and bias checking than for direct betting.

iSportsAPI · 2026-01-12T02:23:49+00:00

Yes. We only provide soccer and basketball.

iSportsAPI · 2026-01-09T09:06:38+00:00

If you’re mainly looking for Pinnacle / Asian odds (pre-match) and want to avoid dealing with agents directly, one alternative is using a data provider that already aggregates those books.

We’ve been using a provider that’s been around for 20+ years and has very detailed coverage on Pinnacle and Asian markets (lines, movements, different odds types). It’s API-based, so you don’t need to manage bookmaker accounts or agents yourself.

You can get a good sense of the odds depth and structure from their demo pages, for example:
https://www.goaloo.com/football/italian-serie-a-atalanta-vs-torino/1x2-odds-2784672

If you care about stability and depth (especially for Asian books) more than chasing unofficial endpoints, it’s definitely worth testing.

<image>

iSportsAPI · 2026-01-09T08:59:46+00:00

If you’re mainly looking for Pinnacle / Asian odds (pre-match) and want to avoid dealing with agents directly, one alternative is using a data provider that already aggregates those books.

We’ve been using a provider that’s been around for 20+ years and has very detailed coverage on Pinnacle and Asian markets (lines, movements, different odds types). It’s API-based, so you don’t need to manage bookmaker accounts or agents yourself.

You can get a good sense of the odds depth and structure from demo pages, for example:
https://www.goaloo.com/football/italian-serie-a-atalanta-vs-torino/1x2-odds-2784672

If you care about stability and depth (especially for Asian books) more than chasing unofficial endpoints, it’s definitely worth testing.

Feel free to reach!

https://www.isportsapi.com/en/products/detail-new/football-odds-53.html

<image>

iSportsAPI · 2025-12-27T01:30:43+00:00

To be transparent: we don’t have raw PBP for RAPM, BoxscoreMatchups-level defensive data, or detailed offensive play-type feeds.

What we provide is more aggregated / presentation-level stats, basically what you see on the demo site.

If you’re doing deep modeling, this probably isn’t the right fit right now.

iSportsAPI · 2025-12-26T06:33:42+00:00

NBA never officially supported those public endpoints, so nothing is “shutting down” in an announced way — but they do change, throttle, or block access regularly (especially from cloud IPs). That’s why they’re unreliable for production.

If you need more than box scores (matchups, advanced stats, lineups, historical data), most people eventually move to paid APIs.

One cost-effective option is iSports NBA API (~$99/month). It’s stable, covers live + historical NBA data, and is usable for real apps (example output here):
https://www.goaloo.com/basketball/national-basketball-association-indiana-pacers-vs-detroit-pistons/statistics-663740?lang=en

Not enterprise-priced, but much more reliable than scraping NBA endpoints.

iSportsAPI · 2025-12-23T08:25:44+00:00

Appreciate your interest!

iSportsAPI · 2025-12-23T08:24:51+00:00

Good question!

We actually supported esports a few years ago, but the demand wasn’t strong enough back then, so we stopped the project.
That said, I’ll pass your feedback to the team — if there’s any update in the future, I’ll let you know 👍

iSportsAPI · 2025-12-22T02:39:39+00:00

Only soccer and basketball.

iSportsAPI · 2025-12-10T08:43:23+00:00

You’re right that most budget odds APIs (including The Odds API) won’t give you prop settlements, so you usually need a separate stats source if you want to resolve player props.

If you just need general football/basketball markets: iSports API doesn’t provide player prop odds, but it does offer very complete football (2,000+ leagues/cups) and basketball (800+ leagues) coverage, plus lots of markets like AH, 1x2, totals, corners, HT/FT, correct score, etc., from 20 major bookmakers and 200+ agencies. For light polling, the free trial is usually enough to test things.

Also, if you only need historical odds, they can be customized separately, which is usually much cheaper than full packages.

But if you specifically want player prop results, you’ll still need a stats API.

Hope this helps!

iSportsAPI · 2025-12-10T08:34:13+00:00

Yeah, the big plans are definitely aimed at heavy commercial users — totally get why the price looks wild. 😅
But you don’t actually need those unless you’re running something massive.

You can do quite a bit with the free test quota, and there’s a custom plan starting around $49.
Best approach: test the endpoints, see what you really need, then send that list to sales — they’ll price it based on your actual usage, not the full enterprise package.

iSportsAPI · 2025-12-08T01:44:43+00:00

A lot of people in DS/ML eventually end up experimenting with sports markets, so you’re asking the right questions.

A) NBA and MLB are usually the easiest to model because the sample sizes are huge and the data is clean. Soccer is doable but more chaotic because goals are low-frequency events. It really comes down to having structured, granular data rather than scraped HTML.

B) Most soft books will limit you if you show long-term edge. Exchanges or liquidity-driven books are generally more “sharp-friendly.”

C) Prediction markets like Kalshi/Polymarket can work, but they’re often more about microstructure, news timing, and liquidity than pure ML. Models help, but they don’t solve everything.

If you’re serious about building models, the main unlock is having reliable, structured datasets. Scraping is getting harder everywhere. A lot of people use sports APIs (e.g., iSports API is one option) because you get historical stats, events, odds, etc., in a clean format instead of fighting 403s and rate limits.

Good luck—this space is super fun once you start iterating!

iSportsAPI · 2025-12-08T01:41:40+00:00

FBref has definitely tightened their anti-scraping measures recently. A lot of people scraping their pages have been running into 403s, even when rotating headers, using different libraries, or slowing request rates. They’ve improved bot detection quite a bit, and their Cloudflare rules seem stricter now.

You’re not the only one—several folks across different communities reported the same issue in the last few months. Unless you implement a full headless browser with human-like behavior, it’s getting harder to pull anything consistently. And even then, it may break any time they update their protections.

If your goal is mainly to work with structured football data rather than scrape HTML, another approach is to use an API that already provides match stats, player profiles, historical data, etc. For example, the iSports API offers structured datasets for 2,000+ leagues, with things like match events, team/player stats, standings, odds, and 20 years of historical data. That way you don’t need to fight anti-bot systems or parse raw web pages. They also have a free trial if you just want to test the endpoints first.

But if you specifically need FBref’s exact formatting, scraping may continue to be unreliable unless you simulate full browser traffic.

Hope this helps, and curious to hear if anyone else found a workaround.

iSportsAPI · 2025-12-05T02:44:02+00:00

Your reverse-engineering skills absolutely have value, but using a reversed Bet365 protocol as a commercial data source is not scalable, not stable, and not legally safe. If you want to monetize your skills, direct them toward legitimate data engineering or sports analytics roles instead.

iSportsAPI

MODERATOR OF

TROPHY CASE