AI Modeling the Sweet Sixteen

Jump right to our Machine Learning Bracket Predictions or Betting Odds or Try the Model Now!

You can download the updated training dataset here.

Get your own free copy of the updated model on Akkio

Madness! Chaos! Well, our AI-driven March Madness bracket is just as busted as yours. The first two rounds of the NCAA Tourney were just as unexpected and crazy as life in the last year. Upsets reigned. The first two rounds produced 12 (!!) upsets where a team 5 seeds or lower than their opponent won. And while we were disappointed to see our model’s pick to win it all (Ohio State) lose in the first round, we couldn’t help but cheer the underdogs along - even those our model didn’t pick.

We can’t help but take another bite at the apple. We’ve taken a look at our model, made some adjustments that we’ll describe here, and have re-forecasted from the Sweet Sixteen forward.

Round 1&2 - Evaluating our Model Performance 

There is no sugarcoating it - our first NCAA bracket was a bust. As the dust cleared on the round of 32, none of our final four teams had even made it to the Sweet 16. 

• 2 seed Ohio State lost to 15 seed Oral Roberts (earning their entry in the Cinderella storybook). Our model had Ohio State winning it all so this loss was particularly devastating. One issue is the model didn’t take Kyle Young’s absence into account. Even then, this was only the ninth time in history a 15 seed won in the first round. 
• 2 seed Illinois went down to 8th seeded Loyola Chicago in the second round, a game they led throughout with shut-down defense. 
• 2 seed Iowa lost to the 7 seed Oregon Ducks by 15 points in a game that never seemed that close. 
• 4 seed Florida State took down 5 seed Colorado with a strong second half of play. We had UNC Greensboro through into that game.
  

Within all the carnage there were a few places where our model performed well. In the First Four, we correctly predicted 3 of the 4 play-in games. For the first round, we got 23/32 of the picks correct. For the second round, we correctly predicted 8 of the 16 games. Breaking down the first and second round in a bit more detail:

• First-round we picked 4 correct upsets: 9 seed Wisconsin over 8 seed North Carolina, 12 seed Oregon State over 5 seed Tennessee (nice pick!), 10 seed Rutgers over 7 seed Clemson, and 10 Maryland over 7 Uconn. 
• Of our 8 first-round misses, 3 were upsets that did not happen with the remainder being upsets that our model did not predict. 
• The best prediction of our model was 12th seeded Oregon State making the Sweet Sixteen - a correctly predicted upset over 4 seed Oklahoma State.

Despite upsets taking out the final-four, our model is currently in the top 8% of models per ESPN’s bracket challenge.    

Building a New Model - Back to Basics

In the sage words of Chumbawamba “I get knocked down, but I get up again.” After we took a break from our misery to watch that music video, we set out to build a new model and earn back our dignity. We made three big changes to the training data - all of which are intended to improve model performance while avoiding teaching it the “wrong” things. 

• Eliminated Games Played from the training set: this was substantially impacted by COVID and caused some of the strangest results in the last model - like a long run by Morehead State that never materialized. 
• Added SOS Rank: the strength of schedule for each team going into the tournament this year (excluding non-division games). Historical data used in training has this stat but including the NCAA tournament.
• Added OPPONENT STATS: The average stats of all opponents that team played during their season
• Added PS/G: average points per game
• Added PS/G Rank: normalized rank of points per game against all other teams
• PA/G: average points allowed per game
• PA/G Rank: normalized rank of points allowed per game against all other teams
  

Taken together these changes had a substantial impact on predicted outcomes. The new model prioritizes the strength of schedule, points scored, and offensive rebounds.

We pressure tested the new model against the first two rounds and found it performed similarly to our old model (20/32 correct in round 1 and 8/16 in round 2). That said, all four of its final four teams are still alive (vs the first model which had zero of its final four still alive). With bracket crushing in mind, this is a positive improvement as correct final four predictions have exponentially more weight.

Try the AI Model Right Now!

You can predict any matchup you like by entering the year and the two team names below and clicking “predict”. 

Pretty cool that you can run predictions against a cloud-deployed machine learning model right in this blog post! To save you some time we did the heavy lifting of running all of the matchups from here on out and have shared them below.

New Sweet Sixteen AI Predictions

This new model ends up following conventional wisdom - it picks the higher seed to win in every game. The seed input only accounts for a very small percentage of the predictive weight of the model, so essentially, we’re seeing that the underlying statistics support the seeding of the teams.

At first glance, this is now a pretty boring bracket. But given how turbulent and upset-prone the first two rounds were, it’d be wild if there are no more upsets to come in the remaining games.

Next Steps

This is the last model we will be making for the NCAA tournament this year, but we will post individual game projections on the Akkio Twitter as those games happen (if the model gets some of the final four wrong, we will run and share its predictions for each of the games.)