It’s back: the third annual comparison of advanced analytics to the NCAA Tournament 1-68 seed list. Before going too far, it’s important to note that these (and other) metrics are intended to quantify different things. Some metrics are predictive in nature – more of a forward-looking thought process (i.e. KenPom, Sagarin, BPI, LRMC). Other metrics are results-based and looking backwards at a team’s resume to date (i.e. KPI, RPI, and ESPN’s Strength of Record). Just because one ranking is further from the seed list doesn’t make it incorrect (or is that implied here).
This comparison is not intended to rank rankings, but rather analyze how the 1-68 seed list compares. No system is perfect, and there are several others (and several very smart people) that are well-written, sound, and progressive. It is also understood that predictive-based metrics are not intending to project who should make the NCAA Tournament. There is no claim here that any metric is “right” or “wrong”. There’s a lot more to be written on metric definitions and outcomes, but that can be saved for another day.
How Metrics Compared to 68-Team Field in 2016-17
The team who is “supposed to win” a game won 77% of the time this season. Upsets and outliers happen and matter. Many of the most quality of wins are outliers. Ultimately, the results have to matter.
The best win of the season according to KPI was BYU’s late-season win at Gonzaga, followed by Butler’s win at Villanova in late February. The largest “outlier” result according to KPI was #340 SIU-Edwardsville (6-24) winning at #128 Grand Canyon (22-9) 76-64 on Nov. 28, 2016.
This year, ESPN’s Strength of Record metric (SOR) is also quantified in the comparison. KPI and SOR both matched on 67/68 NCAA Tournament teams, both with a margin of seeding error of less than one seed line (KPI = 3.64, SOR = 3.93) per team.
Teams Accurately Predicted in the NCAA Tournament Field:
67, KPI
67, ESPN Strength of Record
65, RPI
64, KenPom
64, LRMC
63, ESPN BPI
63, Sagarin
Margin of Difference in Rankings (This is measured as the difference between the given metric and the actual 1-68 List for teams correctly projected in the field):
3.64, KPI
3.93, ESPN Strength of Record
4.60, Sagarin
5.23, RPI
5.52, KenPom
5.78, ESPN BPI
6.03, LRMC
Teams Projected Accurately Within Zero, One, or Two Spots (0.5 seeds) of Actual 1-68 List:
34, KPI
31, ESPN Strength of Record
31, Sagarin
28, KenPom
25, RPI
23, LRMC
21, ESPN BPI
Teams Missed by More Than 8 Spots (equivalent of two seed lines) of Actual 1-68 List:
8, KPI
14, ESPN Strength of Record
14, RPI
16, Sagarin
18, KenPom
18, LRMC
21, ESPN BPI
Full Data on Accuracy of Each Metric:
