Big Ten Geeks: No more BPI, please

If you?ve been watching or reading any of ESPN?s college basketball coverage over the past couple of months, you know what BPI is. That?s because ESPN is shoving this proprietary metric down its audience?s throat. Who should be #1? Let?s check BPI. Is your team on the bubble? Let BPI decide. When SportsCenter loads up a resume of an at-large hopeful, it?s BPI that?s splashed on the screen.

At this point, it?s clear that the Worldwide Leader is going to make us chug BPI until we?re saying it in our sleep. ESPN claims it?s great rating system, better than RPI. It claims that BPI exposes pretenders, and helps you fill out a better bracket.

But I?m calling this what it is-a propaganda campaign.

At face value, BPI sounds like a good system. For one, it was invented by Dean Oliver, the guy who literally wrote the book on advanced basketball statistics. This guy is basketball?s answer to Bill James. If anyone can come up with a great metric, it?s him. What does Oliver have to say about BPI? Let?s read:

There are a number of small details that we have in our methodology to make it reflective of a résumé for a tournament team — these are pretty technical and many people won't be interested, so we won't go into detail, but we think they improve how the tool works.

This is the first, and biggest, problem with BPI. Much of BPI remains a mystery. How much? That?s a mystery, too. This alone might be enough to discredit the entire ranking system. Advanced statistics should follow a similar philosophy to all scientific work, which is to say that it must be available for analysis and critical review. I can?t explain all of the reasons why I might like or dislike various parts of BPI, because I don?t know what those parts are.

I?m not entirely sure what the purpose of keeping so much of BPI a secret is, but I think there are two possibilities. The first is we take Oliver at his word: BPI is complicated, and no one will understand it anyways. This is a self-defeating rationale, because ultimately no one should be using a metric they cannot understand. If no one can understand BPI, then no one should use BPI.

The second reason I can come up with is far more likely-ESPN doesn?t want anyone ripping off its metric and putting the numbers somewhere else. It wants BPI available to only its viewers, or its Insider subscribers. And that?s not unreasonable. ESPN is in the business of sports, and it delivers high-quality programming and content precisely because it is a business. We shouldn?t ignore that. But there is a fine line between protecting intellectual property, and credibility. Take Pomeroy?s rankings, for example. Much of the content is only available with an annual subscription, but that doesn?t stop Pomeroy from fully explaining his methodology. It would take a ton of work, and considerable expense, but one could fully recreate Pomeroy?s ratings if one were so inclined.

And yet, no one has done so. I?ll leave the discussion as to why that hasn?t happened for another time, but it?s clear that the risk that someone will rip off BPI is minimal. In short, there?s no good reason for ESPN to hide parts of the BPI calculation.

On to the next problem:

If a team or its opponent is missing one of its most important players (determined by minutes per game) for a contest, that game is less important for ranking the teams compared to games in which both teams are at full strength.

Again, this sounds good, but it?s easy to see why this is flawed. Let?s consider the case of Indiana. Cody Zeller, Victor Oladipo, and Kevin Ferrell have all played about the same number of minutes this season. But it?s clear that one of these things is not like the others. Zeller and Oladipo are on the shortlist for the Naismith Award. Ferrell, while having a fine season for a freshman, is not going to make the Big Ten Third Team. Yet if Ferrell were to miss a game for injury, BPI would weight that same the same as if Oladipo or Zeller went down. We know that?s the wrong approach.

Now, a fair question is whether the wrong approach is nonetheless better than no approach. Kenpom does not adjust games for injury, after all. And frankly, I don?t know the answer to that question, because BPI does not explain itself. All we get from BPI is one anecdotal example from the 2012 Syracuse team, which certainly is not enough to justify this adjustment.

Indeed, this is symptomatic of BPI. The devil is always in the details, and we have none. For example:

Another way that BPI can rank teams differently than Sagarin or Kenpom is counting close games at home versus on the road. In BPI, a close win at home is better than a close loss on the road against the same opponent.

What does Oliver mean by ?close?? I think we can all agree that a 5-point win at home is better than a 5-point loss on the road. Home court advantage hovers around four points, after all. But this seems to indicate that a 1-point win at home should be viewed as better than a 1-point loss on the road. But again, we know that?s wrong. So why is BPI placing so much emphasis on the win? Again, we don?t know.

And then there?s the final problem I have with BPI: it?s not clear what it wants to be. By including margin-aware metrics, it looks similar in overall form as Pomeroy. But then there?s this nonsensical wrinkle:

BPI gives marginally decreasing credit for bigger wins, with a 30-point win being only about 20 percent better than a 15-point win, not twice as good, which can happen in other methods.

By capturing blowouts, but not overweighting them, BPI credits the ability of good teams to easily beat poor teams without providing incentive to win by 30 when 20 is a safe margin. By capturing both blowouts and close games in this way, BPI summarizes a team's résumé for the NCAA tournament well.

?Incentive?? What a bizarre concern for a rating system that has yet to be given any legitimacy by anyone with any power over the fates of college teams. As much as ESPN would like to change this fact, BPI is not the RPI. There?s no indication that the Selection Committee uses it. And thus, there?s no need to provide any incentives. The last thing that Bo Ryan has on his mind when his Badgers are beating Presbyterian by 40 is what this will do to his team?s BPI, Pomeroy Ranking, or any other metric. Now, if the Selection Committee came out and said that it?s going to rely heavily on Pomeroy (which does reward bigger blowouts over smaller ones) in selecting and seeding teams, then maybe Ryan and other coaches might care, and Club Trillion will reduce its membership.

But in the second place, there?s no reason to believe that?s happening. Even if it were, there?s the problem that predictive power exists in those bigger blowouts in the first place.

And this is another reason why BPI doesn?t make sense. If the goal is to encourage the sportsmanship by inventing a new metric that stops rewarding teams after they win by more than 20 points, fine (although I doubt Division I players, as a group, have a self-esteem problem). But if the goal is to come up with a rating system that appropriately ranks teams by how good they are and how likely they are to win more games in the future, this is precisely the wrong approach. It doesn?t matter if it?s unsportsmanlike-if big blowouts predict more future wins than smaller blowouts, then we shouldn?t ignore the former.

All in all, BPI is a metric shrouded in mystery. Of what little we know about it, much of it is wrong. It?s furthermore unclear as to what exactly it?s trying to measure. Yes, it?s better than RPI, but what isn?t? While I?m all for innovation in this space, BPI is a step in the wrong direction.