BCS Central blog

[an annual feature of the BCS Central blog]

Check out these computer mousepads featuring college logos.

Various self-proclaimed “experts” like to publish predicted BCS numbers, supposedly having the ability to accurately determine how poll voters and computers will react to a weekend’s game results.   A couple of years ago,  the credibility of a web site administrator took a big hit – this person actually assigned negative computer points to teams!   Before the first official BCS rankings of 2010 were released, one self-described “guru” would not have been surprised if Auburn was ranked #2 after beating Arkansas.  It is unknown how much surprise resulted when Auburn debuted at #4.

Poll voters generally stick to their guns.  That is, they make up their minds where a team should be ranked and keep it there, only moving it down if it loses, or moving it up if teams ahead of it lose.  You can see clear evidence of this on the first-place votes page from 2009.  Florida had 53 from the coaches in the preseason, and had 53 thirteen polls later.  There tends to be more fluctuation from week-to-week among those near the bottom of the 25 BCS ranked teams.

The computer page of BCS Central lists properties of the six BCS systems.  There was no guesswork involved – the behavior of these systems over the course of a season was carefully observed to uncover these characteristics.  All six publish conference rankings, and with the exception of Wolfe, all list teams’ strength of schedule.  Only one algorithm – Colley‘s – has been made public.  Colley also provides a mechanism for looking at how specified game results – in isolation – would affect the system’s ratings.

An internet forum of one BCS contending team features an individual who puts a lot of time and effort into highly detailed posts about what the BCS computers’ rankings will look like.   The only real value of these messages is as entertainment, since the output of six independent algorithms can in no way be forecast by the hypothetical product of one.  “Intuition” is said to help make up for the ignorance of the workings of these systems, but in 2010, one set of top-25 projections of Sagarin‘s rankings matched the actual numbers in only 2 cases.

2011 UPDATE:  The projections are even sillier when based upon faulty data.   Recently, the individual mentioned above believed he was using Massey‘s BCS rankings, but neglected to exclude non-FBS teams, so he thought his favorite team was ranked several places below where it was really was.  Another self-proclaimed BCS know-it-all made a similar mistake, taking Sagarin‘s apparent SoS ranking of Stanford at #92 at face value.  (This may have been copied from ABC, which broadcast the incorrect ranking earlier.)  Once again, a person neglected to exclude non-FBS teams, and also forgot that USC‘s rankings do not count in the 2011 season.

2012 UPDATE:  These sites have to publish something and pretend that non-factual items are relevant to college football fans, so the prognosticating continues.  It seems, however, that the administrators of some other sites discovered that their SoS numbers could be proven to be wrong, and since they are unwilling or unable put in the work necessary to produce accurate strength of schedule rankings, have surrendered the task of calculating these important numbers to BCS Central.

The bottom line is that this pair of analogies holds:
Projected BCS numbers are to official BCS ratings as astrology is to astronomy.

This ordering of the six BCS computer systems (and their administrators)  is admittedly primarily subjective.  The rankings are based on how the systems have published data this season.  They’re not intended to highlight which may be more “accurate”, “fair”,  or predictive of game results.  Be sure to look at the computers page and the linked blog posts to get a better  understanding of how any one system is alike or different from the others.

Find jewelry made for fans of your favorite team.

1) Anderson & Hester place all their BCS ratings, strength of schedule ratings, and conference rankings on a single web page.  There is only one ranking system and no numbers are from non-FBS teams.   This makes getting the numbers used for the BCS computer score easy to get, and the data tends to be published within an hour or two after the official BCS release on Sunday.

2) Billingsley places strength of schedule and conference rankings on a separate page.    Only FBS teams are ranked.  New data comes out soon after the official BCS rankings are released.

3) Sagarin is the only system allowed to publish computer ratings before the BCS rankings come out, and the new numbers tend to come out early each Sunday.  Sagarin’s conference rankings are on a separate page.  Ratings from both a points-based system (Predictor) and the BCS algorithm (ELO CHESS) are displayed in the same table.  FCS teams are ranked right along with FBS teams.   There is only one set of numbers for strength of schedule and a single ranking of conferences, and it’s unclear how much influence point differences have on the determination of these ratings.

4) Colley is unique in that he ranks groups of FCS teams, rather than individual ones, with FBS teams.  This makes determination of strength of schedule rankings for FBS teams very cumbersome.  Conference rankings are on a separate page.   New rankings are sometimes published fairly soon after the BCS release on Sunday, but have not come out until Monday in other weeks.

5)  Massey publishes more data and has by far the most sophisticated web site of any of the BCS computer systems.  This is a positive from the perspective of a college football fan in general, but is actually a negative in this review.   One must be extremely careful, as Massey not only has a point-based rating system and ranks lower division teams,  but it’s also quite easy to think you are looking at the BCS numbers when you are not.   Conference rankings and strength of schedule numbers can be found on the same page as the BCS rankings.  Massey rankings are released very late Sunday or early Monday.

6) Wolfe also ranks more non-FBS teams than just those in FCS, but this does not present much of a problem in getting the correct BCS rankings.  Conference rankings are on the same page as the BCS rankings, but Wolfe does not put out any strength of schedule ratings.

In the recent bowl season, there was little variation among the polls and BCS computer systems in how often they ranked a winning bowl team higher.   This is probably due to the fact that there were few games in which two closely-ranked teams faced each other.   Other than in three BCS bowls (BCS NCG, Rose, and Sugar), the difference in the two teams’ rankings was usually substantial.

A cell in the table below has an asterisk (*) to indicate that the component named in the column header ranked the winner of the bowl named in a particular row higher.  In addition to the polls and the six BCS computers systems, Massey‘s Margin Of  Victory and Sagarin‘s Predictor, which take into account points, have been included.

In the first surprise this season,  Billingsley did not have the most computer rankings thrown out for being the highest or lowest for a particular team.  Instead,  Sagarin had more tossed out for both reasons than any other system.  In another upset, Massey, rather than Wolfe, produced the fewest rankings that were not used to determine a team’s BCS computer score.

The computers page of BCS Central has also been updated – the table on the right now shows discarded computer rankings from 2005 through 2010.

First off, this post is not intended to belittle Wes Colley.   (Later reports indicate that Peter Wolfe may share responsiblity)  The error would not have been discovered if, like the other five BCS computer systems, Colley kept his algorithm secret.  He deserves credit for this, and the incident should (but won’t) spur the others to publicize exactly how their BCS ratings are calculated.  I could write once again about how this unfortunate occurrence hurts the credibility of the BCS, but it would have to have some for it to be diminished.

The result of a single FCS game which was not originally considered by Colley resulted in two pairs of  bad rankings.   In Colley‘s rankings, Boise State is now #9, LSU is #10, Nebraska is #17, and Alabama is #18.   There was absolutely no impact on Alabama‘s BCS standing.   Nebraska‘s rating increases to 0.3967, but its BCS ranking of #18 remains the same.

The significant impacts were on Boise State and LSU – they switched places in the correct BCS rankings – Boise State is now #10 and LSU is now #11.

A screen capture of the uncorrected Colley ratings is here.  The uncorrected “official” PDF is here – the “corrected” one is here (but it has even more problems – the “Rank”, “Points”, and “%” columns under “Harris Interactive” are mixed up; the “%” column under “USA Today” is actually a repeat of  points in the Harris Poll; and the headers of the other two columns do not match the data).

Two additions have been made to the “BCS computers” page:  There are now links to the blog posts that discuss each of the six systems.   Additionally, there is a new table that combines the data from 2005-2009 in regards to discarded computer rankings.  For those who have read the blog posts relating to the BCS computer systems before, it should be no surprise that Billingsley‘s rankings were tossed out far more often than any other system.  Also worth noting is that when the numbers of Anderson & Hester were discarded, it was far more likely to happen because the rankings were higher, not lower, than those of the other systems.  Since this new table contains information from past seasons, the “BCS computers” page is now also linked from the “BCS history” page.

As seen in this post, the BCS rankings proved to be poor predictors of the outcome of bowl games.  The two polls ranked the winning team higher half the time – in the very same games. Only two of the six BCS computer systems were more effective than flipping a coin.  Two alternate algorithms which take into account points (Sagarin’s Predictor and Massey’s Margin Of Victory) are also included in the table below.  Both picked three more winners than their BCS counterparts.  The other four BCS algorithms are those of Anderson & Hester, Billingsley, Collley, and Wolfe.

Every component (and the two points-based systems) had Alabama over Texas, Virginia Tech over Tennessee, Pittsburgh over North Carolina, and USC over Boston College.  No system ranked Boise State over TCU, or Wisconsin over Miami (FL), and only MOV placed Ohio State over Oregon and  Florida State over West Virginia.

… for analyzing the performance of members of a competitive league.”   That is how the Massey BCS computer system is described on this page; however that claim and the description of the algorithm may not be totally accurate as most of the information on the page is admittedly “older, and may be out-dated or incomplete.”   However, the “Key” at the top is needed to interpret Massey‘s various abbreviations used on his ratings pages.

Massey‘s most detailed ratings page is here – as the note in large red letters makes clear, the rankings used by the BCS are in the column marked as such.   For any one team, several more measurements than just BCS and strength of schedule rankings are listed.   If one is only concerned with Massey in connection with the BCS, this other ratings page is simpler and thus easier to use.   “FBS” can be chosen from the drop-down menu to list only those teams.   The column with a green background holds BCS ratings and rankings.  A points-based ranking is in the “MOV” column, but this is not used by the BCS.

For the 2009 season, Massey’s rankings were discarded third most often among the six BCS computer systems.