All that is left to wrap up before the start of the 2015-2016 postseason is to watch the Army-Navy Game and see who is handed the Heisman Trophy and the other individual awards. Sunday’s announcement started the chain of events that culminated in the schedule of 38 bowl games, two playoff semifinals grafted onto two other traditional bowl games, and the finale taking place in Glendale, Arizona on January 11.
That means we now have a full season of data through which we can evaluate how well the Pigskin Rating System worked this season at projecting the real quality of teams as they made their respective ways through the 2015 schedule. In some cases we saw earlier than the traditional polls when a team was trending upward; other times it seemed to be a completely false alarm.
Over the season we have seen some teams overvalued, others grossly undervalued, and still more have been accurately pegged. Then again, we have only our own eyeballs and the words of human poll voters against which to compare these numbers, so “accuracy” is ultimately a relative term here.
With that in mind, here are some proposals that will be considered in the offseason when determining how (if at all) the PRS equation might be tweaked to provide a more logical representation of team quality in 2016 and beyond:
- The methodology for weighing offensive and defensive performance seems to be the strongest of the five DOCTS categories (Defense, Offense, Coaching, Talent, and Schedule), strongly correlating to the quality of the respective units. But might there be additional categories that would help provide a more well-rounded understanding within the calculus of the weighted score?
- The coaching metric incorporates both the length of coaching career and the success rate throughout that career. But is it giving too much credit to past glories? The potential for weighting career records in order to favor recent successes might be valuable in determining whether the current state of coaching at each school is more coherently measured in real time.
- Talent is a metric that is dependent on subjective variables, namely the composite rankings issued to recruits before they arrive at school, as well as more quantitative figures such as returning talent. Whether there is a more effective way to measure the talent level of an individual roster will require further examination in the offseason.
- Schedule is another category that remained fairly static throughout the season, supplemented by the addition of the adjusted margin of victory calculations. It might be valuable to add a real-time update element to the schedule rankings that illustrates the quality of schedule for that season rather than depending on past figures to arrive at that mark.
- Adjusted margin of victory also could bear some additional scrutiny. The weighting for FBS versus FCS competition sometimes obfuscates the fact that high-quality FCS opposition can provide a more stringent test than FBS cellar-dwellers. It might also be valuable to evaluate the weighting afforded for home-field advantage, especially in games where teams win by one or two points at home and are credited with a “loss” in the adjustment process. It might also be valuable to evaluate whether Sagarin ratings are the most effective weighting basis; other computer rankings, or a composite that incorporates several of the systems that evaluate both FBS and FCS teams, might help to normalize this further.
The Pigskin Rating System did an admirable job for the most part in its first season of operation, but just like any algorithm there is value in scrutinizing the system in order to determine whether inherent biases exist in the calculations and whether they can be reduced.
No ranking will ever be perfect, but it is laudable in the end to try to strive for objectivity. Thanks for following along throughout the 2015 college football season, and we will be back in January to provide final rankings after the postseason.
You can scroll through the full rankings below, including the breakdown of each category calculated in the Pigskin Rating System. To brush up on the methodology used in the rankings, click here.