Every time a new poll is released, we immediately see the Tweets come flying in about how certain pollsters are biased against a particular team. We've been gathering poll data since 2014 and decided that it was finally time to start analyzing those picks to see if there is any truth to those claims.
We've analyzed thousands of votes to calculate how much each voter in our database under or over-ranks each team. We take the average variance of those picks and once we can determine that a voter consistently under or over-ranks a team, we record that bias to their account. This calculation only accounts for teams that are either in the Top 25 OR on a pollsters ballot. A pollster will never be penalized for not including a team in Others Receiving Votes but they may be for including a team that isn't ranked. A voter is considered biased once they vote the team an average of at least 4 places away from their actual ranking at least 5 times. This shows us a pattern. If a voter doesn't rank a team or they rank a team in Others Receiving Votes, we use a rank of 26 for calculation purposes but also note with a "+" that their vote isn't known.
Each Pollster page on CollegePollTracker now shows up to 3 teams that the selected Pollster is Biased towards and against. You may see fewer than 3 teams (or none at all) if the voter doesn't show a pattern of bias or if we don't have enough information. You can also click on the Pollster's biases to see a further breakdown.
This page lists every team that has shown up in the Top 25 during a week that this Pollster has voted, broken up into 3 categories: Biased Towards, Biased Against, and Neutral/Not Enough Info. It also includes additional data about how far they over or under rank a team and how many week's worth of data we are basing the bias on. You can then take it a step further by clicking on an Individual team.
Now you can see a graph comparing this teams actual rankings with how the selected pollster has ranked them. Below that, we have a list of every week that the team has shown up in the Top 25 during this pollster's time showing the pollster's rank, actual rank and variance between the two. From this data you should be able to come to your own conclusion as to whether a pollster is biased or not.
We've made that easy enough. At the top of each team page there is now a link to View All biases towards a team.
Simply click on that link and you'll be taken to a page that shows you pollsters that are biased towards and against you.
That's perfectly fine. We realize that when you are analyzing data there is a lot that is open for interpretation. Our way of determining if someone is biased may be different from yours. That's why we've chosen to be transparent and show you the data that helps us to make our decisions. This data will hopefully allow you to decide for yourself.
1: Every bias detected isn't a bad thing. Sometimes a pollster notices something that other's take time to adapt to. The data needs to be looked at in each case to see what it really shows.
2: Biases can evolve. Each week when the new poll comes out, we will compile the new ballots into our calculations. In some cases a bias may disappear and in other it may get stronger.
3: Our formulas and calculations may change overtime. If we notice (or have pointed out) weaknesses in the way that we are determining bias we will change it. We will also let you know when changes are made.
