Yeah, But How Does it Work?
My Hockey Rankings Explained
Ohio Hockey Digest’s weekly recap of the Top 25 OHSAA teams in MYHockey Rankings (MHR) has generated some great early-season discussion about high school hockey in Ohio.
Understanding how mathematical ratings work (especially compared to polls by human experts like coaches) is not easy, however. I developed my first “computer ratings” system in 1990, for college hockey, and helped the NCAA develop their Division I tournament criteria, so I may be able to help you understand how MHR works so you can have even more informed debates about the sport we all love.
It is easiest to start with specific examples, so this week let’s tackle the “surprise” new member of the top 25: Normandy. As a member of the Greater Cleveland High School Hockey League’s Blue Division, the Invaders’ ranking above six of the seven GCHSHL Red division teams might seem odd, but if you look at the data from the season so far, their high rating makes sense.
Normandy was ranked for the first time on Wednesday because they finally had five game results in MHR. MHR does not list a team until they have at least a five-game sample, but five games is still not that many to give reliable results. So, a lot of teams will first appear “too high” or “too low” and then move up and down a lot. At around 10 games, a team’s rating starts to “settle down” and be more accurate.
Normandy has a high ranking because they are 5-0-0 (their 6th game vs. Chagrin Falls was entered after Wednesday morning, when MHR does its weekly update). They have been blowing out their opponents with a 35-1 edge in goals.
Normandy’s rating is also extremely “fragile” because most of its opponents have not played very many games. Parma has only played seven other games and Canfield has just one other game listed in MHR. For example, Normandy beat Canfield 9-1, but we do not have much data on how good Canfield really is (in part because some of their games are missing from MHR). We need more games in MHR for all of the GCHSHL Blue teams.
Alter also has a great record (11-0) with some blowouts. We have more data on Alter, and its opponents have played quite a few games, so its rating is already more stable than Normandy’s. Alter’s numbers will keep changing, but I do not expect as much fluctuation as Normandy’s.
The big value of a system like MHR is how it compares teams from different conferences. To really see how good Normandy is compared to the rest of the state, we have to see how they perform against more than just their GCHSHL foes. This usually happens in tournaments, and Normandy will play in one over the holidays. It will face Whitmer of the NHC White, for example.
MHR has an indirect way of comparing Normandy and Whitmer so far, but head-to-head results will make both team’s MHR rating more reliable. MHR does “fancy math” to compare every team with every other even if they have not played each other, resulting in numbers that represent the balance of all of its data.
Let’s look at Normandy and Whitmer in detail to help you understand how MHR works overall.
Normandy is ranked 20th but more important is its RATING of 86.14. Whitmer has a rating of 78.43. The difference is about 7.5 points, which in MHR-speak means there is data that shows Normandy is about 7 goals better than Whitmer. Normandy has beaten CVCA twice, by 5 and 9 goals. CVCA beat Whitmer by 5 goals. Averaging out all of the other scant bits of data that connect Normandy and Whitmer, it comes out to about a 7.5 difference.
When Normandy and Whitmer play on 12/27, we will have much better data to compare those two teams. If Normandy wins by about 7 goals, their rating will stay about the same. If they win by fewer goals, their rating might drop. If they lose to Whitmer, then their rating might drop a lot (and Whitmer’s will increase significantly).
The fact that Normandy’s rating might drop even if they beat Whitmer is important. As MHR gets more data, it becomes more accurate. It does not mean that MHR is “wrong” now to “predict” that Normandy is 7.5 goals better than Whitmer. It just means we do not have much data yet.
Over time, MHR will get a lot more accurate. The more inter-conference games are played, the better comparisons across the whole state we have. I don’t think Normandy has games scheduled against teams from Columbus or Dayton, so we may not have a lot of data in MHR to actually be confident in their rating. Teams in the Northwest Hockey Conference, Great Lakes Hockey League, Capital Hockey Conference and Southwest Ohio High School Hockey League have scheduled a higher ratio of non-conference teams, so expect their MHR ratings to be more reliable.
Let’s keep using MHR to have interesting discussion about high school hockey in Ohio, but we also need to understand how MHR works.
More as the season progresses.
--- Keith Instone for Ohio Hockey Digest
