How to Play
Making Predictions
After creating an account, you'll be able to save your predictions for upcoming matches by checking the box on the right after moving the slider, representing the range of probabilities for the outcome, to your predicted outcome. You can always change your prediction up until an hour before the match begins.
The model's predictions are indicated on the slider for each match with a line, so you can base your predictions on the model's to get a baseline for the matchup.
New matches and match results are automatically updated, so be sure to check back frequently, especially during playoffs when matchups may not be set until a few days before the match happens!
Score & Analyst Rating (AR)
Your score over the course of a season or tournament is the sum of all your match prediction AR values, where AR is a measure of how accurate a prediction was. Scores reset each season and international competition, so everyone starts out on a level playing field.
AR values range from 0 to 100, with a 50/50 prediction yielding 75 points. It rewards well-placed confidence, but punishes misplaced confidence heavily.
Other Stats
Up/Down rating is the percent of predictions in which you predicted the winning team, regardless of the probability you selected.
Raw AR is the average AR across all the matches you've predicted.
Adjusted AR takes into account how experienced you are as an analyst, scaling up to eventually match your Raw AR as you make more and more predictions.
The Math
AR is derived directly from Brier Score, which is a measure of accuracy for probabilistic predictions. The Brier score for a prediction is the squared error between the predicted outcome and the actual outcome, and AR is calculated by scaling brier score to a range of ±100. In the formulas below, \(AR\) is Analyst Rating, \(BS\) is Brier Score, \(N\) is the number of matches predicted, and \(XP\) is the analyst experience factor:
$$Adjusted AR = (100 - 100BS) * XP $$
$$BS = \frac{1}{N} \sum_{t=1}^N (p_t - o_t)^2$$
$$XP = min\left(1 , \frac{1}{3} log_{10} (N)\right)$$
Due to the nature of Brier Scores being based on squared errors, an optimal score can be achieved by being consistenty accurate in your predictions. For instance, among matches you predict with 70% certainty, you should be correct 70% of the time. If you are getting more than 70% correct, you should be more confident in your predictions to increase your score, but if your record isn't keeping up with your predictions, you should be less confident in your predictions.
Elo Model
Elo Basics
The Elo Rating System aims to accurately predict match outcomes by maintaining a rating for each team. The model uses those ratings to derive probabilities for each match outcome, and then depending on the actual match result, the model updates its ratings for the teams.
The model uses the following formula to predict the outcome of a match between Team 1 \(T_1\) and Team 2 \(T_2\) where the teams are represented by their current ratings. The prediction is expressed as a probability between 0 and 1 that \(T_1\) will win, with the opposite being true for \(T_2\).
$$ P(T_1) = \frac{1} {10^ {\frac{-(T_1 - T_2)} {400}} + 1} $$
After the match concludes, the model will calculate an adjustment value that's used to update each team's rating: $$ Adj = K \times M \times (1-P(T_W)) $$ where \(K\) is a fixed parameter in the model, \(M\) is the match score adjustment, and \(T_W\) is the winning team. The winning team's rating gains \(Adj\) while the losing team's rating decreases by as much.
Implementation Specifics
Regional Ratings
International competitions have an interesting way of modifying a whole region's overall rating due to the zero-sum aspect of Elo rating system. Teams that improve their rating will take that point gain back to their regional league, while teams whose ratings drop will take that loss back to their regional league. Effectively, the winning regions steal points from the losing regions, which accumulates over time into some regions having higher average ratings than others.
Match Score Adjustment
Many matches in playoffs and in some regions' regular season games are played as a best-of-X format, where teams play multiple games and the first team to win a majority of the allotted games wins the match. In these cases, we want to adjust the rating change based on how dominant the victory was. The League of Elo model calculates this factor as: $$ M = \left( \frac{W(W-L)} {W+L} \right) ^{0.7} $$ where the \(W\) is the winner's match score and \(L\) is the loser's match score. What this effectively means is that a team that wins a match 3-0 will gain 2.15x what they would have otherwise gained for a single game victory. A 3-1 victory would yield a 1.33x modifier, and a 3-2 victory yields a 0.70x modifier. This compensates teams appropriately for being more or less dominant than expected.
Season Resets
Between seasons, every team's rating regresses 25% toward the regional mean to adjust for any changes in roster, meta, coaching, etc. This also tells the model that there is more uncertainty at the beginning of each new season.
Data Sources
All match data is graciously pulled from Leaguepedia through their API.