Announcing an overhaul of the CoCo's Glicko-2 rating system

So, today, we have a little bit of news to share.

But first, a little backstory.

From the very beginning, when we first launched the CoCo back in late 2019, we set out to do things a little differently. We weren't interested in hewing blindly to tradition; we weren't planning on running our organization in certain ways simply because "that's how it's always been done." Instead, we looked to challenge the status quo, finding ways to take the existing tournament Scrabble scene and improve upon it. We were willing to be bold.

Of course, when you're bold, you always run the risk of making mistakes. There's an old cliché about "moving fast and breaking things" that certainly applies here.

One such breakable thing was the rating system. When we first launched five years ago, we announced that we were using an innovative approach to rating players, one that hadn't been used before in the world of in-person tournament Scrabble play. We introduced what's called a Glicko-2 system - in short, a formula that evaluates players based on the strength of their opponent in each game, as well how they perform against their expected spread in that game. The operative word there is "spread." The idea is, your Scrabble skill isn't just reflected by the binary outcome of a game, whether you win it or lose it - how much you win by matters. A player who wins a game by 100 points is better than a player who wins that same game by 50. The rating system should reflect that.

So we set out to use this new Glicko-2 method. We introduced it at our first tournament, the 2020 Hood River Open. Then there was a long pandemic layoff; after that, we returned in late 2021 and continued to crank out Glicko-2 ratings for event after event. To this date, we've used our system to rate 58 tournaments.

But along the way, some flaws in the math started to emerge. We began hearing complaints from players about their ratings - the most common of which, by far, was that the numbers were too "sticky." People would have great tournaments and see their ratings go up by only a point or two; they'd have terrible tournaments and only lose a couple.

So we dug into the data and realized that, sure enough, these players' complaints had merit. There was a flaw in our formula. I won't go too deep into the math here (because I don't understand it super well myself, honestly), but the gist of it is, the ratings formula used a "standard deviation" variable that controlled how much people's ratings would change. The dev was supposed to gradually decline over time for each player - in other words, the more rating history you have, the clearer your skill level is, and the more your rating should stabilize. But because of an irregularity in our formula, we were instead minimizing everyone's standard deviations. We were basically setting every player's rating like it was nearly set in stone and should only change minimally from here on. This wasn't right.

So, once we discovered this issue, we set out to fix it.

Which brings us to today's news.

This week, we are going back and fixing everything. And I do mean everything. We've revised the rating system so it works as intended, and we're going all the way back to that first Hood River tournament in early 2020 and rating every tournament from scratch. We're putting together the finalized numbers this week, and they will be up on the website soon. In the coming days, everyone's ratings will change - and moving forward, they're going to keep on changing. A lot more than they used to change, too. It's a whole new day.

To be clear, the changes won't be that dramatic for most players. According to the most recent version of the numbers I peeked at, of the 165 people who currently have a CoCo rating, only 14 will see their rating go up by 50 points or more. Only 21 will go down by 50 or more. That means everyone else - about 79% of players - will see a more modest change. And also keep in mind, even if your rating does end up changing significantly, it could easily change back. Your rating might only be high right now because of a recent good tournament, or low because of a bad one. Under this new system, we expect things to even out quickly, and players should soon settle into accurate ratings that reflect their true skill levels.

So - brace yourselves! The new ratings should be out within the next few days. We will follow up shortly. Get ready to see a shiny new number next to your name soon. Thank you to those of you who shared their feedback, encouraging us to get moving and make these necessary changes - and thanks, everyone, for bearing with us during this little changeover period.

Enjoy your new ratings, folks.