PDA

View Full Version : Fairly good analysis of the ranking system. Somewhat Long.


Sanneriet
Aug 6th, 2003, 04:35 PM
If It Ain't Fixed, Don't Break It!
In the past weeks, it has become clear that Kim Clijsters will soon become the women's #1. She will probably do this without ever having won a Slam. What's more, she will do it when another player, Serena Williams, will be holding at least two and probably three Slam titles (depending on when Clijsters takes the #1 ranking). It nearly happened already; Clijsters was one match away from clinching the top ranking at San Diego.

This is, predictably, causing screams of agony from some quarters. People who never cared about the ranking system before are suddenly alert to what they perceive, not always accurately, as its problems.

The irony is, the most recent of the myriad changes the WTA made to the ranking system was designed to prevent this exact possibility. (Well, it had other purposes too, but that was a key aspect.) There were several changes made in the system, but the biggest was to crank the point values awarded for the Slams from their already-stratospheric level to somewhere around low earth orbit.

And it failed.

And it failed predictably.

It's not hard to see why. That's what we'll try to examine in this column.

Let's start with a disclaimer. The author works for Daily Tennis because I know and understand the rankings. It happens that I started following tennis around 1996. It was at the end of 1996 that the WTA made what I consider Great Rankings Mistake: They went from divisor rankings to additive rankings. This had as its purpose an attempt to make the top players play more. (In which it succeeded -- at least temporarily, until they all found themselves injured.) The effect of additive rankings is to cause players who played more events to be ranked higher than more successful players who didn't play as much. This was so patently obvious that the author began, starting with the Australian Open 1997 (the first Slam after the shift to additive rankings), to calculate what the rankings would be had the old divisor still been in place. The author still has that first carefully color-coded printout, now rather tattered, showing that, although Arantxa Sanchez-Vicario was ranked #3 following that Australian Open, she would have been #5 under the divisor. Conchita Martinez, the WTA's #4, would have been #6. Monica Seles, then down to #6, was #3 under the divisor. Etc. Talking about the rankings eventually caused me to get this job -- and to watch with dismay as the WTA made change after change in the rankings without fixing the fundamental problems. Nonetheless, I have studied ranking systems, which means that I probably understand them better, at a deep level, than the WTA.

Buried in my archive of possible future articles, the author has a full-blown explanation of how the rankings work (both men's and women's), and someday may inflict that upon you. But a very brief summary of the women's system will suffice for now: The women's rankings are based on points. Points are awarded in two parts: Round points, based on how far you go in tournaments, and quality points, based on the ranking of the players you beat. Events with more prize money carry more round points (meaning that winning a Slam is worth a lot more than winning a Tier I, which is worth a lot more than a Tier III, etc.), but if you beat #1, you get big quality points even if you beat her at the Podunk $10K Futures. This isn't really an "ideal" way of awarding points (chess uses the ELO rankings; the author likes zero-sum rankings with a fixed set of points which players take from each other; these both are designed to assign points very accurately based on results). Still, the WTA double-points-table system isn't too bad; it's easy to calculate without being too inaccurate in itself.

But once we have the points for a particular event, those results from individual tournaments have to be turned into an overall ranking. It's easy to say who was the best player at a particular tournament: The winner was. But somehow, you have to decide who was best over many tournaments. This is where the complications come in. Under the old pre-1997 divisor ranking system, a player's points from all her tournaments were totalled then divided by the number of events she played (hence the term "divisor") -- if two players each earned 2400 points, but one earned 2400 points in 24 events and the other earned them in 16 events, the player who earned the same points in fewer events was ranked higher. This reflected the fact that when she played, she played better.

The 1997 ranking system changed that. Originally, it just added up all your points. A player who lost first round at 20 events earned one point at each event, and so actually earned more round points for those 20 losses than a player earned for a win at a Tier III event. You couldn't actually lose your way to the top -- but any event, even a first round loss, helped your ranking.

The WTA didn't need long to find that this was a problem; soon, instead of all points counting, they counted only a player's best 18 events, which has since been lowered to best 17 (best 11 in doubles). As a matter of fact, had they stayed with the 1997 ranking system, Clijsters would have gained the #1 ranking well before this. Even the Best 18 ranking system would have made her #1 last week. But the shift to Best 17, while it slightly limits the effects of playing a lot, does nothing to address the ultimate problem, which is that losses don't count. The problem remains under Best 17. Consider this scenario: One player plays 16 events, and reaches the second round at all of them. Another player plays the same 16 events, and reaches the second round also. She plays eight other events in which she loses first round. Who is better, the first player, who has a record of 16-16 and no first round losses, or the second, who has a record of 16-24 and eight first round losses? Surely the first -- but the second will probably be ranked higher. Because none of those first round losses count against her; in fact, they give her a single point used to split the tie between the two.

That situation may not sound like the Serena/Clijsters situation, but in fact it's exactly the same. Serena, when she plays, is more effective. But Clijsters has twice as many events. (As of next week's rankings, 22 for Clijsters to 11 for Serena.) As long as the Belgian's early losses aren't held against her, and Serena has goose eggs in six of her seventeen slots -- well, Clijsters has way, way more wins than Serena. She also has more losses, but again, losses don't count. So Clijsters is on the verge of becoming #1.

The problem has been there since 1997. It's no worse now than when it made Lindsay Davenport #1 at the end of 1998 when Martina Hingis had a better winning percentage but Davenport had more events -- or when, in the spring of 2000, it was Davenport's turn to briefly be more effective. It's also no worse than last year, when Serena was #1 by such a wide margin that the ranking system couldn't mess her up despite her limited schedule. The ranking system last year accurately pegged Serena as the best player in the world. Was it equally good at determining who "should" have been #50? This does not automatically follow; last year's year-end #50 was Magui Serna, who played 28 events and had 12 first round losses and three losses in qualifying and would have been #62 under the divisor.

We've heard people propose various ranking systems to assure that the top player always has Slam titles -- everything from saying that the Slam winners are automatically #1-#4 to proposing to inflate the Slams to some ridiculous value so as to achieve the same effect. Neither is any good, mathematically; .if you automatically rank the Slam winners at the top of the list, then the ranking system is chaotic -- a player cannot with certainty know what she must do to attain a particular ranking. In addition, it must be possible (I didn't say easy, I said possible) to become #1 without a Slam. If it isn't possible, then you render the rest of the Tour essentially just a bunch of practice tournaments, and players will tank at will and soon you don't have any audience -- which means you don't have a Tour.

But it's one thing to be possible and another to be easy. Martina Hingis, based on her tournament winning percentage, probably did deserve to be #1 without a Slam at the end of 2000. Clijsters, by that standard, doesn't deserve to be #1; Best 17 has made it too easy for an undeserving player to reach the top. But the cure must be a rational and continuous ranking system, such as the old divisor -- under which Serena was #1 and will remain #1 for the foreseeable future.

The author does not actually advocate a return to the divisor as such. Of ranking systems I know, Elo (the one used for chess) seems to be the most accurate -- but it's very complicated. My own proposal, balancing Tour needs and the requirement for simplicity, is the modified divisor (total points, if the number of events is less than 17, divide by 17 anyway; if you play more than 17 events, you can knock off a third of an event for each event over 17); this still encourages players to play more but makes losses count. And it's as easy to calculate as the divisor, and easier than the current rankings. (Yes, easier, because under the current rankings, you have to track seventeenth and eighteenth tournament scores.) Other systems are of course possible. (The author has, in fact, investigated over two dozen different ranking systems, each based on a slightly different definition of what it means to be the "best player.")

Should the ranking system be changed to prevent the current "Clijsters problem"? Possibly. But keep this in mind: If you thought the ranking system was actually right last year, then it's right this year. If it's wrong this year, it was wrong last year, too. If a change is made, it should not be yet another quick fix. To date, those have done more harm than good. Literally. By making the players play too much. Just ask Martina Hingis's ankle, or Lindsay Davenport's foot -- or, for that matter, Serena's knee, which is responsible for her losing the top spot.

The analogy to the roofing on the Arkansas fiddle player's house is apt: The hole is always there, but when it ain't raining, the roof don't leak. Last year, when Serena Williams was absolutely dominant, no one worried about the defects in the ranking system. Now, it's raining. The temptation for a quick fix is obviously strong. But a better fix can be found by waiting until the storm is over, and then looking for the right fix. Preferably (in the author's opinion) one that counts losses. It's important to remember a key point: The ranking system is zero-sum -- i.e. when one player moves up, another moves down. There is only one #1 player, and only one #2, and one #3. Having a bad ranking system doesn't create more #1 players, or more Top 100 players for that matter. It just creates "wrong" #1 players, meaning that a good ranking system is to be preferred to the bad. Most players will always "play to" the ranking system, of course (which is what Serena refuses to do, and now it's costing her). But if the ranking system is set up properly, that won't really produce too many problems.

We'll have more on this -- including an analysis of how the different ranking systems affect who stands where -- later this week.

jenglisbe
Aug 6th, 2003, 04:44 PM
I didn't read all of that - yet - but the divisor system has its flaws, too. For one, someone like Serena - who plays only major evens - automatically gets an advantage because a SF in, say, Miami, is worth nearly the same as a win in a lower-tier event. In other words, you are almost punished for playing more events. Is that what we need to encourage??

I agree the current system has problems, and as much as I love Kim, she needs to win a major to be truly deserving of #1...but I also can't think of any ranking system that will be perfect.

KV
Aug 6th, 2003, 04:47 PM
What I can say the ranking of old looks fairer IMO.

shap_half
Aug 6th, 2003, 04:58 PM
Would people still be bitching about how erroneous this ranking system is if Justine was on her way to no.1 because she has a Slam? Or what if Venus was no. 1 seeing as how she both plays less and owns 4 slams?

Or is it because Kim doesn't have a Slam?

The Crow
Aug 6th, 2003, 04:59 PM
So in the divisor system, someone winning the Aus open and then not playing for a year is guaranteed the number 1 ranking the whole year. That doesn't really seem fair either.

IMO if Kim gets to number 1, she deserves it. Even without winning a GS.

Jem
Aug 6th, 2003, 05:04 PM
Sanneriet wrote: One player plays 16 events, and reaches the second round at all of them. Another player plays the same 16 events, and reaches the second round also. She plays eight other events in which she loses first round. Who is better, the first player, who has a record of 16-16 and no first round losses, or the second, who has a record of 16-24 and eight first round losses? Surely the first -- but the second will probably be ranked higher.

I am in agreement with you on the use of having a divisor ranking system that makes each match count. However, I don't necessarily agree with the above statement. Player 2 put herself on the line more than Player 1. She suffered more losses, yes, but she had the same amount of wins. Had Player 1 played more, she might have had similar results.

I would prefer to see a ranking system with a minimum divisor (16 or 17 is OK with me). That sets a minimum standard. Then for players who play more (support the tour in my mind), perhaps you would knock off their divisor by 1 for a certain number of tournaments over the minimum or 2 for even more.

Also, I don't know that agree with your contention that the current ranking system has caused the injuries on the tour. Injuries have been rather commonplace on the tour ever since I started watching tennis in the 1970s. You certainly can't blame Serena or Venus' injuries on overplaying -- they just seem injury prone. I don't even think you can say Davenport's injuries are the result of overplaying. If I remember correctly, she hit her knee with her racket, which ultimately led to her surgery.

If players are getting hurt from playing too much these days, it's usually the fault of some stupid scheduling on their part. For example, Davenport usually plays three California hardcourt tournaments in a row. That's not very bright. A couple of years ago, Seles played non-stop for 6-7 weeks on different continents and wound up hurt. With the current tour set-up, it would be fairly simple to devise a schedule that allows you to fulfill your commitments without overplaying.

As for the rankings, I don't think it will ever be perfect. Right now, I consider Serena Williams the best female player on the Tour -- no question about it. Does she deserve to be ranked No. 1? Perhaps not. She's played 11 tournaments in the past year. Granted she's won three Grand Slams, but after the U.S. Open, she'll have only two Grand Slam titles to her name. I realize the Grand Slams are the most coveted titles in the game, but they are not the be-all, end-all. They are weighted approrpiately in the rankings; the rest of the tour matters, too!

DA FOREHAND
Aug 6th, 2003, 05:12 PM
I think they should copy the ATP system.

liuxuan
Aug 6th, 2003, 05:17 PM
I dont think the system needs too much changing however maybe they could cut the counting tournaments to 16 or 15, and also make the round points at grandslams 750, instead of 650.

I dunno though :bounce:

Im actually glad Kim is gonna take over the #1 spot, cos it will makes things at least more interesting.

The Crow
Aug 6th, 2003, 05:22 PM
Totally agree with you, Jem.

Martian Willow
Aug 6th, 2003, 05:29 PM
So in the divisor system, someone winning the Aus open and then not playing for a year is guaranteed the number 1 ranking the whole year. That doesn't really seem fair either.

That's what I was thinking but, as Jem says, the way round that is to have a minimum divisor...but if you do that, anyone who plays less than 16 (if that's what the divisor is) tournaments a year is still at a disadvantage. The only difference is that there's no advantage in playing more than the divisor. So I don't see it would solve the current problem.

(ie: If Serena plays 12, her ranking points are divided by 16. If Kim plays 25, her points are divided by 25.)

Rollo
Aug 6th, 2003, 05:32 PM
Just as an interesting side note-I recently read where Monica Seles floated a petition in 1997 NOT to have the new ranking system. Her criticism of the system? It promotes quantity over quality.


The easiest thing to do would be to reward winning over losing.

Lets look at how they put out points for a Tier 1.


Winner: 300 points
Finalist: 210
Semis: 135
Quarters: 75
R16: 42


It doesn't reward winning enough. I'd propose:

Winner 300
Finalist: 120
Semis 60
Quarters 30
R16 15.


This way tension builds as the semis and finals near. The Jelena Dokics of the tour would stop being rewarded. Oh-and if you lose your first match-no points.

jenglisbe
Aug 6th, 2003, 05:35 PM
I say make the top 16 results count, and I like the idea of a divisor then...as long as players aren't de-warded (is that a word? LOL!) for winning smaller tournaments. I think players should have to play a decent amount of tournaments - and do well in them - to have a good ranking. Whatever ranking system rewards that is cool by me...

Sanneriet
Aug 6th, 2003, 05:48 PM
I should clarify that this was written by Bob Larson and just pasted by me.

controlfreak
Aug 6th, 2003, 06:44 PM
I don't have any issues with a system that rewards quantity over quality. Like Bob says it all depends on how you define your #1 (or any tennis player). I guess I like to reward stamina and physical toughness in my tennis players :) Anyone who wants to be #1 has got to be prepared to go out there and prove it at least 18 weeks a year, if not more. And if someone wants it really badly, they can play 30 tournaments a year, and the challenge is on for the other contenders to either match that feat or stick to their own pace. People will say this causes injuries and burnout, but in my book, part of being an athlete is knowing when to rest and when to play, and getting the balance right. If they screw up by playing too much and end up injured, then they're not #1 material. Better luck next year.

Rollo
Aug 6th, 2003, 06:50 PM
Anyone who wants to be #1 has got to be prepared to go out there and prove it at least 18 weeks a year, if not more.

Why? Because the WTA says so all of a sudden? Before 1997 MOST computer #1's entered about 15 events a year on average.

DA FOREHAND
Aug 6th, 2003, 06:56 PM
LOL IF YOU want to reward stamina make them play best of five.

I prefer quality over quantity anyday. If you're winning the slams, and other big tier events, and have winning h2h against your nearest rival you should be number one.

To put it simply. The best player that year should be ranked number one. Kim says that player is Serena.

bandabou
Aug 6th, 2003, 07:05 PM
The rollo system seems fair to me. That way Serena would still be head and shoulders above the field and it would be Justine who would be challenging her. That seems more logical,´cause Justine has three biggies: two tier I´s and a slam as does Serena: two slams and a tier I.

Kim with her two tier I´s wouldn´t even be in the discussion.

I like the author´s idea of punishing losses is a good one....