From: desj@CS.Princeton.EDU (David desJardins) Subject: Re: How are "averages" computed? Date: 1997/04/29 Message-ID: #1/2 Sender: desj@aleph.CS.Princeton.EDU References: <33639039.1455@geocities.com> Organization: IDA Center for Communications Research Newsgroups: rec.games.bridge Ed Borodkin writes: > Yet, because of our "average", all other scores were unusual. For > example, the top board received a 7.88 rather than an 8. And the next > 2 pairs (with the same result) each received a 6.25. > > What is the formula for computing these scores and why? (The club > uses the ACBL software.) When there are eight valid scores and a single average, I think the formula is supposed to be to matchpoint the eight valid scores normally (so 7 is the top), and then compute M' = (9/8) * (M + 0.5) - 0.5 for each pair, where M is their score on the 7 top. So if there is an unshared top, it should come out as (9/8) * (7 + 0.5) - 0.5 = 7.94. The next two pairs should get (9/8) * (5.5 + 0.5) - 0.5 = 6.25, and so on. (I can't explain why you observed 7.88 instead of 7.94.) Why is that the formula? My own belief is that it's due to a flawed statistical analysis by the people who make the rules. A more logical formula would be the obvious M' = (8/7) * M, which gives the usual 8 for a sole top, but still gives fractions for other pairs. For example, the next two pairs would then get (8/7) * (5.5) = 6.29. We've been over this before in this newsgroup. I guess I'll briefly repeat the argument, though. The goal as I see it is as follows. You are at one of the eight tables that plays the board normally. You achieve a raw score S, which matchpoints to a matchpoint score M, within the group of eight valid scores. Thus M is between 0 and 7. The director now wants to compute a matchpoint score M', depending only on M, which is an unbiased estimator of the matchpoint score that you would have received if the board had been played at all nine tables. Thus M' is between 0 and 8. (Intuitively, what this means is as follows. You play a bunch of boards and, being a perfect judge of how the rest of the field will play, conclude at the end of play that you expect to score exactly 60% on average against this field. Of course, you will score better on some days and worse on others, but on average let's say you are right and you will score exactly 60% if you could average over many repeated trials. Then the estimator above is unbiased if, when unbeknownst to you some other pair somewhere else in the room had a fouled board and scored average, you *still* expect to score exactly 60% on average, if the event were repeated many many times. This is a logical criterion for the factoring rule, although not the only possible one---see below.) The statistical setting and definition of an "unbiased estimator" is as follows. At each table, there is a random process P_i which independently generates the raw score S_i at table i. Those random processes are supposed to be chosen independently and identically from the "field", which is a probability distribution on the space of score distributions. We'll assume that everything in sight is measurable; in bridge, there are only a finite number of possible raw scores, so everything is in fact finite-dimensional. Now, there are two things that can happen to you and your opponents when you sit down to play. In both scenarios, you and your opponents are supposed to be the same, so you correspond to the same process P_1 chosen from the "field" distribution. Suppose first that all of the other tables play the board normally, according to iid processes P_2, ..., P_9, and then the board is matchpointed on an 8 top. This generates a matchpoint score Ma from 0 to 8, which is a random variable (fixing P_1 and letting the other processes and all of the raw score outputs be chosen at random) and therefore has an expectation, E(Ma). The second scenario has the board played seven other times, according to iid processes P_2, ..., P_8, and the last board achieves no result and scores average. This generates a matchpoint score Mb from 0 to 7, which we will then adjust according to our formula to give a "factored" score Mb' from 0 to 8. This score also has an expectation, E(Mb'). Mb' is an estimator of Ma; it is unbiased if E(Mb') = E(Ma). This means that given who you and your opponents are, you expect to score the same number of matchpoints, on average, regardless of whether table 9 plays the board normally, or receives an average. Of course, since the result at table 9 is random, sometimes Mb' will be greater than Ma, and sometimes it will be less; the only thing that we can reasonably require is that their expectations are equal. Now, Ma can be written as the sum of eight terms: Ma = M2 + M3 + M4 + M5 + M6 + M7 + M8. Where M2 is 1.0 if you get a higher score than your competitors at table 2, 0.5 if you get the same score, and 0.0 if you get a lower score, and similarly for the other tables. Because expectation is linear, E(Ma) = E(M2) + E(M3) + E(M4) + E(M5) + E(M6) + E(M7) + E(M8). This formula holds even though M2, M3, etc., are NOT independent; this is the most fundamental law of statistics. Furthermore, E(M2) = E(M3) = ... = E(M8); these quantities are all equal, because the pairs at table 2 and the pairs at table 3 were selected randomly from the same field, therefore you are just as likely to defeat the pair at table 2 as you are the pair at table 3, and vice versa. So E(Ma) = 8 * E(M2). Exactly the same argument applies to Mb. Mb = M2 + M3 + M4 + M5 + M6 + M7, so E(Mb) = E(M2) + E(M3) + E(M4) + E(M5) + E(M6) + E(M7), so E(Mb) = 7 * E(M2). If we have a *linear* formula relating Mb' to Mb (there are reasons for seeking a linear formula, but they are beyond the scope of this note; for the moment, we can content ourselves with the observation that all candidates are linear), Mb' = A * Mb + B, then E(Mb') = A * E(Mb) + B = 7 * A * E(M2) + B = (7 * A / 8) * E(Ma) + B. But our goal was an unbiased estimator, where E(Mb') = E(Ma). Therefore we must have A = 8/7 and B = 0. This gives my formula above, Mb' = (8/7) * Mb, and not the formula unfortunately in widespread use by the ACBL and WBF. Fortunately, it doesn't matter very much. Also fortunately, there isn't really a "right" formula. While it is logical to use an unbiased estimator, and the above calculation shows what it should be, fairness does not strictly dictate an unbiased rule. An unbiased estimator means that if you have what you expect to be a 60% game on average, then it will still be a 60% game on average when the unbiased estimator is used for factoring. However, the unbiased estimator can be seen to increase the variance of your scores, while preserving the expectation. Thus while your game will still average 60%, the chance that you happen to score over 62% will be (very slightly) increased. Thus the unbiased estimator, while fair from the point of view of expectation, may (very slightly) increase your chance of winning the event. From this point of view, it's even possible that the WBF formula is more fair: if you have a good game so that you are in contention to win, then the WBF formula reduces your expectation, but it also increases your variance, so the effect on your chance of winning is unclear. To know whether it really helps or hurts your chances, one would have to know more about the *distribution* of scores, and also about the differences between pairs in the field, factors that don't come into the analysis of the simple unbiased estimator. (That's probably more than you wanted to know.) David desJardins -- Copyright 1997 David desJardins. Unlimited permission is granted to quote from this posting for non-commercial use as long as attribution is given. From: desj@ccr-p.ida.org (David desJardins) Subject: Re: Factoring Matchpoint Scores Date: 1996/06/22 Message-ID: <4qh5v7$8a1@runner.ccr-p.ida.org>#1/1 References: <31C913BB.64F9@OKbridge.Com> <4qejt9$tv@tang.ccr-p.ida.org> <4qgmpg$itb@dscomsa.desy.de> Organization: IDA Center for Communications Research, Princeton Newsgroups: rec.games.bridge Henk Uijterwaal writes: > If we now look at the formulae, then we see that your formula always > gives you (n-1) matchpoints. This doesn't feel right as the last table > will occasionally beat you. This is absolutely true, but I hope that you will try to carefully understand my answer. There are two things that one might try to do with a matchpoint factoring formula. One goal is to give the correct posterior score on every hand: that is, if a pair receives a particular score S on an n-1 top, then you want to assign them the score S' on an N-1 top which is the average score that they would have received if the board had been played a larger number of times. The second goal is a weaker one: only to find a formula S' = f(S) which has the property that if a pair plays a board on an n-1 top, and achieves a particular result, which is then matchpointed and factored up using the formula, then they will get the same number of matchpoints on average as if they had played the board on an N-1 top in the first place. The first property is an obviously stronger property, which implies the second property. As you point out, the "correct formula" (would you prefer "unbiased formula") has the second property only, not the first, since under the first property a score of n-1 must transform to something less than N-1. However, Neuberg's formula does not even have the second property (as I have proven in my previous posting), and therefore cannot have the first property either. The problem is that the first property is ill-defined. The only way that I know to make it well-defined is to assume that all pairs have equal skill, so that all of the scores (including the score S to be factored up) can be regarded as sampled from the same distribution. To my mind, this is a bad assumption, because the purpose of bridge tournaments is precisely to identify those players with greater skill. If you do make such an assumption, you can get a formula, by integrating over the (n-1)-dimensional simplex, which has both the first and the second property. I would be happy to work this out if anyone is interested. However, it won't be a linear formula, like all of those that have been discussed so far (I think it will involve gamma functions), and hence will be difficult to use in practice. > The last time we discussed this topic, you planned to write the WBF > scoring committee and I looked up the address for you. Did you ever > write? I decided that if I can't even convince you, the chances of successfully explaining the issues to the WBF scoring committee were close to zero, so I gave up. Do you think there are any professional statisticians on the committee? David desJardins -- Copyright 1996 David desJardins. Unlimited permission is granted to quote from this posting for non-commercial use as long as attribution is given. From: desj@ccr-p.ida.org (David desJardins) Subject: Re: Factoring Matchpoint Scores Date: 1996/06/21 Message-ID: <4qfepj$4r2@runner.ccr-p.ida.org>#1/1 References: <31C913BB.64F9@OKbridge.Com> <4qejt9$tv@tang.ccr-p.ida.org> <4qep6b$qcj@sjx-ixn3.ix.netcom.com> Organization: IDA Center for Communications Research, Princeton Newsgroups: rec.games.bridge Nick France writes: > Which is right is just a matter of whether you want one universal > result for all the partnership or a result based on the opponents of > each individual partnership. There is no right mathematical answer. There is a fair answer (i.e., an unbiased estimator) and an unfair answer (i.e., a biased estimator). Suppose an event has 50 tables and 100 pairs. Two of the pairs are strong and the other 98 are weak. The 98 weak pairs are all exactly equal in ability; the two strong pairs (denoted Pair A and Pair B) are also equal in ability, but will beat the weak pairs on average. All of the pairs play 26 boards, each board played 13 times, but one board is unplayable at one table due to an irregularity, so there are only 12 scores for that board. Pair A happened to play the irregular board, while Pair B did not. If the scores are factored up according to the ACBL/WBF formula, then Pair A will, on average, score less than Pair B will score, on average. (Although still above the field average.) If the scores are factored up according to the formula I posted, then Pair A will, on average, score the same as Pair B. My formula is essentially the unique formula with this property. I consider this an important difference and one which has a fundamental bearing on fairness, and not just a matter of "what you want." On the other hand, if you don't consider this form of fairness an important property, then you can factor up however you like. David desJardins -- Copyright 1996 David desJardins. Unlimited permission is granted to quote from this posting for non-commercial use as long as attribution is given. From: desj@ccr-p.ida.org (David desJardins) Subject: Re: Factoring Matchpoint Scores Date: 1996/06/21 Message-ID: <4qejt9$tv@tang.ccr-p.ida.org>#1/2 References: <31C913BB.64F9@OKbridge.Com> Organization: IDA Center for Communications Research, Princeton Newsgroups: rec.games.bridge Matthew Clegg writes: > Given that we have observed 12 scores from a distribution, let's > compute the expected number of matchpoints that would be received by a > pair if a 13th score were to be drawn from the same distribution. Certainly the correct first-order goal is to give a pair the average score that it would have been expected to receive in a larger section, given the available information. > Suppose in an n-table event that the number of scores less than a > given score s is x and the number of scores equal to s is y, so the > number of matchpoints awarded to s is x + y/2. Normally the pair with the worst score in a section gets 0 matchpoints; they don't get a half matchpoint for tying themselves. So, actually, the number of matchpoints awarded to s is x + (y-1)/2. > If a new score is drawn from the distribution defined by the first n > scores, then with probability (x/n) it will be less than s and with > probability (y/n) it will equal s. Therefore, the expected number of > matchpoints awarded to s will be (x+y/2)*(1 + 1/n). Actually, x + (y-1)/2 + x/n + y/2n. > So, to factor an n-table event up to an N-table event, this logic > implies that one should multiply each matchpoint score by (N/n). Only if you correct for the half matchpoints. If you do that, you get a score of the form S' = (N/n) * (S + 1/2) - 1/2 which is the so-called "Neuberg's formula." However, this still isn't right. The reason is that there is a selection bias imposed when a score is factored based on including itself in the sample. The unbiased way to do the factoring is to exclude s itself from the sample. This gives the better formula: S' = (N-1)/(n-1) * S Here's an example (the same one that I used two years ago) that illustrates why the latter formula is correct. Suppose a particular board is played in several sections of different sizes. Suppose for simplicity that the obvious contract is 4S, and there are only two possible results, +420 or -50. Suppose that the field is such that +420 will be achieved with probability P, and -50 with probability (1-P). It isn't necessary to know what P is. Now take a particular pair in an n-table section (i.e., n-1 top). If that pair happens to score +420, then they should receive (n-1)*(1-P/2) matchpoints on average. And they will: each other time the board is played, they will get 1 matchpoint with probability P, and 1/2 matchpoint with probability 1-P; adding this up n-1 times gives the desired result. Now, suppose that pair had played in an N-table section (i.e., N-1 top). Then, on average, their score would have been worth (N-1)*(1-P/2) matchpoints. Thus the factoring formula should have the property that if you compute their score in the smaller section, then factor it up to the larger section, then on average it is equal to the average result if they had just played in the larger section. Since both formulas are linear, we can simply test this for the candidates: Neuberg's formula: S' = (N/n) * (S + 1/2) - 1/2 E(S') = (N/n) * (E(S) + 1/2) - 1/2 = (N/n) ((n-1)*(1-P/2) + 1/2) - 1/2 = (N-N/n) * (1-P/2) + 1/2 (N/n-1) Correct formula: S' = (N-1)/(n-1) * S E(S') = (N-1)/(n-1) * E(S) = (N-1)/(n-1) * (n-1)*(1-P/2) = (N-1) * (1-P/2) The correct formula gives the pair the same average result if they play in the smaller section and factor up, as if they play in the larger section. Neuberg's formula means that above-average scores are reduced if factored up, and below-average scores increased; this makes it harder to win in a smaller section. However, there are other reasons why it's easier to win in a smaller section. Since each board is played a smaller number of times, the variance of the score is higher. Therefore, the probability of getting an exceptionally high score goes up. So, while it's clear that an above-average pair playing in the small section will get a lower score on average using Neuberg's formula than the correct formula, it's not clear whether they will actually win more or less often. That depends on additional hypotheses. > If so, why does this formula differ from the formula used by the ACBL > and WBF? I think that they use Neuberg's formula because they don't understand the statistics very well, not from any conscious decision to compensate players in larger sections for their lower variance. The last time this came up, it was pretty clear that some people simply couldn't understand the above argument. David desJardins -- Copyright 1996 David desJardins. Unlimited permission is granted to quote from this posting for non-commercial use as long as attribution is given.