From: desj@CS.Princeton.EDU (David desJardins)
Subject: Re: How are "averages" computed?
Date: 1997/04/29
Message-ID: #1/2
Sender: desj@aleph.CS.Princeton.EDU
References: <33639039.1455@geocities.com>
Organization: IDA Center for Communications Research
Newsgroups: rec.games.bridge
Ed Borodkin writes:
> Yet, because of our "average", all other scores were unusual. For
> example, the top board received a 7.88 rather than an 8. And the next
> 2 pairs (with the same result) each received a 6.25.
>
> What is the formula for computing these scores and why? (The club
> uses the ACBL software.)
When there are eight valid scores and a single average, I think the
formula is supposed to be to matchpoint the eight valid scores normally
(so 7 is the top), and then compute M' = (9/8) * (M + 0.5) - 0.5 for
each pair, where M is their score on the 7 top. So if there is an
unshared top, it should come out as (9/8) * (7 + 0.5) - 0.5 = 7.94. The
next two pairs should get (9/8) * (5.5 + 0.5) - 0.5 = 6.25, and so on.
(I can't explain why you observed 7.88 instead of 7.94.)
Why is that the formula? My own belief is that it's due to a flawed
statistical analysis by the people who make the rules. A more logical
formula would be the obvious M' = (8/7) * M, which gives the usual 8 for
a sole top, but still gives fractions for other pairs. For example, the
next two pairs would then get (8/7) * (5.5) = 6.29.
We've been over this before in this newsgroup. I guess I'll briefly
repeat the argument, though. The goal as I see it is as follows. You
are at one of the eight tables that plays the board normally. You
achieve a raw score S, which matchpoints to a matchpoint score M, within
the group of eight valid scores. Thus M is between 0 and 7. The
director now wants to compute a matchpoint score M', depending only on
M, which is an unbiased estimator of the matchpoint score that you would
have received if the board had been played at all nine tables. Thus M'
is between 0 and 8.
(Intuitively, what this means is as follows. You play a bunch of boards
and, being a perfect judge of how the rest of the field will play,
conclude at the end of play that you expect to score exactly 60% on
average against this field. Of course, you will score better on some
days and worse on others, but on average let's say you are right and you
will score exactly 60% if you could average over many repeated trials.
Then the estimator above is unbiased if, when unbeknownst to you some
other pair somewhere else in the room had a fouled board and scored
average, you *still* expect to score exactly 60% on average, if the
event were repeated many many times. This is a logical criterion for
the factoring rule, although not the only possible one---see below.)
The statistical setting and definition of an "unbiased estimator" is as
follows. At each table, there is a random process P_i which
independently generates the raw score S_i at table i. Those random
processes are supposed to be chosen independently and identically from
the "field", which is a probability distribution on the space of score
distributions. We'll assume that everything in sight is measurable; in
bridge, there are only a finite number of possible raw scores, so
everything is in fact finite-dimensional.
Now, there are two things that can happen to you and your opponents when
you sit down to play. In both scenarios, you and your opponents are
supposed to be the same, so you correspond to the same process P_1
chosen from the "field" distribution. Suppose first that all of the
other tables play the board normally, according to iid processes P_2,
..., P_9, and then the board is matchpointed on an 8 top. This
generates a matchpoint score Ma from 0 to 8, which is a random variable
(fixing P_1 and letting the other processes and all of the raw score
outputs be chosen at random) and therefore has an expectation, E(Ma).
The second scenario has the board played seven other times, according to
iid processes P_2, ..., P_8, and the last board achieves no result and
scores average. This generates a matchpoint score Mb from 0 to 7, which
we will then adjust according to our formula to give a "factored" score
Mb' from 0 to 8. This score also has an expectation, E(Mb').
Mb' is an estimator of Ma; it is unbiased if E(Mb') = E(Ma). This means
that given who you and your opponents are, you expect to score the same
number of matchpoints, on average, regardless of whether table 9 plays
the board normally, or receives an average. Of course, since the result
at table 9 is random, sometimes Mb' will be greater than Ma, and
sometimes it will be less; the only thing that we can reasonably require
is that their expectations are equal.
Now, Ma can be written as the sum of eight terms: Ma = M2 + M3 + M4 + M5
+ M6 + M7 + M8. Where M2 is 1.0 if you get a higher score than your
competitors at table 2, 0.5 if you get the same score, and 0.0 if you
get a lower score, and similarly for the other tables. Because
expectation is linear, E(Ma) = E(M2) + E(M3) + E(M4) + E(M5) + E(M6) +
E(M7) + E(M8). This formula holds even though M2, M3, etc., are NOT
independent; this is the most fundamental law of statistics.
Furthermore, E(M2) = E(M3) = ... = E(M8); these quantities are all
equal, because the pairs at table 2 and the pairs at table 3 were
selected randomly from the same field, therefore you are just as likely
to defeat the pair at table 2 as you are the pair at table 3, and vice
versa. So E(Ma) = 8 * E(M2).
Exactly the same argument applies to Mb. Mb = M2 + M3 + M4 + M5 + M6 +
M7, so E(Mb) = E(M2) + E(M3) + E(M4) + E(M5) + E(M6) + E(M7), so E(Mb) =
7 * E(M2).
If we have a *linear* formula relating Mb' to Mb (there are reasons for
seeking a linear formula, but they are beyond the scope of this note;
for the moment, we can content ourselves with the observation that all
candidates are linear), Mb' = A * Mb + B, then E(Mb') = A * E(Mb) + B =
7 * A * E(M2) + B = (7 * A / 8) * E(Ma) + B. But our goal was an
unbiased estimator, where E(Mb') = E(Ma). Therefore we must have A =
8/7 and B = 0. This gives my formula above, Mb' = (8/7) * Mb, and not
the formula unfortunately in widespread use by the ACBL and WBF.
Fortunately, it doesn't matter very much. Also fortunately, there isn't
really a "right" formula. While it is logical to use an unbiased
estimator, and the above calculation shows what it should be, fairness
does not strictly dictate an unbiased rule. An unbiased estimator means
that if you have what you expect to be a 60% game on average, then it
will still be a 60% game on average when the unbiased estimator is used
for factoring. However, the unbiased estimator can be seen to increase
the variance of your scores, while preserving the expectation. Thus
while your game will still average 60%, the chance that you happen to
score over 62% will be (very slightly) increased. Thus the unbiased
estimator, while fair from the point of view of expectation, may (very
slightly) increase your chance of winning the event. From this point of
view, it's even possible that the WBF formula is more fair: if you have
a good game so that you are in contention to win, then the WBF formula
reduces your expectation, but it also increases your variance, so the
effect on your chance of winning is unclear. To know whether it really
helps or hurts your chances, one would have to know more about the
*distribution* of scores, and also about the differences between pairs
in the field, factors that don't come into the analysis of the simple
unbiased estimator.
(That's probably more than you wanted to know.)
David desJardins
--
Copyright 1997 David desJardins. Unlimited permission is granted to quote
from this posting for non-commercial use as long as attribution is given.
From: desj@ccr-p.ida.org (David desJardins)
Subject: Re: Factoring Matchpoint Scores
Date: 1996/06/22
Message-ID: <4qh5v7$8a1@runner.ccr-p.ida.org>#1/1
References: <31C913BB.64F9@OKbridge.Com> <4qejt9$tv@tang.ccr-p.ida.org>
<4qgmpg$itb@dscomsa.desy.de>
Organization: IDA Center for Communications Research, Princeton
Newsgroups: rec.games.bridge
Henk Uijterwaal writes:
> If we now look at the formulae, then we see that your formula always
> gives you (n-1) matchpoints. This doesn't feel right as the last table
> will occasionally beat you.
This is absolutely true, but I hope that you will try to carefully
understand my answer.
There are two things that one might try to do with a matchpoint
factoring formula. One goal is to give the correct posterior score on
every hand: that is, if a pair receives a particular score S on an n-1
top, then you want to assign them the score S' on an N-1 top which is
the average score that they would have received if the board had been
played a larger number of times. The second goal is a weaker one: only
to find a formula S' = f(S) which has the property that if a pair plays
a board on an n-1 top, and achieves a particular result, which is then
matchpointed and factored up using the formula, then they will get the
same number of matchpoints on average as if they had played the board on
an N-1 top in the first place.
The first property is an obviously stronger property, which implies the
second property. As you point out, the "correct formula" (would you
prefer "unbiased formula") has the second property only, not the first,
since under the first property a score of n-1 must transform to
something less than N-1. However, Neuberg's formula does not even have
the second property (as I have proven in my previous posting), and
therefore cannot have the first property either.
The problem is that the first property is ill-defined. The only way
that I know to make it well-defined is to assume that all pairs have
equal skill, so that all of the scores (including the score S to be
factored up) can be regarded as sampled from the same distribution.
To my mind, this is a bad assumption, because the purpose of bridge
tournaments is precisely to identify those players with greater skill.
If you do make such an assumption, you can get a formula, by integrating
over the (n-1)-dimensional simplex, which has both the first and the
second property. I would be happy to work this out if anyone is
interested. However, it won't be a linear formula, like all of those
that have been discussed so far (I think it will involve gamma
functions), and hence will be difficult to use in practice.
> The last time we discussed this topic, you planned to write the WBF
> scoring committee and I looked up the address for you. Did you ever
> write?
I decided that if I can't even convince you, the chances of successfully
explaining the issues to the WBF scoring committee were close to zero,
so I gave up. Do you think there are any professional statisticians on
the committee?
David desJardins
--
Copyright 1996 David desJardins. Unlimited permission is granted to quote
from this posting for non-commercial use as long as attribution is given.
From: desj@ccr-p.ida.org (David desJardins)
Subject: Re: Factoring Matchpoint Scores
Date: 1996/06/21
Message-ID: <4qfepj$4r2@runner.ccr-p.ida.org>#1/1
References: <31C913BB.64F9@OKbridge.Com> <4qejt9$tv@tang.ccr-p.ida.org>
<4qep6b$qcj@sjx-ixn3.ix.netcom.com>
Organization: IDA Center for Communications Research, Princeton
Newsgroups: rec.games.bridge
Nick France writes:
> Which is right is just a matter of whether you want one universal
> result for all the partnership or a result based on the opponents of
> each individual partnership. There is no right mathematical answer.
There is a fair answer (i.e., an unbiased estimator) and an unfair
answer (i.e., a biased estimator). Suppose an event has 50 tables and
100 pairs. Two of the pairs are strong and the other 98 are weak. The
98 weak pairs are all exactly equal in ability; the two strong pairs
(denoted Pair A and Pair B) are also equal in ability, but will beat the
weak pairs on average. All of the pairs play 26 boards, each board
played 13 times, but one board is unplayable at one table due to an
irregularity, so there are only 12 scores for that board. Pair A
happened to play the irregular board, while Pair B did not.
If the scores are factored up according to the ACBL/WBF formula, then
Pair A will, on average, score less than Pair B will score, on average.
(Although still above the field average.) If the scores are factored up
according to the formula I posted, then Pair A will, on average, score
the same as Pair B. My formula is essentially the unique formula with
this property. I consider this an important difference and one which
has a fundamental bearing on fairness, and not just a matter of "what
you want." On the other hand, if you don't consider this form of
fairness an important property, then you can factor up however you like.
David desJardins
--
Copyright 1996 David desJardins. Unlimited permission is granted to quote
from this posting for non-commercial use as long as attribution is given.
From: desj@ccr-p.ida.org (David desJardins)
Subject: Re: Factoring Matchpoint Scores
Date: 1996/06/21
Message-ID: <4qejt9$tv@tang.ccr-p.ida.org>#1/2
References: <31C913BB.64F9@OKbridge.Com>
Organization: IDA Center for Communications Research, Princeton
Newsgroups: rec.games.bridge
Matthew Clegg writes:
> Given that we have observed 12 scores from a distribution, let's
> compute the expected number of matchpoints that would be received by a
> pair if a 13th score were to be drawn from the same distribution.
Certainly the correct first-order goal is to give a pair the average
score that it would have been expected to receive in a larger section,
given the available information.
> Suppose in an n-table event that the number of scores less than a
> given score s is x and the number of scores equal to s is y, so the
> number of matchpoints awarded to s is x + y/2.
Normally the pair with the worst score in a section gets 0 matchpoints;
they don't get a half matchpoint for tying themselves. So, actually,
the number of matchpoints awarded to s is x + (y-1)/2.
> If a new score is drawn from the distribution defined by the first n
> scores, then with probability (x/n) it will be less than s and with
> probability (y/n) it will equal s. Therefore, the expected number of
> matchpoints awarded to s will be (x+y/2)*(1 + 1/n).
Actually, x + (y-1)/2 + x/n + y/2n.
> So, to factor an n-table event up to an N-table event, this logic
> implies that one should multiply each matchpoint score by (N/n).
Only if you correct for the half matchpoints. If you do that, you get a
score of the form
S' = (N/n) * (S + 1/2) - 1/2
which is the so-called "Neuberg's formula."
However, this still isn't right. The reason is that there is a
selection bias imposed when a score is factored based on including
itself in the sample. The unbiased way to do the factoring is to
exclude s itself from the sample. This gives the better formula:
S' = (N-1)/(n-1) * S
Here's an example (the same one that I used two years ago) that
illustrates why the latter formula is correct. Suppose a particular
board is played in several sections of different sizes. Suppose for
simplicity that the obvious contract is 4S, and there are only two
possible results, +420 or -50. Suppose that the field is such that +420
will be achieved with probability P, and -50 with probability (1-P). It
isn't necessary to know what P is.
Now take a particular pair in an n-table section (i.e., n-1 top). If
that pair happens to score +420, then they should receive (n-1)*(1-P/2)
matchpoints on average. And they will: each other time the board is
played, they will get 1 matchpoint with probability P, and 1/2
matchpoint with probability 1-P; adding this up n-1 times gives the
desired result.
Now, suppose that pair had played in an N-table section (i.e., N-1
top). Then, on average, their score would have been worth (N-1)*(1-P/2)
matchpoints. Thus the factoring formula should have the property that
if you compute their score in the smaller section, then factor it up to
the larger section, then on average it is equal to the average result if
they had just played in the larger section. Since both formulas are
linear, we can simply test this for the candidates:
Neuberg's formula:
S' = (N/n) * (S + 1/2) - 1/2
E(S') = (N/n) * (E(S) + 1/2) - 1/2
= (N/n) ((n-1)*(1-P/2) + 1/2) - 1/2
= (N-N/n) * (1-P/2) + 1/2 (N/n-1)
Correct formula:
S' = (N-1)/(n-1) * S
E(S') = (N-1)/(n-1) * E(S)
= (N-1)/(n-1) * (n-1)*(1-P/2)
= (N-1) * (1-P/2)
The correct formula gives the pair the same average result if they play
in the smaller section and factor up, as if they play in the larger
section. Neuberg's formula means that above-average scores are
reduced if factored up, and below-average scores increased; this makes
it harder to win in a smaller section.
However, there are other reasons why it's easier to win in a smaller
section. Since each board is played a smaller number of times, the
variance of the score is higher. Therefore, the probability of getting
an exceptionally high score goes up. So, while it's clear that an
above-average pair playing in the small section will get a lower score
on average using Neuberg's formula than the correct formula, it's not
clear whether they will actually win more or less often. That depends
on additional hypotheses.
> If so, why does this formula differ from the formula used by the ACBL
> and WBF?
I think that they use Neuberg's formula because they don't understand
the statistics very well, not from any conscious decision to compensate
players in larger sections for their lower variance. The last time this
came up, it was pretty clear that some people simply couldn't understand
the above argument.
David desJardins
--
Copyright 1996 David desJardins. Unlimited permission is granted to quote
from this posting for non-commercial use as long as attribution is given.