Statistical Graphics and more

Presenting at the Dutch Chemometrics Society annual meeting late May this year, I heard a talk of Klaas Faber on the “Athletes Biological Passport” – especially targeting the Pechstein case. Now that the Swiss court finally confirms the ruling, things popped up again. Faber, being the expert of Pechstein, talks about “torture the data until they confess”.

Regardless of Pechstein being guilty or not, there are some problems with the passport from the statistical point of view.

The two major problems with the passport which I remember from Faber’s talk are:

1. The sample from which the confidence intervals are created is based on ordinary people, or at least average sports men. This is certainly due to the fact that we need a large enough sample, but is it representative for the few top athletes – doped or not?
2. Assuming the confidence intervals are created for sportsmen who did not use illegal methods to enhance their performance, we a-priori know that we mistakenly will convict x% clean sportsmen given the (100-x)% interval.

Not only as a statistician I have a problem with the above mentioned points, as statistics is used to convict someone merely by the fact that he/she is off the limits with his/her biological measurements, without any causal connection proven that these values are caused by doping.

Let me finish with a simple example of a sample of a normal distribution of size 100,000 illustrating the dilemma. Plotted in a boxplot, we get the following:

What is marked as “outlier” by the boxplot is, for most cases, not any different from the adjacent values at the whiskers. Getting more and more to the fringes, we might find that some values really “look like” outliers. For this sample we would “convict” 763 cases according to the boxplot definition, although all come from a “perfect” normal distribution.

In the end, much seems to be determined by the credibility of the different sports associations. Very much points to a doping case for Contador, but the UCI seems to cover up for him – the ISU did the opposite for Pechstein.