Posted on

Imprisoned Because of Bad Statistics

57 Flares Twitter 38 Facebook 15 Reddit 1 StumbleUpon 1 LinkedIn 1 Google+ 1 57 Flares ×

deadly

The infamous Lucia de Berk case officially ended three years ago, but are there still lessons to learn from her ordeal?

The name Lucia de Berk will always be associated with a huge miscarriage of justice in the Netherlands. Accused of murdering patients at the Juliana Children’s Hospital where she worked as a nurse, De Berk was arrested in 2001 and found guilty of seven counts of murder in addition to three counts of attempted murder that were added to her rap sheet in 2004. The case relied heavily on bad statistics that were wrongly interpreted, eventually leading to a life sentence.

A campaign involving philosophy professor emeritus  Ton Derksen, geriatrician Metta de Noo-Derksen and statistics professor  Richard Gill resulted in De Berk’s retrial in 2008. She was acquitted on all counts in 2010.  The court officially stated that not only was De Berk innocent, but there had never been any murders in the first place. It went even further to say that De Berk and other nurses had battled to save lives which were then lost due to errors by medical specialists.

Three years later the landscape of the Dutch legal system has changed, due in part to De Berk’s case. Today we hear from Richard Gill to find out what more needs to be done in order to fix the justice system in the Netherlands.

Bad statistics in court
The Lucia de Berk case is most famous for its statistical failings with several blunders paving the way to the erroneous guilty verdict. One of the striking facts of the case was the lack of evidence. There were no witnesses, confessions or even a motive. So the statistics became crucial.

The chance that De Berk’s presence at so many deaths and resuscitations was just coincidence was first calculated by the director of the Juliana Children’s Hospital to be 1 in 7 billion and later by psychology of law professor Henk Elffers to be 1 in 342 million. This number is problematic for several technical reasons that have been explained in scientific journals and books, and was later revised by professional statisticians down to 1 in 9.

But, while the figure 1 in 342 million was splashed across newspapers and featured prominently in the minds of everybody involved in the case, its interpretation was another major difficulty. What does this number actually mean?

The common misconception was that this was the chance of De Berk being innocent, transforming the chance of her being present when a patient died to the chance that she caused the patient to die. For example, there is a very low chance of winning the lottery, but that is not the same as saying that there is a very low chance that the winner played fairly. Making such a mistake is not uncommon in court cases—it is known as the prosecutor’s fallacy—and in 1999 it was responsible for wrongly convicting Sally Clark of the murder of her two young sons in the UK. Clark’s story is especially tragic: after her acquittal in 2003 she developed an alcohol addiction and died of alcohol poisoning in 2007.

In 2004 the De Berk case reached the Court of Appeals where the statistical analysis was removed due to its unreliability but the original verdict was upheld. The court stated that a “statistical probability calculation did not play any role” in its conviction and it had instead based its decision solely on medical evidence. Officially, the role of statistics (and statistical mistakes) in the De Berk case ends here. Yet statistics continued to influence things.

Professor Richard Gill admits that while there was no explicit calculation in the report, the court’s arguments were “based on lawyers’ and doctors’ amateur statistical reasoning.” He cites an example of one particular death where De Berk was accused. The only medical expert to consider it unnatural did so because De Berk was present and she had been present at so many other deaths. Otherwise the expert would have considered the death to have been natural.

Lessons from Lucia
Lucia de BerkSo, three years on, where do things stand today?

Predictably, due to the De Berk case and other recent miscarriages of justice in the Netherlands, calls were made for an examination of the legal procedure. As a result, the Expert Witness in Criminal Cases Act was introduced in 2010 which contains a national public register of expert witnesses for use in legal cases. There is more communication between statisticians and police, as well as more caution in the use of statistical reasoning in courts.

The modifications, however, have not reached the very top of the legal chain. The Dutch Supreme Court still requires a “novum”—a new fact that would have impacted the court’s original decision—to be presented before it will reopen any previously closed cases. The definition of a novum is restrictive: a new opinion or argument based on old evidence does not qualify. This is to avoid undermining the authority of the judiciary system.

This does not satisfy Stijn Franken, professor of criminal law and criminal procedure and De Berk’s former defence lawyer. In a lecture given to law graduates, he states that “easier access to the revision system is to be preferred above the rather obligatory communication on legal certainty and confidence in the judiciary.”

To increase the chance of reopening closed cases, many have called for an independent body to be set up that will be responsible for reviewing closed cases and deciding whether to reopen them. Such a system exists in the UK, where it is called the Criminal Cases Review Committee. This suggestion was rejected with the explanation that it would lead to more bureaucracy.

Gill still believes that another case similar to De Berk’s could happen again. Geert-Jan Knoops, a defence lawyer involved in another overturned Dutch murder sentence, says “the number of miscarriages of justice in the Netherlands may be far higher than we have been inclined to think.” There is little hope of finding out if the cases can never be reopened.

The Minister of Justice apologised in person to Lucia de Berk after her acquittal. She received an undisclosed sum ascompensation from the Ministry of Justice.

Swept under the rug
justiceWhile the legal system has taken steps to learn from the De Berk case, others have not shouldered any responsibility. “In my opinion the medical world is largely to blame,” Gill says. “No one in the medical world talks about the Lucia case, no one asks what lessons could be learnt.”

This attitude is reflected in the reluctance to make public the final medical reports used by the court. A report by Jan Meulenbelt, professor of child toxicology, examines three of the most important deaths in the De Berk case. In it, Meulenbelt not only rules out foul play by anyone, including De Berk, but he also finds multiple mistakes made by medical specialists, including communication failures, misdiagnoses and errors in medication doses.

Could the Juliana Children’s Hospital have known this when they initially pressed charges against De Berk in 2001? Gill suspects that “the police and the courts right from the start were misled by a lot of untruths in the information they obtained from the hospital and its doctors, and never seemed to doubt them later.”

In the year leading up to De Berk’s arrest in 2001, the hospital has been accused of compiling a document of “suspicious incidents” in which she was involved. Yet at the same time an independent investigation was conducted by the Health Care Inspectorate, which found nothing suspicious. This investigation never surfaced in court, nor did the errors that Meulenbelt’s report uncovered.

In an open letter to Haga Hospital (of which the Juliana Children’s Hospital is now a part) Gill suggested a thorough investigation of the hospital’s actions in the year before De Berk’s arrest. This request has been ignored.

Another peculiarity is the inclusion of only certain parts of the available data from the hospital. A striking omission was the seven deaths in the same wards (under a different name) during the three years before De Berk began working there. During the three years that she did work there, there were only six deaths. Missing data was one of the technical mistakes that led to the incorrect number of 1 in 342 million and is known in statistics as selection bias.

The Dutch statistical community has explained why this is wrong. The Dutch judicial community has accepted that this is wrong. Why has the Dutch medical community—where the faulty data originated—said nothing?

“What is behind all this, is, I believe, the medical culture of denial of ever making errors,” Gill says. There are an estimated 2,000 deaths a year in the Netherlands as a result of avoidable medical errors. “The vast majority of them are known to the doctors but denied by hospitals. This means that everyone tells one another half-truths and this is a culture which allows individual half-truths to be combined into a collective lie very easily, without anyone being personally responsible.”

The hospital has never apologised to Lucia de Berk. After her acquittal, it released a statement saying it was “responsible for good patient care in a safe climate.”

References:
T. Derksen, “Lucia de B. Reconstructie van een gerechtelijke dwaling” Veen Magazines BV. (2006)
Ronald Meester, Marieke Collins, Richard Gill, & Michiel van Lambalgen (2006). On the (ab)use of statistics in the legal case against the nurse Lucia
de B arXiv arXiv: math/0607340v1

M. Zegers, M.C. de Bruijne, C. Wagner, L.H. Hoonhout, R. Waaijman, M. Smits, F.A. Hout, L. Zwaan, I. Christiaans-Dingelhoff, D.R. Timmermans, P.P Groenewegen, G. van der Wa (2009). Adverse events and potentially preventable deaths in Dutch hospitals: results of a retrospective patient record review study Qual Saf Health Care DOI: 10.1136/qshc.2007.025924
S. Franken, “Finding the Truth in Dutch Courtrooms – How Does One Deal with Miscarriages of Justice,” Utrecht L. Rev. 218 (2008)
B. Keulen, N. Kwakman, “Experts and expert registers in five European countries,” contribution to the Annual Legal Research Network Conference (2010)

Photography: ANP

This article originally appeared in the tablet issue Dangerous Minds | The Criminal Issue

statistical illiteracy, bad statistics, uk miscarriages of justice, worst miscarriage of justice, prosecutors fallacy

57 Flares Twitter 38 Facebook 15 Reddit 1 StumbleUpon 1 LinkedIn 1 Google+ 1 57 Flares ×
(1 votes, average: 5.00 out of 5)

  • Pingback: Imprisoned Because of Bad Statistics | Dr. Goul...()

  • Pingback: Somewhere else, part xxxxx | Freakonometrics()

  • Flippy

    Just because a doctor or nurse has patients that die, that doesn’t mean that the doctor or nurse is terribly incompetent. In fact, it can the exact opposite case, that the doctor or nurse is highly competent. This may be because people with the most serious conditions go to see the most reputable doctors and nurses. Not so competent ones get the patients with mild conditions or they may send the hardest cases to the competent doctors. Thus, the competent doctors may end up with the hardest cases, the ones that die with the most frequency, while the not-so-competent doctors end up with the easiest cases, which have the least likely of dying.