Halloween Science

This is a detailed critique of Trick or Treatment by Edward Ernst and Simon Singh. It took some 6 months to write, but it fully exposes the contradictions, inaccuracies and lack of a scientific approach in their arguments against alternative medicine. The analysis can be downloaded by clicking on the image of the title page.

Simon Singh and Edzard Ernst claim that

Although there are already plenty of books that claim to tell you the truth about alternative medicine, we are confident that ours offers an unparalleled level of rigour, authority and independence. We are both trained scientists, so we will examine the various alternative therapies in a scrupulous manner.” (p.3).

Halloween Science assesses the validity of this claim, both in general and specifically in respect of homeopathy, by analysing the authors’ own arguments and evidence for accuracy, consistency and reliability. Where information is lacking in Trick or Treatment?, the critique incorporates evidence from other primary sources (where possible) or reputable secondary sources. Some conclusions reached on the basis of Ernst and Singh’s own statements are also supported by reference to other sources. All sources are referenced.

There are nineteen major faults in the case presented by Trick or Treatment?. Faults 1-3 relate to their treatment of evidence,

Faults 4-7 relate to their treatment of science in general,

Faults 8-14 relate to their treatment of definitions, and

Faults 15-19 relate to use of analytical tools.

The authors frequently rely on figures, trials, events, quotations, statements, opinions and explanations which are unsupported by reference to sources.
This evidence is frequently misleading as a result of being presented out of context.
The authors use different criteria when assessing the validity of evidence, depending on whether the evidence supports their views or not.
The authors commit the common fallacy of confusing absence of proof with proof of absence.
The importance of theory is minimized or even ignored, when discussing both science in general and individual alternative therapies.
The authors assume that orthodox medicine is scientific, but offer no justification for this position.
There is evidence that the authors do not understand the principles and practice of orthodox medicine.
Alternative medicine is defined in four different ways in the course of the book.
Other significant terms, such as ‘science’, ‘disease’, ‘cure’, ‘effectiveness’ and ‘orthodox medicine’ are undefined.
This allows arguments to be built on vague preconceptions rather than on clearly defined principles.
The differences between orthodox medical and alternative medical definitions is not taken into account, despite their impact on the design of trials.
The authors fail to present the ideas of evidence-based medicine accurately.
The authors fail to present the nature and development of homeopathy accurately, raising doubts about their presentation of the other therapies.
They also call into question the principles of orthodox drug therapy, despite the fact that the tests used by this therapy underpin much of their argument.
The authors fail to prove that their main tool, the randomised controlled trial (RCT), is valid for testing curative interventions, while presenting evidence that there are serious problems with using it for this purpose.
They show that a tool derived from these trials, the meta-analysis, is prone to lack of objectivity, yet they rely on this for some of their conclusions.
Their conclusions are also dependent on the concept of the placebo effect, but they make it clear that this effect has no scientific basis and is so unpredictable as to have questionable scientific validity in this context.
They acknowledge the importance of individuality in the curative process, but deny its significance for the design of analytical tools.
They fail to take into account the need for analysis of evidence from clinical practice.

In conclusion, Ernst and Singh fail to provide a secure theoretical or evidential base for their argument, and have used analytical tools inadequate (in this context) for achieving objective and reliable conclusions. The result of these weaknesses is that their argument relies heavily on preconceptions and variable definitions and opinion, a problem exacerbated by a tendency to confirmation bias on the authors’ part. As a result, Trick or Treatment? has no validity as a scientific examination of alternative medicine.

The title page of Halloween Science by William Alderson