One of the best things about blogging is that I don’t feel obligated to cover a topic completely in one post because I know I can always write another one or revisit the topic later. It also allows me to look at what I like to call “variations on a theme” of various kinds of quackery (or anything else, for that matter). View this as a post looking at one such variation on a theme.
The theme this time is the tendency of antivaccine activists to demonstrate their utter cluelessness when it comes to designing clinical studies. This cluelessness virtually always manifests itself in the frequent call by antivaccine quacks for what they seem to view as the Holy Grail of evidence to “prove” that vaccines cause autism, other neurodevelopmental disorders, sudden infant death syndrome, asthma, diabetes, autoimmune diseases, and just about any other chronic condition you can think of, namely the so-called “vaxed/unvaxed” study. The reason they seek such a study is because it is an article of faith among the antivaccine movement that vaccines are degrading the health of our children, making them the “sickest generation ever” and causing all the aforementioned conditions. As I mentioned the last time I discussed this, years ago (and ) antivaccinations would call for an actual randomized, double-blind, placebo-controlled “vaxed versus unvaxed” clinical trial. Of course, as Imention every time this topic comes up, antivaccine activists appear to be completely ignorant of the concept of , which renders such a trial completely unethical and therefore undoable, at least in developed countries with actual laws protecting the welfare of clinical trial subjects, particularly children. Remember: Clinical equipoise means that there must be genuine uncertainty over which group in a clinical trial is receiving the superior treatment, and there is no doubt that leaving one group of children unvaccinated leaves them susceptible to vaccine-preventable diseases and thus does them harm. That’s why doing such a trial would be completely unethical and we have to rely on the accumulated knowledge from epidemiological studies to determine whether vaccines are causing adverse outcomes.
Some antivaccine activists have realized that there’s no way a randomized clinical “vaxed vs. unvaxed” trial will ever be done. Grudgingly (and I do mean grudgingly), they accept that virtually every other physician does; so they try to find evidence of vaccine harm that they so desperately seek in other ways. Most recently, it was a that it was even more risibly pathetic than previous efforts, such as J.B. Handley’s and a by a German homeopath named Andreas Bachmair for VaccineInjury.info, both of which were presented as showing that unvaccinated children are healthier than vaccinated children and both of which showed nothing of the sort. This time around, it’s a “study” being promoted by the surprisingly honestly named Vaccine Resistance Movement (VRM) called the . Not surprisingly, this new antivaccine “study” follows the long, dishonorable tradition of horribly designed and outright incompetent “studies” done by antivaccinationists since time immemorial (at least to me).
I’ll show you what I mean.
Whenever a clinical trial is designed there are several important things that need to be prespecified. To put it simply, you need:
Now, to be fair in the case of pilot observational studies, the need for a specific hypothesis can be somewhat less critical, as it can be quite possible to do such studies as a “hypothesis-generating” study. Even so, you have to have some. So does the VRM study specify these key variables? What do you think?
Let’s take a look at the VRM announcement of its study on its :
The purpose of this worldwide study is to determine the incidence/rate of autism amongst those unvaccinated children & adults surveyed. We are also concurrently tracking the incidence/rate of autism amongst those vaccinated children & adults surveyed; cross-referencing dietary factors, pre-existing medical conditions/allergies, family health/vaccine history (multiple generations), breast/bottle feeding, We hope to gain new insights into the causality & manifestations of autism with an exhaustive, scientific approach.
“Scientific approach.” You keep using that term. I do not think it means what you think it means.
Elsewhere, :
It is our goal, through this study, to determine an accurate percentage of those unvaccinated autistic children & adults vs. those unvaccinated children & adults who have not become autistic. Essentially what we are trying to identify is the healthy trend in unvaccinated children & adults.
OK, fair enough—on first glance. The devil, as always, is in the details. That’s something that’s true of any sort of science. How does VRM plan to measure these percentages? Well, that’s where the big problem is. As was the case with all the previous studies I’ve discussed, apparently VRM thinks that an Internet survey is a valid way to collect data that can be used to up with accurate estimates of the prevalence of autism:
Just to clarify this study is open to EVERYONE. The absence of autism in your family/household or the fact you haven’t vaccinated your children does not preclude you from taking part in the study. The control parameters are wide open this time. All the data we receive will be pertinent in determining the paths that lead to autism. VRM will be continuing to collect your invaluable data for as long as possible. That being said we still encourage everyone to enroll in the study while it’s openly available.
And:
In order to guarantee accuracy of the data acquired study participants must allow for access to family medical records (*Note: only in cases where autism has occurred). This disclosure will be entirely confidential in nature and at the discretion of the study organizers. Any shared information will be used strictly to confirm an autism diagnosis. No data beyond that which the family agrees to disclose shall be included in the study results. All participants in this study will be designated a number to assure complete anonymity. Any sensitive information including family names, personal references, e-mail addresses etc will be inaccessible to the public. Only study administrators will be privy to access. Contact will remain exclusively confidential between participants & VRM. At any time a participant may withdraw from the study group; whereupon their data will be returned to them and/or destroyed.
Out of curiosity, I took a look at the . As you can see if you take a look yourself, this “study” (such as it is), it’s yet another example of bad experimental design coupled with bad survey design. Take a look:
Category of Participant (choose one selection):
In clinical trials and observational studies, it is absolutely critical to define one’s terms. What does VRM mean by “partial schedule of shots”? It’s not defined. Then, later on, the survey lists the individual vaccines and when the person received them. Of course, anyone should be able to see another problem here. This survey includes “ALL AGES.” The vaccine schedule has changed significantly over the years, as new vaccines were added and the recommended timing was changed over the years. So what it means to receive a “partial schedule of shots” will be very different depending upon how hold the subject is.
That’s not all (of course). Later in the survey, the subjects are asked for “pre-existing medical conditions/genetic predisposition.” The list includes a veritable laundry list of conditions that antivaccinationists consider to be “predisposing” to autism, including mercury amalgams, “medicated childbirth” (e.g., induced labor), antibiotics, and more. Elsewhere, under dietary history, the subjects are asked, for example, if they:
There are more—oh, so many more—examples listed. I simply chose some of the sillier ones. Let’s just put it this way. The survey is so vague as to be meaningless. After all, most people don’t even know what is and isn’t a “genetically modified food,” and there’s no evidence that GMOs are associated with autism.
As with all these antivaccine vaccinated versus unvaccinated”studies,” the key problem (besides the undefined hypothesis and endpoints) is that the subjects are self-selected. More than that, they’re self-selected from people who tend to be antivaccine and learn about such surveys through antivaccine websites and Facebook pages. Add to that the multiple comparisons, and the “researchers” (and I do use that term very loosely) are virtually guaranteed to find all sorts of spurious associations. They’ll be meaningless, of course, because the methodology is so poor, so utterly lacking in anything resembling valid methods. None of this stops antivaccine groups, though. They forge boldly ahead into the depths of pseudoscience in search of “evidence,” no matter how risibly bad, that vaccines cause autism.
Seeing such perversions of science, I sometimes wonder if scientists should just do a large epidemiological study of vaccinated versus unvaccinated children, if only to shut antivaccinationists up and shut down studies like this, even though such a study would be . (Also, there are no compelling preliminary data to justify such a study.) Then I remember how antivaccine activists react to negative studies. They ignore them or attack them as fatally flawed, usually due to “bias” or—of course!—the influence of big pharma. Scientifically pandering to antivaccine loons achieves exactly nothing. Unfortunately, that means we’re likely to continue to see such epically silly self-selected Internet polls masquerading as research.
And don’t even get me started on where the Institutional Review Board approval is for this VRM “study,” given that it says that participants are giving the VRM permission to examine their medical records.