A study by an Australian research group published last September in Molecular Psychiatry claimed to have found a genetic screen capable of predicting autism with 71.7% accuracy. The screen was made up of 237 single nucleotide polymorphisms (SNPs), or DNA variations of only one base pair, that were found to be connected to autism. Of these, the sequences of only the most influential 30 were released. A test like this could be used to evaluate risk for autism in a wide variety of populations, which would be a huge step forward in both treatment and understanding of autism spectrum disorders.
Unfortunately, it looks as if this step may have been taken a bit prematurely. Another article, by a group out of Massachusetts General Hospital and published in this month’s Molecular Psychiatry, has tried to replicate the results of the earlier study, using a similar set-up and applying the genetic screen released by the Australians to a much larger sample. This time though, the screen failed to predict autism.
The new study approached the controversial screen, which had already met with some criticism, on three fronts. First, they performed a meta-analysis of the 30 SNPs released, combining a number of different statistical tests to evaluate each polymorphism’s connection to autism in the 5,417 samples tested. They found no connection between autism diagnosis and any of the supposed 30 most powerful predictors. The authors of the original study declined to release the sequences for all 237 polymorphisms, so only those initially published could be considered.
Second, the group looked at whether or not those 30 SNPs as a group could be used as a genetic classifier, a means of testing to genetically classify someone as having an autism spectrum disorder or not having one. Even as a whole, the genetic screen proposed was unable to accurately predict autism within the data used.
The group’s final analysis looked at those cellular pathways the original study proposed as being affected by the SNPs they were studying. These pathways came from a variety of processes, ranging from calcium signaling to purine metabolism and long-term depression. Yet again though, the new study found no significant connection between the pathways proposed and autism spectrum disorders. These pathways were what the original study used to identify those SNPs that could predict autism, so if no connection ever really existed, the whole study runs the risk of being inherently flawed.
So now the inevitable question is what went wrong with that first study? How did they find a whole series of genes that appeared to be connected to autism without any real connection existing? Well the answer may lie in their sampling techniques. The group made a few errors they really shouldn’t have. For one thing, they initially applied their genetic screen to the same population from which it was derived. To get a solid and reliable result, you really need to use a different population, as Leonid Kruglyak pointed out to The Scientist. At the same time, the group’s autism cases and controls came from two ethnically different backgrounds, meaning that the differences found could be based in differing ancestry rather than the disorder. This particular problem had already been pointed out in a letter to the editor in Molecular Psychiatry this past April.
Now it must be said that, while this new study certainly raises a number of very good questions about the strength of the Australian group’s results, the Massachusetts study has its own problems. In particular, they were working with a bit of a handicap, since they were not given the full list of 237 SNPs. While they did have the thirty most powerful predictors, that still only accounts for 58% of the screen’s power. While that is fairly high, it still takes a 42% hit to the screen’s ability to identify autism. The Australian group is already working on getting its response published, a response which will contain a full listing of the 237 SNPs.
The results that the Australian study initially published were very promising, and seeing them called into question on such firm grounds is unfortunate. But as the Massachusetts study’s authors note in their paper’s closing, there are a large number of highly successful and hopeful results coming out every year surrounding autism and other psychiatric disorders. Studies like this really need to be tested as rigorously as possible, and refuted when necessary, to protect the people that this type of research is trying to help.
The Scientist article: