Professor Richie Poulton
"Generally people assume that when you get a failure to replicate, it is all bad news. What we see here is a classic case of 'Don't throw the baby out with the bathwater'."
Failing to replicate a research finding is not all bad and could, in fact, shed new light on the subject being investigated.
One such recent failure may lead to better understanding of mechanisms that underlie the interplay of nature and nurture, says Professor Richie Poulton, director of the Dunedin Multidisciplinary Health and Development Research Unit.
The unit, which has followed around a thousand people from birth for 37 years, sought to test the findings from a recent US-based study on genes and depression.
That study showed how some adults were protected against developing depression after childhood trauma if they had a particular version of the corticotropin-releasing hormone receptor 1 gene (CRHR1).
The Dunedin researchers collaborated with colleagues who run the British-based E-Risk cohort study to test the original US finding in two high quality data sets. What they found was both frustrating and fascinating, says Poulton.
The British cohort used the same (retrospective) measures of childhood trauma that had been used in the US study, and replicated the original findings. In contrast, the Dunedin study did not, despite using supposedly gold-standard prospective measures of childhood trauma. The researchers then tried to work out what might have influenced their conflicting results. They focused on the different methodologies used to glean early childhood experiences and came up with new ideas about how people's memories work.
It has long been recognised that negative and ruminative memories are important factors in developing depression. The researchers now suggest that variations in the CRHR1 gene may influence how these aversive memories are consolidated and this might help explain why some people develop adult depression after being maltreated as a child whereas others do not. "The value of what we found here goes beyond the topic of depression," says Poulton. "Generally people assume that when you get a failure to replicate, it is all bad news. What we see here is a classic case of 'Don't throw the baby out with the bathwater'.
"Failure to replicate may be explained by, among other things, using different methods to measure exposure to environmental risk – this is a key element that many people tend to gloss over. "Apparent failures, far from gut-wrenching, might be a gift because they make you think more deeply about what might be going on and, perhaps, in different directions."
Poulton says the replication issue is currently a major talking point in the world of behavioural genetics and bears directly on the value of trying to model gene-environment interactions. For example, a 2003 gene-environment paper published by the unit is now being questioned by American researchers, igniting a worldwide controversy about studying the interplay of nature and nurture.
"Our latest paper investigating depression allows us to dip our toes into this raging international debate in a way that says 'Hold on, the debate may be more nuanced than most people appreciate'."
The paper combined the strengths of two different studies on two different sides of the globe to begin to address these questions and pushed the boundaries a bit, says Poulton.
"Here we have a wonderful example of how different methods can throw up different results – and how important it is to compare apples with apples.
"By looking carefully at why a failure to replicate occurs you may get a hitherto unrecognised hint of what mechanism may be underlying the basic association you are studying."
Funding
- Health Research Council of New Zealand
- UK Medical Research Council
- US National Institute of Mental Health
- William T Grant Foundation
- National Alliance for Research on Schizophrenia and Depression (NARSAD)
- Young Investigator Award