Loading presentation...

Present Remotely

Send the link below via email or IM

Copy

Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.

DeleteCancel

Make your likes visible on Facebook?

Connect your Facebook account to Prezi and let your likes appear on your timeline.
You can change this under Settings & Account at any time.

No, thanks

Controlling the false discovery rate in behavior genetics research

No description
by

Sara Boxhoorn

on 25 September 2012

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of Controlling the false discovery rate in behavior genetics research

Controlling the False Discovery Rate in Behavioural Genetics.
Benjamini et al. (2001) Controlling the False Discovery Rate in Behavior Genetics Research Outline: The Problem of Multiple Comparisons 96% significance rate in novel studies compared to only 27% significance rate in replication studies (Duncan & Keller, 2011) Epigenetics:

Epigenetic patterns shape the diversity of gene expression programs

"Different environmental exposures, including
early parental care, could impact epigenetic patterns,
with important implications for mental health in humans."
McGowan & Szyf (2010) Genes May Determine
How We Vote (Yahoo news) Lawyer suggests abortion if a test
could prove fetus has "gay gene'
(San Francisco Chronicle)


Scientific Evidence?
"A 1993 study, for example, found that gay brothers tended to share the same region of the X-chromosome, called Xq28, suggesting that there may be a gene in that region that affects sexual orientation." small effects per gene How Big is the Problem ? Alternative solutions... Epigenetics:

Epigenetic patterns shape the diversity of gene expression programs

"Different environmental exposures, including
early parental care, could impact epigenetic patterns,
with important implications for mental health in humans."
McGowan & Szyf (2010) small effects per gene What Not to Do !
Allow the probability of making even one false discovery to be as high as half. Then follow
the original study with a more limited confirmation study.

The following confirmation study fails the replicate the findings of the initial liberal study

In some cases the follow up is not performed Play ignorant....
Argue that multiplicity is not a problem!

Massive false discovery rates!
The probability to discover a real strain difference (the power) is greatly reduced when screening a large family of endpoints The bigger picture
Why is multiplicity is a core problem
What not to do
Solutions - BH and BL Procedures
More powerful procedures
Conclusions Benjamini et al (2001) Sara Boxhoorn and Rachel King The Bigger Picture Lingering
Time Early
Activity Lingering
Speed Center
Rest Movement
Speed Diversity Why is it a Core issue ? Epigenetics:

Epigenetic patterns shape the diversity of gene expression programs

"Different environmental exposures, including
early parental care, could impact epigenetic patterns,
with important implications for mental health in humans."
McGowan & Szyf (2010) small effects per gene Effect different procedures More powerful procedures Post hoc checking Epigenetics:

Epigenetic patterns shape the diversity of gene expression programs

"Different environmental exposures, including
early parental care, could impact epigenetic patterns,
with important implications for mental health in humans."
McGowan & Szyf (2010) small effects per gene The Use of Confidence Statements In most analysis a decision is made about statistical significance by looking at whether zero is in the confidence interval Therefore the same problem of multiple tests! In the Media ....... New ecologically relevant parameters are tested to study open field behaviour in different mouse strains.
(Drai, Benjamini, Elmer & Golani, 2001) Resampling procedures: specifically designed to cope with correlated test statistics

Bayesian methods (often based on BH-procedures): e.g. p (FDR | data) Be aware: choice of variables influences FDR procedures!

Data look suspicious? Perform sensitivity analysis: recalculate FDR constant by only considering relevant variables Summary

The media can misrepresent scientific studies and report many false positives before replication has proven them wrong. Discussion Question:
Williams, Jones & Tukey (1999) that differences passing the Bonferonni correction are "highly significant" whereas those passing the FDR are simply "significant"
Would this be a valid interpretation of the FDR criterion? False Discovery Rate procedures provide a balance between the concern for too many false discoveries and missing discoveries due to low power. Summary Inflation of the Error Rate ? ? ? ? ? Genome wide studies fail to replicate reported associations in candidate research Does the behaviour of different mouse strains differ
across lab environments ?
(Crabbe, Wahlsten & Dudek, 1999)

56 hypotheses
"Despite our efforts to equate laboratory
environments, significant and, in some cases, large effects of site were found for nearly all variables." Compromise Why the Bonferroni
is not the
optimal solution........ WHY? Example 1 Table 1. blue, P < 0.00001; purple, P < 0.001; gold, P < 0.01; dashes with no shading,P > 0.01. Example 2 Examples to illustrate... - BH Procedure - BL Procedure How does it work? p values are sorted and ranked for every p value one calculates a constant
reflects the number of tests and size of the p value relative to the other p values 0.05 * i/m significance level = first p value < constant BH procedure requires either independent or positively dependent test statistics BL procedure requires no assumption at all Multiplicity is key to explaining the false discovery rate because of the large number of comparisons per study. The traditional approach (Bonferroni) is too conservative. Leading some researchers to ignore the problem of multiplicity. The BH procedure controls FDR when test statistics are independent. The BL procedure requires no such assumption. Other approaches are even more powerful than BL.
Full transcript