E3 PreliminaryWeak / uncertainPEM unclearMethods-PaperPeer-reviewedMachine draft
An experimental study of determinants of the extent of disagreement within clinical guideline development groups.
Hutchings, A, Raine, R, Sanderson, C et al. · Quality & safety in health care · 2005 · DOI
Quick Summary
This study looked at how doctors and mental health professionals reach agreement on which treatments are appropriate for different conditions, including chronic fatigue syndrome (ME/CFS). Researchers tested whether different ways of running these decision-making meetings affected how much the professionals disagreed with each other. They found that the way meetings were structured didn't really change how much disagreement there was—doctors were more likely to agree on some treatments (like cognitive behavioral therapy) than others.
Why It Matters
Understanding how clinical guidelines for ME/CFS are developed is important for patients, as these guidelines influence treatment recommendations and research priorities. This study reveals that the structured consensus process produces consistent results regardless of how it's organized, which suggests guideline recommendations for ME/CFS have stability and are not arbitrarily influenced by procedural choices.
Observed Findings
- Consensus was stronger for cognitive behavioral therapy and behavioral therapy than for brief psychodynamic therapy and antidepressants
- Disagreement levels were lower when discussing depressed patients and patients willing to try any treatment
- None of the three design features (literature review provision, group composition, or resource assumptions) significantly affected the degree of disagreement (all p>0.3)
- Disagreement patterns differed meaningfully between different treatments, indicating treatment type—not procedural format—influences professional consensus
Inferred Conclusions
- Formal consensus development methodology produces robust, stable results that are not dependent on specific implementation choices such as literature review provision or group composition
- Treatment modality and clinical presentation (patient willingness, depression status) are more important determinants of professional disagreement than procedural design features
Remaining Questions
- Why did disagreement differ between treatments—does this reflect genuine evidence quality differences or professional training/philosophy differences?
- How well does professional consensus (measured in this controlled setting) predict actual clinical appropriateness or patient outcomes for ME/CFS treatments?
- Would findings differ if patient advocates or people with ME/CFS were included in the consensus development groups?
What This Study Does Not Prove
This study does not prove that any particular treatment for ME/CFS is effective or appropriate—it only examines the process by which experts reach agreement. It also does not establish whether disagreement among professionals reflects genuine uncertainty about treatment value or differences in clinical experience. The study cannot determine whether consensus among professionals correlates with actual patient outcomes.
Tags
Method Flag:Weak Case Definition
Metadata
- DOI
- 10.1136/qshc.2004.013227
- PMID
- 16076786
- Review status
- Machine draft
- Evidence level
- Early hypothesis, preprint, editorial, or weak support
- Last updated
- 8 April 2026
About the PEM badge: “PEM required” means post-exertional malaise was an explicit required diagnostic criterion for participant inclusion in this study — not that PEM was studied, observed, or discussed. Studies using criteria that do not require PEM (e.g. Fukuda, Oxford) are tagged “PEM not required”. How the atlas works →
Spotted an error in this entry? Report it →