Even before the addition of total fabrications (death panels) and flat-out inanities ("Keep your government hands off my Medicare") dragged the health-care reform debate through the looking-glass, there's good reason to suspect that the reliability of the data our policymakers are using may be seriously flawed.


The Congressional Budget Office, often cited as an impartial source of data on spending, has "underestimated the amount of savings and overestimated the costs" of major health-care initiatives, according to a researcher from the National Opinion Research Center of the University of Chicago. The study analyzed three major Medicare reform initiatives over the past 30 years and cites flaws in the CBO's methodology that led to substantial miscalculations. (commonwealth fund.org)


A
pair of studies in August's Health Affairs (healthaffairs.org) paint an even more grim picture of flawed research design and questionable conclusions repeated in the media. One study looked at research and initiatives in areas such as pay-for-performance, health-information technology and drug formularies and assessed the rigor of the study de-signs. Its verdicts range from widespread "weak designs" to "almost no evidence" for studies that were widely reported in both "the highest-impact peer-reviewed journals" and the consumer media. The methods of many published studies are sufficiently flawed that " ... it might not be possible for health-care professionals, article peer reviewers, editors, journalists, the public, or policymakers to know what studies to believe or disbelieve."


In spite of (or is that because of?) the screaming matches passing for town hall meetings across the country, as a spokesperson for Pollster.com recently said, "Most outside Washington have no idea what the overhaul will look like, what it will cost and how it could affect them personally." Apparently, those in Washington may be in the same straits.