printAlmost every bit of research cited in support of some or another ridiculous claim in the popular press seems abysmally designed. At least to this former student of research methodology.
Samples are too minuscule to claim generalizeability of findings beyond the sample, to the broader, targeted population. Likewise, you just know sample selection is poor, too. Variables are not often operationalized in an intelligent way. The actual hypothesis frequently sounds wacky. On and on.
It transpires that the same methodological flaws that “bedevil most social sciences, and some hard sciences, too,” have infected the dismal scientists of microeconomics.
A recent examination in the Economic Journal, of almost 7,000 empirical economics studies, found that in half of the areas of research, nearly 90% of those studies were underpowered, ie, that they used samples too small to judge whether a particular effect was really there. Of the studies that avoided this pitfall, 80% were found to have exaggerated the reported results. Another study, published in Science, which attempted to replicate 18 economics experiments, failed for seven of them.