Practices | Techniques | Description |
---|---|---|
Confounding Referencing | • The misleading use of references which either overstates or gives an entirely false impression of support for a claim or obstructs evidence appraisal. | |
Cryptic references | • An opaque reference that provides insufficient information to easily locate the original source and which serves to obstruct evidence appraisal. | |
Faux sources / False authority | • A faux source involves providing an incorrect source for key data. The concept overlaps with an appeal to a false authority, where an alleged authority is used as evidence to support a claim, which, in fact, is not an authority on the facts relevant to the claim. | |
Out-of-place citations | • References that give a false impression of support for a proposition as a result of being misplaced in the text. These take various forms and can be used to validate illicit generalisations or simply provide a faux source for a key proposition. | |
Vapid out-of-place citations | • A hybrid confounding reference (combining an out-of-place citation and a faux source) which contains relatively useless contextual information that fails to support, and has no direct relevance, to the claim in the text. | |
Source laundering | • Provision of a relatively independent source which obscures the use of industry data as the underlying support for the proposition. | |
..... | ..... | |
Inaccessible source | • The use of a source that is not publicly available. | |
Misleading Summaries | • Inaccurate reporting of objectives, findings, and conclusions of sources. | |
Absence of evidence as evidence of absence | • A logical fallacy aimed at representing a relationship that has not been satisfactorily explored as evidence that no relationship exists (usually used in combination with other techniques, such as omission of qualifying information). | |
False attribution of focus | • Misrepresentation of the focus of studies. | |
Omission of important qualifying information | • A specific variant of strategic ignorance characterised by precise but inaccurate reporting of study findings in which important qualifying information that significantly changes the implications of the findings is omitted. | |
Selective quotation | • Reporting extracts either out of context or by omitting qualifying information to give a misleading impression of either the study quoted or the research upon which it is based. | |
Simple misstatement of key/study findings | • Erroneously and unambiguously claiming that a study has produced a specific finding. | |
‘The Tweezers Method’ | • The practice of picking phrases out of context from peer-reviewed studies with the effect of changing the emphasis and/or intended meaning of the original text. | |
Acalculiac rounding-up | • Rounding-up estimates without cause or explanation. | |
Double-counting | • Counting an economic impact (or part of an impact) more than once. | |
Illicit Generalisation | • A logical fallacy where the underlying evidence is insufficiently developed to support an inductive generalisation. | |
Evidential Landscaping | • Either promoting alternative evidence (a parallel evidence base) to shift the evidential basis upon which the policy is being discussed and evaluated or purposefully excluding relevant evidence | |
Data dredging (misuse of raw data) | • Presenting and/or analysing data to depict relationships or trends that either misrepresent actual relationships or obscure other contradictory relationships and/or trends in the data. | |
Unmodelled data (misuse of raw data) | • Homespun trend analysis summarising patterns across time that ignores key confounding variables or pre-existing/underlying trends. In this latter sense, unmodelled data may involve a faux counterfactual, where the impact of an intervention is not appropriately explored by comparing the world in which the intervention occurred with the world in which it did not. | |
Observational Selection/Cherry-Picking | • The practice of highlighting individual studies or data to support a pre-determined conclusion, whilst ignoring contradictory (and typically stronger) evidence. | |
The ‘Hens’ teeth’ technique | • An egregious form of cherry-picking that involves foregrounding obscure, outlying studies. | |
Passé Source | • Cherry-picking an older source to support an assumption, which although fairly reflecting the state of scientific knowledge when published has since been superseded by developments in the evidence-base. | |
Strategic ignorance | • The technique of ignoring findings and evidence-backed observations in cited sources that contradict unsupported or weakly supported claims. | |
Syncopated Estimation | • Missing or failing to fully articulate key steps in economic modelling (including, but not limited to, the failure to: provide a range of estimates to reflect uncertainties in assumptions; fairly review the literature relevant to specifying assumptions; provide a clear and comprehensive assessment of assumptions). | |
Black-box Computation (information asymmetries) | • Opaque, unverifiable steps in economic modelling. | |
Inaccessible Data (information asymmetries) | • The reliance on privately held data in economic assessments. |