Tag: policy evaluation

Citational chains

“Within the academy, the word theory has a lot of capital. I have always been interested in how the word theory itself is distributed; how some materials are understood as theory and not others. […] As a student of theory, I learned that theory is used to refer to a rather narrow body of work. Some work becomes theory because it refers to other work that is known as theory. A citational chain is created around theory: you become a theorist by citing other theorists that cite other theorists.” (Ahmed, 2017, p. 8)

Reference

Ahmed, S. (2017). Living a feminist life. Duke University Press.

Quant methods aren’t positivist.

A paragraph from the archive (Fugard, 2020, p. 525–6):

“Theories in social science are underdetermined by evidence. They are produced by people who have biases and differing values, so two theorists may interpret the same evidence in different ways. Quantitative research is no exception since researchers have to choose what to measure and how to model it. For non-experimental data, many causal models are mathematically equivalent, so assumptions beyond the data are required (Pearl, 1995). Although there are criteria for preferring one theory over another, such as accuracy, simplicity, and scope, applying them is not always straightforward, even in natural sciences (Kuhn, 1977).”

References

Fugard, A. (2020). Should trans people be postmodernist in the streets but positivist in the spreadsheets? A reply to Sullivan. International Journal of Social Research Methodology, 23, 525–531.

Kuhn, T. S. (1977). Objectivity, value judgment, and theory choice. In The essential tension: Selected studies in scientific tradition and change (pp. 320–339). The University of Chicago Press.

Pearl, J. (1995). Causal diagrams for empirical research. Biometrika, 82(4), 669–688.

The problem with politics and evaluation

“The problem is that politics and evaluation are both intrinsically connected and intrinsically separated. We need independence if we’re to be competent, objective, and credible. However, independence can’t always be preserved without a fight; this doesn’t make us a lot of friends; and yet we need friends, partners, and teamwork to be effective in a political universe.”
—Eleanor Chelimsky (2008, p. 414). A Clash of Cultures: Improving the “Fit” Between Evaluative Independence and the Political Requirements of a Democratic Society. American Journal of Evaluation, 29(4), 400–415.

Chocolate and diabetes

Increased consumption of dark, but not milk, chocolate was “associated with” lower risk of type 2 diabetes (Lin, et al., 2024). Refresh your teaching examples for causal inference using non-experimental data 🤓

Liu, B., Zong, G., Zhu, L., Hu, Y., Manson, J. E., Wang, M., Rimm, E. B., Hu, F. B., & Sun, Q. (2024). Chocolate intake and risk of type 2 diabetes: prospective cohort studies. BMJ, e078386.