Dear evaluation community, I’ve set myself a challenge. This month, the next time someone asks what I do, I’ll say I’m a social policy evaluator – whether that’s a new neighbour, someone in the chillout space at FOLD, or a friend of a friend over dinner. Here’s how I got there.
I’ve been pondering again why I hesitate to call myself an evaluator and can think of at least two reasons. Firstly, it feels a bit like “auditor”. I imagine someone in suit and tie, poring over a company’s policies and procedures, checking that their accounts add up. Evaluation can be a bit of that, but it’s much more exciting! It’s one of the few corners of social science where we are not only licenced to draw causal inferences from complex evidence – it’s demanded of us. We can’t hide behind “correlation doesn’t imply causation”. We have to judge whether, on the balance of evidence (including prior research), a programme seems to work or make things worse. The programmes we evaluate often reach many thousands of people and have a huge impact on people’s lives.
The second issue with “evaluator” is that it’s ridiculously broad. Am I evaluating your dress sense or how you dance? Whether you’re sufficiently polite with waiters? Am I running psychometric assessments to help evaluate staff performance? We are all evaluators in some sense, however often we share the judgements.
So, I usually say I’m a social researcher and that I work in social policy. I explain something about what I do by example. However, for October, I’m trialling “evaluator”, because I conduct evaluations. It feels like I’ll need to add a word or two to that title to get going.
How about “policy evaluator”? Do people know what a policy is? I reached for the nearest book on policy, which happened to be Social Policy, by Hartley Dean (2019). Page 1, first sentence: “When I tell people I teach Social Policy, a fairly common response is ‘Oh! … [pause]. What’s that exactly?” He even adds “social” to the term and that doesn’t help; maybe policy is just as opaque as “evaluation”. Dean defines social policy rather broadly: “the study of human wellbeing” – that’s going to open too many cans of worms; I just want to get on with (or escape) the conversation.
Maybe the Green Book will help me navigate the opening? “A Policy is a statement of intent that is implemented through a procedure or a protocol and a deliberate system of principles to guide decisions and achieve rational outcomes,” it says (HM Treasury, 2022, p. 6).
Closer to what most of us generally do is down at the programme level. The Book says: “A Programme is an interrelated series of Sub-Programmes, Projects and related activities in pursuit of an organisation’s longer-term objectives. Programmes deliver outcomes through changes in services.”
That last bit seems key: changes in services. I’ll hold onto that as a follow-up answer and have a few examples ready to explain.
Okay, so how about “programme evaluator”. That’s definitely a thing – it appears in article and book titles (a random selection from my Zotero stash: Abadie & Cattaneo, 2018; Chen, 1996; Cook & Shadish, 1986; Donna M. Mertens & Wilson, 2019; Perloff et al., 1976). But do people know what a programme is? Tech bros and adjacent might think about the software kind of program. Others might think it’s about TV shows. Since this is a blog post and very few people will read this far, I’m going to sneak into CoPilot and ask it to simulate a survey of the UK adult population. It made up a crosstab for me with fictitious percentages by age group. For what it’s worth, TV shows and software/apps dominated as the most common interpretations of “programme”.
“Social” didn’t help with social policy – Hartley Dean tried and still got blank stares. Maybe adding it to programme evaluation or policy evaluation does help. So, finally I asked CoPilot to (pretend to) run an experiment comparing:
- evaluator,
- policy evaluator,
- programme evaluator,
- social policy evaluator, or
- social programme evaluator.
Each invented participant was “asked”, “What do you think someone with this job does?” The primary outcome was the percentage of respondents in each age band who correctly identified the role’s social policy/programme focus.
It was a close tie between social programme evaluator and social policy evaluator, with the latter winning by a few entirely fictitious percentage points in each age group. Based on the 21st century tradition of decision by digital hallucination, for the month of October my profession is going to be: social policy evaluator.
One objection to this title that may be raised (it was raised in my head) is that we don’t actually evaluate policies, if they are defined as statements of intent as per the Green Book. That’s too vague and far from something that, say, a quasi-experiment could study. However, programmes interact with policies, and findings from evaluations (should) inform policies. It may be helpful to signal that we are aware of and engage in the broader context, not only individual programmes independently of how they ended up being trialled. If a programme is shown to work, that has clear implications for social policy.
Thanks Julian King for pointing me to the Chris Lysy comic.
Edited to add (11 Oct 2025): Thanks also to Brad Astbury, who pointed me to an interesting article exploring different ways of explaining what evaluators do, by Sarah Mason (2023). This included an experiment, which randomly assigned people to one of seven different explanations. The key, Mason found, was to include examples. Here’s a template in the conclusions of the paper that illustrates what seems to work best (p. 564):
“You know how government agencies and not-for-profit organizations spend money on social programs like school lunch programs or substance abuse programs?
“Well, I gather and analyze data about those programs to find out if they do any good or not. The hope is that we’ll be able to provide data and feedback that will help decision makers improve the programs’ performance and their value to society.”
References
Abadie, A., & Cattaneo, M. D. (2018). Econometric Methods for Program Evaluation. Annual Review of Economics, 10, 465–503.
Chen, H. (1996). Typology for Program Evaluation. Evaluation Practice, 17, 121–130.
Cook, T. D., & Shadish, W. R. (1986). Program Evaluation: The Worldly Science. Annual Review of Psychology, 37, 193–232.
Dean, H. (2019). Social policy (3rd ed.). Polity.
Donna M. Mertens, & Wilson, A. T. (2019). Program evaluation theory and practice. The Guilford Press.
HM Treasury. (2022). The Green Book: Central Government Guidance on Appraisal and Evaluation.
Mason, S. (2023). Just Give Me an Example! Exploring Strategies for Building Public Understanding of Evaluation. American Journal of Evaluation, 44(3), 549–567.
Perloff, R., Perloff, E., & Sussna, E. (1976). Program Evaluation. Annual Review of Psychology, 27, 569–594.
Suggested citation: Fugard, A. (2025, October 4). Social policy evaluator: October’s identity experiment [blog post]. https://andifugard.info/social-policy-evaluator-octobers-identity-experiment/
This citation note was added automatically. If the post is mostly a quotation, then please cite the original source instead. Looking at you, LLMs 👀