FIGJAM-based practice

Alternative to evidence-based practice: FIGJAM-based practice.MM0726_Fig_Jam__66623_std
(F**k I’m Good, Just Ask Me.)

Evidence is for the bureaucrats.
Trust us, we’re experts.
Join the school of the FIGJAM.
Throw your positivist randomised trials on the fire.

“I used the FIGJAM approach and I felt better.”

Coming to a social enterprise near you soon.

Book review: Power, interest and psychology, by David Smail (2005)

Power, interest and psychology argues that psychotherapists need to take seriously how the social forces of interest and power affect how we all – therapists and clients alike – think, feel, and behave. The main targets of the book are what Smail believes to be the over-ambition and limited reach of therapists’ actions: warmth and empathy, debating beliefs, the “transference” – exploring and making explicit how the relationship between therapist and client might mirror relationships outside the room. The various therapeutic techniques, he argues, are dwarfed by the harsh social environment outside the therapy room. I read this book with interest as a (non-clinical, academic) lecturer who works with many kinds of psychotherapists and counsellors.

Smail rejects interventions which assume that insight leads to change, that we have individual will power which therapy can encourage, and that conscious thoughts accessible in therapy precede action. But what about clients who show improvement during the first few sessions of therapies which use these forms of intervention? He argues (pp. 24–25) that

“such initial gains tend not to last… Rather like tender plants that thrive only in a greenhouse, it seems that people find that there is still a cold and hostile world waiting for them at the end of their therapy sessions…”

The exceptions cited are clients who are young, attractive, verbal, intelligent, and successful – people who tend to be privileged by society. There is some research support for his clinical experience, for instance showing that cognitive ability positively correlates with outcomes (e.g., Mathiassen et al., 2012). A counterargument is evidence showing that “early responders” tend to sustain  better outcomes at long term follow up (Haas, Hill, Lambert, & Morrell, 2002; Lambert, 2005). However these correlational studies are open to attack: perhaps the “early response” just signals existing social and material resources which were easily activated by therapy (friends, family, money, etc.).

Therapy, Smail argues, tries to boost the perception of clients’ power to change, when in reality it is actual power that clients often need: power over material resources, finances; control in the workplace; personal characteristics such as confidence and intellect; a good home and family life; a love life; and an active social life (Hagan & Smail, 1997). These are areas which often cannot be influenced by talk in the clinic.

So why has individual therapy grown so popular? Smail argues – and emphasises that it’s nothing to be ashamed of – that therapists rely on income to put food on the table and pay the rent, just like their clients. He illustrates with the example of Sigmund Freud (p. 3) who wrote that

“My mood also depends very strongly on my earnings… I have come to know the helplessness of poverty and continually fear it.”

Freud, he argues, changed his theories so as not to blame clients’ parents since they paid the bills. Smail also argues that there is a great mysticism around therapy (p. 8): “rituals of therapeutic cure… bear a strong resemblance to the spells and incantations of sorcerers”, with practitioners rarely explaining in plain language to clients how their techniques supposedly work. Together these interests help sustain psychotherapy.

Is it really true that therapists can only intervene in the room with the individual client? Couple therapy takes the first step beyond the individual by bringing a romantic partner into the room, and there is evidence it helps with relationship problems (Snyder, Castellani, & Whisman, 2006). Child and adolescent mental health services frequently intervene in the family (Carr, 2009). Multi-family therapy (Asen & Scholz, 2010) brings communities of people into a room and encourages families to help each other as the therapists gradually “decentralise” themselves. There is an awareness of the importance of the systems around people suffering distress.

Another path outside the clinic is via “homework”, such as practicing social skills, which is (ideally) jointly agreed and set in a range of different types of therapies (Ronan & Kazantzis, 2006). Outcomes are better when therapies include homework than when no homework is included (Kazantzis, Whittington, & Dattilio, 2010). Smail, however, no doubt would argue that each of these interventions is limited when there are more material challenges at work such as poverty. What then would the homework consist of? Get a job? Make more money?

“The world is in a bloody mess,” concludes the book, “and even though I know, as do many others, what it would look like if it weren’t, I have no more viable idea than anyone else how to get there.” But there are constructive ideas in this text. Awareness that the causes of many of our actions is a mystery can be positive, for example in terms of accepting that social power flows through us and we shouldn’t blame ourselves for our situation or how we feel. A rich analysis is provided of the sources of this social power. The positive and convincing argument of the book is that the main hope of exercising power is through cooperation with others on all levels from friendship through to political activism. Indeed there is some evidence that activists who “advocate a social or political cause” tend to experience more positive emotions than non-activists (Klar & Kasser, 2009).

To what extent broader societal processes are within the scope of psychotherapy will no doubt continue to be debated. But whatever the scope, Smail suggests (p. 84) that the “appropriate role for therapeutic psychology is to record, celebrate and wonder at the extraordinary diversity of human character” – which sounds to me like a valuable starting point for therapeutic research and practice.


Asen, E., & Scholz, M. (2010). Multi-family therapy: concept and techniques. Hove: Routledge.

Carr, A. (2009). The effectiveness of family therapy and systemic interventions for child-focused problems. Journal of Family Therapy, 31, 3–45.

Haas, E., Hill, R. D., Lambert, M. J., & Morrell, B. (2002). Do early responders to psychotherapy maintain treatment gains? Journal of Clinical Psychology, 58, 1157–72. doi:10.1002/jclp.10044

Hagan, T., & Smail, D. (1997). Power-Mapping I . Background and Basic Methodology. Journal of Community & Applied Social Psychology, 7, 257–267.

Kazantzis, N., Whittington, C., & Dattilio, F. (2010). Meta-Analysis of Homework Effects in Cognitive and Behavioral Therapy: A Replication and Extension. Clinical Psychology: Science and Practice, 17, 144–156. doi:10.1111/j.1468-2850.2010.01204.x

Klar, M., & Kasser, T. (2009). Some Benefits of Being an Activist: Measuring Activism and Its Role in Psychological Well-Being. Political Psychology, 30(5), 755–777. doi:10.1111/j.1467-9221.2009.00724.x

Lambert, M. J. (2005). Early response in psychotherapy: further evidence for the importance of common factors rather than “placebo effects”. Journal of Clinical Psychology, 61(7), 855–69. doi:10.1002/jclp.20130

Mathiassen, B., Brøndbo, P. H., Waterloo, K., Martinussen, M., Eriksen, M., Hanssen-Bauer, K., & Kvernmo, S. (2012). IQ as a moderator of outcome in severity of children’s mental health status after treatment in outpatient clinics. Child and Adolescent Psychiatry and Mental Health, 6(22), 1–7. doi:10.1186/1753-2000-6-22

Ronan, K. R., & Kazantzis, N. (2006). The use of between-session (homework) activities in psychotherapy: Conclusions from the Journal of Psychotherapy. Journal of Psychotherapy Integration, 16, 254–259. doi:10.1037/1053-0479.16.2.254

Smail, D. (2005). Power, interest and psychology: elements of a social materialist understanding of distress. Ross-on-Wye: PCCS Books.

Snyder, D. K., Castellani, A. M., & Whisman, M. a. (2006). Current status and future directions in couple therapy. Annual Review of Psychology, 57, 317–44. doi:10.1146/annurev.psych.56.091103.070154

Lightly edited 3 Feb 2019

An argument against payment-by-outcomes for mental health

I have just seen a report on Payment by Results (PbR) for the adult Improving Access to Psychological Therapies (IAPT) programme and have concerns about the approach. The conclusion of the summary is that “the system appears feasible and the currency appears to be fit for purpose” which seems to suggest that the approach is going ahead.

This IAPT PbR proposal is outcomes based, so that the more improvement shown by service users, as partly determined by patient-reported outcome measures (PROMs), the more money service providers would receive. This is a worry as there is evidence that linking measures to targets has a tendency to cause the measures to stop measuring what it is hoped that they measure. For instance targets on ambulance response times have led to statistically unlikely spikes at exactly the target, suggesting times have been changed [1]. A national phonics screen has a statistically unlikely spike just at the cutoff score, suggesting that teachers have rounded marks up where they fell just below [2]. The effect has been around for such a long time that it has a name, Goodhart’s law: “Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes” [3]. Faced with funding cuts, how many NHS managers will be forced to “game” performance-based payment systems to ensure their service survives?

PROMs have been criticised by therapists for leading to an “administratively created reality” [5] and being clinically unhelpful, perhaps even damaging. However, evidence is building that feeding back results from PROMs to clinicians is helpful for improving care [4]. It would be very sad indeed if this useful tool were destroyed by payment systems, just as many mental health practitioners — and more importantly, service users — are seeing the benefits. Linking outcomes algorithmically to finances at all seems to be a bad idea in general — it’s especially bad when PROMs are just beginning to be trusted in routine practice.


[1] G. Bevan and C. Hood, “What’s measured is what matters: targets and gaming in the English public health care system,” Public Adm., vol. 84, no. 3, pp. 517–538, 2006.

[2] L. Townley and D. Gotts, “Topic Note: 2012 Phonics Screening Check Research report,” 2013.

[3] C. A. E. Goodhart, “Monetary relationships: A view from Threadneedle Street.” 1975.

[4] C. Knaup, M. Koesters, D. Schoefer, T. Becker, and B. Puschner, “Effect of feedback of treatment outcome in specialist mental healthcare: meta-analysis.,” Br. J. Psychiatry, vol. 195, no. 1, pp. 15–22, Jul. 2009.

[5] J. McLeod, “An administratively created reality: Some problems with the use of self-report questionnaire measures of adjustment in counselling/psychotherapy outcome research,” Couns. Psychother. Res., vol. 1, no. 3, pp. 215–226, Dec. 2001.

What I think’s wrong with adult mental health Payment by Results (PbR)

(Usual disclaimer: these are my personal views, etc.)

Here’s a simple guide to PbR for some background.

In adult mental health in England there is a collection of “clusters” characterizing mental health service users who (it is hoped) have similar levels of need. These will eventually be linked to tariffs – price (which hopefully relates to cost) – and used by CCGs when they commission services. Key to the approach is a questionnaire which asks clinicians to rate problems (e.g., “Problems associated with hallucinations and delusions”) and their severity, and an algorithm mapping these to the clusters. Some more detail is available over there.

I’m not convinced by the approach. Here’s why:

  1. The model used to link score profiles to clusters has a large number of predictors (1,204) which means it is likely it is “overfitting”, i.e., any predictions made are unlikely to generalise beyond the sample on which it was developed. At its worst there are around 1.5 cases per predictor.
  2. There is evidence that clinicians disagree with the cluster predictions. Investigations around this have seemingly ignored the fact that there is additional information in cluster descriptions such as an ICD-10 clinical diagnosis, for instance “Likely to include F60 Personality disorder”. This information is not part of the scores used as an input to the algorithm which assigns clusters. Without understanding how clinicians use this information it is not possible to improve the approach.
  3. The methodology used to validate the model is circular. Clinicians were trained in an algorithm to choose a cluster on the basis of clinician-completed questionnaire scores. They followed this process with service users in routine practice, first completing a questionnaire, then recording the cluster chosen. A statistical approach was used to model the relationship between scores and clusters chosen. The end result is a statistical model predicting what the clinicians were initially trained to do. The validation method used is to look at correlations between what clinicians (trained in an algorithm) do and what a computer (which relearned the algorithm) does. This is circular.
  4. The clusters are supposed to characterise patients with similar needs, e.g., in terms of duration and complexity of interventions. Is there any evidence that they do? Seemingly not but I hope I’m wrong. It’s clearly essential that clusters actually do this, since PbR is supposed to be used for commissioning services and deciding how much services get paid. It is crucial to look at variation in costs as well as averages.
  5. The questionnaire (“tool”) used for deciding how much services get paid has also been proposed as an outcomes measure. This is despite the fact that the proposed approach derived by a factor analysis is psychometrically poor. The proposers recognise this (page 30): “it has been well established within the literature that the HoNOS is not typically associated with a high level of internal consistency due to its original intended purpose of being a scale with independent items encompassing a variety of health related problems.” Their factor analysis confirms this. Goodhart’s law suggests that even if the psychometrics were fine, the measure would cease to measure what it claims to measure once linked to costs.

I don’t think it has to be like this but it all makes very depressing reading.

What happens when you set targets – examples

(Updated 25/5/2016)

1. Ambulance response times (see Bevan and Hood, 2006)

ambulance target
Statistically unlikely spike exactly at the target suggesting something has happened to the data.

2. Phonics test scores (see Dorothy Bishop’s blog post)

A higher score is better performance. There’s a dip just below the pass mark of 32 and then a big spike, suggesting scores have been changed.

3. Call centre response times (see Caulcutt, 2004)

“The Times reported in October 2003 that the telecommunications regulator, Oftel, intended to investigate the workings of one of the newly established directory enquiry companies. According to the report: “Sixty call centre workers at the 118 118 directory enquiries service will be sacked in an attempt to head off a scandal over staff who deliberately gave out wrong numbers to boost their pay”. Why did they do this? It appears that the motivation was provided by a bonus system that rewarded employees for dealing with calls in less than 40 seconds.”

4. Final high school exam scores

Matura scores in Poland, 2013. To pass you need 30% or above.


(See here, spotted via @MaxCRoser.)

5. “‘G4S cheats’ made 1,000 FAKE 999 calls to boost performance figures”

“Staff at scandal-hit G4S boosted performance figures by making hundreds of fake calls to a 999 centre run by the firm.

“Five employees have been suspended after allegedly making more than 1,000 “test calls” – many reportedly at quiet times when they could be picked up quickly.

“Without them G4S would have missed key targets of answering 92% of calls within 10 seconds in November and December 2015, so incurring a financial penalty.”

(Mirror article)


In general, Goodhart’s (1975) law applies: “Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes”



Bevan, G., & Hood, C. (2006). What’s measured is what matters: targets and gaming in the English public health care system. Public administration, 84(3), 517-538.

Caulcutt, R. (2004). Managing by fact. Significance, 1(1), 36-38.

Goodhart, C. A. E. (1975). Monetary relationships: A view from Threadneedle Street. In Papers in Monetary Economics, Vol 1, Reserve Bank of Australia.

Computing number needed to treat from control group recovery rates and Cohen’s d

Furukawa and Leucht (2011) give a  formula for calculating the number needed to treat (NNT), i.e., (p. 1)

“the number of patients one would need to treat with the intervention in question in order to have one more success (or one less failure) than if treated in the control intervention”

based on the control group event rate (CER; for instance proportion of cases showing recovery) and Cohen’s d – an effect size in standard deviation units.

R code below:

NNT = function(d, CER) {
1 / ( pnorm( d - qnorm(1-CER) ) - CER )


Furukawa, T. A., & Leucht, S. (2011). How to obtain NNT from Cohen’s d: comparison of two methods. PloS one, 6(4), e19070.

Psychotherapy techniques – beyond the brands

The therapeutic brands are misleading as there’s a lot of overlap in techniques.

I like this simple table from Mick Power (2010, p. 49) of the different techniques, expressed in a cross-brand way.


Power argues that

“… therapy heightens access to cognitive–emotional structures and processes that relate to past and present significant objects and significant others including the therapist. In the context of this heightened access, there is the common therapeutic goal that patients will relearn, cope more successfully with, view more realistically, reinterpret or reconstruct; that is, in some way view more constructively the object, person or situation that has been the source of their distress or conflict.”

Fonagy and Bateman (2006, p. 425) go somewhere similar with the interrelationship part of this:

“It is possible that psychotherapy in general is effective because it arouses the attachment system at the same time it applies interpersonal demands (psychotherapy technique), which require the patient to mentalize, to confront and experience negative affect, and to confront and review issues of morality (superego). Why might this be helpful? We speculate that thinking about feelings, thoughts, and beliefs in the context of attachment is helpful because in this “paradoxical” brain state there may be more access to modifying preset ways of conceptualizing the contents of one’s own and other’s minds, as well as issues of morality and social judgment. Activating the attachment system harnesses brain biological processes partially to remove the dominance of constraints on the present from the past (long-term memory) and creates the possibility of rethinking, reconfiguring intersubjective relationship networks.”

Both are jargon heavy, though.


Fonagy, P., & Bateman, A. W. (2006). Mechanisms of change in mentalization‐based treatment of BPD. Journal of Clinical Psychology, 62, 411-430.

Power, M. (2010). Emotion-focused cognitive therapy. London: Wiley

Each investigation is an original research project

“There have been many studies of social factors in relation to schizophrenia. These include attempts to discover whether schizophrenia occurs more or less frequently in one or other ethnic groups, social class, sex, ordinal position in the family, and so on. The conclusion from such studies has often been that social factors do not play a significant role in the ‘aetiology of schizophrenia’. This begs the question, and moreover such studies do not get close enough to the relevant situation. If the police wish to determine whether a man has died of natural causes or has committed suicide, or been murdered, they do not look up prevalence or incidence figures. They investigate the circumstances attendant upon each single case in turn. Each investigation is an original research project, and it comes to an end when enough evidence has been gathered to answer the relevant questions.” (Laing, 1967, Politics of Experience.)

Haverstock Hill

“Paul Zeal told me of how, one day, he and Laing had spotted me careering down Haverstock Hill on my bike: bobble-hat on the hawk-head, a Dr Who oversized overcoat flapping in the breeze, with my “Unfit to Plead” badge attached to the lapel, no doubt heading for another night at the Vortex, or the consoling disillusions of the White Heart Lane terraces. This sight(ing) had provoked Laing to casually share with Paul, “What a strange bunch we are!” And the strangeness, the incipient unreality, of the psychotic world is what Laing had a considerable capacity to acknowledge, whilst neither rejecting, nor reinforcing it.”

Chris Oakley (2012), Where did it all go wrong? In R.D. Laing 50 Years since The Divided Self, edited by Theodor Itten and Courtenay Young.

Factors thought to maintain children and young people’s mental health problems

From Alan Carr’s Handbook of Child and Adolescent Clinical Psychology (2nd ed) pp. 62-70:

Family system factors

  • Inadvertent reinforcement, e.g., frequently inquiring about mood or commenting on negative conduct
  • Insecure attachment, whereby children don’t experience carers as a secure base
  • Coercive interaction, e.g., escalating negative interaction leading to withdrawal and relief, reinforcing behaviour just before relief
  • Over-involvement, parental criticism and emotional over-involvement
  • Disengagement, low frequency carer-child interaction
  • Inconsistent parental discipline, leading to problems internalising rules
  • Confused communication, e.g., indirect rather than direct communication
  • Triangulation (not always negative), e.g., a “coalition” where one carer is peripheral
  • Chaotic organisation
  • Absent carer
  • Carer relationship discord

Parental factors

  • Parents with similar problems as child act as a role model maintaining the behaviour
  • Resources for parenting compromised by mental health issues or criminality
  • Misinterpreted crying (interpreted as intentionally punishing carer)
  • Low self esteem
  • External locus of control
  • “Immature defences”
  • Unemployment (failure to meet financial needs; impact on status)
  • Boredom in work
  • Excessive stress in work
  • Role strain with parallel “homemaking” and working

Social network factors

  • Lack of social support, e.g., lack of positive interactions with extended family/friends
  • Chronic life stress
  • Unsuitable education placement, e.g., understaffed schools
  • “Deviant” peer-group e.g., peers using drugs
  • Community problems, e.g., social disadvantage, racism, social exclusion, high crime rates

Problem maintaining treatment system factors

  • Family members’ denial of problems
  • Poor working alliance with clinicians
  • Rejection of formulation and/or treatment plan
  • Failure of communication between MDT members
  • Failure of inter-agency network
  • Conflicting formulations in multidisciplinary team and inter-agency work
  • Culturally insensitive clinicians

Also flip side of these, protective factors, such as good physical health, high intellectual ability, high self-esteem, humour, positive engagement with treatment agencies, protective peer group, …