Exploring the validity of sentiment analysis in psychotherapy

What happens if you apply the Multilingual Language Model Toolkit for Sentiment Analysis (XLM-T), “a transformer-based NLP model derived from the Cross-lingual Language Model based on RoBERTa”, to psychotherapy transcripts? Eberhardt et al. (2024) investigate. Here’s a slightly simplified Table 1, showing correlations between positive and negative sentiment and a patient-reported emotions scale. Green gives between-patient correlations and pink within-patient across sessions. Note the wide confidence intervals.

Eberhardt, S. T., Schaffrath, J., Moggia, D., Schwartz, B., Jaehde, M., Rubel, J. A., Baur, T., André, E., & Lutz, W. (2024). Decoding emotions: Exploring the validity of sentiment analysis in psychotherapy. Psychotherapy Research.

Predictors of Transgender Prejudice: A Meta-Analysis

Hailey Hatch et al. (2022) conducted a systematic review and meta-analysis of predictors of transphobia. The final analysis consisted of 82 studies with a total of 36,285 participants. Main findings in the table below. A higher score indicates more transphobia. Right wing authoritarianism (RWA) is one of the strongest predictors, “characterized by the tendency to submit to authority figures, to adhere to conventional social norms, and to aggress against those who may be considered threatening or perceived to go against conventional norms”.

References

Hatch, H. A., Warner, R. H., Broussard, K. A., & Harton, H. C. (2022). Predictors of Transgender Prejudice: A Meta-Analysis. Sex Roles, 87(11–12), 583–602. [Unpaywalled]

Do “Growth Mindset” interventions improve students’ academic attainment?

“We conducted a systematic review and multiple meta-analyses of the growth mindset intervention literature. Our goal was to answer two questions: (a) Do growth mindset interventions generally improve students’ academic achievement? and (b) Are growth mindset intervention effects due to instilling growth mindsets in students or are apparent effects due to shortcomings in study designs, analyses, and reporting? To answer these questions, we systematically reviewed the literature and conducted multiple meta-analyses imposing varying degrees of quality control. Our results indicated that apparent effects of growth mindset interventions are possibly due to inadequate study designs, reporting flaws, and bias. In particular, the systematic review yielded several concerning patterns of threats to internal validity.”

Here’s a pic:

Can you bullshit a bullshitter?

You can bullshit a bullshitter, except if they also have high cognitive ability, according to Littrell et al. (2021).

Littrell, S., Risko, E. F., & Fugelsang, J. A. (2021). ‘You can’t bullshit a bullshitter’ (or can you?): Bullshitting frequency predicts receptivity to various types of misleading information. British Journal of Social Psychology, 60(4), 1484–1505.

No reasoning task is too irritating to be completed

Consider the sentence: “No head injury is too trivial to be ignored.” What does it mean?

Now consider: “No missile is too small to be banned.”

Go back to the first sentence – are you sure it means what you thought it did?

See Wason, P. C., & Reich, S. S. (1979). A Verbal Illusion. Quarterly Journal of Experimental Psychology, 31(4), 591–597.

Formal education and training

“[…] formal education and training rarely enhances competence. Instead, the so-called educational system mainly performs sociological functions, like controlling access to protected occupations and legitimising huge disparities in quality of life. These, in turn, have the effect of compelling most people, against their better judgement, to participate in the unethical activities of which modern society is so largely composed – the manufacture and marketing of junk foods, junk toys, junk education and junk research.”

– John Raven (2003, p. 360)

References

John, R. (2003). CPD – What should we be developing? The Psychologist, 16(7), 360–362.

Myers-Briggs

People are rightly critical of the Myers–Briggs Type Indicator (MBTI). But some of the types are moderately correlated with the Big Five dimensions, which are seen as more credible in differential psychology. MBTI extraversion correlates with… wait for it… Big Five extraversion (50% shared variance). MBTI intuition correlates with openness to new experiences (40% shared variance). The opposite poles correlate as you’d expect.

Here are the key correlations (Furnham et al., 2003, p. 580, gender and linear effects of age have been partialed out):

“Neuroticism was most highly correlated with MBTI Extraversion (r = -.30, p = .001) and Introversion (r = .31, p < .001). Costa and McCrae’s Extraversion was most highly correlated with Myers-Briggs Extraversion (r = .71, p < .001) and Introversion (r=-.72, p < .001). Openness was most highly correlated with Sensing (r = -.66, p < .001) and Intuition (r = .64, p < .001). Agreeableness was most highly correlated with Thinking (r=-41, p < .001) and Feeling (r = .28, p < .001). Conscientiousness was most highly correlated with Judgment (r = .46, p<.001) and Perception (r=-.46, p < .001).”

Dichotomising is still silly, particularly for scores close to thresholds, where a light breeze might flip someone’s type from, say, I to E or vice verse. But the same can be said of any discretisation taken too seriously. Consider also clinical bands on mental health questionnaires and attachment styles on the Experience in Close Relationships Scale.

Also silly are tautologous non-explanations of the form: they behave that way because they’re E. Someone is E because they ticked a bunch of boxes saying they consider themselves extraverted! The types are defined transparently in terms of thoughts, feelings, and behaviour. They help structure self-report, but don’t explain why people are the way they are. Explanations require mechanisms.

References

Furnham, A., Moutafi, J., & Crump, J. (2003). The relationship between the revised NEO-Personality Inventory and the Myers-Briggs Type Indicator. Social Behavior and Personality, 31, 577–584.

Emotional blunting cured by reading original research

“Antidepressants can cause ‘emotional blunting’, study shows”, said the Grauniad today. The study by Langley et al. (2023) randomised 66 healthy participants (i.e., not requiring antidepressants) to either SSRI (20 mg of escitalopram – a high dose) or placebo, daily for 21 days.

I have two brief observations:

Almost all the statistical analyses were classical, with 95% confidence intervals including zero interpreted as not statistically significant. The authors’ headline-grabbing finding, concerning reinforcement sensitivity, was for one of four outcomes modelled using hierarchical Bayesian modelling. A 90% highest density interval excluded zero. However, the 95% interval included zero, so following the conventions of the rest of the paper would be counted as a null effect. There was no comment on this in the paper, which strikes me as a little odd, particularly given the large number of preregistered study outcome measures (16 primary, 44 secondary, 32 other) and consequent risk of a false positive.

Additionally, the headlines and study’s conclusion claim that their findings may explain the emotional blunting sometimes reported by users of SSRIs. But I don’t see how emotional blunting relates to the probabilistic reversal learning task the authors used.

I hope the study receives critical scrutiny in the press.

References

Langley, C., Armand, S., Luo, Q. et al. Chronic escitalopram in healthy volunteers has specific effects on reinforcement sensitivity: a double-blind, placebo-controlled semi-randomised studyNeuropsychopharmacol. (2023).