Research Design in the Social Sciences

“This book introduces a new way of thinking about research designs in the social sciences. Our hope is that this approach will make it easier to develop and to share strong research designs.

“At the heart of our approach is the MIDA framework, in which a research design is characterized by four elements: a model, an inquiry, a data strategy, and an answer strategy. We have to understand each of the four on their own and also how they interrelate.”

Uses {DeclareDesign} in R.

Blair, G., Coppock, A., & Humphreys, M. (2023). Research Design in the Social Sciences: Declaration, Diagnosis, and Redesign. Princeton University Press.

Texts I like for learning statistics and using R

Fitting regression models, GLMs, etc.

Fox, J., & Weisberg, S. (2019). An R companion to applied regression (3rd ed.). London: SAGE Publications Ltd.

See also online material, including free appendices and R code.

Data transformation and visualisation

Healy, K. (2019). Data Visualization: A Practical Introduction. Princeton University Press. (Free online version.)

Wickham, H., Mine Cetinkaya-Rundel, & Grolemund, G. (2023). R for data science: Import, tidy, transform, visualize, and model data (2nd ed.). O’Reilly Media. (Free online version.)

Chang, W. (2020). R Graphics Cookbook (2nd ed.). Sebastopol, CA: O’Reilly. (Free online version.)

Lüdecke D (2018). ggeffects: Tidy Data Frames of Marginal Effects from Regression ModelsJournal of Open Source Software3(26), 772. doi: 10.21105/joss.00772

This is very handy for getting predictions from models, focusing on the effect of predictors of interest whilst holding covariates at some fixed values like a mean or (for factors) mode.

See also the package website for illustrative examples.

Gelman, A. (2011). Tables as graphs: The Ramanujan principleSignificance, 8, 183.

Nordmann, E. & DeBruine, L. (2023) Applied Data Skills.

Missing data imputation

Van Buuren, S. (2018). Flexible Imputation of Missing Data. Second Edition.. Chapman & Hall/CRC. Boca Raton, FL. (Free online version.)

See also the package website.

P-values

Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016). Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. European Journal of Epidemiology31, 337–350.

“… correct use and interpretation of these statistics requires an attention to detail which seems to tax the patience of working scientists.”

Colquhoun, D. (2014). An investigation of the false discovery rate and the misinterpretation of p-values. Royal Society Open Science, 1, 140216. doi: 10.1098/rsos.140216

This generated lots of debate – I like how it attempts to use Bayes rule to turn p-values into something useful and the explanation in terms of diagnostic test properties. See also this on PPV and NPV.

Rafi, Z., & Greenland, S. (2020). Semantic and cognitive tools to aid statistical science: replace confidence and significance by compatibility and surprise. BMC Medical Research Methodology, 20(1), 244. doi: 10.1186/s12874-020-01105-9

Interesting proposal to use s-values, calculated from p-values as −log₂(p). It’s a simple transformation: p is probability of getting all heads from −log₂(p) fair coin tosses. For example if p = 0.5 then s = 1; toss a coin once then the probability of head is 0.5. If p = 0.03125 then s = 5; toss a coin 5 times then the probability of all heads is 0.03125. But the s-value is supposedly easier to think about. I’m not sure if it really is, but I like the idea!

Updated 27 November 2024

Book review: Power, interest and psychology, by David Smail (2005)

Power, interest and psychology argues that psychotherapists need to take seriously how the social forces of interest and power affect how we all – therapists and clients alike – think, feel, and behave. The main targets of the book are what Smail believes to be the over-ambition and limited reach of therapists’ actions: warmth and empathy, debating beliefs, the “transference” – exploring and making explicit how the relationship between therapist and client might mirror relationships outside the room. The various therapeutic techniques, he argues, are dwarfed by the harsh social environment outside the therapy room. I read this book with interest as a (non-clinical, academic) lecturer who works with many kinds of psychotherapists and counsellors.

Smail rejects interventions which assume that insight leads to change, that we have individual will power which therapy can encourage, and that conscious thoughts accessible in therapy precede action. But what about clients who show improvement during the first few sessions of therapies which use these forms of intervention? He argues (pp. 24–25) that

“such initial gains tend not to last… Rather like tender plants that thrive only in a greenhouse, it seems that people find that there is still a cold and hostile world waiting for them at the end of their therapy sessions…”

The exceptions cited are clients who are young, attractive, verbal, intelligent, and successful – people who tend to be privileged by society. There is some research support for his clinical experience, for instance showing that cognitive ability positively correlates with outcomes (e.g., Mathiassen et al., 2012). A counterargument is evidence showing that “early responders” tend to sustain  better outcomes at long term follow up (Haas, Hill, Lambert, & Morrell, 2002; Lambert, 2005). However these correlational studies are open to attack: perhaps the “early response” just signals existing social and material resources which were easily activated by therapy (friends, family, money, etc.).

Therapy, Smail argues, tries to boost the perception of clients’ power to change, when in reality it is actual power that clients often need: power over material resources, finances; control in the workplace; personal characteristics such as confidence and intellect; a good home and family life; a love life; and an active social life (Hagan & Smail, 1997). These are areas which often cannot be influenced by talk in the clinic.

So why has individual therapy grown so popular? Smail argues – and emphasises that it’s nothing to be ashamed of – that therapists rely on income to put food on the table and pay the rent, just like their clients. He illustrates with the example of Sigmund Freud (p. 3) who wrote that

“My mood also depends very strongly on my earnings… I have come to know the helplessness of poverty and continually fear it.”

Freud, he argues, changed his theories so as not to blame clients’ parents since they paid the bills. Smail also argues that there is a great mysticism around therapy (p. 8): “rituals of therapeutic cure… bear a strong resemblance to the spells and incantations of sorcerers”, with practitioners rarely explaining in plain language to clients how their techniques supposedly work. Together these interests help sustain psychotherapy.

Is it really true that therapists can only intervene in the room with the individual client? Couple therapy takes the first step beyond the individual by bringing a romantic partner into the room, and there is evidence it helps with relationship problems (Snyder, Castellani, & Whisman, 2006). Child and adolescent mental health services frequently intervene in the family (Carr, 2009). Multi-family therapy (Asen & Scholz, 2010) brings communities of people into a room and encourages families to help each other as the therapists gradually “decentralise” themselves. There is an awareness of the importance of the systems around people suffering distress.

Another path outside the clinic is via “homework”, such as practicing social skills, which is (ideally) jointly agreed and set in a range of different types of therapies (Ronan & Kazantzis, 2006). Outcomes are better when therapies include homework than when no homework is included (Kazantzis, Whittington, & Dattilio, 2010). Smail, however, no doubt would argue that each of these interventions is limited when there are more material challenges at work such as poverty. What then would the homework consist of? Get a job? Make more money?

“The world is in a bloody mess,” concludes the book, “and even though I know, as do many others, what it would look like if it weren’t, I have no more viable idea than anyone else how to get there.” But there are constructive ideas in this text. Awareness that the causes of many of our actions is a mystery can be positive, for example in terms of accepting that social power flows through us and we shouldn’t blame ourselves for our situation or how we feel. A rich analysis is provided of the sources of this social power. The positive and convincing argument of the book is that the main hope of exercising power is through cooperation with others on all levels from friendship through to political activism. Indeed there is some evidence that activists who “advocate a social or political cause” tend to experience more positive emotions than non-activists (Klar & Kasser, 2009).

To what extent broader societal processes are within the scope of psychotherapy will no doubt continue to be debated. But whatever the scope, Smail suggests (p. 84) that the “appropriate role for therapeutic psychology is to record, celebrate and wonder at the extraordinary diversity of human character” – which sounds to me like a valuable starting point for therapeutic research and practice.

References

Asen, E., & Scholz, M. (2010). Multi-family therapy: concept and techniques. Hove: Routledge.

Carr, A. (2009). The effectiveness of family therapy and systemic interventions for child-focused problems. Journal of Family Therapy, 31, 3–45.

Haas, E., Hill, R. D., Lambert, M. J., & Morrell, B. (2002). Do early responders to psychotherapy maintain treatment gains? Journal of Clinical Psychology, 58, 1157–72. doi:10.1002/jclp.10044

Hagan, T., & Smail, D. (1997). Power-Mapping I . Background and Basic Methodology. Journal of Community & Applied Social Psychology, 7, 257–267.

Kazantzis, N., Whittington, C., & Dattilio, F. (2010). Meta-Analysis of Homework Effects in Cognitive and Behavioral Therapy: A Replication and Extension. Clinical Psychology: Science and Practice, 17, 144–156. doi:10.1111/j.1468-2850.2010.01204.x

Klar, M., & Kasser, T. (2009). Some Benefits of Being an Activist: Measuring Activism and Its Role in Psychological Well-Being. Political Psychology, 30(5), 755–777. doi:10.1111/j.1467-9221.2009.00724.x

Lambert, M. J. (2005). Early response in psychotherapy: further evidence for the importance of common factors rather than “placebo effects”. Journal of Clinical Psychology, 61(7), 855–69. doi:10.1002/jclp.20130

Mathiassen, B., Brøndbo, P. H., Waterloo, K., Martinussen, M., Eriksen, M., Hanssen-Bauer, K., & Kvernmo, S. (2012). IQ as a moderator of outcome in severity of children’s mental health status after treatment in outpatient clinics. Child and Adolescent Psychiatry and Mental Health, 6(22), 1–7. doi:10.1186/1753-2000-6-22

Ronan, K. R., & Kazantzis, N. (2006). The use of between-session (homework) activities in psychotherapy: Conclusions from the Journal of Psychotherapy. Journal of Psychotherapy Integration, 16, 254–259. doi:10.1037/1053-0479.16.2.254

Smail, D. (2005). Power, interest and psychology: elements of a social materialist understanding of distress. Ross-on-Wye: PCCS Books.

Snyder, D. K., Castellani, A. M., & Whisman, M. a. (2006). Current status and future directions in couple therapy. Annual Review of Psychology, 57, 317–44. doi:10.1146/annurev.psych.56.091103.070154

Lightly edited 3 Feb 2019