Advice to empiricists from Alan Grafen

Spotted in Grafen, A. (1987). Measuring sexual selection: why bother.

To field workers, I would say: whatever you do, don’t go out and measure a parameter in somebody else’s model. Few things could be more fruitless. Your role is not to be a theoretician’s technician. Instead you are at least the theoretician’s equal and would be better employed doing one of three things. First, study an as yet unmodelled problem. Show the world that this problem is interesting and important, and outline what you see as its main features. Let theoreticians then busy themselves, working out in more and more detail that you had indeed got it nearly all right in the first place. Meanwhile, you can have discovered another interesting problem. Make the theoreticians come to you. After all, your function now is to provide the material which will determine what theoreticians will find interesting in five years’ time and beyond.

The second useful role of the field worker is to cut the Gordian knots of the theoreticians. For example, a debate is raging overhead about whether or not sufficient variability can be maintained in heritable fitness to make it worth while for a female to pay costs to choose a male solely for the inherited fitness of her offspring. Ultimately, this is a question that in theory will depend on the plausibility of various parameter values and details of the models. An empirical demonstration in reasonably natural conditions that fitness is often inherited from father to offspring in the absence of paternal care would radically change the nature of the debate. No longer would the plausibility of the conditions for maintaining this heritability be the hot topic, but rather what forces are actually responsible for it, and how strong are they? The empiricist must try to stop theoreticians wasting their time on empirical questions, if it is thought they could spend their time more profitably.

The third role of the empiricist is to undermine the theoretician by casting doubt on his accepted premises. What empirical demonstration would most embarass this model? How can I make this theory seem truly pointless? And so on. Go for the jugular.

Grafen, A. (1987). Measuring sexual selection: why bother?
J.W. Bradbury, M.B. Andersson (Eds.), Sexual Selection: Testing the Alternatives, John Wiley, Chichester (1987), pp. 221-233.

Individuals versus aggregrates

“Winwood Reade is good upon the subject,” said Holmes. “He remarks that, while the individual man is an insoluble puzzle, in the aggregate he becomes a mathematical certainty. You can, for example, never foretell what any one man will do, but you can say with precision what an average number will be up to. Individuals vary, but percentages remain constant. So says the statistician.”

—The Sign of Four by Sir Arthur Conan Doyle (hat-tip MP)

“… something more besides [psycho]analysis…”

“When the ego has taken its defensive measures against an affect for the purpose of avoiding unpleasure, something more besides analysis is required to undo them, if the result is to be permanent. This child must learn to tolerate larger and larger quantities of unpleasure without immediately having recourse to his defense mechanisms. It must, however, be admitted that theoretically it is the business of education rather than of analysis to teach him this lesson.”
—Anna Freud (1966, pp. 64-65)

Reference

Freud, A. (1966). The ego and the mechanisms of defense (revised ed.). New York: International Universities Press.

Famous women statisticians (or those who should be famous)

One of my day jobs is teaching psychology students how to do data analysis. Occasionally I quote famous statisticians, for instance to illustrate ways of thinking about analysis, the subjective nature of modelling data, and other fun things. I mention the likes of William Gosset (Guinness and t-tests), George Box (all models are wrong), and Bruno de Finetti (probabilities don’t exist).

Most – often all – of my students are women. Most of my current collection of quotations are from men. This is a problem. So, I’m currently looking for examples of famous statisticians who are women (statistician broadly interpreted; including data scientists, economists, quantitative social scientists). Here’s my current list. Suggestions for others would be most welcome, especially if you have a quotation I can use (turns out that statisticians write in maths most of the the time so it can be hard to find nice quotes).

  • Daphne Koller (Professor in Stanford University; wide range, e.g., conditional independence models, feature selection)
  • Deirdre McCloskey (Professor, economist, writes on stats amongst many other topics)
  • Elizabeth A. Stuart (Professor, statistician, causal inference methodology in public health and education)
  • Fiona Steele (Professor in Stats at LSE, e.g., multilevel modelling)
  • Florence Nightingale (data visualisation and public health stats)
  • Gertrude Mary Cox (experimental design and analysis of experiments)
  • Helena Chmura Kraemer (Professor of Biostatistics in Psychiatry, Emerita, at Stanford)
  • Hilary Mason (“enthusiastic member of the larger conspiracy to evolve the emerging discipline of data science”)
  • Hilary Parker (data analyst at Etsy; PhD in biostatistics, genomics; useR)
  • Irini Moustaki (Professor in Social Statistics at LSE)
  • Jane Hillston (Professor of quantitative modelling at Edinburgh University; invented the stochastic process algebra, PEPA)
  • Jennifer Neville (e.g., data mining for relational data such as networks/graphs)
  • Juliet Popper Shaffer (work on corrections for multiple hypothesis testing)
  • Pat Dugard (e.g., randomisation stats for single case and small-N multiple baseline studies)
  • Rachel Schutt (Senior Vice President of Data Science at News Corp)
  • Stella Cunliffe (worked in Guinness and first woman president of RSS)
  • Susan A. Murphy (e.g., clinical trial design; methods for multi-stage decision making)
  • Victoria Stodden (e.g., reproducibility of models, codes)

Example Quotations:

“The newly mathematized statistics became a fetish in fields that wanted to be sciences. During the 1920s, when sociology was a young science, quantification was a way of claiming status, as it became also in economics, fresh from putting aside its old name of political economy, and in psychology, fresh from a separation from philosophy. In the 1920s and 1930s even the social anthropologists counted coconuts.”
– Deirdre McCloskey, The Trouble with Mathematics and Statistics in Economics

“The Cabinet Ministers, the army of their subordinates… have for the most part received a university education, but no education in statistical method. We legislate without knowing what we are doing. The War Office has some of the finest statistics in the world. What comes of them? Little or nothing. Why? Because the Heads do not know how to make anything of them. […] What we want is not so much (or at least not at present) an accumulation of facts, as to teach men who are to govern the country the use of statistical facts.”
– Florence Nightingale in a letter to Benjamin Jowett; quoted by Kopf, E. W. (1916). Florence Nightingale as statistician. Publications of the American Statistical Association, 15, 388–404.

“To understand God’s thoughts we must study statistics, for these are the measure of His purpose.”
– Florence Nightingale

“The statistician who supposes that his [sic] main contribution to the planning of an experiment will involve statistical theory, finds repeatedly that he makes his most valuable contribution simply by persuading the investigator to explain why he wishes to do the experiment, by persuading him to justify the experimental treatments, and to explain why it is that the experiment, when completed, will assist him in his research.”
– Gertrude Mary Cox

“It is no use, as statisticians, our being sniffy about the slapdash methods of many sociologists unless we are prepared to try to guide them into more scientifically acceptable thought. To do this, there must be interaction between them and us.”
– Stella V. Cunliffe (1976, p. 9). Interaction. Journal of the Royal Statistical Society. Series A (General), 139, 1–19.

Brand names in psychology

‘The “brand names” tend to make difficult the analysis and comparison of these mechanisms or the exchange of knowledge between research groups. One can argue that it has caused and causes an enormous amount of duplication of effort. Physicists did not divide quantum mechanics into the Heisenberg Brand, the Schrodinger Brand,and the Dirac Brand, but analyzed in detail the relations among these and use one or the other according to their computational power in particular situations. When specific “brand name” choices have arisen (wave v. particle theories of light, Ampere’s v. Faraday”s theories of electromagnetism, phlogisten v. oxygen theories of production), they used experimental techniques to analyze both similarities and differences and to sort them out.’

Herbert Simon (spotted over here)

Power analysis for astronomers

Stephen Senn (2002) explains the need for statistical power analysis.

“An astronomer does not know the magnitude of new stars until he has found them, but the magnitude of star he is looking for determines how much he has to spend on a telescope.”

Senn, S. J. (2002). Power is indeed irrelevant in interpreting completed studies. BMJ, 325, 1304–1304

Tea

“… it is the simplest and most difficult thing in the world for one person, genuinely being his or her self, to give, in fact and not just in appearance, another person, realised in his or her own being by the giver, a cup of tea, really, and not in appearance.”
– R. D. Laing, Self and Others

NATs

From Power, M. (2010) [Emotion-Focused Cognitive Therapy, Wiley-Blackwell]:

“It is hard to know from the relevant literature how often cognitive therapists have patients […] whose emotions and moods do not appear to be triggered by reportable [Negative Automatic Thoughts (NATs)]. […] But whether the answer is that there are many or very few such NAT-free cases, the fact that a proportion of any cognitive therapist’s caseload must consist of such cases raises the question of what therapists do when this happens. Perhaps the comment about Freudian patients that they always came to have Freudian dreams and Jungian patients came to have Jungian dreams might be applicable; perhaps, clients may be suggestible enough to begin to have NATs if you persist long enough pursuing them as a therapist – if the clients can stand such therapy for that long. Of course, we know from the work on false memories that the therapeutic encounter is an extremely powerful one and that some clients may even falsely recollect memories of abuse, alien abduction or such, if that is the line being pushed by the therapist (Power, 2002). So having a few negative thoughts is small fry compared to alien abduction or imagined abuse…

“The point that we wish to make is that there are many strengths to the cognition focused approach, but there may be many limitations because of the failure to give emotion its rightful place…”

“We believe (Power & Dalgleish, 1997, 2008) that the problem is that the basic theory is wrong and that it is too simple. The cognitive therapies over-emphasize the role of thought in emotional disorders, and they lack an adequate theory of emotion:…”

Measurements presuppose theories

(cited in Gillies’ “Philosophical theories of probability”): 

“Against this view [operationalism] it can be shown that measurements presuppose theories. There is no measurement without a theory and no operation which can be satisfactorily described in non-theoretical terms. The attempts to do so are always circular; for example, the description of the measurement of length needs a (rudimentary) theory of heat and temperature-measurement; but these in turn involve measurements of length.” (Popper, Conjectures and Refutations, 1963)

Punched card equipment and questionnaires

“It can be said that some of the questionnaires used in these surveys contain everything but the proverbial kitchen sink, and once such a questionnaire has been filled in by a sizable group its author has the ‘basic’ data at hand for a half dozen articles. If he is fortunate enough to have punched card equipment, it becomes the misfortune of his professional contemporaries to find the literature being filled with results of cross tabulations which are so lacking in rationale as to be nonsensical. The ‘hypothesis’ step in scientific reasoning and research seems to be all too frequently ignored by the users of these techniques.”

– Quinn McNemar (1946, p. 293) [Opinion-attitude methodology. Psychological Bulletin, 43(4), 289–374].