Personal website – any views expressed herein are my own unless otherwise obvious
Using my favourite tricks to setup the simulation:
Knitted R over here.
This plot provides some clues (group 0 is never treated; group 3 treated at time 3 and group 4 at time 4):
Zimmer, S. A., Powell, R. J., & Velásquez, I. C. (2024). Exploring Complex Survey Data Analysis Using R: A Tidy Introduction with {srvyr} and {survey}. Chapman & Hall: CRC Press.
“For our purposes, the same physical object or person at a different time is a different unit.” (Guido Imbens and Donald Rubin, 2015, p. 4)
“I guess I wont send that note now, for the mind is such a new place, last night feels obsolete.” (Emily Dickinson, 1870)
Dickinson, E. (1870). Letter to Elizabeth Holland.
Imbens, G. W., & Rubin, D. B. (2015). Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction. Cambridge University Press.
I happened upon an interesting question on Philosophy Stack Exchange: “What are the main views among academics on the question of the relation between logic and human cognition?” Here follows (a lightly edited version of) my answer.
Van Benthem (2008) offers examples of how cognition and social practices influence the development of logics, citing, e.g., Prior’s work on temporal structures and Lewis and Stalnaker on comparative orderings of possible worlds. He concedes that much of the work in this direction is informal (p. 74):
Logicians analyzing natural language, or computer scientists modeling “common sense”, tend to go by their own intuitions, anecdotal evidence from colleagues, email surveys of sometimes surprising naiveness, and other easy procedures that avoid the laboratories and statistical packages of the world of careful experimental design. But even so, experimental evidence is relevant, in that these theories can be, and sometimes are, modified under pressure of evidence from actual usage, even when it comes through these home-grown sources.
He also points to work in the other direction, where cognitive scientists take non-classical logics and use them to model social and cognitive phenomena. An example I know about (it was one of the reasons I chose my PhD topic and supervisor) is Stenning and van Lambalgen’s (2005) analysis of the “suppression task” in psychology, using nonmonotonic logic. Probability theory may be viewed as a kind of logic and it is used in cognitive science too; try e.g., Pfeifer (2013) and references that follow.
One key element of cognitive science studies that use logics is how people interpret tasks, and often have different interpretations, which can be modelled as choosing a logic and setting its parameters. Geis and Zwicky (1971) observe that logic students sometimes map the natural language indicative “if A, then B” to a biconditional in classical logic, so even in logic class interpretations aren’t naive.
Cognitive scientists don’t expect non-specialists to be aware of the logics that model their interpretations and inferences, e.g., here’s Da Silva Neves, Bonnefon, and Raufaste (2002, p. 110) discussing the relationship between potential properties of people’s reasoning systems and how people feel that they reason:
… even if we expect human inference to corroborate these properties, we know of no sufficient reason to think that lay reasoners would recognize any rationality postulate as valid, neither that they would conscientiously use them to guide their reasoning. […W]e assume that human inference is constrained by knowledge organisation in memory and that its formal properties emerge from a spreading activation process operating directly on knowledge structures.
van Benthem, J. (2008). Logic and reasoning: Do the facts matter? Studia logica, 88, 67-84.
Da Silva Neves, R., Bonnefon, J. F., & Raufaste, E. (2002). An empirical test of patterns for nonmonotonic inference. Annals of Mathematics and Artificial Intelligence, 34, 107-130.
Geis, M. L., & Zwicky, A. M. (1971). On invited inferences. Linguistic inquiry, 2(4), 561-566.
Pfeifer, N. (2013). The new psychology of reasoning: A mental probability logical perspective. Thinking & Reasoning, 19(3-4), 329-345.
Stenning, K., & van Lambalgen, M. (2005). Semantic interpretation as computation in nonmonotonic logic: The real meaning of the suppression task. Cognitive Science, 29(6), 919-960.
I was interested to see a recent review of process tracing in health research (Johnson, Beach, & Al-Janabi, 2024). How are its users getting on…?
Of the 19 evaluations satisfying review criteria, only 2 got full marks:
These were praised for how they justified using process tracing, their complete and detailed description of methods, their use of multiple types of evidence, and how they considered and attempted to rule out alternative or counterfactual explanations of outcomes. Finally, they linked their findings back to the theory. I have added them to my reading stack.
The review’s conclusions are damning:
“… a majority of studies reported do not describe in-depth how they conduct their process trace, and fewer still support their claims consistently and comprehensively with verifiable and well organised evidence. This is somewhat counterintuitive given that one of the features of process tracing is to support the more robust depth of analysis to explore hypotheses and confirm or counter with supportive evidence. However, this is not just a feature of studies reporting process tracing in health. Applications of process tracing in policy studies and political science more generally have similar trends, with many studies providing little detail of their methodology or a mechanistic depiction of the theory.”
As ever, to be continued…
Alvarado, M., Penney, T. L., Unwin, N., Murphy, M. M., & Adams, J. (2021). Evidence of a health risk ‘signalling effect’ following the introduction of a sugar-sweetened beverage tax. Food Policy, 102.
Johnson, R., Beach, D., & Al-Janabi, H. (2024). How is process tracing applied in health research? A systematic scoping review. Social Science & Medicine, in press, journal pre-proof.
Ryu, S.-K., & Chung, S.-G. (2021). Korea’s Early COVID-19 Response: Findings and Implications. International Journal of Environmental Research and Public Health, 18(16), 8316.
‘Highlighting QCA’s reliance on two-element Boolean algebra and propositional logic brings the weaknesses of QCA into sharp relief. The distinction between QCA and conventional statistics is not a difference between “set-theoretic” methods and what QCA researchers call “non-set-theoretic” methods (Schneider and Wagemann 2012) […]. Conventional statistics are set-theoretic methods. […] What truly sets QCA apart from conventional statistics is the former’s embrace of the limitations of two-element Boolean algebra and propositional logic. QCA exists in a nether region between qualitative and quantitative research; it exhibits neither the depth of small-n qualitative research nor the power of quantitative research.’ (Clarke 2020, p. 553)
Clarke, K. A. (2020). Logical Constraints: The Limitations of QCA in Social Science Research. Political Analysis, 28(4), 552–568.
Will McPhail [@realmikejfox] (2019, October 20). “Listen to me. Gender is a construct, society is a construct, money is a construct. But bedtime is very, very real.” [Instagram photo]. Retrieved from instagram.com/p/B32EjQKD7UT/
“CORELS (Certifiable Optimal RulE ListS) is a custom discrete optimization technique for building rule lists over a categorical feature space. Our algorithm provides the optimal solution, with a certificate of optimality. By leveraging algorithmic bounds, efficient data structures, and computational reuse, we achieve several orders of magnitude speedup in time and a massive reduction of memory consumption. Our approach produces optimal rule lists on practical problems in seconds. This framework is a novel alternative to CART and other decision tree methods.”
New paper:
Arel-Bundock, V., Greifer, N., & Heiss, A. (2024). How to Interpret Statistical Models Using marginaleffects for R and Python. Journal of Statistical Software, 111(9), 1–32.
Abstract:
“The parameters of a statistical model can sometimes be difficult to interpret substantively, especially when that model includes nonlinear components, interactions, or transformations. Analysts who fit such complex models often seek to transform raw parameter estimates into quantities that are easier for domain experts and stakeholders to understand. This article presents a simple conceptual framework to describe a vast array of such quantities of interest, which are reported under imprecise and inconsistent terminology across disciplines: predictions, marginal predictions, marginal means, marginal effects, conditional effects, slopes, contrasts, risk ratios, etc. We introduce marginaleffects, a package for R and Python which offers a simple and powerful interface to compute all of those quantities, and to conduct (non-)linear hypothesis and equivalence tests on them. marginaleffects is lightweight; extensible; it works well in combination with other R and Python packages; and it supports over 100 classes of models, including linear, generalized linear, generalized additive, mixed effects, Bayesian, and several machine learning models.”