Pinference: Probability Inference for Propositional Logic. “Implementation of T. Hailperin’s procedure to calculate lower and upper bounds of the probability for a propositional-logic expression, given equality and inequality constraints on the probabilities for other expressions. Truth-valuation is included as a special case. Applications range from decision-making and probabilistic reasoning, to pedagogical for probability and logic courses.”
RobinCar2: ROBust INference for Covariate Adjustment in Randomized Clinical Trials. “Performs robust estimation and inference when using covariate adjustment and/or covariate-adaptive randomization in randomized controlled trials.” Though see this post.
Here’s a quick ggplot in R. Foyle (Derry) is an interesting outlier, though there’s a very small number of restaurants in the data – maybe data quality. Cynon Valley (south Wales) has 318 chippies.
Interesting paper urging researchers to go beyond the household as a level of analysis (and data collection).
“Overall, older adults have better mental health when LAT [living apart together] than when single, but little difference in mental health is found across LAT, cohabitation, and marital partnerships. Whereas older singles moving into LAT experience mental health improvements, those moving from LAT to singlehood suffer mental health declines. Although the mental health benefits of moving into LAT are smaller than those of entering cohabitation and particularly marriage, exiting LAT is associated with smaller mental health declines compared with exiting cohabitation and marriage. No statistically significant gender difference is found in the mental health benefits of LAT.”
Just out in Statistics in Medicine, this is a helpful review of approaches to adjusting for observed confounding in non-randomised evaluations (and other studies). Includes a concise summary of assumptions, using the Neyman–Rubin potential outcomes framework.
Useful blog post (2022, last updated 2024) by Iris Eekhout showing how to update the predictor matrix and setup passive imputation using {mice} so that:
Items are imputed using (a) other items within a questionnaire and (b) other questionnaires’ total scores.
A total score is passively imputed on each run which becomes part of the MI process.
Periodically I listen to contemporary debates in evaluation and wonder how much progress there has been in the past fifty years. Here’s a picture from Carol Weiss (1972, Figure 3–3, p. 50; reprinted in 1996):
Weiss called it a model of a programme; now we would use the term programme theory or theory of change. About half of the next page (Weiss, 1972, p. 51) is filled with intriguing footnotes. The first notes the similarity between the diagram above and a statistical approach:
“As the similarity between Figure 3–3 and a path diagram suggests, path analysis is a useful way of estimating the strength of the linkages. When the necessary conditions for each path analysis are met, path coefficients will represent the magnitude of effects for each arrow.”
On path analysis, Weiss cites Blalock’s (1969) Theory construction from verbal to mathematical formulations. Structural equation modelling, path analysis plus confirmatory factor analysis, turned out to be a popular way to test theories in social science (see, e.g., MacCallum & Austin, 2000; Zyphur, Bonner, & Tay, 2023).
The same footnote also summarises the Automatic Interaction Detector (AID), an early example of a data-driven partitioning procedure:
“Another analytic device, particularly useful when one step in the chain is totally or largely conditional on the one before, is the Automatic Interaction Detector (AID). Unlike path analysis, this type of ‘branch’ analysis shows whether the program is producing strong effects for subsamples of the group. This is a matter of both theoretical and programmatic interest, because if there are differential effects, special programs can be developed to fit special groups.”
AID and its successors are data hungry and liable to overfitting data if misused; however, there is a rich literature on how to use them and stay out of trouble. Loh (2014) reviews fifty years of that literature, beginning with AID (p. 330):
“Despite their novelty, or perhaps owing to it, AID and THAID did not initially attract much interest in the statistics community. Einhorn (1972) showed by simulation that AID can severely overfit the data. Doyle (1973) pointed out that if two or more variables are highly correlated, at most one may appear in the tree structure. This problem of masking can lead to spurious conclusions about the relative importance of the variables. Bishop et al. (1975) criticized AID for ignoring the inherent sampling variability of the data. Around the same time though, the idea of recursive partitioning was gaining steam in the computer science and engineering communities as more efficient algorithms for carrying out the search for splits began to appear (Chou 1969; Henrichon & Fu, 1973; Meisel & Michalopoulos, 1977; Payne & Meisel, 1977; Sethi & Chatterjee, 1991).”
(Example newer algorithms are implemented in {rpart} in R.) Weiss’s footnote and these statistical approaches are clear examples of trying to tackle the “what works best for whom” problem, which for some reason (along with “context”) is now associated, like a Pavlovian response, with “realist” evaluation. We need to do better.
References
Blalock, H. M., Jr. (1969). Theory construction from verbal to mathematical formulations. Prentice-Hall Inc.
MacCallum, R. C., & Austin, J. T. (2000). Applications of structural equation modeling in psychological research. Annual Review of Psychology, 51(1), 201-226.
Weiss, C. H. (1972). Evaluation research: Methods of assessing program effectiveness. Prentice-Hall, Inc.
Weiss, C. H. (1996). Excerpts from evaluation research: Methods of assessing program effectiveness. Evaluation Practice, 17(2), 173–175.
Zyphur, M. J., Bonner, C. V., & Tay, L. (2023). Structural equation modeling in organizational research: The state of our science and some proposals for its future. Annual Review of Organizational Psychology and Organizational Behavior, 10(1), 495-517.
That feeling when you want to see the 648 permutations of the words “evidence”, “based”, “policy”, and “making”, conjoined by space, hyphen, or just stuck together without either to make a closed compound word.
Zimmer, S. A., Powell, R. J., & Velásquez, I. C. (2024). Exploring Complex Survey Data Analysis Using R: A Tidy Introduction with {srvyr} and {survey}. Chapman & Hall: CRC Press.
