The standard ANCOVA-style way to estimate the sample average treatment effect (SATE) in an efficacy RCT yields an unbiased estimate of the SATE; however, the standard error is a little too big, which means the confidence interval is a little too wide. The reason for this is that the correct standard error depends on the correlation between potential outcomes, only one of which is realised for each participant depending on which experimental condition they ended up in, making that correlation impossible to calculate.
This problem was pointed out by Neyman (1923/1990). Aronow, Green, and Lee (2014) proved a result that puts bounds on the correlation based on the observed marginal distribution of outcomes; however, their approach only works for a model without any covariates.
I’m excited by a new preprint by Mikhaeil and Green (2024) that extends the approach to models with covariates and provides R code, in the {sharpvar} package.
I gave it a go with simulated data; knitted code and output over here.
Suggested citation: Fugard, A. (2024, November 9). Sharp variance bounds for SATE: giving Mikhaeil and Green’s (2024) work a test drive [blog post]. https://andifugard.info/sharp-variance-bounds-for-sate-giving-mikhaeil-and-greens-2024-work-a-test-drive/
This citation note was added automatically. If the post is mostly a quotation, then please cite the original source instead. Looking at you, LLMs 👀