Tag: policy evaluation

Hume’s is-ought thesis states that we cannot infer a normative statement about what we should do from a descriptive statement about what is the case. This post spells out some of the detail about what the thesis means, which should be of interest to policy evaluators since we’re in the business of producing evidence and offering advice on policy that’s consistent with that evidence.

Let’s start with Prior’s (1960) puzzle (mildly edited): what kind of statement is “Either it’s raining or cricket should be banned”? It combines a descriptive statement (“it’s raining”) with a normative one (“cricket should be banned”), but is the disjunction of the two descriptive or normative?

Suppose Prior’s puzzle is a descriptive statement. Then by adding it to the premise “it’s not raining”, we can derive the purely normative conclusion that “cricket should be banned”. Since the disjunction is true, but one of its disjuncts is false, then the other disjunct must be true. We have just drawn an is-ought inference.

In symbols,

\(\displaystyle \neg R, R \lor \mathsf{O} B\ \models\ \mathsf{O} B\),

where \(R\) denotes “it’s raining”, \(\mathsf{O} B\) denotes that we ought to ban cricket (the \(\mathsf{O}\) is the “ought” from deontic logic), \(\lor\) is disjunction (or), \(\neg\) is negation, and \(\models\) is logical consequence.

This is maybe easier to see by rewriting the disjunction as a conditional, “If it’s not raining, then cricket should be banned” (since \(\neg \phi \lor \psi = \phi \rightarrow \psi\)):

\(\displaystyle \neg R, \neg R \rightarrow \mathsf{O} B\ \models\ \mathsf{O} B\),

where \(\rightarrow\) is the material conditional. The conclusion is then drawn by modus ponens. This pattern is interesting for evaluation, since it mirrors the logic:

1. The programme works*.
2. If the programme works*, we should roll it out nationally.
3. Therefore, we should roll it out nationally.

Where works* includes a range of statements about impact and process evaluation evidence, whether that evidence can be generalised to a broader population, whether the programme avoids causing harm, its cost‑effectiveness, and other considerations.

In symbols,

\(\displaystyle W^*, W^* \rightarrow \mathsf{O} R\ \models\ \mathsf{O} R\),

where \(W^*\) denotes the programme works* and \(R\) denotes that we should roll it out.

Suppose instead that Prior’s puzzle is a normative statement. Starting from the purely descriptive premise “it’s raining”, we can infer “Either it’s raining or cricket should be banned” by disjunction introduction. Again, we have drawn an is-ought inference.

In symbols,

\(\displaystyle R\ \models\ R \lor \mathsf{O} B\).

Rewriting using implication,

\(\displaystyle R\ \models\ \neg R \rightarrow \mathsf{O} B\),

the conclusion is one of the “paradoxes” of the material conditional: true under classical logic because the antecedent is false, but people tend to judge the conditional to be neither true nor false but irrelevant (Johnson-Laird & Tagart, 1969).

The solution is, “Either it’s raining or cricket should be banned” is neither descriptive nor normative: it’s mixed. The same applies to “If the programme works*, we should roll it out nationally”. Three refinements of the is-ought thesis are provided in a logic-heavy book by Schurz (1997), which has been on my reading stack for years. A more digestible summary is provided by Schurz (2014, pp. 2–3):

(H1) No non-logically true purely normative conclusion can be derived from a consistent set of purely descriptive premises.

(H2) Every mixed conclusion [i.e., combining descriptive and normative statements] which follows logically from a set of purely descriptive premises is normatively irrelevant in the sense that all of its normative subformulas are replaceable by other arbitrary subformulas, while preserving the validity of the inference [“salva validitate of the inference” in Schurz’s original].

(H3) No non-tautologous descriptive statement can be inferred from a consistent set of purely normative premises.

H1 is the thesis that applies most to evaluation: to make an evaluative judgement, you need mixed premises that blend normative and descriptive statements. H2 deals with weird uses of logic. I’ve used the principle of irrelevance it contains to help understand how people reason about sentences like “If Alex posted the letter, then he posted the letter or set fire to the letter”, which are true in classical logic but which people often judge to be false (Fugard et al., 2011). There’s a blog post about it yonder. H3 says that knowing or believing what should be true doesn’t tell you what is factually true.

There is, however, a well‑known problem with using the material conditional in combination with oughts (Chisholm, 1963), e.g., \(W^* \rightarrow \mathsf{O} R\) used above. Consider the following sentences and formalisations:

1. It ought to be that Jones goes to assist his neighbours: \(\mathsf{O} g\).
2. It ought to be that if Jones goes, then he tells them he is coming: \(\mathsf{O} (g \rightarrow t)\).
3. If Jones doesn’t go, then he ought not tell them he is coming. \(\neg g \rightarrow \mathsf{O} \neg t\).
4. Jones doesn’t go: \(\neg g\).

The English‑language statements feel consistent with each other. They are also independent in the sense that no one sentence follows from any of the others. A formalisation should preserve both features.

There are three paths through, using what has come to be known as standard deontic logic (SDL). McNamara and Van De Putte (2025, Section 2.1) provides an introduction to SDL. Section 4.1 provides the illustration of Chisholm’s (1963) problem, which I’ve just spelt out a little. Here’s a summary of the paths:

Path A	Path B	Path C
(1′) \(\mathsf{O}g\) (2′) \(\mathsf{O}(g \rightarrow t)\) (3′) \(\neg g \rightarrow \mathsf{O}\neg t\) (4′) \(\neg g\)	(1′) \(\mathsf{O}g\) (2′) \(\mathsf{O}(g \rightarrow t)\) (3″) \(\mathsf{O}(\neg g \rightarrow \neg t)\) (4′) \(\neg g\)	(1′) \(\mathsf{O}g\) (2″) \(g \rightarrow \mathsf{O}t\) (3′) \(\neg g \rightarrow \mathsf{O}\neg t\) (4′) \(\neg g\)
From (1′), (2′), \(\mathsf{O}t\). From (3′), (4′), \(\mathsf{O}\neg t\). Consistency is lost.	(1′) implies (3″). Independence is lost.	(4′) implies (2″). Independence is lost.

Following Path A, we deduce that Jones ought to tell them he is coming and ought not tell them he is coming, which is suspect. This uses the following SDL rule:

\(\mathsf{O}(\varphi \rightarrow \psi) \rightarrow (\mathsf{O}\varphi \rightarrow \mathsf{O}\psi)\text{,} \tag{OB-K}\)

which gives us \(\mathsf{O}g \rightarrow \mathsf{O} t\) (from 2′). This (alongside 1′) gives us \(\mathsf{O}t\) by modus ponens. We can also get \(\mathsf{O}\neg t\) using modus ponens (3′ and 4′). It also contradicts one of the axioms of SDL: if you should do something, then you shouldn’t also do the negation of that something:

\(\mathsf{O} \phi \rightarrow \neg \mathsf{O}\neg \phi \tag{NC}\)

Paths B and C attempt to use the same expression for the conditional oughts in sentences 2 and 3. Path B uses

\(\mathsf{O}(\phi \rightarrow \psi)\).

Path C uses

\(\phi \rightarrow \mathsf{O}\psi\),

which we encountered above when exploring Prior’s (1960) puzzle.

The problem with both attempts to save SDL is that the sentences become dependent, whereas in the original informal English they aren’t. For Path B, \(\mathsf{O}(\neg g \rightarrow \neg t)\) is vacuously true (when rewritten using OB-K) because \(\mathsf{O}g\) is true – a paradox of the material conditional again. Similarly for Path C, \(g \rightarrow \mathsf{O}t\) is vacuously true because \(\neg g\) is.

So however we choose to formalise the sentence “if you should do A, then you should do B” or “if A, then you should do B”, it is not something that can be captured within SDL – a conclusion reached in the late 1960s (Parent & Torre, 2018, p. 20). This conclusion is familiar from work in the psychology of reasoning, where the material conditional is replaced with systems that behave more like everyday inference.

One approach uses defeasible logics, which allow us to retract a conclusion when new information arrives and to treat some premises as having greater priority than others (e.g., Neves et al., 2002). Another approach uses probability logics, which model reasoning under uncertainty (e.g., Pfeifer & Kleiter, 2009). Probability logics typically rely on an underlying three‑valued semantics, in which a statement can be true, false, or neither. The third value is usually interpreted as something like “irrelevant” or “undetermined”.

A recent review of deontic logic (McNamara & Van De Putte, 2025) concludes that “there are a number of outstanding problems for deontic logic. Some see this as a serious defect; others see it merely as a serious challenge, even an attractive one.” I’ve been reading attempts to solve some of these problems, e.g., Horty (2012), which applies defeasible logic to deontic reasoning. See the follow-up.

References

Chisholm, R. M. (1963). Contrary-to-duty imperatives and deontic logic. Analysis, 24, 33–36.

Fugard, A., Pfeifer, N., & Mayerhofer, B. (2011). Probabilistic theories of reasoning need pragmatics too: modulating relevance in uncertain conditionals. Journal of Pragmatics, 43, 2034–2042.

Horty, J. F. (2012). Reasons as defaults. Oxford University Press.

Johnson-Laird, P., & Tagart, J. (1969). How implication is understood. The American Journal of Psychology, 82, 367–373.

McNamara, P., & Van De Putte, F. (2025). Deontic logic. In E. N. Zalta & U. Nodelman (Eds), The Stanford encyclopedia of philosophy (Winter 2025). Metaphysics Research Lab, Stanford University.

Neves, R. D. S., Bonnefon, J.-F., & Raufaste, E. (2002). An Empirical Test of Patterns for Nonmonotonic Inference. Annals of Mathematics and Artificial Intelligence, 34, 107–130.

Parent, X., & Torre, L. van der. (2018). Introduction to Deontic Logic  and Normative Systems. College Publications.

Pfeifer, N., & Kleiter, G. D. (2009). Framing human inference by coherence based probability logic. Journal of Applied Logic, 7, 206–217.

Prior, A. N. (1960). The autonomy of ethics. Australasian Journal of Philosophy, 38(3), 199–206.

Schurz, G. (1997). The Is-Ought Problem: An Investigation in Philosophical Logic. Springer.

Schurz, G. (2014). Cognitive success: Instrumental justifications of normative systems of reasoning. Frontiers in Psychology, 5(625).