Jach's personal blog

(Largely containing a mind-dump to myselves: past, present, and future)
Current favorite quote: "Supposedly smart people are weirdly ignorant of Bayes' Rule." William B Vogt, 2010

Absence of Evidence is Evidence of Absence

(Edit: A much simpler proof is shown at the very bottom.) A lot of people get this wrong. It is true that absence of proof is not proof of absence; indeed to argue otherwise is almost incoherent. Was the absence of Andrew Wiles' proof taken as proof of absence that such a proof could exist, or absence of the theorem's truth, or what? In formal logic, if Evidence logically implies Proposition, it is not the case that not-Evidence logically implies not-Fact. But if we're talking about evidence about some proposition, the calculus is not governed by formal logic. You're in Probability Theory now, and the absence of evidence is in fact evidence of absence. I can prove it to you.

What is evidence? Evidence is some event, fact, or observation E that, when coupled with saying something about some other proposition X, leads to the following equation: $$Evidence\ about\ X = \frac{P(E|X,C)}{P(E|\sim X,C)}$$. C is any background context. So the equation says evidence about X is the probability of such evidence happening given some X is true, divided by the same probability but assuming X is false. As an example, if E is "raining", and X is "there are storm clouds in the sky", it's obvious that the ratio will be greater than 1--the odds of it raining given no storm clouds.

Bayes' Theorem says this: $$P(A|B,C) = P(A|C)*\frac{P(B|A,C)}{P(B|C)}$$. Look familiar? Let A be X, and B be E. Now if we want to compare two hypotheses, X and ~X, we can write as a ratio: $$\frac{P(X|E,C)}{P(\sim X|E,C)} = \frac{P(X|C)*\frac{P(E|X,C)}{P(E|C)}}{P(\sim X|C)*\frac{P(E|\sim X,C)}{P(E|C)}} = \frac{P(X|C)*P(E|X,C)}{P(\sim X|C)*P(E|\sim X, C)}$$. Now this is very familiar.

If we wanted to go further, we could write as $$\frac{P(X|E,C)}{P(\sim X|E,C)} = \frac{P(X|C)}{P(\sim X|C)} * Evidence$$, and then say your Prior Belief about X is defined as $$P(X|C)/(1-P(X|C)) = P(X|C)/P(\sim X|C)$$, and we could take the log base 2 of everything, which turns the multiplication into addition, and suddenly we get a nice common-sense equation that says $$Belief\ about\ X\ after\ seeing\ E = Prior\ Belief + Evidence$$ that is measured in units of bits. But this is just a fancy aside.

Along with Bayes' Theorem, we'll use another theorem of probability theory: marginalization. What is $$P(X|C)$$ regardless of whether we have evidence E or not? (Having absence of evidence is the same as having ~E.) Marginalization tells us: $$P(X|C) = P(X|E,C)*P(E|C) + P(X|\sim E,C)*P(\sim E|C)$$.

Now we can combine everything together. Let us assume this fact: having evidence E raises the probability of X. What does this fact give us, mathematically? We have to start with our fractional equation, and say that "our belief about x after seeing evidence is greater than our prior belief alone." This means that our amount of evidence is greater than our prior belief.

$$\frac{P(X|E,C)}{P(\sim X|E,C)} > \frac{P(X|C)}{P(\sim X|C)} \Rightarrow \frac{P(X|E,C)}{P(\sim X|E,C)} > \frac{P(X|E,C)*P(E|C) + P(X|\sim E,C)*P(\sim E|C)}{P(\sim X|E,C)*P(E|C) + P(\sim X|\sim E,C)*P(\sim E|C)} \Rightarrow $$

$$1 > \frac{P(\sim X|E,C)*\big[P(X|E,C)*P(E|C) + P(X|\sim E,C)*P(\sim E|C)\big]}{P(X|E,C)*\big[P(\sim X|E,C)*P(E|C) + P(\sim X|\sim E,C)*P(\sim E|C)\big]}\Rightarrow$$

$$1 > \frac{P(\sim X|E,C)*P(X|E,C)*P(E|C) + P(\sim X|E,C)*P(X|\sim E,C)*P(\sim E|C)}{P(X|E,C)*P(\sim X|E,C)*P(E|C) + P(X|E,C)*P(\sim X|\sim E,C)*P(\sim E|C)}\Rightarrow$$

$$1 > \frac{(1-P(X|E,C))*P(X|E,C)*P(E|C) + (1-P(X|E,C))*P(X|\sim E,C)*P(\sim E|C)}{P(X|E,C)*(1-P(X|E,C))*P(E|C) + P(X|E,C)*(1-P(X|\sim E,C))*P(\sim E|C)}\Rightarrow$$

$$1 > \frac{P(X|E,C)*P(E|C) - P(X|E,C)*P(X|E,C)*P(E|C) + P(X|\sim E,C)*P(\sim E|C) - P(X|E,C)*P(X|\sim E,C)*P(\sim E|C)}{P(X|E,C)*P(E|C) - P(X|E,C)*P(X|E,C)*P(E|C) + P(X|E,C)*P(\sim E|C) - P(X|\sim E,C)*P(X|E,C)*P(\sim E|C)} \Rightarrow$$

(Note that our inequality implies the top is less than the bottom, which lets us add/subtract to both top/bottom and retain the same quantity.)

$$1 > \frac{P(X|E,C)*P(E|C) + P(X|\sim E,C)*P(\sim E|C) }{P(X|E,C)*P(E|C) + P(X|E,C)*P(\sim E|C)} \Rightarrow$$

$$1 > \frac{P(X|C)}{P(X|E,C)*\big[P(E|C) + P(\sim E|C)\big]} \Rightarrow$$

$$1 > \frac{P(X|C)}{P(X|E,C)} \Rightarrow$$

$$P(X|C) < P(X|E,C)$$

In other words, observing E, or having the presence of E, increases the probability of X. This is what we wanted.

The proof also shows us that if we failed to observe E, that is if we have ~E, that is we have "absence of E", then $$P(X|C) > P(X|\sim E,C)$$. In other words, the absence of E decreases the probability of X.

If X is a proposition that states "Unicorns existed in the history of the Earth", then ~X says "Unicorns did not exist in the history of the earth, i.e. Unicorns are absent in the history of the earth." If we assume as rational humans the following: "if we discovered fossil records that consisted with a creature that the common man would call a unicorn, then our belief in unicorns existing in the history of the world would increase in confidence", then by the above proof, we have to also must agree to this: "if we fail to observe fossil records, then our belief in unicorns existing would decrease in confidence."

Obviously, we have failed and continue to fail to find any fossil records for unicorns. Does this fact alone prove that unicorns don't exist? No! It only proves that unicorns existing is less likely. If you wanted to prove unicorns don't exist, you have to make an additional assumption that if unicorns existed, then logically we'd have found fossil records by now, and by modus tollens we can show that unicorns don't exist. But without this extra assumption, our lack of fossil records is merely evidence against their existence. Similarly we can come up with other pieces of evidence in favor of unicorns--such as finding a live one in the places horses generally live, or finding out that the genetics of a horse allow for growing a horn some times, or something else, and because we have not observed any of these pieces of supporting evidence, our confidence must go down from wherever it was at in the first place.

The only way to escape this fate is to reject fossil records as having any bearing on an undecided person's belief in unicorns. But if you want to sway the undecided person's belief, you have to use something as evidence. If you say "I'm just going to be very confident that unicorns existed and I'm not going to look at any piece of evidence supporting (or rejecting) my claim", then you're stupid and ugly.

If you admit something as evidence for X, not seeing that evidence decreases your confidence in X and increases it in ~X. Absence of evidence is evidence of absence.

Edit: Here's the simpler proof:


I got it from here.

Posted on 2012-10-17 by Jach

Tags: bayes, math, philosophy, probability, rationality


Trackback URL:

Back to the top

Back to the first comment

Comment using the form below

(Only if you want to be notified of further responses, never displayed.)

Your Comment:

LaTeX allowed in comments, use $$\$\$...\$\$$$ to wrap inline and $$[math]...[/math]$$ to wrap blocks.