Absence of Evidence is Evidence of Absence

(Edit: A much simpler proof is shown at the very bottom.) A lot of people get this wrong. It is true that absence of proof is not proof of absence; indeed to argue otherwise is almost incoherent. Was the absence of Andrew Wiles' proof taken as proof of absence that such a proof could exist, or absence of the theorem's truth, or what? In formal logic, if Evidence logically implies Proposition, it is not the case that not-Evidence logically implies not-Fact. But if we're talking about evidence about some proposition, the calculus is not governed by formal logic. You're in Probability Theory now, and the absence of evidence is in fact evidence of absence. I can prove it to you.

What is evidence? Evidence is some event, fact, or observation E that, when coupled with saying something about some other proposition X, leads to the following equation: $$Evidence\ about\ X = \frac{P(E|X,C)}{P(E|\sim X,C)}$$. C is any background context. So the equation says evidence about X is the probability of such evidence happening given some X is true, divided by the same probability but assuming X is false. As an example, if E is "raining", and X is "there are storm clouds in the sky", it's obvious that the ratio will be greater than 1--the odds of it raining given no storm clouds.

Bayes' Theorem says this: $$P(A|B,C) = P(A|C)*\frac{P(B|A,C)}{P(B|C)}$$. Look familiar? Let A be X, and B be E. Now if we want to compare two hypotheses, X and ~X, we can write as a ratio: $$\frac{P(X|E,C)}{P(\sim X|E,C)} = \frac{P(X|C)*\frac{P(E|X,C)}{P(E|C)}}{P(\sim X|C)*\frac{P(E|\sim X,C)}{P(E|C)}} = \frac{P(X|C)*P(E|X,C)}{P(\sim X|C)*P(E|\sim X, C)}$$. Now this is very familiar.

See Full Post and Comments

Looking past preconceptions

"The single most important kind of intelligence is the ability to see past your own strongly-held preconceptions and your tribe's conventional wisdom and engage reality as it actually is and facts as they actually are." --esr

If this is true, I would say the second-most important kind of intelligence is the same as the first-most, with this difference: "...the ability to see past your ... preconceptions .. and consider reality differently to how you consider it now." We all know that rape is bad, right? What if rape was good? Are you intelligent enough to have a conversation about this, without becoming emotional or using emotion-packed arguments? I am. And at the end of whatever speculation, deduction, or arm-chair philosophizing I participate in, I still think that rape is bad. But I'm willing to seriously consider the alternative. In fact I have, more that once. Is it a form of mental masturbation? Just as much as devil's advocacy is. But it's a nice and easy way to characterize uncivilized peoples (such as those Muslims protesting free speech) as not only wrong, but stupid. They're also ugly too. They can't mentally consider that what they believe is false, and they'll pay the price.

In fact I have considered this before, and this "type of intelligence" is really just part of the aspect of smartness otherwise known as intellectualism, something that is very much lacking in modern middle eastern and east asian cultures. ESR is wrong in characterizing a type of intellectualism as the most important aspect of smartness (and my generalization of "supposing" contradictory realities being second-most important is wrong, too). Intellectualism, while in my opinion very important for human progress, is nevertheless inferior to raw amounts of synthesis, analysis, and memory that combined get a ton of shit done. Intellectualism can easily be at odds with productivity (it certainly is with me, and I have met others who struggle with getting shit done because they want to learn something new but not necessarily in a deep way), and if productivity doesn't factor into your equation for the importance of some aspect of smartness, I have no idea what you're trying to do.

See Full Post and Comments

Fixing a trivial Clojure Error...

I had this happen to me this morning:

$ lein repl

REPL started; server listening on localhost port 18383

IllegalArgumentException Don't know how to create ISeq from: java.lang.Character  clojure.lang.RT.seqFrom (RT.java:494)

clojure.core=> (pst)

IllegalArgumentException Don't know how to create ISeq from: java.lang.Character

        clojure.lang.RT.seqFrom (RT.java:494)

        clojure.lang.RT.seq (RT.java:475)

        clojure.core/seq (core.clj:133)

        clojure.core/concat/fn--3804 (core.clj:662)

        clojure.lang.LazySeq.sval (LazySeq.java:42)

        clojure.lang.LazySeq.seq (LazySeq.java:60)

        clojure.lang.Cons.next (Cons.java:39)

        clojure.lang.RT.boundedLength (RT.java:1633)

        clojure.core/apply (core.clj:603)

        clojure.core/load-libs (core.clj:5302)

        clojure.core/apply (core.clj:603)

        clojure.core/require (core.clj:5381)

nil

clojure.core=>

What the hell, right? Turns out the problem was in my project.clj file! I had defined my main namespace like this: :main "com.ns.core"
But Leinigen wants it defined like this: :main com.ns.core
That is, without the quotes. Silly me! Also silly REPL for a horrible error message. I'm only blogging about this because pasting that error along with "lein repl" into Google didn't give me the solution, so hopefully this helps someone else.

See Full Post and Comments

Winning is fun

I took a bet recently, it was: "The U.S. unemployment rate will be below 8.0% for September 2012" I bet 0.10 bitcoins in favor of the statement, others bet more against, and so I ended up winning 1.078 bitcoins when the result was known, which is a 978% profit.

Why did I bet the way I did? Because I realize it's pretty much the last real month to put out something in favor of the President, and the existence of Shadow Stats is evidence that the number can be more-or-less whatever those in power want it to be, and those in power are happy with Obama. It was fun to win. So I wasn't very surprised when the results came back.

Was I overconfident? Probably. I'll know better when other bets come around for which I have similar degrees of evidence, and then I can learn just how strong the evidence is.

See Full Post and Comments