Jach's personal blog

(Largely containing a mind-dump to myselves: past, present, and future)
Current favorite quote: "Supposedly smart people are weirdly ignorant of Bayes' Rule." William B Vogt, 2010

Logit and Expit

Logit, or log-odds, is a sometimes useful function for dealing with probabilities in a more intuitive fashion. Expit is its inverse. This was kind of a tricky algebraic proof for me, but I am out of practice... For the following, assume I'm using log() function in base 2, because I'm a programmer.

[math]logit(p) = log(odds(p)) = log(\frac{p}{1-p}) \\
expit(p) = \frac{exp(p)}{1+exp(p)} = \frac{2^p}{1+2^p}

In order to prove these are inverses, I am going to prove that

p &=& logit(expit(p)) \\
&=& log(\frac{expit(p)}{1-expit(p)}) \\
&=& log\Big[\frac{\frac{exp(p)}{1+exp(p)}}{1-\frac{exp(p)}{1+exp(p)}}\Big] \\
&=& log(\frac{exp(p)}{1+exp(p)}) - log(1-\frac{exp(p)}{1+exp(p)}) \\
&=& log(exp(p)) - log(1+exp(p)) - log(1-\frac{exp(p)}{1+exp(p)}) \\
&=& p - log(1+exp(p)) - log(1-\frac{exp(p)}{1+exp(p)}) \\
&=& p - log(1+exp(p)) - log(1-(\frac{exp(p)+1-1}{1+exp(p)})) \\
&=& p - log(1+exp(p)) - log(1-(\frac{exp(p)+1}{1+exp(p)}-\frac{1}{1+exp(p)})) \\
&=& p - log(1+exp(p)) - log(1-(1-\frac{1}{1+exp(p)})) \\
&=& p - log(1+exp(p)) - log(\frac{1}{1+exp(p)}) \\
&=& p - log(1+exp(p)) - (log(1)-log(1+exp(p))) \\
&=& p - log(1+exp(p)) - log(1) + log(1+exp(p)) \\
&=& p - 0 + (0 - 0) \\
&=& p \\

So not strictly that hard from number of reductions, the hardest part for me was the x/(1+x) to 1-1/(1+x) step which wasn't obvious. Anyway I think I should prove the other way too, p = expit(logit(p)), to be done, but not now. Maybe in a future edit.

As a preview for where this is going, I want to write a post justifying from proofs that bayes' theorem can be rewritten as Belief(subject x after evidence e) = Belief(x prior) + Evidence(e about x), where Belief(x) = logit(x) and Evidence(e about x) = log(P(e|x)/P(e|!x)). But right now I'm struggling to find how that shouldn't be a proportional symbol, even though in some test calculations with the fully expanded Bayes theorem vs this, the results are equal...

Posted on 2017-11-19 by Jach

Tags: bayes, math


Trackback URL:

Back to the top

Back to the first comment

Comment using the form below

(Only if you want to be notified of further responses, never displayed.)

Your Comment:

LaTeX allowed in comments, use $$\$\$...\$\$$$ to wrap inline and $$[math]...[/math]$$ to wrap blocks.