H(p) = -plogp - (1-p)log(1-p)
by Scott Aaronson
I was reading an obituary of Shannon,
asking myself:
why did it take till 1948?
That the semantics of a message
are irrelevant to how to transmit it,
that the distribution from which the message was
chosen is all that matters,
is obvious.
Should've been obvious to the Babylonians,
Greeks, Egyptians.
Why, among the thousands of ideas put forward
by Aristotle, Augustine, Maimonides,
Kant, Comte, Marx, Nietzsche,
Mary Baker Eddy,
was there nowhere the almost trivial observation
that to encrypt perfectly you need a key
the size of the message itself?
or that almost all functions
have exponential circuit complexity?
I wandered out of the library
and saw a girl sketching the landscape,
breasts straining against the V of her sweater,
surrounded by laughing guys
who'd probably never even seen the entropy formula,
and then I knew.
