I postulate that our brains have a shock-detector. The shock value of a sure thing is zero, so it doesn't even register. The shock value of an impossible thing is infinite. To accommodate a wide range of shocks, the response of the shock detector is logarithmic. So for an event of probability \(p\) we register the shock value of \(sh = log (1/p)\).
Shannon entropy is the average shock value you are likely to experience. It should be called boredom, because it peaks when any outcome is as likely as any other.
Follow
The way this fails to match up with intuitive meaning of boredom is that it claims that white noise is the least boring thing possible.
