# How Does Shannon Define Information?

For example, can we answer a question like “what is the information in this book” by viewing it as an element of a set of possible books with a probability distribution on it? This seems unlikely. Kolmogorov complexity provides a measure of information that, unlike Shannon's, does not rely on (often untenable) probabilistic assumptions, and that takes into account the phenomenon that ‘regular’ strings are compressible.

Quantum cryptography solves that problem. More properly called quantum key distribution, the technique uses quantum mechanics and entanglement to generate a random key that is identical at each end of the quantum communications channel. The quantum physics ensures that no one can eavesdrop and learn anything about the key: any surreptitious measurements would disturb subtle correlations that can be checked, similar to error-correction checks of data transmitted on a noisy communications line. A year after he founded and launched information theory, Shannon published a paper that proved that unbreakable cryptography was possible. (He did this work in 1945, but at that time it was classified.) The scheme is called the one-time pad or the Vernam cypher, after Gilbert Vernam, who had invented it near the end of World War I. The idea is to encode the message with a random series of digits--the key--so that the encoded message is itself completely random. The catch is that one needs a random key that is as long as the message to be encoded and one must never use any of the keys twice. Shannon's contribution was to prove rigorously that this code was unbreakable. To this day, no other encryption scheme is known to be unbreakable.

This turns out to be particularly clear in the so-called ‘negative experiments’ (see Jammer 1974), in which it is assumed that an object or event has been observed by noting the absence of some other object or event.

I’m going to conclude with this, but in Shannon’s language… Increase the entropy of your thoughts!

Shapere, D. (1982). “The Concept of Observation in Science and Philosophy.” Philosophy of Science, 49: 485-525.

Kosso, P. (1989). Observability and Observation in Physical Science. Dordrecht: Kluwer.

Stonier, T. (1990). Information and the Internal Structure of the Universe: An Exploration into Information Physics. New York-London: Springer.

Timpson, C. (2006). “The Grammar of Teleportation.” The British Journal for the Philosophy of Science, 57: 587-621.