`Information is Physical' - R. Landauer
The science of information was initially motivated not by interest in our physical universe, but rather by the blossoming field of computer science. It dealt with the minipulation of bits, 0's and 1's, and at outset, seemed to share little relation with the physical universe. Sure, most people believed that all information must be encoded in some physical medium (except perhaps the dualists who regarded the mind as separate from the brain), but it had little direct relevance to concepts such as force, energy, or any of other myriad of observables that's traditionally associated with the physical world.
Yet, remarkably, the mere knowledge, or possession of bits, can be directly related so such quantities. This relation is no better demonstrated than Szilard's engine, whose fuel is only information... and nothing else.
The mechanics of the is simple to describe. We consider a single gas molecule trapped in a chamber of volume where is the length of the chamber, and its cross-sectional area. The chamber is in thermal equilibrium with the outside environment at some temperature . From statistical mechanics, we know that gas molecule obeys the Ideal Gas Law:
Where is the pressure it exerts, and $k$ is Boltzmann's constant. Since all components are in thermal equillibirum, the second law of thermal dynamics seems to suggest that there exists no possible way to extract work from the system.... however, what is we were capable of measuring the location of the gas particle inside at each point in time?
Consider Maxwell's demon, who is capable of doing just that. When he sees that the has molecule on the left hand side of the chamber, he quickly drops down a shutter that separates the chamber into two separate compartments. The demon the inserts a piston into the right side chamber and releases the shutter. Since the pressure in the left chamber is now double the pressure outside, we would expect the gas molecule inside the slowly push the piston till its pressure once again equalises that of the enviroment. Clearly, if the demon were to attach the piston to some machine, work can be extracted.... In particular, we know that the work done , obeys . Substituting the observation that , and the ideal gas law, we have
.
Thus, each cycle of this engine produces Joules of energy.... from a system that is in thermal equaillibrium.
Wait you say! Alarm bells should sound immediately. The intial and final state of the system appears exactly identical, and yet we have extracted energy from the system. Is this finally the perpetual motion machine that every is dreaming of?
To solve this paradox, we must look closely at each of the individual operations. Somewhere within the sequence, energy must have been discretely injected into our engine... but where? The laws of physics for any closed system in reversible, and thus our approach is to identify whereabouts the irreversible operation hides. The obvious candidates seem the measurement process, or perhaps the installation of the shutter or piston. Yet, while it such actions are likely to be irreversible in practice, there's no reason why they need to be in principle... elastic collisions can function as measurements, and everything else can be adiabatically. So at outset, it looks like the second law of thermodynamics has indeed been shattered!
There is however, one component of the system we've so far failed to address... the demon himself. Somewhere within the mind of this demon, he will need to store the knowledge of the result of his measurement - whether the gas molecule in question is in the left or right side of the chamber. Provided we regard this mind as being physical, it can be modelled by having to demon store his knowledge of some physical system with two configurations - a single bit, with states 0 and 1. Each cycle of the Szilard requires the demon to assign a value to the bit depending on the result of his measurement, and therefore erase whatever value was stored in the bit previously. This assignment process... mapping both 0 and 1 to a single value, is fundamentally irreversible.
The Energy Cost of Deleting Information:
In 1961, Landauer's published a remarkable argument that energy is indeed required erase information. Take, for example, the single bit that the demon holds. A physical process that assigns a specific value to the bit must first erase the previous value from memory. Whatever physical process that enacts this erasure, it must work equally well regardless of whether the bit is in state 0 or state 1. The initial system exists in two possible configurations, and thus possesses an entropy of . After erasure, the system exists in a single state, and hence has entropy 0. This, the entropy of the system has decreased.
Landauer argued, on the basis of that entropy does not decrease, that this loss of entropy must correspond to a gain of at least $k ln 2$ units of entropy elsewhere. Thus, the acting of erasing the information would incur Joules. Coincidentally, this is the exact amount of energy generated by the Szilard's engine at each cycle.
Information, theory, it appears, has saved us from a rather embarrassing perpetual motion machine!
Although Szilard's engine does not allow us to generate energy out of nothing, it does provide that subtle link between energy and information. Mere knowledge of the state of a system gives us the capacity to extract power from it. The catch is that the mere storage of that knowledge requires power. There's only a net gain if our hard drive lives in a colder enviroment than the Engine, and thus a temperature gradient is still required for work.