At the heart of complexity and complex systems is the manipulation of patterns – predictable structures – using an internal memory, but the exact structure of this memory is not uniquely defined. Here, we consider the thermodynamics involved in this manipulation, such as by living things which exploit patterns in their surroundings, and use the energy released to generate new structure. We physically substantiate the intuition of Occam's razor by identifying the thermodynamic consequences of using a complicated memory.
To do this, we introduce a general framework for the manipulation of patterns, consisting of generators that use energy to produce patterns, and extractors that release energy by destroying patterns. Both must contain internal memory about what has happened in the pattern so far, in order that they can accurately generate (or anticipate) the upcoming parts of the pattern. When it comes to generating a pattern, updating the internal memory incurs an energy cost. This cost is lowest when the memory contains no more information than required to accurately generate the pattern. We thus identify that simpler is better when it comes to producing patterns, providing a physical motivation to support Occam's razor.
- When is simpler thermodynamically better?
Andrew J. P. Garner, Jayne Thompson, Vlatko Vedral, Mile Gu
