Increasing the gap between descriptional complexity and algorithmic probability

Abstract:

The coding theorem is a fundamental result of algorithmic information theory. A well-known theorem of Gács shows that the analog of the coding theorem fails for continuous sample spaces. This means that descriptional monotonic complexity does not coincide within an additive constant with the negative logarithm of algorithmic probability. Gács’s proof provided a lower bound on the difference between these values. He showed that for infinitely many finite binary strings, this difference was greater than a version of the inverse Ackermann function applied to string length. This paper establishes that this lower bound can be substantially improved. The inverse Ackermann function can be replaced with a function $O(\operatorname {log}(\operatorname {log}(x)))$. This shows that in continuous sample spaces, descriptional monotonic complexity and algorithmic probability are very different. While this proof builds on the original work by Gács, it does have a number of new features; in particular, the algorithm at the heart of the proof works on sets of strings as opposed to individual strings.