The Nearly Perfect Prediction Theorem

Like stock-market traders trying to guess today’s opening price and movements, we aim to predict the current and future values of an unknown function based on the observed history of previous values. The true function $f:{\mathbb{R}}\to {\mathbb{R}}$, not necessarily continuous, will be revealed only gradually.

Figure 1.

The nearly perfect prediction theorem, a remarkable result of Christopher Hardin and Alan Taylor HT08HT13 shows that there is a nearly perfect prediction strategy, predicting the current and immediate future values of every function almost always correctly.

For predicting the present, we define a prediction strategy $P:{\mathbb{R}}^{<{\mathbb{R}}}\to {\mathbb{R}}$ on the set ${\mathbb{R}}^{<{\mathbb{R}}}$ of possible histories $f\upharpoonright t$, each a real-valued function on domain $(-\infty ,t)$ for some $t\in {\mathbb{R}}$, assigning them each to a corresponding present-value prediction $P(f\upharpoonright t)\in {\mathbb{R}}$. For every function $f$, these are almost always correct:

$$\begin{equation*} f(t)=P(f\upharpoonright t). \end{equation*}$$

Specifically, the predictions will be correct except on a countable nowhere dense set of exceptional points. What is more, every real number $s$ will be followed by an interval $(s,s+\varepsilon )$ on which the predictions are correct, and indeed there will be arbitrarily long intervals on which the predictions are perfectly correct. It is incredible.

To define the prediction strategy, simply fix a well order $\trianglelefteq$ on the set of all functions $f:{\mathbb{R}}\to {\mathbb{R}}$, and let $P$ predict in accordance with the $\trianglelefteq$-least function standing in agreement with the observed data. That is, given historical information $f\upharpoonright t$, let $P(f\upharpoonright t)=f^*(t)$, where $f^*$ is the $\trianglelefteq$-least function compatible with the prior data, meaning $f^*\upharpoonright t=f\upharpoonright t$.

To see that these predictions are almost always correct, suppose that the true function is $f:{\mathbb{R}}\to {\mathbb{R}}$. As time passes, more of the function is revealed and so the class of compatible functions becomes more restrictive. Consequently $s<t$ implies $f^*_s\trianglelefteq f^*_t$, where these are the $\trianglelefteq$-least functions in agreement with the data at those times. Further, if the prediction at time $s$ is incorrect, then $f^*_s$ would be incompatible with the data up to $t$, and so $f^*_s\neq f^*_t$. So the prediction functions climb strictly higher in the hierarchy $f^*_s\vartriangleleft f^*_t$ at every exceptional point, and since this hierarchy is well ordered, the exceptional points themselves must therefore be well ordered in the real numbers.

From this, all the amazing consequences follow. Above every real number $s$ there is a next exceptional point, if any, and so there is an open interval $(s,s+\varepsilon )$ on which all predictions are correct. The exceptional points are thus a countable nowhere dense set, countable since we can associate each exceptional $s$ with a rational number in this interval. Since there must also be an absolutely smallest exceptional point $r$, if any, all predictions made at times $t\in (-\infty ,r)$ are correct, and so there are arbitrarily long intervals on which all the predictions are correct. The real line is thus exhausted by a countable well-ordered sequence of intervals, which abut each other and their limits, with the predictions fully correct in the interior of each interval. Amazing!

Next, to predict the immediate future values of the function, we extend our earlier idea by defining a continuation prediction strategy $F:{\mathbb{R}}^{<{\mathbb{R}}}\to {\mathbb{R}}^{\mathbb{R}}$, which maps each history $f\upharpoonright t$ to its predicted continuation $F(f\upharpoonright t)=f^*$, the $\trianglelefteq$-least function standing in agreement with the observed data $f\upharpoonright t=f^*\upharpoonright t$.

Figure 2.

Incredible as it may seem, these predicted continuations will almost always agree with the true function $f:{\mathbb{R}}\to {\mathbb{R}}$ in an interval extending strictly beyond $t$:

$$\begin{equation*} \exists \epsilon >0\quad \forall x<t+\epsilon \qquad f(x)=F(f\upharpoonright t)(x). \end{equation*}$$

We observed earlier in effect that if $s<t$, then $F(f\upharpoonright s)\trianglelefteq F(f\upharpoonright t)$, since any function compatible with the later data $f\upharpoonright t$ is also compatible with the earlier data $f\upharpoonright s$. As time passes, therefore, we are climbing in the $\trianglelefteq$ hierarchy, and we climb strictly higher whenever the prediction at $s$ is not fully correct between $s$ and $t$. So again the continuation prediction exceptional points will be well ordered in the real numbers, and consequently again they will form a countable nowhere dense set. So the continuation predictions are almost always correct. And again, following every point $s$ will be an interval $(s,s+\epsilon )$ on which all predicted continuations are correct for the duration, and there will again be an absolutely least exceptional point $r$, meaning that the predicted continuations are fully correct for the entire interval $(-\infty ,r)$, which includes arbitrarily long intervals on which the predictions are correct.

I find it interesting to notice a certain robust stability—whenever a correct continuation prediction is made, then the very same prediction will continue to be made for the duration of that correctness, since it will continue as the $\trianglelefteq$-least function in agreement with the observed data as long as it remains correct.

The theorem is interesting even just for continuous functions. Although predicting the present value $f(t)$ of a continuous function is trivial, of course, since it is just the limit of $f(x)$ as $x\to t$ from below, the nearly perfect prediction theorem enables us to predict how a continuous function will continue into the immediate future, which is not trivial. We can ensure that strategies $P$ and $F$ predict in accordance with continuity when possible simply by placing the continuous functions at the bottom of the $\trianglelefteq$ hierarchy.

Figure 3.

Stock traders in the platonic realm.

Meanwhile, the reader will have noticed that the proof of the nearly perfect prediction theorem is not constructive, since it relies on the axiom of choice via the well-order theorem. The proof does not provide a useful concrete prediction strategy—it reveals little about the actual predictions to be made. I regret to say that you had best keep your day job.

Rather, we proved the theorem as a pure-existence result, showing that there is a nearly perfect prediction strategy as a matter of abstract mathematical ontology. I am confident that the iridescent mathematical stock traders of the platonic realm are truly making bank.