Wednesday, June 8, 2016

Some Perils of Mathematical Modeling

Mathematical models can be great. They do, however, have some limitations. Suppose, for example, that you are trying to predict some data that you suspect has some mathematical relationship and you want to know the future behavior. A mathematical model might be useful to predict the future results of the data. Your predictive abilities will only be as good as the model (or formula) that you are using. Presumably, if your model accounts for past data, it should work for future data as well. We'll keep this fairly simple.

Lets say that you start with an initial condition and it starts at zero. The next data point to come out is a one. So at x = 0, y = 0 and x=1, y=1. This gives us a nice formula: x = y. We are ready to predict the future. Our guess is that when x = 2, y =2. Our graph of the function looks like this:

This provides nice steady increase. If it is a graph of your investments, you will not be getting rich very quick, but you might not be getting poor either. If it is global temperatures, it might cause some concern. If it is crop yields per square meter, then it is steady and predictable.

But when x =2 comes out, it turns out that y = 0. Our prediction was off by 2. Our graph comparing our prediction with actual results looks like this:

This looks like a simple problem to fix. We simply change our equation to y = -x^2 + 2x. This equation also works for the first three values. Our graph comparing our prediction with actual results now looks like this:

Those curves are pretty close. We seem to be on the right track. Let's expand our prediction graph and predict what is going to happen in the future:

We predict that the next point on the graph will be -3. It looks as though the graph is going increasingly downward. If this is your return on investment, then it looks like you better get out of the market now. If this is global temperatures, then stock up on winter clothes.

In fact, the next point is -1. Again, we are off by 2. Out graph comparing our prediction with actual results looks like this:

This is a not so easy fix. We change our equation to y = (x^3)/3 - 2x^2 + 8x/3. This gives us the following graph:

This is not exact but it is close. If we look down the road, we can predict the following:

So if this is our investments, we should just ride it out because things look better down the road. If it is global temperatures, then hang on because things will get a lot hotter really quick.

When the next number comes in, it comes in as 0, exactly as our model predicted:

Surely, we are on the right track.

The next number, however, comes in as 1 rather then the 5 our model predicted.

Something is wrong again. If we look at our various model graphs, we can see that they end up going all over the place:

Clearly, while each of these graphs works for a bit, they all fail in the end. They all end up flying off on a tangent. This is even more clear when we look at the long term trajectories:

All of these graphs were based on the actual data, but they differ markedly in their projections (all of which turn out to be wrong in the long term). Remember that the extreme models accounted for almost the same range of data, but after a point made widely divergent predictions.

So, one take away is that the models, at some point, break down. We could make the models much more complicated and account for the first twenty points but they would then still go wildly wrong. The general point would remain. If you are looking at a fluctuating phenomenon and suddenly your model becomes monotonically increasing or decreasing (that is, it stops fluctuating) then that is the point where your model probably has broken down.