THE SCIENCE BEHIND CHATELET
Uncertainty in physical problems
In the pristine world of abstraction, the laws of physics can be written down as equations. By carefully manipulating the math, physicists and engineers are able to describe millions of physical systems. Given a few input parameters, such as the initial velocity, position and material properties of objects and the substrates through which they move, mathematical modeling makes it possible to predict the future, at least in terms of how systems are expected to evolve. It also provides tools for analyzing the past and present, because if the parameters of an equation can be found that make its predictions match actual data, then conclusions can be drawn about the structure of the system, such as what the substrate through which objects are moving must be made of. One way of testing a hypothesis is to use it, through an equation, to predict the future and see how well it pans out. The ability to fit equations to data in order to understand systems and make predictions is probably the most important tenet of modern analytical science.
But it breaks down. The real world is much more complicated than physical abstractions. Consider the problem of breaking a set of billiard balls: theoretically, if you know the exact position of all of the balls, their mass and diameter and the initial position and velocity of the cue ball, then it should be possible to compute the exact time and location that every ball will hit the side of the table and where it will go from there. In fact, there are many computer simulations which will do so convincingly. The problem is that they are not real. Far too many things are going on to make a simulation exactly match a true experiment. Like, for instance, the effect of inhomogeneity (bumps!) in the felt covering of the table on the trajectory of balls. In reality, balls will never arrive exactly where they are supposed to. The typical way of dealing with this problem is to say that, in fact, it is impossible to get the prediction absolutely right. There is going to be uncertainty on the inputs into the system (the precise path of the cue ball will never be exactly known) and uncertainty on the outputs (balls are not going to arrive on time, and even if they did there is no way to measure their arrival to infinite precision). But everyone still trusts the laws of conservation of mass and momentum.
At the same time, it is obvious that these laws aren’t enough. Not to push a point, but if quantum mechanics is invoked, there is a miniscule possibility that one of the balls might even tunnel through the edge of the table and arrive on the floor. But that’s ridiculous. And this is a set of billiard balls. Imagine how hard it is to build a mathematical model which completely predicts how the energy of a Magnitude 8 earthquake and all of its aftershocks will travel through six thousand miles of the interior of the earth! Even with a hundred years of data to work with.
Admitting then that physics is largely insufficient to describe the real world, even if it were possible to put together a perfect mathematical model for a complicated system, computing a single simulation might take days or weeks. High end simulations of 3D commercial seismic experiments can take a month. If you are going to try this, you’d better make sure that all of your inputs are exactly right. And we know they can’t be.The other problem is that often the crux of the problem is not to run a simulation assuming complete knowledge of the problem setup, but to try to go backwards through a system to figure out what the interior parameters of the equation must have been in order to create an outcome that was measured. (Given that the balls hit the side of the table at such and such a time, what was the initial shot?) This information is used to either study the system, or to try to figure out what is going to happen next, or both. There are a number of problems with this approach. One is that most physical equations are partial differential equations (often coupled together) which cannot be solved using simple algebra. Another is that there are usually a large number of variables involved that play against each other so it is difficult to separate the effect of one from the effect of another. Lastly, and probably the most troublesome fact, is that in most interesting situations there are a nearly infinite number of combinations of input variables that lead to almost the exact same output and the only way to sort between them is to have some kind of knowledge, a priori, of what the right answer should be.
Enter the world of probabilistic inference. As its name suggests, it is made up of two concepts. The first is that the world is best described not just in terms of equations, but also in terms of deviations from those equations which can be rigorized as probability distributions (so the word probabilistic). The second is inference: that is, answers must be arrived at not purely by “inversion”, that is, going backward through equations, but by reasoning in the face of evidence. Evidence is, of course, data – and now we’ll talk about reasoning.
Physicists and engineers are used to the idea that the inputs into an equation are often uncertain. One of the ways to get around this is to run through the equation (or the program which computes it) multiple times with inputs that are drawn at random from initial (prior) probability distributions. The result is a suite of outputs (samples) which represent the likely outcomes of the system (a posterior distribution). This is fine, if you are willing to believe that uncertainty only lies in the inputs and outputs of a system. What if it lies in the description of the system itself? What if some parts of the system (such as how many bumps in the felt a ball is likely to encounter) can only realistically be described probabilistically? What then? Is it OK to build that uncertainty directly into uncertainty in the output? But what if the bumps in the felt interact directly with slight differences in the diameter of the balls? How do you sort all of that out?
It doesn’t have to be a hopelessly deadend (or endless) problem. Our job at Chatelet Resources is to build computer models which explicitly insert uncertainty into physical relationships at the same level as the mathematical physics. We assume that physics itself is a little bit slippery. Instead of trying to force it into explaining every possible phenomenon, we let it bend to accommodate real world deviations from abstraction. We put our effort into precisely calibrating probability distributions which capture those deviations (albeit only in statistical form) and trying to understand how different causes of deviation affect each other. This allows us to capture the full richness of the system and, as a side benefit, to do so using simple math. We know that simplified equations are wrong (billiard balls are not point masses), but we compensate for that by learning how they tend to behave.
Instead of a single equation (or set of coupled equations), we model physical systems using a probabilistic Knowledge Representation that is made up of simplified physical equations coupled to probability distributions that are learned from real data. In this way, we can combine academic book learning (what should happen) with real world experience (what actually does).
We build a Knowledge Representation (KR) by first consulting “domain experts,” that is, real live people who have years of experience in a field and who understand both the physics and the ways in which it tends to break down. Once we have mocked up the KR, we do a series of statistical tests using historical data to validate our assumptions, in which process factors are correlated with each other and the probability distributions that best describe their interactions. Finally, we “train” the model using as much data as we can find to calibrate it. The result is a specific Generative Model which can be applied to future data. The model is called “generative” because in the next step it is used to generate and evaluate the probability of millions of hypothetical scenarios.
Once a Generative Model is established, it has two jobs. The first is to extract a reasonable set of initial conditions (such as the velocity of the cue ball) from its interior probability distributions and run it all the way through to a final output (set of balls hitting the sides of the table), drawing randomly when necessary (the number of bumps in the felt which each ball might encounter). Its second job is to be able to evaluate the probability of any particular outcome given a particular set of initial conditions. (If the cue ball was shot this way, how likely is this pattern of arrivals?).Answering these questions requires inference, that is, the process of coming to a conclusion based on evidence and reasoning. “Evidence” in this context means a particular set of data that needs to be explained, often to support a forecast. “Reasoning” is done by building an Inference Algorithm which is tightly coupled to the Generative Model. The inference algorithm uses the Generative Model to propose hypothesis (What if the cue were shot like this? – cue balls are often shot this way) and then evaluate the probability of the data in light of the hypothesis. (No way, it is very unlikely to see five sinks with that lousy shot!) There are, of course, tricks, but basically the algorithm searches around, drawing hypothesis and either accepting or rejecting them at a rate that eventually becomes proportional to their true probability of occurrence. The final result is a “posterior” distribution on the initial shot (it probably happened like this, but could have been a few degrees to the right or left) as well as a set of posterior distributions on the interior happenings of the model (ball 16 almost always hits three felt bumps).
An answer like this, from a well calibrated model is much more powerful than a single solution to an equation. Or even a suite of solutions arrived at assuming uncertainty on the inputs and outputs of a simulation but not the simulation itself. It is powerful because a) it is more likely to be correct! b)it lends itself to a depth of understanding that is complete and realistic and c)all of the assumptions that were called into play to build the model are transparent and can be re-evaluated as necessary.
Each of the companies under the Chatelet Umbrella uses its own adaptation of this kind of thinking to solve a problem that has high financial, scientific or humanitarian value and for which traditional modeling and analysis have proved insufficient.