In part two of this series I discussed Bayesian inference. Specifically, I discussed how Bayesian inference provided us with a mechanism for deciding in what we should place our confidence given all the information we possess and will yet obtain. This was all framed in the context of confidence. I’d like to discuss an alternative way of looking at Bayesian inference – namely optimization.
Optimization
Optimization largely rules our world. Virtually all of management, engineering, politics, and much of science is about optimization. Optimization, in this sense, is the process of determining the optimal solution given all the objectives and constraints. In management, the process may not be that formal. Perhaps there is a board of directors who gather around a table to discuss the optimal set of policies, the direction to go, etc. In politics it is likely similar. The President of the U.S. surrounds himself with experts on a particular topic, they then engage in discussion, and hope to land on the optimal answer given the objectives and constraints.
For engineers the process is much more formal and precise. Usually optimization takes the form of a cost function – a function incorporating, mathematically, all the objectives and constraints. An algorithm (and there are many) is then employed to “solve” the function resulting in the optimal solution. To demonstrate, here’s an example:
Suppose you are designing an aircraft. There are numerous design possibilities, a canard style, V-tail empennages, aspect ratio of the wings, length of fuselage, coating of the surfaces, wingtips, where to place the turbines, height of vertical stabilizer (if having one at all), etc. etc. We would like to find the optimal answer amongst all these parameters such that we maximize lift, maximize cargo space, maximize safety, minimize energy consumption, etc. Of course we also have constraints. We cannot physically manufacture a flexible fixed wing that is 800 ft long and thin as a toothpick. To solve the problem, we can write down a big, long, nasty equation that would mathematically characterize the physics, constraints, and objectives and then pick our favorite optimization algorithm and wait for it to churn out the answer (which may take a long time).
Bayesian inference is one algorithm that can be applied to such an optimization problem. Typically one would choose this algorithm amidst a cost function that was stochastic in nature, having noise and/or error in the system, that expressed our confidence.
Finding the Truth, Optimization Style
In some sense, the Bayesian inference mechanism I discussed in previous posts could be seen as an optimization method for finding the truth. If we assume that all the new information we regularly encounter has some (even if very little) truth therein, and we apply that information in the regular Bayesian inference sense, we could then reliably conclude that we have found the “truth,” with some probability (level of confidence), given all the information.
This is highly related to a comment FireTag made on my previous post. He asked
So there are routes to evolve our beliefs toward truth no matter where we start or whatever the order of our search algorithm?
In the context of this question, Bayesian inference can easily be seen as a search algorithm. And, in fact, if we used a Sequential Monte Carlo method, it really does feel like a search algorithm.
In expanding this notion, my response, in part, was:
Absolutely (at least in my book)! Though certainly some search algorithms are definitely worse than others and some starting places better than others! Otherwise what prayer in the world do we have (unless you’re absolutely certain that YOU’VE got it right, but I sure don’t)? I view my religion/spirituality as a compass that (I hope) points me in a good direction. My hope is that if/when the absolute truth is made manifest to me I will be humble enough (and my definition of humble is “openness to the truth”) to recognize it because/in spite of my current confidence distribution.
From this perspective, we might view the church (or whatever church you belong to), the Gospel, this life, and all our associated experience as tools to help us optimize for, and draw nearer to the truth given the objectives and constraints of our personal limitations and the limitations of this mortal existence. While I have encountered a very few number of Mormons who claim that we have ALL the truth, this is not the claim of the LDS church. Most of us, I believe, accept there are things we don’t yet know and don’t yet understand. The real challenge is to have an appropriate confidence distribution such that you will accept that truth when it is made known to you.
However, I finished my response to FireTag with the following:
However, this really opens another can of worms – namely, what is truth? My explanation thus far has been about our perception of truth which may or may not correlate with objective or absolute truth. To argue over whether or not our perception of truth is objective truth is to argue over what forms of evidence are acceptable and what weight we should apply to that evidence (which is the conclusion of this post and is an argument with no victor).
Comments 10
re are numerous design possibilities, a canard style
I was assured that such a design is the only optimum approach. 😉
#1 — If you have enough computing power in the avionics to take care of the instability, then maybe.
Guess that is why Burt Rutan’s number one engineer was a numbers guy 🙂 Saab seems to do it without computing power in the avionics though.
jmb:
Good points, and I’d also like to think about the design optimization problem from the point of view of the Great Designer. What is the optimal design for Him to adopt if one of His constraints/goals is to let the “wings”, “airfoils”, “pilot consoles”, and “air traffic control systems” optimize to their goals independently of Him?
“From this perspective, we might view the church (or whatever church you belong to), the Gospel, this life, and all our associated experience as tools to help us optimize for, and draw nearer to the truth given the objectives and constraints of our personal limitations and the limitations of this mortal existence.”
Thanks for this thought-provoking post. As an educator, manager, and minister, I related to what you said about optimizing.
I also think there is possibility to increase understanding and reduce conflict when we look at disagreements between groups/tribes as differences in tools used to optimize rather than gigantic divides where one tribe then feels the need to wipe out or treat badly another tribe. This is the difference in knowing that while we may own the best tool that we know of to get to optimize our ability to know and live the truth, we don’t “own” the truth. In a world where different religions fight with one another or where different kinds of people are marginalized this understanding could bring some healing.
With that said, I think there is something to be said about optimization as it pertains to Mystery. We don’t have the “big picture,” or the “whole truth” because God is bigger than we understand. So, I think learning how to listen to Spirit is all the more important because the Spirit can coordinate the optimization process much better than we can given our human limitations. Otherwise, we fall into the Elephant Trap–two people are blindfolded and not told what they are about to touch. One touches the tail of the elephant, the other touches the back. They are then asked to describe the elephant. Because they have different experiences with the reality of the elephant, their descriptions will differ unless they “see” beyond the part they are touching to the whole of what they are touching.
Re Stephen and Thomas
Well, it depends on the type of the canard and obviously the objectives (though I know you were kidding). Some canards actually provide lift, others just act as a horizontal stabilizer (normally on the empennage). There is nothing inherently unstable about a canard configuration. Most of the stability in aircraft design comes from the design of the main wings and empennage. The tradeoffs are very involved from a design perspective, but Burt Rutan seems to favor them. The one big advantage, however, is that if a canard style stalls, the canard stalls first sending the nose of the aircraft down which helps it regain speed and pull out of the stall. Most fighter jets have control canards.
Re FireTag
A VERY good point. I would say I think it very much a possibility that the optimum design for him would be exactly what we’ve got in this world. However, I would also interpret that to mean there is likely much more flexibility in the requirements than we give him credit for. This doesn’t sit very well with claims about the “one true way” etc.
Re TH
I couldn’t agree more. This is exactly the kind of understanding and conclusion I was going for. Well said.
To sum all of this up, I have a lovely quote. I wish I had remembered this quote earlier I would have included it in the post. It is from Ling-Pei Kung, an influential engineer.
JMB 275:
Wonderful quote. Now embed that in the notion that what you describe is a Russian doll inside other Russian dolls, interacting as each other’s external influences (and interanl “genetic” influences). Then assume the Russian dolls also have kalaidoscope properties, so that no matter how you view the entities within you see the same infinite complexity expressed in infinite ways — physically, spiritually, politically, mathematically, economically, artistically, and templates even the angels imagine as dimly as we imagine angels. Life everywhere, everyway.