The people I work with like to think of them as being smart, and they are. Most of all they like believing that they are right, and that they have solved important problems. The truth is a bit more complicated than that. The truth is that we know a lot less than we would like to admit. Actually we know hell of a lot less. Admitting this to us is scary.
Denial can be beautiful
But only when you’re a fantastic liar
― Kim Holden,
Still, many of us would like to convince ourselves of the opposite for the comfort it provides the soul. This is a selfish and self-serving mentality, which most clearly comes from the desire to have accomplished more than we have. Often the honesty in admitting our flawed knowledge and capability for understanding is too much to bear and we submerge it in falsehoods. This is basic human nature and it is inescapable.
Humankind cannot bear too much reality.
It seems to be a lot easier to metaphorically put our heads in the sand. A lot of the time we go to great lengths to convince ourselves of the opposite of the truth, to convince ourselves that we are the master’s of the universe. Instead we can only achieve the mastery we crave though the opposite. We should never consider our knowledge and capability to be flawless, but flawed and incomplete.
Integrity is telling myself the truth. And honesty is telling the truth to other people.
This comes up all the time when you’re doing V&V. When V&V gets into the assessment mode we constantly butt our heads into the people putting their heads in the sand. They want to think that everything is OK and mastery is at hand, and the problems that exist are under control. The reaction to finding problems is often full of emotion and anger because the truth is so unpleasant. It confronts the control of reality they have worked hard to build for themselves.
It takes strength and courage to admit the truth.
Often the control they have is local and it is really OK. For a lot of modeling the results are calibrated heavily and as long as the analysis is done close to where the calibrating data was taken there is credibility. In the long run this is a difficult circumstance that is unsustainable. The problem is that the calibrated modeling is often quite successful. 
The people applying calibrated models are often lauded as the models of success. The problems with this are deep and pernicious. We want to do much more than calibrate results, we want to understand and explore the unknown. The only way to do that is systematically uncover our failings, and shortcomings with a ken focus on exposing the limits we have. The practical success of calibrated modeling stands squarely in the way of pushing the bounds of knowledge.
A program that produces incorrect results twice as fast is infinitely slower.
–John Osterhout
A close analog to this process exists in code development. In debugging code finding a bug is to be celebrated, but one should never believe that all the bugs have been located. A more healthy and reasonable philosophy is to assume that more bugs are hiding waiting to be discovered. Anyone who thinks their code is bug-free is delusional. Such attitudes are rightly greeted with skepticism. The same skepticism should be offered to those who think their modeling is similarly “bug-free”.
Most men would rather deny a hard truth than face it.
It is so very difficult to achieve because of the human element; it is because people would rather think that things are better than they actually are. It takes leadership to overcome these issues. An environment that provides the impetus for improving modeling quality means confronting issues that make people uncomfortable. Specific measures need to be taken that reward people for finding “bugs” in the modeling capability. Finding problems in the modeling needs to be expected, instead of confirmations of mastery.
Quality means doing it right when no one is looking.
We are encouraged by everything around us to work on things that are important. Given the intrinsic differences between the messaging we are given explicitly and implicitly, its hard to really decide what’s important. Of course, if you work on what’s important you will personally make a bit more money. You really make a lot of money if you work specifically in the money making industry…
These words are spoken whenever we go into planning “reportable” milestones in virtually every project I know about. If we are getting a certain chunk of money, we are expected to provide milestones that report our progress. It is a reasoned and reasonable thing, but the execution is horribly botched by the expectations that are grafted onto the milestone. Along with the guidance in the title of this post, we are told, “these milestones must always be successful, so choose your completion criteria carefully.” Along with this we make sure that these milestones don’t contain too much risk.
The real danger in the philosophy we have adopted is the creeping intrusion of mediocrity into everything we do. Nothing is important enough to take risks with. The thoughts expressed through these words are driving a mindless march toward mediocrity, once great research institutions are being thrust headfirst into the realm of milquetoast also-rans. The scientific and engineering superiority of the United States is leaving in lockstep with every successfully completed milestone built this way.
Science depends on venturing bravely into the unknown, a task of inherent risk, and massive potential reward. The reward and risk are linked intimately; with nothing risked, nothing is gained. By making milestones both important and free of risk, we sap vitality from our work. Instead of wisely and competently stewarding the resources we are trusted with, they are squandered on work that is shallow and uninspired. Rather than being the best we can do, it becomes the thing we can surely do.
When push comes to shove, these milestones are always done, and always first in line for resource allocation. At the same time we have neutered them from the outset. The strategy (if you can call it that!) is self-defeating, and only yields the short-term benefit of the appearance of success. This appearance of success is believed to be necessary for continuing the supply of resources.
If you haven’t heard of “wicked problems” before it’s a concept that you should familiarize your self with. Simply put, a wicked problem is a problem that can’t be stated or defined without attempting to solve it. Even then your definition will be woefully incomplete. Wicked problems are recursive. Every attempt to solve the problem yields a better definition of the problem. They are the metaphorical onion where peeling back every layer produces another layer.
In code development this often takes the form of refactoring where the original design of part of the software is redone based on the experience gained through its earlier implementation. You understand the use of and form that the software should take once you’ve tried to write it (or twice or thrice or…). The point is that the implementation is better the second or third time based on the experience of the earlier work. In essence this is embracing failure in its proper role as a learning experience. A working, but ultimately failed form of the software is the best experience for producing a better piece of software.
This principle applies far more broadly to scientific endeavors. An archetypical scientific wicked problem is climate change not simply because the complexity of the scientific aspects of the problem, but also the political and cultural dynamics stirred up. In this way climate change connects back to the traditional wicked problems from the social sciences. A more purely scientific problem that is wicked is turbulence because of its enormous depth in terms of physics, engineering and math with an almost metaphysical level of intractability arising naturally. Turbulence is also connected to a wealth of engineering endeavors with immense economic consequences.
Maintaining the perspective of wickedness as being fundamental is useful as it drives home the belief that your deep knowledge is intrinsically limited. The way that experts look at V&V (or any other wicked problem) is based on their own experience, but is not “right” or “correct” in and of itself. It is simply a workable structure that fits the way they have attacked the problem over time.
This is truly sad considering the transformative potential bound up in those hopeful, unrealistic, dreams we allowed ourselves to express. We could be doing things that are magnificent; instead we withdraw to the world of the possible and bureaucratically controlled, politically viable reality. The projects we hopefully envisioned would be transformative and create a far greater future than the path we are currently on. We are told that the people in Washington can’t envision anything greater either. Perhaps they are just like us, simply unwilling to honestly voice anything greater than our currently pedestrian path.
This is why the future is so bleak; the dreams are there, but no one has faith that these dreams can be realized. Support for working on the dream is missing, why start something that will never be finished? People have recently realized that the future was supposed to bring flying cars and instead we got mini-supercomputers in our pockets (that do very little computing). Of course it doesn’t quite look like “Blade Runner” or “Minority Report” either. The problem is that it looks like the dystopian parts of those movies have a greater chance of reality than the cool parts.
that more often than not, such symmetry is missing. Quite often the value doesn’t seem to be aligned with the cost (that watch cost how much!?). The key is to look deeper, the costs and the value are aligned, but the truth is often hidden by shame. The shame is often the explicit admission that we value things that are so inherently superficial and ultimately damaging to our future.
Much of the actual value we take in items is hidden. It also matters greatly who does the evaluation. This is clearest with luxury items where people are willing pay a great deal for a brand name because of its cachet. This is a how top designer and brands make their money; people are willing to pay significant premium for having the product with a name. Examples can be found with shoes, cars, watches, and handbags among many items. It becomes a sign of distinction just to own certain 
items.
In the area of computing other opportunity costs are in evidence. We have made significant effort to make high performance computing about big computers solving meaningless benchmarks. We have defined weak scaling as a success. These have come at significant costs such as the diminishment of the efficiency of doing most of our computational work because we have failed to focus on performance at the basic node level. Our pursuit of mistaken goals in high performance computing has also driven a divestment from many important areas of algorithmic research. Perhaps the most powerful tool for effectively using computers, the algorithm, has been stripped of its vitality in the process.
The question is of supreme importance moving forward with supercomputing. The Drucker quote fairly well captures both the importance of answering the metrics question moving forward, but also consideration of what happened in the recent path. I’ve come to the conclusion that the devotion to “weak scaling” is probably doing a lot of damage to high performance computing.
solution verification/error estimation, validation, and uncertainty quantification are a good place to start, but inadequate for many projects. An example might be weather or climate modeling where data assimilation is important enough to warrant its own category. In computation of social science, the geometry is irrelevant and needs to be replaced with an appropriate description of the environment things like agents are placed in. In other cases the experimental work is sufficiently complex 
and focused that it should be expanded into far greater detail included a data focus. The point is that PCMM is not a fixed framework, but an idea of how to organize your activity as to not leave important things out.
How might I use PCMM to do something that isn’t V&V related first? Say, like writing a new code?
I would consider what the application of the code is intended to be and how much further than the original intent might be supported? How essential is the geometric fidelity to the quality of simulations? How well are the basic physical models, and supporting constitutive relations established? Is the numerical method and the equations supported by mathematical rigor? Are numerical errors well understood where the equations and method are to be applied? What experiment exist for validation, and will new validation experiments be conducted? What sort of quantities of interest are needed and how will their uncertainty be assessed? For every question how critical is the quality of the answer, and what is the level of decision to be made with the results? Might any of this change over time, and can those changes be accommodated in the desired code?
The national security-industrial complex. This is the new and improved version of the original complex. Where the military-industrial complex is sold through nationalism and the ability to exercise raw power, this complex has the selling point of irrational fear. Make no mistake; this one is coupled to the other. The activities of this complex are cloaked in secrecy and a vast unspecified (black) budget that is completely justified in the eyes of the cowering-terrified masses of voters. Americans have agreed in overwhelming numbers to part with liberty and a great deal of money in service of this complex. The true aim of this complex is control by the powerful as well as profit. Again, the threats we are facing are vastly overblown.
I would argue that the security state we are creating is more of a threat to our way of life than any of the enemies they are “protecting” us from. We have allowed ourselves to become fearful as a Nation and we will suffer a fate befitting this trend. Recently and increasingly this complex is supporting the militarized and aggressive police.
The financial-investment-industrial complex. You know who these guys are. This is the industry that almost melted down the economy five years ago. Bailouts happened and the incomes of most of the guilty were saved (if temporarily diminished slightly). Reform was promised and never delivered upon. Almost none of the shady, sleazy or corrupt practices that enriched the financial managers and preyed upon common citizens resulted in any legal action. New financial vehicles have been created to prey on the citizenry (e.g., student loans). More than nuclear weapons, this vehicle of mass destruction looms over the economy waiting for the next bubble to burst. The national-security-police apparatus was used to crust any resistence and treated the Occupy Wall Street movement like domestic terrorists. So much for democracy.
This complex has the money to buy immense favor, does almost nothing for society, and unlike the others produces only rich people. Somehow they have conned to electorate into believing that they create economic opportunity. Once finance did this, now all the opportunity is in the hands of the “shareholders” who are mostly financial managers. Increasingly business interests is just code for ability for executives to be compensated in repugnantly extravagant ways. Much of the deplorable situation with education can be traced to student loans as investment vehicles along with the concentration of wealth cascading to enhanced support of a cadre of private universities that serve the rich. No other complex is so singularly responsible for the erosion of the middle class as this one.
The medical-industrial complex. This complex is the two-edged sword of overwhelming good on one side and horrific greed on the other. Medicine does enormous good and healing people is a worthy profession. Too bad it has become the cover for a massive plundering operation. Medicine in the USA consumes nearly to over twice the amount of GDP than it does in other comparable Nations. In return for this massive cost we get substandard care. Our health outcomes and quality of care is actually lower than comparable nations. We get a lousy deal and the medical industry makes even more money.
The criminal-justice-prison-industrial complex. The embarrassment we should be feeling over the incarnation rate of our citizens is absent. Do we really feel that the American public is so awful that they should be imprisoned at a rate two to ten times larger than comparable countries? Are Americans inherently violent and criminal in our conduct? Is that the way we are exceptional? The answer to all these questions is no. The problem is that power and money flow from locking up Americans. It has also become a convenient way of reinstituting “Jim Crow” laws.
This all began with the failed war on drugs. Ultimately this produced a bunch of people to imprison. Eventually the prisons became privatized and a way to make money. Prisons had a lobby. At the same time law enforcement found ways of increasing their power through the same war. Increased weaponry, increased surveillance, increased powers of seizure. All of this coupled together to produce an orgy of imprisonment, and a deluge of cash to industry. This has been amplified by the rise of the national security industrial complex that provides an additional source of enhanced power.
The Regularized Singularity 