In working on an algorithm (or a code) we are well advised to think carefully about requirements and priorities. It is my experience that these can be stated clearly as a set of adjectives about the method or code that form the title of this post. Moreover the order of these adjectives forms the order of the priorities from the users of the code. The priorities of those funding code development are quite often the direct opposite!

There is nothing quite so useless as doing with great efficiency something that should not be done at all.

– Peter Drucker

None of these priorities can be ignored. For example if the efficiency becomes too poor, the code won’t be used because time is money. A code that is too inaccurate won’t be used no matter how robust it is (these go together, with accurate and robust being a sort of “holy grail”).

 Extraordinary claims require extraordinary evidence.

― Carl Sagan

The problem I’d like to raise your attention to be that the people handing out money to do the work seem inverts these priorities. This creates a distinct problem with making the work useful and impactful. The over-riding concern is the high-performance computing imperative, which is encoded in the call for efficiency. There is an operating assumption that all of the other characteristics are well in hand, and its just a matter of getting a computer (big and) fast enough to crush our problems out of sight and out of mind. Ironically, the meeting devoted to this dim-sighted worldview is this week, SC14. Thankfully, I’m not there.

All opinions are not equal. Some are a very great deal more robust, sophisticated and well supported in logic and argument than others.

― Douglas Adams

Robust. A robust code runs to completion. Robustness in its most refined and crudest sense is stability. The refined sense of robustness is the numerical stability that is so keenly important, but it is so much more. It gives an answer come hell or high water even if that answer is complete crap. Nothing upsets your users more than no answer; a bad answer is better than none at all. Making a code robust is hard work and difficult especially if you have morals and standards. It is an imperative.

Physical. Getting physical answers is often thought of as being equal to robustness, and it should be. It isn’t so this needs to be a separate category. In a sense physical answers are a better source of robustness, that is, an upgrade. An example is that the density of a material must remain positive-definite, or velocities remain sub-luminal, and things like that. A deeper take on physicality involves quantities staying inside bounds imposed, or satisfaction of the second law of thermodynamics.

Robustness and physicality of solutions with a code are ideally defined by the basic properties of an algorithm. Upwind differencing is a good example where this happens (ideally). The reality is that achieving these goals usually takes a lot of tests and checks with corrective actions where the robustness or physicality is risked or violated. This makes for ugly algorithms and ugly codes. It makes purists very unhappy. Making them happy is the place where the engineering becomes science (or math).

A computational study is unlikely to lead to real scientific progress unless the software environment is convenient enough to encourage one to vary parameters, modify the problem, play around.

– Nick Trefethen

Flexible. You can write your code to solve one problem, if the problem is important enough, and you solve it well enough it might be a success. If your code can solve a lot of problems robustly (and physically) you might be even more successful. A code that is a jack-of-all-trades is usually a huge success. This requires a lot of thought, experience and effort to pull off. This usually gets into issues of meshing, material modeling, input, output, and generality in initial and boundary conditions. Usually the code gets messy in the process.

All that it is reasonable to ask for in a scientific calculation is stability, not accuracy. – Nick Trefethen

Accuracy. This is where things get interesting. For the most part accuracy is in conflict with every single characteristic we discuss. It is a huge challenge to both ends of the spectrum to robustness and efficiency. Accuracy makes codes more prone to failure (loss of robustness) and the failures have more modes. Accuracy is expensive and makes codes run longer. It can either cause more communication, or more complexity (or both). It makes numerical linear algebra harder and far more fragile. It makes boundary conditions, and initial conditions harder and encourages much more simplification of everything else. If it can be achieved accuracy is fantastic, but it is still a huge challenge to pull of.

The fundamental law of computer science: As machines become more powerful, the efficiency of algorithms grows more important, not less.

– Nick Trefethen

Efficiency. The code runs fast and uses the computers well. This is always hard to do, a beautiful piece of code that clearly describes an algorithm turns into a giant plate of spaghetti, but runs like the wind. To get performance you end up throwing out that wonderful inheritance hierarchy you were so proud of. To get performance you get rid of all those options that you put into the code. This requirement is also in conflict with everything else. It is also the focus of the funding agencies. Almost no one is thinking productively about how all of this (doesn’t) fit together. We just assume that faster supercomputers are awesome and better.

Q.E.D.

God help us.

 

A secret’s worth depends on the people from whom it must be kept.

― Carlos Ruiz Zafan

Secrecy is a necessary element in the conduct of National Security. Some information is either too dangerous to too many to be allowed to be freely shared. It needs to be effective. The mis-classification of information is a threat because it undermines the cases where classification is necessary and appropriate.

Secrecy also carries a massive cost that should be factored into the consideration. When something is secret is isn’t subject to the sort of review and consideration that public information is. As a result aspects like the quality of the related work suffer, or achieving quality is more expensive. The United States kept a large number of documents classified for decades and hidden from view in spite of having no classified information and having great historical value. Other documents were classified solely due to their embarrassing nature to the government. Some of these were the same.

Man is not what he thinks he is, he is what he hides.

― André Malraux

It isn’t a secret that the United States has engaged in a veritable orgy of classification since 9/11. What is less well known is the massive implied classification through other data categories such as “official use only (OUO)”. This designation is itself largely unregulated as such is quite prone to abuse.

The smart way to keep people passive and obedient is to strictly limit the spectrum of acceptable opinion, but allow very lively debate within that spectrum….

― Noam Chomsky

OUO is also used to manage things like export control. Despite its importance, the law managing export control is terrible. It is poorly written, poorly managed, and its application is primarily driven by fear rather than rational thought. It is too important to treat this way, and I believe its implementation is actually a threat to our security. It might be useful to describe explicitly the ways that secrecy and classification are abused. Some of these sins are a nuisance, and some of them border on unethical, immoral or illegal.

The ones with no imagination are always the quickest to justify themselves

― Haruki Murakami

Legitimacy (pride). Some information seeks legitimacy through being declared classified in some manner. For example, shoddy work can be implied to be legit through its classification.

It is in the nature of the human being to seek a justification for his actions.

― Aleksandr Solzhenitsyn,

Importance (envy). Some information or the work related to the information is implied to be more important because it is classified. I see this a lot. It is a way of making the case that your work is so important that it needs to be protected.

I have grown to love secrecy. It seems to be the one thing that can make modern life mysterious or marvelous to us. The commonest thing is delightful if only one hides it.

― Oscar Wilde

Hiding (greed). Some information is deliberately hidden through classification, the words to remember here are “need to know”. This is used to hide things that people don’t want too many eyes on. When I encounter this it disgusts me.

It is almost always the cover-up rather than the event that causes trouble.

–Howard Baker

Cover-up (lust). Fortunately, I have not seen this form in person, but it often involves things that are embarrassing or illegal. You’ve seen this in the news, I’m sure it happens a lot more than we think.

 Withholding information is the essence of tyranny. Control of the flow of information is the tool of the dictatorship.

― Bruce Cavil

Control (wrath). This is a common use of classification. It is a valid reason in many cases, but it is also misused when it is applied to keep communication from happening. Another place where the words “need to know” appear.

Safety (gluttony). This is the “cover your ass” version of classification. Basically, you’re not sure so you declare it to be classified because it’s the “safe” thing to do. At some level there isn’t anything wrong with this if you rectify the situation promptly. In many areas there is clear guidance that allows a better final determination to be made. In other areas, like OUO there is no clear guidance, and the safety sin reigns through the fear associated with awfully written laws.

The awareness of the ambiguity of one’s highest achievements – as well as one’s deepest failures – is a definite symptom of maturity.

― Paul Tillich

Ambiguity (sloth). This goes to the heart of the management of classified documents. In the example of export control we have allowed ambiguity and resultant fear to rule for years. There is no standard and no clear guidance. As a result the application of classification is uneven and ultimately self-conflicting.  

A blank canvas…has unlimited possibilities.

― Stephanie Perkins

Again something at work has inspired me to write. It’s a persistent theme among authors, artists and scientists regarding the concept of the fresh start (blank page, empty canvas, original idea). I think its worth considering how truly “fresh” these really are. This idea came up during a technical planning meeting where one of participants viewed this new project as being offered a blank page.

Are we every really offered a blank page? Or is there more to it?

Once we stepped over that threshold, conflict erupted over the choices available with little conclusion. A large part of the issue was the axioms each person was working with. Across the board we all took a different set of decisions to be axiomatic. At some time in the past these “axioms” were choices, and became axiomatic through success. Someone’s past success becomes the model for future success, and the choices that led to that success become unstated decisions we are generally completely unaware of. These form the foundation of future work and often become culturally iconic in nature.

Take the basic framework for discretization as an operative example: at Sandia this is the finite element method; at Los Alamos it is finite volumes. At Sandia we talk “elements”, at Los Alamos it is “cells”. From there we continued further down the proverbial rabbit hole to discuss what sort of elements (tets or hexes). Sandia is a hex shop, causing all sorts of headaches, but enabling other things, or simply the way a difficult problem was tackled. Tets would improve some things, but produce other problems. For some ,the decisions are flexible, for others there isn’t a choice, the use of a certain type of element is virtually axiomatic. None of these things allows a blank slate, all of them are deeply informed and biased toward specific decisions of made in some cases decades ago.

It’s so fine and yet so terrible to stand in front of a blank canvas.

– Paul Cezanne

 

If the decision was a success it stands a chance of ultimately becoming axiomatic. If it wasn’t a success, you probably don’t even know about it much less the details of what went wrong. Failures fade away after a few years. The institutional memory is crumbling right in front of us. One might even say that our Labs are developing a sort of organizational Alzheimer’s. For this reason the low risk path is to follow in the footsteps of the successes. In some cases the failure was idiosyncratic or not remotely related to the choice, but other effects. This leads to deep sustained problems with progress, and the options available to deal with deep problems.

 

Creativity is always a leap of faith. You’re faced with a blank page, blank easel, or an empty stage.

– Julia Cameron

 

The other day I risked a lot by comparing the choices we’ve collectively made in the past as “original sin”. In other words what is computing’s original sin? Of course this is a dangerous path to tread, but the concept is important. We don’t have a blank slate; our choices are shaped, if not made by decisions of the past. We are living, if not suffering due to decisions made years or decades ago. This is true in computing as much as any other area.

 

Human material existence is limited by ideas, not by stuff

–Steven Pinker

The key to the original advances in computing was first the drive to use computers to solve important problems. Once important problems were solved, the method of solution provided the proof it could be done. Others could then in good faith follow in those footsteps and build upon that experience. At Los Alamos in the 40’s and early 50’s this happened in a chain of Von Neumann, to Richtmyer to Lax and Harlow. Each person’s work built upon the others progress. One might consider Von Neumann to have worked with a blank page, but he was building upon the work of Richardson as well as Courant, Friedrichs and Lewy. Their work was itself based upon the efforts of Newton, Gauss, Hilbert and countless others.

 

You might not write well every day, but you can always edit a bad page. You can’t edit a blank page.

– Jodi Picoult

 

The key is that we are all shaped by history and the success (and failures) of the past. We are shaped by our culture and biases. We are shaped by whom we meet and what we experience. The blank page is merely a vehicle for us to produce a record of this influence. Most of the time we aren’t even conscious of all the implicit decisions we commit in creative process.

 

Habits

Good habits are worth being fanatical about.

― John Irving

 In case you’re wondering about my writing habit and blog. I can explain a bit more. If you aren’t, stop reading. In the sense of authorship I force myself to face the blank page every day as an exercise in self-improvement. I read Charles Durhigg’s book “Habits” and realized that I needed better habits. I thought about what would make me better and set about building them up. I have a list of things to do every day, “write” “exercise” “walk” “meditate” “read” and so on.

 

We become what we repeatedly do.

― Sean Covey

 

The blog is a concrete way of putting the writing to work. Usually, I have an idea the night before, and draft most of the thoughts during my morning dog walk (dogs make good motivators for walks). I still need to craft (hopefully) coherent words and sentences forming the theme. The blog allows me to publish the writing with a minimal effort, and forces me to take editing a bit more seriously. The whole thing is an effort to improve my writing both in style and ease of production.

 

A man who can’t bear to share his habits is a man who needs to quit them.

― Stephen King

 

The topics are things that I’m working on, or thinking about, or simply pisses me off. It’s a way of working them out in more detail and trying to produce a logical structure for the thought process. My wife thinks its good because I don’t bug her with this “shit” anymore, but in all honesty it just makes room for different “shit” to bug her about!

 

 

I much prefer the sharpest criticism of a single intelligent man to the thoughtless approval of the masses.

― Johannes Kepler

For some reason I’m having more “WTF” moments at work lately. Perhaps something is up, or I’m just paying attention to things. Yesterday we had a discussion about reviews, and people’s intense desire to avoid them. The topic came up because there have been numerous efforts to encourage and amplify technical review recently. There are a couple of reasons for this, mostly positive, but a tinge of negativity lies just below the surface. It might be useful to peel back the layers a bit and look at the dark underbelly.

The pleasure of criticizing takes away from us the pleasure of being moved by some very fine things.

― Jean de La Bruyère

First, the reasons for an increased emphasis on reviews should be examined. It turns out that most of the problems are lying on the surface. The general assumption is that peer review is one of the cornerstones of quality in science. It is a powerful mechanism for communication in both directions; the reviewers are experts you’d like to promote your ideas with, and the reviewers usually have something useful to say to you, at least if they are doing it right. Nonetheless, as many of you know, peer review can be emotionally draining, and painful to go through. A second, less positive aspect is the organizational desire to escape embarrassment from either shoddy or fraudulent work, which should be smoked out via peer review. A third aspect that also comes from the “dark side” of peer review is a sort of smoke and mirrors of using it to craft a veneer of due diligence and implied quality (more on this later). These are reviews that are mandated by organizations, and of course, such a mandate shouldn’t be needed to have this happen, but as such the mandate is actually a sign that a problem exists. Technical organizations should “know” that technical review is essential to its fundamental health.

I touched on this topic a couple of weeks ago (https://williamjrider.wordpress.com/2014/11/02/why-does-vv-get-me-in-trouble/), but classic peer review has many problems. Some of these are due to abuse of the anonymous nature and inadequate policing of this abuse by editorial boards. The difficult part of the review for the reviewer isn’t the critique as most papers always have weaknesses, but rather balancing it with appropriate praise. In all honesty, I personally struggle with this. It is the balance between doing a fair and complete job of reviewing while not being unfairly harsh in criticism. Despite my conscious efforts to deal with the problem, I probably fail to hit the mark. As I note later, no one ever taught me how to do a review; I discovered it through osmosis.

A more difficult topic is the organizational imperative to avoid embarrassment. Mandated reviews are a terrible way to handle this problem, and a terrible reason for reviews. The need to have work reviewed should flow from the basic duty of scientists to communicate their work to peers and receive feedback. The mandated review for the purpose of ferreting out fraud or garbage is unnecessarily confrontational. It puts a negative spin on the entire topic of review. The real core of the issue is management, which should be the responsible agent in knowing what it going on in the first place. A review isn’t a police action, and using it as such undermines the purpose of review in subtle and pernicious ways. Given the sorry state of peer review, these are hits it can’t take.

 People ask you for criticism, but they only want praise.

― W. Somerset Maugham

The biggest problems with peer reviews are “bullshit reviews”. These are reviews that are mandated by organizations for the organization. These always get graded and the grades have consequences. The review teams know this thus the reviews are always on a curve, a very generous curve. Any and all criticism is completely muted and soft because of the repercussions. Any harsh critique even if warranted puts the reviewers (and their compensation for the review at risk). As a result of this dynamic, these reviews are quite close to a complete waste of time.

Because of the risk associated with the entire process, the organizations approach the review in an overly risk-averse manner, and control the whole thing. It ends up being all spin, and little content. Together with the dynamic created with the reviewers, the whole thing spirals into a wasteful mess that does no one any good. Even worse, the whole process has a corrosive impact on the perception of reviews. They end up having no up side; it is all down side and nothing useful comes out of them. All of this even though the risk from the reviews has been removed through a thoroughly incestuous process.

Don’t criticize what you can’t understand.

― Bob Dylan

An element in the overall dynamic is the societal image of external review as a sideshow meant to embarrass. The congressional hearing is emblematic of the worst sort of review. The whole point is grandstanding and/or destroying those being reviewed. Given this societal model, it is no wonder that reviews have a bad name. No one likes to be invited to their own execution.

When virtues are pointed out first, flaws seem less insurmountable.

― Judith Martin

What can be done about this? The answers are simple, but complex within the environment we find ourselves. First of all, people should be trained or educated in conducting, accepting and responding to reviews. Despite its importance to the scientific process, we are never trained how to conduct, accept or responds to a review (response happens a bit in a typical graduate education). Today, it is a purely experiential process. Next, we should stop including the scoring of reviews in any organizational “score”. Instead the quality of the review including the production or hard-hitting critique should be expected as a normal part of organizational functioning.

It’s easy to attack and destroy an act of creation. It’s a lot more difficult to perform one.

― Chuck Palahniuk

People, projects and organizations willing and capable of undergoing honest, critical review are usually much better than those who aren’t. The unwilling or softball review is itself is a better indicator of problems than a negative review itself. Peer review is essential for science and we must fix it. It is an essential element in our quality process that we cannot afford to remain completely broken.

 

I’m a progressive. In almost every way that I can imagine, I favor progress over the status quo. This is true for science, music, art, and literature, among other things. The one place where I tend to be status quo are work and personal relationships that form the foundation for my progressive attitudes. These foundations are formed by several “social contracts” that serve to define the roles and expectations. Without this foundation, the progress I so cherish is threatened because people naturally retreat to conservatism for stability.

Without deviation from the norm, progress is not possible.

― Frank Zappa

What I’ve come to realize is that the shortsighted, short is demolishing many of these social contracts –term thinking dominating our governance. Our social contracts are the basis of trust and faith in our institutions whether they are the rule of government, or the place we work. In each case we are left with a severe corrosion of the intrinsic faith once granted these cornerstones of public life. The cost is enormous, and may have created a self-perpetuating cycle of loss of trust precipitating more acts that undermine trust.

 Stagnation is self-abdication.

― Ryan Talbot

Take the Labs where I’ve worked. At one time the Lab’s were trusted with the (nuclear) defense of the Nation. This happened in a time of immense threat and danger, yet the oversight was minimal. A substantial resource was given to the Labs to pursue the mission, and the Labs performed marvelously. The Labs fulfilled their social contract with the Nation, and similarly the Labs created a social contract with its employees. Serve here, and we will take care of you. You will be given engaging work, paid well, and ultimately allowed to retire comfortably. Shape your scientific explorations in service of the National security mission, and you will be provided resources. Beyond the direct success in the nuclear work, the scientific work was part of the Nation’s preeminence internationally and produced much of the foundation for great economic success. We have almost systematically destroyed everything good about these Labs.

It takes strength and courage to admit the truth.

―Rick Riordan

Even before the Cold War ended, this social contract began to unravel. The trust eroded and the money came with increasing strings attached. Similarly, the social contract with the employees became too “expensive” to fulfill. Over time the lack of trust and the associated “accountability” has spiraled out of control (we will spend ten dollars to save one). This has precipitated the no risk, no failure allowed environment that is choking innovation and progress out of our work. Increasingly the support for a career at the Labs is being removed, and it’s turning into just another job (not a bad one, but nothing special either).

These developments are paralleled by changes across the economy. They are manifestations of the short-term quarterly return mentality ruling industry. Research and development without immediate impact on the bottom line are increasingly missing from industrial research (missing from government research too). Employees are commodities whose life and career prospects is none of the business concern. The Labs benchmark themselves to these industries and share these attitudes because it benefits the short-term balance sheet.

What gets lost? Almost everything. Progress, quality, security, you name it. Our short-term balance sheet looks better, but our long-term prospects look dismal. The scary thing is that these developments help drive conservative thinking, which in turn drives these developments. As much as anything this could explain our Nation’s 50-year march to the right. We have taken the virtuous cycle we were granted, and developed a viscous cycle. It is a cycle that we need to get out of before it crushes our future.

Any defensiveness is a sign of failure. You can’t move forward if you are defensive.

― Bryant McGill,

We got here through overconfidence and loss of trust can we get out of it by combining realism with trust in each other. Right now, the signs are particularly bad with nothing looking like realism, or trust being part of the current public discourse on anything.  

The people I work with like to think of them as being smart, and they are. Most of all they like believing that they are right, and that they have solved important problems. The truth is a bit more complicated than that. The truth is that we know a lot less than we would like to admit. Actually we know hell of a lot less. Admitting this to us is scary.

Denial can be beautiful

But only when you’re a fantastic liar

― Kim Holden,

Still, many of us would like to convince ourselves of the opposite for the comfort it provides the soul. This is a selfish and self-serving mentality, which most clearly comes from the desire to have accomplished more than we have. Often the honesty in admitting our flawed knowledge and capability for understanding is too much to bear and we submerge it in falsehoods. This is basic human nature and it is inescapable.

Humankind cannot bear too much reality.

― T.S. Eliot

It seems to be a lot easier to metaphorically put our heads in the sand. A lot of the time we go to great lengths to convince ourselves of the opposite of the truth, to convince ourselves that we are the master’s of the universe. Instead we can only achieve the mastery we crave though the opposite. We should never consider our knowledge and capability to be flawless, but flawed and incomplete.

Integrity is telling myself the truth. And honesty is telling the truth to other people.

― Spencer Johnson

This comes up all the time when you’re doing V&V. When V&V gets into the assessment mode we constantly butt our heads into the people putting their heads in the sand. They want to think that everything is OK and mastery is at hand, and the problems that exist are under control. The reaction to finding problems is often full of emotion and anger because the truth is so unpleasant. It confronts the control of reality they have worked hard to build for themselves.

It takes strength and courage to admit the truth.

― Rick Riordan

Often the control they have is local and it is really OK. For a lot of modeling the results are calibrated heavily and as long as the analysis is done close to where the calibrating data was taken there is credibility. In the long run this is a difficult circumstance that is unsustainable. The problem is that the calibrated modeling is often quite successful. The people applying calibrated models are often lauded as the models of success. The problems with this are deep and pernicious. We want to do much more than calibrate results, we want to understand and explore the unknown. The only way to do that is systematically uncover our failings, and shortcomings with a ken focus on exposing the limits we have. The practical success of calibrated modeling stands squarely in the way of pushing the bounds of knowledge.

A program that produces incorrect results twice as fast is infinitely slower.

–John Osterhout

A close analog to this process exists in code development. In debugging code finding a bug is to be celebrated, but one should never believe that all the bugs have been located. A more healthy and reasonable philosophy is to assume that more bugs are hiding waiting to be discovered. Anyone who thinks their code is bug-free is delusional. Such attitudes are rightly greeted with skepticism. The same skepticism should be offered to those who think their modeling is similarly “bug-free”.

Most men would rather deny a hard truth than face it.

― George R.R. Martin

It is so very difficult to achieve because of the human element; it is because people would rather think that things are better than they actually are. It takes leadership to overcome these issues. An environment that provides the impetus for improving modeling quality means confronting issues that make people uncomfortable. Specific measures need to be taken that reward people for finding “bugs” in the modeling capability. Finding problems in the modeling needs to be expected, instead of confirmations of mastery.

Quality means doing it right when no one is looking.

― Henry Ford

This is your life and its ending one moment at a time.
― Chuck Palahniuk,

We are encouraged by everything around us to work on things that are important. Given the intrinsic differences between the messaging we are given explicitly and implicitly, its hard to really decide what’s important. Of course, if you work on what’s important you will personally make a bit more money.  You really make a lot of money if you work specifically in the money making industry…

Where I’ve worked, the explicit message is that nuclear weapons are important. Science for nuclear weapons should be the sweet spot, but its not really. Part of the problem is that science sells itself to scientists. They actually have to pay us less to get us to do science. On the other hand, the implicit message for all of society is money is important. Nuclear weapon’ labs are no different, money is the most important things there too. Its become pretty clear that the same superficial value system present society-wide is at work there too.

At a personal level you also like to do interesting work. Sometimes interesting and challenging are equated. You get into science because it is both interesting and you’re good at it.  Someone will pay you to do something you’re good at and is interesting. You convince yourself that it is good for society, good for your Nation, and the right thing to do. As you get older, some of these things start to come into question.

The messages you begin to get implicitly start to undermine all those things you convinced yourself of. Interesting work keeps you sane; important work keeps you paid. Maybe there isn’t much more that you can hope for.

Nothing of me is original. I am the combined effort of everyone I’ve ever known.
― Chuck Palahniuk

 

These words are spoken whenever we go into planning “reportable” milestones in virtually every project I know about. If we are getting a certain chunk of money, we are expected to provide milestones that report our progress. It is a reasoned and reasonable thing, but the execution is horribly botched by the expectations that are grafted onto the milestone. Along with the guidance in the title of this post, we are told, “these milestones must always be successful, so choose your completion criteria carefully.” Along with this we make sure that these milestones don’t contain too much risk.

That would be dangerous.

People who don’t take risks generally make about two big mistakes a year. People who do take risks generally make about two big mistakes a year.

― Peter F. Drucker

Dangerous for whom, I wonder? Dangerous for what reason I ask?

 The desire for safety stands against every great and noble enterprise.

― Tacitus

The real danger in the philosophy we have adopted is the creeping intrusion of mediocrity into everything we do. Nothing is important enough to take risks with. The thoughts expressed through these words are driving a mindless march toward mediocrity, once great research institutions are being thrust headfirst into the realm of milquetoast also-rans. The scientific and engineering superiority of the United States is leaving in lockstep with every successfully completed milestone built this way.

There is no discovery without risk and what you risk reveals what you value.

― Jeanette Winterson

Science depends on venturing bravely into the unknown, a task of inherent risk, and massive potential reward. The reward and risk are linked intimately; with nothing risked, nothing is gained. By making milestones both important and free of risk, we sap vitality from our work. Instead of wisely and competently stewarding the resources we are trusted with, they are squandered on work that is shallow and uninspired. Rather than being the best we can do, it becomes the thing we can surely do.

Never was anything great achieved without danger.

― Niccolò Machiavelli

When push comes to shove, these milestones are always done, and always first in line for resource allocation. At the same time we have neutered them from the outset. The strategy (if you can call it that!) is self-defeating, and only yields the short-term benefit of the appearance of success. This appearance of success is believed to be necessary for continuing the supply of resources.

The long-term cost is systematic atrophy of the core identity of the Labs as centers of excellence. Today the Labs are being transformed into centers of compliance. No one can solely take the blame for this, it is the handiwork of the entire system.

You cannot swim for new horizons until you have courage to lose sight of the shore.

― William Faulkner

 

 

 

If I had an hour to solve a problem I’d spend 55 minutes thinking about the problem and 5 minutes thinking about solutions.

― Albert Einstein

If you haven’t heard of “wicked problems” before it’s a concept that you should familiarize your self with. Simply put, a wicked problem is a problem that can’t be stated or defined without attempting to solve it. Even then your definition will be woefully incomplete. Wicked problems are recursive. Every attempt to solve the problem yields a better definition of the problem. They are the metaphorical onion where peeling back every layer produces another layer.

Wicked problems were introduced as a way of defining the sorts of issues dealt with in the social sciences. There the human element provides a source of infinite complexity. Issues such a poverty and race relations are the archetype of wicked problems. Virtually every response to solving the original problem creates more problems itself as well as informing future approaches. The concept defines the difficulty in providing well-defined, finite solutions that are simple and easy to apply.

The formulation of the problem is often more essential than its solution, which may be merely a matter of mathematical or experimental skill.

― Albert Einstein

There are ten classical characteristics of wicked problems to consider in deciding if your problem is indeed “wicked”. Defining the wicked problem is difficult to impossible.

2. You don’t know when to stop solving the wicked problem (because its never solved, just solved well enough, or you simply run out of time).
3. No proposed solution can be complete, but can be defined in terms of quality (good or bad, good enough).
4. Testing solutions to the wicked problem is inconclusive and limited.
5. Every solution to the wicked problem to a problem has consequences and implications. These are unpredictable.
6. The set of potential solutions to the wicked problem do not form a well-defined or bounded list.
7. Each wicked problem is different.
8. Each wicked problem is connected to other problems.
9. There are multiple explanations for the causes of wicked problems. Some of these explanations are conflicting.
10. Wicked problems have consequences if things are wrong, thus solutions are expected to be right the first time. Failures are punished due to consequences.

Over time we have come to realize that science and technology has its own cadre of wicked problems. I first encountered the concept in code development. Steve McConnell made the observation that code development is wicked in his classic book “Code Complete”. You can’t even anticipate all the problems you’ll have to solve in developing a code until you start to code. Deep aspects of the code will unveil themselves in the process of implementation and testing.

In code development this often takes the form of refactoring where the original design of part of the software is redone based on the experience gained through its earlier implementation. You understand the use of and form that the software should take once you’ve tried to write it (or twice or thrice or…). The point is that the implementation is better the second or third time based on the experience of the earlier work. In essence this is embracing failure in its proper role as a learning experience. A working, but ultimately failed form of the software is the best experience for producing a better piece of software.

We can not solve our problems with the same level of thinking that created them

― Albert Einstein

This principle applies far more broadly to scientific endeavors. An archetypical scientific wicked problem is climate change not simply because the complexity of the scientific aspects of the problem, but also the political and cultural dynamics stirred up. In this way climate change connects back to the traditional wicked problems from the social sciences. A more purely scientific problem that is wicked is turbulence because of its enormous depth in terms of physics, engineering and math with an almost metaphysical level of intractability arising naturally. Turbulence is also connected to a wealth of engineering endeavors with immense economic consequences.

This brings me to the practice of verification and validation with uncertainty quantification. This is most certainly wicked as well. One of the clear signs of wickedness is the PCMM (Predictive Capability Maturity Model). PCMM has evolved continuously toward ever-greater complexity as experience grows without any limits in sight. Other frameworks for wrangling the complexity of V&V exist with entirely different structures.

We easily start to see the entire engagement with V&V is inherently wicked. The more you do V&V and solve problems using its principles, the more complex and deep the topic seems. This makes V&V appear to be more daunting than it actually is. Experts in V&V with a deep knowledge have difficulty in pulling themselves back to a novice’s perspective. Ultimately this is counter-productive as it makes the topic difficult to approach once it matures.

Another aspect of wicked problems that manifests with V&V is cultural and political consequences. V&V intersects with the culture of engineering and science in aggressive ways. V&V pushes the cultural norms of conducting science and engineering. Ultimately V&V collides with the political side of things with decision-making or resource allocation. Once this occurs you’re down the proverbial rabbit’s hole, and complexity explodes. You will then understand intuitively that you have found a wicked problem.

Maintaining the perspective of wickedness as being fundamental is useful as it drives home the belief that your deep knowledge is intrinsically limited. The way that experts look at V&V (or any other wicked problem) is based on their own experience, but is not “right” or “correct” in and of itself. It is simply a workable structure that fits the way they have attacked the problem over time.

 I suppose it is tempting, if the only tool you have is a hammer, to treat everything as if it were a nail.

― Abraham Maslow

It is more shameful to distrust our friends than to be deceived by them.

― Confucius

I’ve continued to think about that planning session a couple of days ago. It was utterly stunning that a group of people could simultaneously come up with the same idea of what needed to be done, and be totally convinced that it was impossible. Not technically impossible, but culturally, and practically impossible due to political constraints. Let’s be clear, the solution we came up with would be transformative to how we execute our mission. It would drive meaningful use and need for the cutting edge (exascale) computing, we are blamed for not having. Despite all of this, it seems unthinkable to even attempt it.

Success is most often achieved by those who don’t know that failure is inevitable.

― Coco Chanel,

How can this be? I think it boils down to trust, or more properly the lack of it. We have become a low trust society and the current trends seem to be pushing us to even lower levels of confidence and faith in our fellow citizens. Democrats don’t trust Republicans and vice versa, white doesn’t trust black, young and old lack trust for each other. The rich don’t trust the poor, and even trustworthy professions like medicine, science and engineering aren’t trusted. We imprison a massive portion of our population to a point that exceeds the rate everywhere else on Earth. Everywhere we look the evidence points to little confidence or faith in the best intentions of our fellow citizens.

The implicit costs of all of this lack of trust are immense. I’m only seeing a small amount of it in the lack of high-minded risk-taking encouraged in science. People won’t engage in anything risky or overly challenging because there isn’t any faith that they won’t be skewered if they don’t deliver on promises. Ultimately everyone is afraid of appearing in some congressional hearing and being made into a political football to be kicked around.

The fear-based politics and shame baiting does everything to make things worse. The ever-growing evidence that our democracy is devolving into oligarchical, kleptocratic system of privilege over substance only drives a deep-seated feeling of untrustworthiness. Given the Orwellian nightmare unfolding with surveillance of every moment of our lives coupled with the capacity of drones to kill people at will makes every untrusting thought seem more rational.

At one time, we trusted massive resources to people to accomplish big things. They were accountable for doing the best job possible to achieve the goals. The difference with today is that we hadn’t broken down every step, every month-week-day of effort into an accounting superstructure designed to track every single action. Just because we can easily create a system to account for something doesn’t make it a good idea. The impact of this is to be a straightjacket on every effort, and assure that we accomplish much less rather than just breakeven. That’s right, I would be happy if we just stopped losing ground. Every year we become less productive.

If we could muster the courage to trust each other, we would be able to craft a much better, much happier, much richer future.

 The best way to find out if you can trust somebody is to trust them.

― Ernest Hemingway