We are losing the ability to understand anything that’s even vaguely complex.

― Chuck Klosterman

I get asked, “what do you do?” quite often in conversation, and I realize the truth needs to be packaged carefully for most people. One of my issues is that advertise what I do on my body with some incredibly nerdy tattoos including an equation that describes one form of the second law of thermodynamics. What I do is complex and highly technical full of incredible subtlety. Even when talking with someone from a nearby technical background the subtlety of approximating physical laws numerically in a manner suitable for computing can be daunting. For someone without a technical background it is positively alien. This character comes to play rather acutely in the design and construction of research programs where complex, technical and subtle does not sell. This is especially true in today’s world where expertise and knowledge is regarded as suspicious, dangerous and threatening to so many. In today’s world one of the biggest insults to hurl at some one is to accuse them of being one of the “elite”. Increasingly it is clear that this isn’t just an American issue, but Worldwide in its scope. It is a clear and present threat to a better future.

I’ve written often about the sorry state of high performance computing. Our computing programs are blunt and naïve constructed to squeeze money out of funding agencies and legislatures rather then get the job done. The brutal simplicity of the arguments used to support funding is breathtaking. Rather than construct programs to be effective and efficient getting the best from every dollar spent, we construct programs to be marketed at the lowest common denominator. For this reason something subtle, complex and technical like numerical approximation gets no play. In today’s world subtlety is utterly objectionable and a complete buzz kill. We don’t care that it’s the right thing to do, or that it is massively greater in return than simply building giant monstrosities of computing. It would take an expert from the numerical elite to explain it, and those people are untrustworthy nerds, so we will simply get the money to waste on the monstrosities instead. So here I am, an expert and one of the elite using my knowledge and experience to make recommendations on how to be more effective and efficient. You’ve been warned.

Truth is much too complicated to allow anything but approximations.

— John Von Neumann

If we want to succeed at remaining a high performance computing superpower, we need change our approach and fast. Part of what is needed is a greater focus on numerical approximation. This is part of deep need to refocus on the more valuable aspects of the scientific computing ecosystem. The first thing to recognize is that our current hardware first focus is oriented on the least valuable part of the ecosystem, the computer itself. A computer is necessary, but horribly insufficient for high performance computing supremacy. The real value for scientific computing is the opposite end of the spectrum where work is grounded in physics, engineering and applied mathematics.

Although this may seem a paradox, all exact science is dominated by the idea of approximation.

— Bertrand Russell

I’ve made this argument before and it is instructive to unpack it. The model solved via simulation is the single most important aspect of the simulation. If the model is flawed, no amount of raw computer speed, numerical accuracy, or efficient computer code can rescue the solution and make it better. The model must be changed, improved, or corrected to produce better answers. If a model is correct the accuracy, robustness, fidelity and efficiency of its numerical solution is essential. Everything upstream of the numerical solution aimed toward the computer hardware is less important. We can move down the chain of activities all of which are necessary seeing the same effect, the further you get from the model of reality, the less efficient the measures are. This whole thing is referred to an ecosystem these days and every bit of it needs to be in place. What also needs to be in place is a sense of the value of each activity, and priority placed toward those that have the greatest impact, or the greatest opportunity. Instead of doing this today, we are focused on the thing with least impact, farthest from reality and starving the most valuable parts of the ecosystem. One might argue that the hardware is a subject of opportunity, but the truth is the opposite. The environment for improving the performance of hardware is at a historical nadir; Moore’s law is dead, dead, dead. Our focus on hardware is throwing money at an opportunity that has passed into history.

I’m a physicist, and we have something called Moore’s Law, which says computer power doubles every 18 months. So every Christmas, we more or less assume that our toys and appliances are more or less twice as powerful as the previous Christmas.

— Michio Kaku

At some point, Moore’s law will break down.

— Seth Lloyd

There is one word to describe this strategy, stupid!

At the core of the argument is a strategy that favors brute force over subtleties understood mainly by experts (or the elite!). Today the brute force argument always takes the lead over anything that might require some level of explanation. In modeling and simulation the esoteric activities such as the actual modeling and its numerical solution are quite subtle and technical in detail compared to the raw computing power that can be understood with ease by the layperson. This is the reason the computing power gets the lead in the program, not because of its efficacy in improving the bottom line. As a result our high performance-computing world is dominated by meaningless discussions of computing power defined by a meaningless benchmark. The political dynamics is basically a modern day “missile gap” like we had during the Cold War. It has exactly as much virtue as the original “missile gap”; it is a pure marketing and political tool with absolutely no technical or strategic validity aside from its ability to free up funding.

Each piece, or part, of the whole of nature is always merely an approximation to the complete truth, or the complete truth so far as we know it. In fact, everything we know is only some kind of approximation because we know that we do not know all the laws as yet.

— Richard P. Feynman

Once you have an entire program founded on bullshit arguments, it is hard to work your way back to technical brilliance. It is easier to double down on the bullshit and simply define everything in terms of the original fallacies. A big part of the problem is the application of modern verification and validation to the process. Both verification and validation are modern practices to accumulate evidence on the accuracy, correctness and fidelity of computational simulations. Validation is the comparison of simulation with experiments and in this comparison the relative correctness of models is determined. Verification determines the correctness and accuracy of the numerical solution of the model. Together the two activities should help energize high quality work. In reality most programs consider them to be nuisances and box checking exercises to be finished and ignored as soon as possible. Programs like to say they are doing V&V, but don’t want to emphasize or pay for doing it well. V&V is a mark of quality, but the programs want its approval rather than attend to its result. Even worse, if the results are poor or indicate problems, they are likely to be ignored or dismissed as being inconvenient. Programs get away with this because the practice of V&V is technical and subtle and in the modern world highly susceptible to bullshit.

Far better an approximate answer to the right question, which is often vague, than an exact answer to the wrong question, which can always be made precise.

— John W. Tukey

Numerical methods for solving models are even more technical and subtle. As such they are the focus of suspicion and ignorance. For high performance computing today they are considered to be yesterday’s work and largely a finished, completed product now simply needing a bigger computer to do better. In a sense this notion is correct, the bigger computer will produce a better result. The issue is that using the computer power, as the route to improvement is inefficient under the best of circumstances. We are not living under of the best of circumstances! Things are far from efficient, as we have been losing the share of computer power advances useful for modeling and simulation for decades now. Let us be clear, when we receive an ever-smaller proportion of the maximum computing power as each year passes. Thirty years ago we would commonly get 10, 20 or even 50 percent of the peak performance of the cutting edge supercomputers. Today even one percent of the peak performance is exceptional, and most codes doing real application work are significantly less than that. Worse yet, this dismal performance is getting worse with every passing year. This is one element of the autopsy of Moore’s law that we have been avoiding while its corpse rots before us.

So we are prioritizing improvement in an area where the payoffs are fleeting and suboptimal. Even these improvements are harder and harder to achieve as computers become ever more parallel and memory access costs become ever more extreme. Simultaneously we are starving more efficient means of improvement of resources and emphasis. Numerical methods and algorithms are two key areas not getting any significant attention or priority. Moreover support for these areas is actually diminishing so that support for the inefficient hardware path can be increased. Let’s not mince words; we are emphasizing a crude naïve and inefficient route to improvement at the cost of a complex and subtle route that is far more efficient and effective.

Numerical approximations and algorithms are complex and highly technical things poorly understood by non-experts even if they are scientists. The relative merits of one method or algorithm compared to another is difficult to articulate. The merits and comparison is highly technical and subtle. Since creating new methods and algorithms makes progress, this means improvements are hard to explain and articulate to non-experts. In some cases both methods and algorithms can produce breakthrough results and produce huge speed-ups. These cases are easy to explain. More generally a new method or algorithm produces subtle improvements like more robustness or flexibility or accuracy than the older options. Most of these changes are not obvious, but making this progress over time leads to enormous improvements that swamp the progress made by faster computers.

An expert is someone who knows some of the worst mistakes that can be made in his subject, and how to avoid them.

― Werner Heisenberg

The huge breakthroughs are far and few between but provide much greater value than any hardware over similar periods of time. To get these huge breakthroughs requires continual investment in research for extended periods of time. For much of the time the research is mostly a failure producing small or non-existent improvements, until they don’t. Without the continual investment, the failure and the expertise failure produces, the breakthroughs will not happen. They are mostly serendipitous and the end product of many unsuccessful ideas. Today the failures and lack of progress is not supported; we exist in a system where insufficient trust exists to support the sort of failure needed for progress. The result is the addiction to Moore’s law and its seemingly guaranteed payoff because it frees us from subtlety.

Often a sign of expertise is noticing what doesn’t happen.

― Malcolm Gladwell

A huge aspect of expertise is the taste for subtlety. Expertise is built upon mistakes and failure just as basic learning is. Without the trust to allow people to gloriously make professional mistakes and fail in the pursuit of knowledge, we cannot develop expertise or progress. All of this lands heavily on the most effective and difficult aspects of scientific computing, the modeling and solution of the models numerically. Progress on these aspects is both highly rewarding in terms of improvement, and very risky being prone to failure. To compound matters progress is often highly subjective itself needing great expertise to explain and be understood. In an environment where the elite are suspect and expertise is not trusted such work is unsupported. This is exactly what we see, the most important and effective aspects of high performance computing are being starved in favor of brutish and naïve aspects, which sell well. The price we pay for our lack of trust is an enormous waste of time, money and effort.

Wise people understand the need to consult experts; only fools are confident they know everything.

― Ken Poirot

Again, I’ll note that we still have so much to do. Numerical approximations for existing models are inadequate and desperately in need of improvement. We are burdened by theory that is insufficient and heavily challenged by our models. Our models are all flawed and the proper conduct of science should energize them to improve.

…all models are approximations. Essentially, all models are wrong, but some are useful. However, the approximate nature of the model must always be borne in mind… [Co-author with Norman R. Draper]

— George E.P. Box

The fundamental law of computer science: As machines become more powerful, the efficiency of algorithms grows more important, not less.

― Nick Trefethen

Modern modeling and simulation is viewed as a transformative technology for science and engineering. Invariably the utility of modeling and simulation is grounded on the solution of models via numerical approximations. The fact that numerical approximation is the key to unlocking its potential seems largely lost in the modern perspective, and engaged in any increasingly naïve manner. For example much of the dialog around high performance computing is predicated on the notion of convergence. In principle, the more computing power one applies to solving a problem, the better the solution. This is applied axiomatically and relies upon a deep mathematical result in numerical approximation. This heritage and emphasis is not considered in the conversation to the detriment of its intellectual depth.

Where all think alike there is little danger of innovation.

― Edward Abbey

At this point, the mathematics and specifics of numerical approximation is then systematically ignored by the dialog. The impact of this willful ignorance is felt across the modeling and simulation world, a general lack of progress and emphasis on numerical approximation is evident. We have produced a situation where the most valuable aspect of numerical modeling is not getting focused attention. People are behaving as if the major problems are all solved and not worthy of attention or resources. The nature of the numerical approximation is the second most important and impactful aspect of modeling and simulation work. Virtually all the emphasis today is on the computers themselves based on the assumption of their utility in producing better answers. The most important aspect is the modeling itself; the nature and fidelity of the models define the power of the whole process. Once a model has been defined, the numerical solution of the model is the second most important aspect. The nature of this numerical solution is most dependent on the approximation methodology rather than the power of the computer.

The uncreative mind can spot wrong answers, but it takes a very creative mind to spot wrong questions.

― Anthony Jay

People act as if the numerical error is so small as not to be important on one hand, while encouraging great focus on computing power where the implicit reasoning for the computing power is founded on reducing numerical error. To make matters worse with this corrupt logic, the most effective way to reduce numerical error is being starved for attention and resources having little or no priority. The truth is that numerical errors are still too large, and increasing computing power is lousy way and inefficient to make them smaller. We are committed to a low-risk path that is also highly inefficient because the argument is accessible to the most naïve people in the room.

What is important is seldom urgent and what is urgent is seldom important.

― Dwight D. Eisenhower

Another way of getting to the heart of the issue is the efficacy of using gains in computer power to get better solutions. Increases in computing power are a terrible way to produce better results; it is woefully inefficient. One simply needs to examine the rate of solution improvement based on scaling arguments. First, we need to recognize that practical problems converge quite slowly in terms of the application of enhanced computational resources. For almost any problem of true real world applicability, high-order convergence (higher than first-order) is never seen. Generally we might expect solutions to improve at first-order with the inverse of mesh size. If we look at three dimensional, time dependent problems and we want to halve the numerical error, we need to apply at least 16 times the computing power. Usually convergence rates are less than first order, so the situation is actually even worse. As a result we are investing an immense amount in progressing in an incredibly inefficient manner, and starving more efficient means of progress. To put more teeth on the impact of current programs, the exascale initiative wants to compute things fifty times better, which will only result is reducing errors by slightly more than one half. So we will spend huge effort and billions of dollars in making numerical errors smaller by half. What an utterly shitty return on investment! This is doubly shitty when you realize that so much more could be done to improve matter by other means.

The first thing we need to recognize for progress is relative efficacy of different modes of investment. The most effective way to progress in modeling and simulation are better models. Better models require work on theory and experiment with deeply innovative thinking based on inspiration and evidence of limitations of current theory and modeling. For existing and any new models the next step is solving the models numerically. This involves detailed and innovative numerical approximations of the models. The power of modeling and simulation with computers is predicated on the ability to solve complex models that cannot be understood analytically (or analytically without severe restrictions or assumptions). The fidelity of the numerical approximations is the single most effective way to improve results once modeling errors have been addressed. Numerical approximations can make a huge difference in the accuracy of simulations far more effectively than computer power.

Don’t tell me about your effort. Show me your results.

― Tim Fargo

So why are we so hell bent on investing in a more inefficient manner of progressing? Our mindless addiction to Moore’s law providing improvements in computing power over the last fifty years for what in effect has been free for the modeling and simulation community.

Our modeling and simulation programs are addicted to Moore’s law as surely as a crackhead is addicted to crack. Moore’s law has provided a means to progress without planning or intervention for decades, time passes and capability grows almost if by magic. The problem we have is that Moore’s law is dead, and rather than moving on, the modeling and simulation community is attempting to raise the dead. By this analogy, the exascale program is basically designed to create zombie computers that completely suck to use. They are not built to get results or do science, they are built to get exascale performance on some sort of bullshit benchmark.

This gets to the core of the issue, our appetite for risk and failure. Improving numerical approximations is risky and depends on breakthroughs and innovative thinking. Moore’s law has sheltered the modeling and simulation community from risk and failure in computing hardware for a very long time. If you want innovation you need to accept risk and failure; innovation without risk and failure simply does not happen. We are intolerant of risk and failure as a society, and this intolerance dooms innovation literally strangling it to death in its crib. Moore’s law allowed progress without risk, as if it came for free. The exascale program will be the funeral pyre for Moore’s law and we are threatening the future of modeling and simulation with our unhealthy addiction to it.

If failure is not an option, then neither is success.

― Seth Godin

There is only one thing that makes a dream impossible to achieve: the fear of failure.

― Paulo Coelho

The key thing to realize about this discussion is that improving numerical approximations is risky and highly prone to failure. You can invest in improving numerical approximations for a very long time without any seeming progress until one gets a quantum leap in performance. The issue in the modern world is the lack of predictability to such improvements. Breakthroughs cannot be predicted and cannot be relied upon to happen on a regular schedule. The breakthrough requires innovative thinking and a lot of trial and error. The ultimate quantum leap in performance is founded on many failures and false starts. If these failures are engaged in a mode where we continually learn and adapt our approach, we eventually solve problems. The problem is that it must be approached as an article of faith, and cannot be planned. Today’s management environment is completely intolerant of such things, and demands continual results. The result is squalid incrementalism and an utter lack of innovative leaps forward.

Civilizations… cannot flourish if they are beset with troublesome infections of mistaken beliefs.

― Harry G. Frankfurt

What is the payoff for methods improvement?

If we improve a method we can achieve significantly better results without a finer computational mesh. This results in a large saving in computational cost as long as the improved method isn’t too expensive. As I mentioned before one needs 16 times the computational resources to knock error down by half for a 3-D time dependent calculation. If I produce a method with half the error, it can be more efficient if it is less than 16 times as expensive. In other word, the method can use 16 times the computational resource and still be more efficient. This is a lot of headroom to work with!

The most dangerous ideas are not those that challenge the status quo. The most dangerous ideas are those so embedded in the status quo, so wrapped in a cloud of inevitability, that we forget they are ideas at all.

― Jacob M. Appel

For some cases the payoff is far more extreme than these simple arguments. The archetype of this extreme payoff is the difference between first and second order monotone schemes. For general fluid flows, second-order monotone schemes produce results that are almost infinitely more accurate than first-order. The reason for this stunning claim are acute differences in the results comes from the impact of the form of the truncation error expressed via the modified equations (the equations solved more accurately by the numerical methods). For first-order methods there is a large viscous effect that makes all flows laminar. Second-order methods are necessary for simulating high Reynolds number turbulent flows because their dissipation doesn’t interfere directly with the fundamental physics.

As technology advances, the ingenious ideas that make progress possible vanish into the inner workings of our machines, where only experts may be aware of their existence. Numerical algorithms, being exceptionally uninteresting and incomprehensible to the public, vanish exceptionally fast.

― Nick Trefethen

We don’t generally have good tools for numerical error approximation in non-standard (or unresolved) cases. One digestion of one of the key problems is found in Banks, Aslam, Rider where sub-first-order convergence is described and analyzed for solutions of a discontinuous problem for the one-way wave equation. The key result in this paper is the nature of mesh convergence for discontinuous or non-differentiable solutions. In this case we see sub-linear fractional order convergence. The key result is a general relationship between the convergence rate and the formal order of accuracy for the method, , which is . This comes from the analysis of the solution to the modified equation including the leading order truncation error. For nonlinear discontinuous solutions, the observed result is first-order where one establishes a balance between the regularization and the self-steepening in shock waves. At present there is no theory of what this looks like theoretically. Seemingly this system of equations could be analyzed as we did for the linear equations. Perhaps this might provide guidance for numerical method development. It would seemingly be worthy progress if we could analyze such systems more theoretically providing a way to understand actual accuracy.

Another key limitation of existing theory is chaotic solutions classically associated with turbulent or turbulent-like flows. These solutions are extremely (perhaps even infinitely) sensitive to initial conditions. It is impossible to get convergence results for point values, and the only convergence is for integral measures. These measures are generally convergent very slowly and they are highly mesh-dependent. This issue is huge in high performance computing. One area of study is measure-valued solutions where convergence is examined statistically. This is a completely reasonable approach for convergence of general solutions to hyperbolic PDE’s.

The much less well-appreciated aspect comes with the practice of direct numerical simulation of turbulence (DNS really of anything). One might think that having a DNS would mean that the solution is completely resolved and highly accurate. They are not! Indeed they are not highly convergent even for integral measures. Generally speaking, one gets first-order accuracy or less under mesh refinement. The problem is the highly sensitive nature of the solutions and the scaling of the mesh with the Kolmogorov scale, which is a mean squared measure of the turbulence scale. Clearly there are effects that come from scales that are much smaller than the Kolmogorov scale associated with highly intermittent behavior. To fully resolve such flows would require the scale of turbulence to be described by the maximum norm of the velocity gradient instead of the RMS.

If you want something new, you have to stop doing something old

― Peter F. Drucker

When we get to the real foundational aspects of numerical error and limitations, we come to the fundamental theorem of numerical analysis. For PDEs it only applies to linear equations and basically states that consistency and stability is equivalent to convergence. Everything is tied to this. Consistency means you are solving the equations in a valid and correct approximation, stability is getting a result that doesn’t blow up. What is missing is the theoretical application to more general nonlinear equations along with deeper relationships to accuracy, consistency and stability. This theorem was derived back in the early 1950’s and we probably need something more, but there is no effort or emphasis on this today. We need great effort and immensely talented people to progress. While I’m convinced that we have no limit on talent today, we lack effort and perhaps don’t develop or encourage the talent to develop appropriately.

Beyond the issues with hardware emphasis, today’s focus on software is almost equally harmful to progress. Our programs are working steadfastly on maintaining large volumes of source code full of the ideas of the past. Instead of building on the theory, methods, algorithms and idea of the past, we are simply worshiping them. This is the construction of a false ideology. We would do far greater homage to the work of the past if we were building on that work. The theory is not done by a long shot. Our current attitudes toward high performance computing are a travesty, and embodied in a national program that makes the situation worse only to serve the interests of the willfully naive. We are undermining the very foundation upon which the utility of computing is built. We are going to end up wasting a lot of money and getting very little value for it.

We now live in a world where counter-intuitive bullshitting is valorized, where the pose of argument is more important than the actual pursuit of truth, where clever answers take precedence over profound questions.

― Ta-Nahisi Coates

Margolin, Len G., and William J. Rider. “A rationale for implicit turbulence modelling.” International Journal for Numerical Methods in Fluids 39, no. 9 (2002): 821-841.

Grinstein, Fernando F., Len G. Margolin, and William J. Rider, eds. Implicit large eddy simulation: computing turbulent fluid dynamics. Cambridge university press, 2007.

Banks, Jeffrey W., T. Aslam, and William J. Rider. “On sub-linear convergence for linearly degenerate waves in capturing schemes.” Journal of Computational Physics 227, no. 14 (2008): 6985-7002.

Fjordholm, Ulrik S., Roger Käppeli, Siddhartha Mishra, and Eitan Tadmor. “Construction of approximate entropy measure-valued solutions for hyperbolic systems of conservation laws.” Foundations of Computational Mathematics (2015): 1-65.

Lax, Peter D., and Robert D. Richtmyer. “Survey of the stability of linear finite difference equations.” Communications on pure and applied mathematics 9, no. 2 (1956): 267-293.

 

There is a cult of ignorance in the United States, and there has always been. The strain of anti-intellectualism has been a constant thread winding its way through our political and cultural life, nurtured by the false notion that democracy means that ‘my ignorance is just as good as your knowledge.

― Isaac Asimov

On Saturday I participated in the March for Science in downtown Albuquerque along with many other marches across the World. This was advertised as a non-partisan event, but to anyone there it was clearly and completely partisan and biased. Two things united the people at the march: a philosophy of progressive and liberalism and opposition to conservatism and Donald Trump. The election of a wealthy paragon of vulgarity and ignorance has done wonders for uniting the left wing of politics. Of course, the left wing in the United States is really a moderate wing, made to seem liberal by the extreme views of the right. Among the greater proponents of the left wing are science as an engine of knowledge and progress. The reason for this dichotomy is the right wing’s embrace of ignorance, fear and bigotry as its electoral tools. The right is really the party of money and the rich with fear, bigotry and ignorance wielded as tools to “inspire” enough of the people to vote against their best (long term) interests. Part of this embrace is a logical opposition to virtually every principle science holds dear.

The premise that a march for science should be non-partisan is utterly wrong on the face of it; science is and has always been a completely political thing. The reasoning for this is simple and persuasive. Politics is the way human beings settle their affairs, assign priorities and make decisions. Politics is an essential human endeavor. Science is equally human in its composition being a structured vehicle for societal curiosity leading to the creation of understanding and knowledge. When the political dynamic is arrayed in the manner we see today, science is absolutely and utterly political. We have two opposing views of the future, one consistent with science favoring knowledge and progress, with the other inconsistent with science favoring fear and ignorance. In such an environment science is completely partisan and political. To expect things to be different is foolish and naïve.

The essence of Conservatism is fear: fear of what is not understood, fear of change, fear of the new, fear of what’s different, fear of the other.

― Gene Pozniak

One of the key things to understand is that science has always been a political thing although the contrast has been turned up in recent years. The thing driving the political context is the rightward movement of the Republican, which has led to their embrace of extreme views including religiosity, ignorance and bigotry. Of course, these extreme views are not really the core of the GOP’s soul, money is, but the cult of ignorance and anti-science is useful in propelling their political interests. The Republican Party has embraced extremism in a virulent form because it pushes its supporters to unthinking devotion and obedience. They will support their party without regard for their own best interests. The republican voter base hurts their economic standing in favor of policies that empower their hatreds and bigotry while calming their fear. All forms of fact and truth have become utterly unimportant unless they support their world-view. The upshot is the rule of a political class hell bent on establishing a ruling class in the United States composed of the wealthy. Most of the people voting for the Republican candidates are simply duped by their support of extreme fear, hate and bigotry. The Democratic Party is only marginally better since they have been seduced by the same money leaving voters with no one to work for them. The rejection of science by the right will ultimately be the undoing of the Nation as other nations will eventually usurp the United States militarily and economically.

Increasingly science and progress are rejected because they invariantly upset the status quo. When the rich and powerful make the rules and have all the advantage, any perturbation in the balance of society must be resisted. The social structure and the system of laws are already setup in service to the ruling class. Any change is viewed as a danger. Science in all its forms is a threat to power. The only science that

is favored should serve the needs of the powerful. Increasingly the science supported by the ruling class is carefully controlled and obsessively applied in nature. Pure research and acquisition of knowledge is dangerous and feared because its results cannot be controlled. Pure research has the potential to upset the order the powerful have carefully established. This order is an engine to enrich them and further tap into the wealth of society for their own benefit. Science is political because it is an engine of progress, and progress is the enemy of the powerful.

The South, which is peopled with ardent and irascible beings, is becoming more irritated and alarmed.

― Alexis de Tocqueville

In a deep way the latest election is simply the culmination of forty years of political evolution (I can’t help but note the irony of using evolution to describe Republicans!). Corruption, hatred and abuse of power are a common thread for this entire time. The foundation of this change happened under Richard Nixon whose “Southern Strategy” allowed the GOP to embrace the toxic legacy of bigotry that defines the old Confederacy as its own brand. Along with the change of the GOP into the embodiment of Confederate values came a tidal wave of ignorance ushered in by Ronald Reagan whose generally unintellectual approach is revered by the Republicans almost as if he was a demi-God. In this way Donald Trump is almost the perfect Republican, corrupt and hateful to the core like Nixon, ignorant and uneducated like Reagan with the mantle of wealth their voters admire. We now see a political movement completely devoted to fear, hatred and ignorance as their brand. They wear these values with pride and attack the elite whosevalues of progress, love and knowledge are viewed as weakness. In this lens it is no wonder that science is rejected.

Ignorance is a progressive thinker’s enemy; acceptance, awareness, accountability and action are the tools of the informed; of people who want change and do all they can to achieve it.

― Carlos Wallace

The assault on science is based on its incompatibility with the values of modern extreme conservatism. Again and again knowledge and progress will challenge the conservative mind with truths they do not want to hear. The best way to avoid this problem is kill the knowledge before it is produced. We can find example after example of science being silenced because it is likely to produce results that do not match their view of the world. Among the key engines of the ignorance of conservatism is its alliance with extreme religious views. Historically religion and science are frequently at odds because the faith and truth are often incompatible. This isn’t necessarily all religious faith, but rather that stemming from a fundamentalist approach, which is usually grounded in old and antiquated notions (i.e., classically conservative and opposing anything looking like progress). Fervent religious belief cannot deal with truths that do not align with dictums. The best way to avoid this problem is get rid of the truth. When the government is controlled by extremists this translates to reducing and controlling
science to avoid such truths.

Every gun that is made, every warship launched, every rocket fired signifies in the final sense, a theft from those who hunger and are not fed, those who are cold and are not clothed. This world in arms is not spending money alone. It is spending the sweat of its laborers, the genius of its scientists, the hopes of its children. This is not a way of life at all in any true sense. Under the clouds of war, it is humanity hanging on a cross of iron.

― Dwight D. Eisenhower

With this background we can see how this imprints onto science rather clearly. Fear and money both like science that leads to National security and defense, so the right supports activities that either provide people with protection or allow us to kill more effectively and efficiently. The problem is that the right only supports this kind of work in its most naked and applied sense. The right doesnot support deeper research that forms the foundation allowing us to develop technology. As a result our ability to be the best at killing people is at risk in the long run. Eventually the foundation of science used to create all our weapons will run out, and we no longer will be the top dogs. The basic research used for weapons work today is largely a relic of the 1960’s and 1970’s. The wholesale diminishment in societal support for research during the 1980’s and onward will start to hurt us more obviously. In addition we have poisoned the research environment in a fairly bipartisan way leading to a huge drop in the effectiveness and efficiency of the fewer research dollars spent.

The right wing has become the enemy of intellectualism. Increasingly they have formed open opposition of educating citizens in anything beyond a rote traditional form of learning. They have systematically undermined a system that teaches people to think for themselves. Science is a form of high thinking that the right opposes for vehemently, and as such to be diminished as a threat to their agenda. At the apex of the educational system are universities. Universities are the factories for the elite they hate so much. The right wing has been engaged in an all out assault on universities in part because they view them as the center of left wing views. Attacking and contracting science is part of this assault. In addition to a systematic attack on universities is an increasing categorization of certain research as unlawful because its results will almost certainly oppose right wing views. Examples of this include drug research (e.g. marijuana in particular), anything sexual, climate research, health effects of firearms, evolution, and the list grows. The deepest wounds to science are more subtle. They have created an environment that poisons intellectual approaches and undermines the education of the population because educated intellectual people naturally oppose their ideas.

It is a well known fact that reality has liberal bias.

― Stephen Colbert

Let’s get to the core of science that the right opposes. One of the areas where science rubs the right wing the wrong way could broadly be characterized as ecological and environmental research. The archetype of this is climate research and the concept of climate change. In general, research in Earth sciences leads to conclusions that upset the status quo hurting the ability of traditional industry to make money (with the exception of geological work associated with energy and mining). Much of the ecological research has informed us how human activity is damaging the environment. Industry does not want to adopt practices that preserve the environment primarily due to greed. Moreover the religious extremism opposes ecological research because it opposes the dictums of their faith as chosen people who may exploit the Earth to the limits of their desire. Climate change is the single greatest research threat to the conservative worldview. The fact that mankind is a threat to the Earth is blasphemous to the right wing extremists either impacting their greed or the religious conviction. The denial of climate change is based primarily on religious faith and greed, the pillars of modern right wing extremism.

An area where science and public policy come into great conflict is public health. Americans love their firearms and possess them in great numbers. Americans also die in great numbers at the end of their firearms. While gun violence captures the attention and imagination, most gun deaths are self-inflicted, or accidental. It is a public health issue of great consequence. Science should be studying it, and yet it is not allowed by federal funding. The reason is the gun lobby (i.e., the NRA), which own the right wing. They don’t want the facts known and have made government funded research on the topic effectively illegal. They are worried that knowledge of the level of public health threat will result in a desire for regulation, and a public outcry. Instead of working with facts and knowledge, we simply suppress it.

Another huge area of intrusion of politics into science is a drug especially illegal ones. We have made research into certain drugs and their medical benefit illegal (i.e., marijuana). Several things are obvious to even the most casual observer, marijuana is not a particularly dangerous drug (definitely not “Schedule 1”), and it has medical benefits. The right (and a lot of the left) wing is opposed to adding science to the discussion. This is yet another example where facts and truth are excluded from the dialog. We have a number of purely political reasons for this. A huge one is the war on drugs, which is supported by law enforcement and prison lobbies (and the people enriched by these “industries”). These lobbies work to damage society as a whole, and enable the implicit bigotry in how laws are enforced and people are imprisoned ignoring the damage to society at large.

The impact of the drug laws has enabled the Jim Crow laws of the past to be enforced by a new mechanism. Again, the impact of the illegal nature of the drugs is ignored because their illegality serves conservative political interests. Their illegal nature is boon to criminal cartels that use them to generate huge incomes. Instead of cutting the cartels source of money out and defuse criminal violence, we keep it fully charged because law enforcement wants power, not a solution to the problem. A perverse fact of modern life is the greater threat to public health of legal drugs (i.e., the opioid crisis), and illegal drugs like marijuana could mitigate the legal drug crisis. We maintain a ridiculous perspective on these drugs by suppressing research. Other drugs like alcohol are legal while the public health impacts are obviously severe. We don’t have the ability to understand the benefits of these drugs from scientific research.

Fundamentalism isn’t about religion, it’s about power.

― Salman Rushdie

Medicine is not immune from the political aspects especially as the money associated with it becomes enormous. Increasingly the whole of medical research starts to run into issues associated with morality. This is true with both the beginning and the end of life. Genetics is fraught with moral implications that naturally drive political response. We already see huge issues on the left with the response to GMO’s. The left wing response to GMO’s is reactionary and Luddite in character. At the same time the right’s view of GMO is cavalier and geared toward money motives above all else. Properly managed GMO’s could be a boon for mankind if we can balance the risks and benefits. The increasingly destructive political dialog without compromise assures no good outcomes.

We are living in modern times throughout the world and yet are dominated by medieval minds.

― Eqbal Ahmad

In no place does politics become more poisonous than with sex (at least in the United States). Whether we are talking about reproductive rights, public health, or sexual education, science is ignored in favor of religious moralization delivered through politics. We have the power to make huge difference in people’s lives through giving them control over their reproduction, yet this progress is undermined by the right wing. Ultimately the right’s position is to criminalize sex by using reproduction as apunishment. This works to destroy people’s potential for economic advancement and burden the World with poor, unwanted children. Sex education for children is another example where ignorance is promoted as the societal response. Science could make the World far better and more prosperous, and the right wing stands in the way. It has everything to do with sex and nothing to do with reproduction.

The ludicrous idea of abstinence-based education is the rights approach even though it is utterly and completely ineffective and actually damages people. Those children are then spared the knowledge of their sexuality and their reproductive rights through an educational system that suppresses information. Rather than teach our children about sex in a way that honors their intelligence and arms them to deal with life, we send them out ignorant. This ignorance is yet another denial of reality and science by the right. The right wing does not want to seem like they are endorsing basic human instinct around reproduction and sex for pleasure. The result is to create more problems, more STD’s, more unwanted children, and more abortions. It gets worse because we also assure that a new generation of people will reach sexual maturity without the knowledge that could make their lives better. We know how to teach people to take charge of their reproductive decisions, sexual health and pleasure. Our government denies them this knowledge largely driven by antiquated moral and religious ideals, which only serve to give the right voters.

Take a stand for science, knowledge and progress by embracing politics. In today’s world nothing is more dangerous than truth and facts. People whose fundamental principles and philosophy are utterly opposed to the foundational principles of science govern us. Conflict between their approach to governance and science is inevitable. People who are completely terrified of reality increasingly rule us. Science is one of the key ways humans understand their reality. Science is a completely human thing and political to its core. We are rapidly moving to a world where it is radical to simply speak facts and the truth. Before long doing science may be an act of dissidence against the ruling class. As I noted above in many cases it already is.

If by a “Liberal” they mean someone who looks ahead and not behind, someone who welcomes new ideas without rigid reactions, someone who cares about the welfare of the people-their health, their housing, their schools, their jobs, their civil rights and their civil liberties-someone who believes we can break through the stalemate and suspicions that grip us in our policies abroad, if that is what they mean by a “Liberal”, then I’m proud to say I’m a “Liberal.”

― John F. Kennedy

Great research depends on taking big risks with a large chance of failure and mission-focus is a clear way to get there. Failure is the key to learning, and research is fundamentally learning. We need to avoid failing too early by shying away from solving the harder problems and devotion to a mission provides the drive to keep focus on results that provide value to that mission.

There is only one thing that makes a dream impossible to achieve: the fear of failure.

― Paulo Coelho

For a lot of people working at a National Lab there are two divergent paths for work, the research path that leads to lots of publishing, deep technical work and strong external connection, or the mission path that leads to internal focus and technical shallowness. The research path is for the more talented and intellectual people who can compete in this difficult world. For the less talented, creative or intelligent people, the mission world offers greater security at the price of intellectual impoverishment. Those who fail at the research focus can fall back onto the mission work and be employed comfortably after such failure. This perspective is a cynical truth for those who work at the Labs and represents a false dichotomy. If properly harnessed the mission focus and empower and energize better research, but it must be mindfully approached.

The measure of greatness in a scientific idea is the extent to which it stimulates thought and opens up new lines of research.

― Paul A.M. Dirac

As I stated, I believe the dichotomy of mission versus research is false. The mission imposes limitations and constraints on research. In a nutshell, the mission imposes a fixed problem to solve and one must adapt the solution to impact this mission. Conversely, pure research is unconstrained by a mission, which encourages people to change the problem to fit a solution. The fixed problem, adaptive solution mindset is much better for engaging innovation and producing breakthrough results. It also means a great amount of risk and lots of failure. Pure research can chase unique results, but the utility of those results is often highly suspect. This sort of research entails less risk and less failure as well. If the results are necessarily impactful on the mission, the utility is obvious. The difficulty is noting the broader aspects of research applicability that mission application might hide.

Examples of great mission-focused research abound and our modern world is testimony to the breakthrough nature of Cold War defense focused research. The shape of the modern world is a testament to the power of mission-focused work to succeed. Ubiquitous aspects of modernity such as the Internet, cell phones and GPS all owe their existence to Cold War research focused on some completely different mission. All of these technologies were created through steadfast focus on utility that drove innovation as a mode of problem solving. This model for creating value has fallen into disrepair due to its uncertainty and risk. Risk is something we have lost the capacity to withstand as a result the failure necessary to learn and succeed with research never happens.

Failure is a greater teacher than success

― Clarissa Pinkola Estés

Mission focused research falls prey to concerns over risk. In many mission organizations there is a fear of taking on too much risk in adopting research results into the mission delivery. The thought is that the research might not pan out and the mission will suffer as result. This is both shortsighted and foolhardy. The truth is vastly different than this fear-based reaction and the only thing that suffers from shying away from research in mission-based work is the quality of the mission-based work. Doing research causes people to work with deep knowledge and understanding of their area of endeavor. Research is basically the process of learning taken to the extreme of discovery. In the process of getting to discovery one becomes an expert in what is known and capable of doing exceptional work. Today to much mission focused work is technically shallow and risk adverse. It is over-managed and underled in the pursuit of false belief that risk and failure are bad things.

There is a key tension to maintain in harnessing this engine of knowledge. The successful delivery of value and success to the mission work must take priority. Those conducting the research should have a deep commitment to the mission and its success. Ultimately the success at the mission work must supersede the research objectives. Even better the research objectives need to be guided by the mission needs. In this sense the mission acts to constrain the research and shape its direction and focus. This sort of dynamic must be carefully and adroitly managed if it can be achieved. Unconstrained research without mission focus is quite tempting and much simpler to manage. It is also less successful at producing real value for society. Almost every breakthrough of great significance was the result of results-focused work although many of the breakthroughs had far greater reach beyond their intended use.

An expert is someone who knows some of the worst mistakes that can be made in his subject, and how to avoid them.

– Werner Heisenberg

In my own experience the drive to connect mission and research can provide powerful incentives for personal enrichment. For much of my early career the topic of turbulence was utterly terrifying, and I avoided it like the plague. It seemed like a deep, complex and ultimately unsolvable problem that I was afraid of. As I began to become deeply engaged with a mission organization at Los Alamos it became clear to me that I had to understand it. Turbulence is ubiquitous in highly energetic systems governed by the equations of fluid dynamics. The modeling of turbulence is almost always done using dissipative techniques, which end up destroying most of the fidelity in numerical methods used to compute the underlying ostensibly non-turbulent flow. These high fidelity numerical methods were my focus at the time. Of course these energy rich flows are naturally turbulent. I came to the conclusion that I had to tackle understanding turbulence.

One Winter break my laptop broke leaving without the ability to work on my computer codes over the break (those were the days!). So I went back to my office (those were the days!) and grabbed seven books on turbulence that had been languishing on my bookshelves unread due to my overwhelming fear of the topic. I started to read these books cover to cover, one by one and learn about turbulence. I’ve included some of these references below for your edification. The best and most eye opening was Uriel Frisch’s “Turbulence: the Legacy of A. N. Kolmogorov”. In the end, the mist began to clear and turbulence began to lose its fearful nature. Like most things one fears; the lack of knowledge of a thing gives it power and turbulence was no different. Turbulence is actually kind of a sad thing; its not understood and very little progress is being made.

The main point is that the mission focus energized me to attack the topic despite my fear of it. The result was a deeply rewarding and successful research path resulting in many highly cited papers and a book. All of a sudden the topic that had terrified me was understood and I could actually conduct research in it. All of this happened because I took contributing work to the mission as an imperative. I did not have the option of turning my back on the topic because of my discomfort over it. I also learned a valuable lesion about fearsome technical topics; most of them are fearsome because we don’t know what we are doing and overelaborate the theory. Today the best things we know about turbulence are simple, and old discovered by Kolmogorov as he evaded the Nazis in 1941.

People who don’t take risks generally make about two big mistakes a year. People who do take risks generally make about two big mistakes a year.

― Peter F. Drucker

In today’s world we have allowed a system to come into power that funds useless research. We have created vast swaths of safe research topics that ultimately produce rafts of papers, but little or no real utility for society. A big driver behind this mentality is the need to fund “sure things” that can’t fail. This pushes research into the realm of squalid incrementalism. Incremental research is safe and almost never prone to the risk of failure. It is also an actual waste of money that can produce the appearance and guise of success without actual achievement. Our unremitting fearful society including the worry of the appearance of scandal has driven us to this horrible point. Research has become cowardly and uninspired so that it doesn’t ever fail. Being mission-focused is too hard and too risky because the mission is too important to ever fail at. The true attitude should be that the mission is too important not fail at!

The main reason of fear of failure lies in people’s being unaware of the impact of misfortune on achieving success

― Sunday Adelaja

The current sorry state in high performance computing is a direct result of the current milieu where mission-focus is neglected in favor of carefully managed projects with sure things as targets. Project management is not leadership, and without leadership we will continue to steadfastly underachieve. For example, we have utterly eviscerated applied mathematics by pushing a product-oriented approach that demands the delivery of results in software. Producing software in the conduct of applied mathematics used to be a necessary side activity instead of the core of value and work. Today software is the main thing produced and actual mathematics is often virtually absent. Actual mathematical research is difficult, failure prone and hard to measure. Software on the other hand is tangible and managed. It is still is hard to do, but ultimately software is only as valuable as what it contains, and increasingly our software is full of someone else’s old ideas. We are collectively stewarding other people’s old intellectual content, and not producing our own, nor progressing in our knowledge.

This trend would be bad enough on its own, but it is the tip of a proverbial iceberg of underachievement. The second pillar for underachievement in high performance computing is, ironically, a devotion to computer hardware. Again computer hardware is tangible and easy to measure. To a naïve person (or congressman) the ability to measure our ability to do things with computers should be a one-to-one match with the raw power of our computers. Nothing could be farther from the truth as computing is a completely multi-disciplinary field depending on a huge swath of science for success. The computer hardware is actually one of the least important components in our modeling and simulation competence. Instead of producing a program that strives for true success in modeling and simulation based on real mission value, we have constructed programs that are intellectually vacuous because they are easier to fund and explain to unsophisticated people. The hardware program more naturally lends itself to management and simple metrics of success. It can be sold to uninformed people. Its current form is the abdication of leadership and antithetical to the concept of mission-focus. Our approach to high performance computing is only likely to achieve supremacy for the Chinese in the field.

Success is stumbling from failure to failure with no loss of enthusiasm.

― Winston S. Churchill

What is gained by this mission focus? The focus on mission-focused research means the problem being solved is fixed and unwavering, and the results and knowledge must contribute to the solution of this problem. This forces the research to adapt itself to the needs of the problem rather than the problem to the research. The result of this model is the tendency to confront difficult thorny issues rather than shirk them. At the same time this form of research can also lead to failure and risk manifesting itself. This tendency is the rub, and leads to people shying away from it. We are societally incapable of supporting failure as a viable outcome. The result is the utter and complete inability to do anything hard. This all stems from a false sense of the connection between risk, failure and achievement.

If a single characteristic is contributing to a societal feeling that we have lost greatness, it is that we cannot accept failure. Without failure, great things cannot be achieved. Failure is the vehicle of achievement and learning whether we are talking about individuals, organizations or nations. The inability to accept failure as a possible outcome is the tacit acceptance of not wanting to do anything that matters, or anything great. The road to greatness is paved with many failures and the unerring drive to learn and grow from these failures. For the complex missions we are charged with, the commitment to mission focus in research means accepting failure as a necessary outcome of endeavor. This is the hard message that our spineless politicians and managers cannot give us. The inability to grasp this core truth is utter societal cowardice. True leadership would provide us the necessary support and encouragement to be courageous and steadfast. Instead we succumb to fear and the false belief that achievement can be managed and had without risk.

Research is about learning at a fundamental, deep level, and learning is powered by failure. Without failure you cannot effectively learn, and without learning you cannot do research. Failure is one of the core attributes of risk. Without the risk of failing there is a certainty of achieving less. This lower achievement has become the socially acceptable norm for work. Acting in a risky way is a sure path to being punished, and we are being conditioned to not risk and not fail. For this reason the mission-focused research is shunned. The sort of conditions that mission-focused research produces are no longer acceptable and our effective social contract with the rest of society has destroyed it.

If we are to successfully do great things again as people, as organizations, as laboratories and as a nation, the irony is that we need to fail a lot more. One way to assure the sort of failure we need is mission-focused research where providing value to a difficult mission is the primal goal of research. Better research is founded on devotion to meaningful outcomes, taking big risks and tolerating lots of failure.

Only those who dare to fail greatly can ever achieve greatly.

― Robert F. Kennedy

Launder, Brian Edward, and Dudley Brian Spalding. “Lectures in mathematical models of turbulence.” (1972).

Frisch, Uriel, and Russell J. Donnelly. “Turbulence: the legacy of AN Kolmogorov.” (1996): 82-84.

Pope, Stephen B. “Turbulent flows.” (2001): 2020.

Grinstein, Fernando F., Len G. Margolin, and William J. Rider, eds. Implicit large eddy simulation: computing turbulent fluid dynamics. Cambridge university press, 2007.

Margolin, Len G., and William J. Rider. “A rationale for implicit turbulence modelling.” International Journal for Numerical Methods in Fluids 39, no. 9 (2002): 821-841.

Margolin, L. G., W. J. Rider, and F. F. Grinstein. “Modeling turbulent flow with implicit LES.” Journal of Turbulence 7 (2006): N15.

Many times in life I’ve regretted the things I’ve said without thinking. But I’ve never regretted the things I said nearly as much as the words I left unspoken.

― Lisa Kleypas

I’m a scientist and I tackle lots of difficult intellectual topics, higher-level mathematics and deep physical principle daily. All of these things are very complex and require immense amounts of training, experience and effort. For most people the things I do, think about, or work on are difficult to understand or put into context. None of this is the hardest thing I do every day. The thing that we trip up on, and fail at more than anything is simple, communication. Scientists fail to communicate effectively with each other in a myriad of ways leading to huge problems in marshaling our collective efforts. Given that we can barely communicate with each other, the prospect of communicating with the public becomes almost impossible.

Listen with curiosity. Speak with honesty. Act with integrity. The greatest problem with communication is we don’t listen to understand. We listen to reply. When we listen with curiosity, we don’t listen with the intent to reply. We listen for what’s behind the words.

― Roy T. Bennett

It is as much a problem of listening as talking, and we do neither very well. It is arguable that the art of listening is something in deep crisis society wide. We seem to be quite capable of expressing a wealth of opinions to each other, but incapable of listening and attempting to understand each other. This makes every problem we have worse and stymies our efforts to solve them. In the sciences, these issues are generally compounded by the nature of the people capable of carrying out the deep scientific work. With all that effort and energy put toward the intellectual labor and their basic nature as people, little is left over to do the heavy lifting of communication. This leaves this essential effort languishing from a combination of lack of effort and outright incompetence.

If you had to identify, in one word, the reason why the human race has not achieved, and never will achieve, its full potential, that word would be ‘meetings.

― Dave Barry

Meetings are at the heart of an effective organization, and each meeting is an opportunity to clarify issues, set new directions, sharpen focus, create alignment, and move objectives forward.

― Paul Axtell

A big conduit for communication is the “meeting” a proper target of derision and ridicule. We all spend way too much time in horrible meetings that masquerade as communication. In addition to time wasting, the worst thing about them is that they give people the impression that communication has taken place when it hasn’t. The meeting doesn’t provide effective broadcast of information and it’s even worse as a medium for listening. Our current management culture seems to have gotten the idea that a meeting is sufficient to do the communication job. Meetings seem efficient in the sense that everyone is there and words are spoken, and even time for questions is granted. With the meeting, the managers go through the motions. The problems with this approach are vast and boundless. The first issue is the general sense that the messaging is targeted for a large audience and lacks the texture that individuals require. The message isn’t targeted to people’s acute and individual interests. Conversations don’t happen naturally, and people’s questions are usually equally limited in scope. To make matters worse, the managers think they have done their communication job.

People who enjoy meetings should not be in charge of anything.

― Thomas Sowell

The tendency to do everything through meetings results in the illusion that communication has happened. The same happens with mass e-mails where management assumes the exchange of information was successful. A lot of the necessary vehicles for communication are overlooked or discounted in the process. Managers avoid the one-on-one conversations needed to establish deep personal connections and understanding. We have filled manager’s schedules with lots of activities involving other managers and paperwork, but not prioritized and valued the task of communication. We have strongly tended to try to make it efficient, and not held it in the esteem it deserves. Many hold office hours where people can talk to them rather than the more effective habit of seeking people out. All of these mechanisms give the advantage to the extroverts among us, and fail to engage the quiet introverted souls or the hardened cynics whose views and efforts have equal value and validity. All of this gets to a core message that communication is pervasive and difficult. We have many means of communicating and all of them should be utilized. We also need to assure and verify that communication has taken place and is two ways.

Employees hate meetings because they reveal that self-promotion, sycophancy, dissimulation and constantly talking nonsense in a loud confident voice are more impressive than merely being good at the job – and it is depressing to lack these skills but even more depressing to discover one’s self using them.

― Michael Foley

We haven’t touched on the other context of the word “meeting” for scientists. The technical talk, seminar or conference version of the word has great importance. The act of formally giving a presentation of technical work is one of the key ways of delivering information in a professional setting. It forms a core of opportunity for peer review in a setting that allows for free exchange. Conferences are an orgy of this and should form a backbone of information exchange. Instead conferences have become a bone of contention. People are assumed to only have a role there as speakers, and not part of the audience. Again the role of listening as an important aspect of communication is completely disregarded in the dynamic. The digestion of information and learning or providing peer feedback provide none of the justification for going to conferences, yet these all provide invaluable conduits for communication in the technical world.

Americans no longer talk to each other, they entertain each other. They do not exchange ideas, they exchange images. They do not argue with propositions; they argue with good looks, celebrities and commercials.

― Neil Postman

Part of the irony of this problem is the role of science in providing technology that makes communication easier than ever. The issue is the content of that
communication, which varies depending on people’s innate taste for clarity and focus. We have issues with transparency of communication even with automatic and pervasive use of all the latest technological innovations. These days we have e-mail, instant messaging, blogging, Internet content, various applications (Twitter, Snapchat,…), social media and other vehicles information transfer through people. The key to making the technology work to enable better performance still comes down to people’s willingness to pass along ideas within the vehicles available. This problem is persistent whether communications are on Twitter or in-person. Again the asymmetry between broadcasting and receiving is amplified by the technology. I am personally guilty of the sin that I’m pointing out, we never prize listening as a key aspect of communicating. If no one listens, it doesn’t matter who is talking.

We are still woeful in our ability to confront difficult issues. Concepts like “ghosting” or “breadcrumbing” are useful for online relationships, and actually have great meaning in non-romantic settings. For difficult issues there are a strong tendency to avoid discussion and contention, this is just like ghosting where people disappear to avoid a real breakup. Breadcrumbing is pretending to have a deep relationship that really has no basis in fact and gets nursed along by a stream of limited communications. Both practices are emblematic of deeply toxic engagements that exhibit a lack of humanity, empathy and understanding. Each of these characteristics is deeply helpful in aiding the depth of communication, and things scientists and engineers are generally quite ill equipped to deal with.

Scientists and engineers tend to be introverts and not necessarily good at communicating. Nothing about technology changes these facts; it only gives more mechanics for communication to be poorly executed. All the various communication vehicles require skill to execute and basic ideas to be presented. The ideas are often complicated by their esoteric nature meaning that standard communication is already stressed. Add controversy and tension to the communication will simply lead to greater stress and encourage bad behavior. This greater stress will serve to bait people into various ghosting or breadcrumbing exercises. All of these elements go into undermining the depth of communication possible if all the means of achieving it are exercised.

I work on vastly complex projects involving big ideas and massively deep technical issues. The concepts and ideas in the project are deep and need vast wells of technical expertise to even understand much less solve. In spite of this the biggest issue holding us back is communication. We are more challenged by having access to information and knowledge of the full nature of the problems we face. The complex issues typically need the expertise of many people to understand, and solutions are equally dependent on wide ranging expertise. More than simply understanding being fuelled by communication, the solutions need collaborations across many technical disciplines to work together. Instead of harnessing the possibility and potential that communication has to unleash productivity and innovation, we embrace secrecy and information hiding that only empowers control and destroys potential. Information hiding has become the standard in reaction to the possibilities of greater transparency that the modern world offers.

When the trust account is high, communication is easy, instant, and effective.

― Stephen R. Covey

One of the key issues contributing to our systematic communication issues is conflict aversion. If we talk about our collective problems, we run the risk of conflict. Problems and issues that hold us back are often the source of conflict. Our problems are usually associated with vested interests, or solving the problems might involve some sort of trade space where some people win or lose. Most of the time we can avoid the conflict for a bit longer by ignoring it. Facing the problems means entering into conflict, and conflict terrifies people (this is where ghosting and breadcrumbing come in quite often). The avoidance is usually illusory; eventually the situation will devolve to the point where the problems can no longer be ignored. Usually the situation is much worse, and the solution is much more painful. We need to embrace means of making facing up to problems sooner rather than later, and seek solutions when problems are small and well confined.

Given that scientists can barely communicate with each other effectively, the general public is almost impossible to manage. In many ways the modern world acts to amplify the issues that the technical world has with communication to an almost unbearable level. Perhaps excellence in communication is too much to ask, but the inabilities to talk and listen effectively with the public are hurting science. If science is hurt than society also suffers from the lack of progress and knowledge advancement science provides. When science fails, everyone suffers. Ultimately we need to have understanding and empathy across our societal divides whether it is scientists and lay people or red and blue. Our failure to focus on effective, deep two-way communication is limiting our ability to succeed at almost everything.

The single biggest problem in communication is the illusion that it has taken place.

― George Bernard Shaw

Experiment is the sole source of truth. It alone can teach us something new; it alone can give us certainty.

― Henri Poincaré

In looking at the dynamic surrounding verification and validation recently I’ve noticed a very grim evolution of the narrative. Two things have happened to undermine the maturity of V&V. One I’ve spoken about in the past, the tendency to drop verification and focus solely on validation, which is bad enough. In the absence of verification, validation starts to become rather strained and drift toward calibration. Assurances that one is properly solving the model they are claiming to be solving are unsupported by evidence. This is bad enough all by itself. The use of V&V as a vehicle for improving modeling and simulation credibility is threatened by this alone, but something worse looms even larger.

A more common and pervasive trend is the conflation of validation with uncertainty quantification. It has become very common for uncertainty quantification (UQ) to be defined as the whole of validation. To some extent this is fueled by a focus on high performance computing where UQ provides a huge appetite for computational cycles thus eliciting lots of love and support in HPC circles. Validation must be about experiments and a broad cross section of uncertainties that may only be examined through a devotion to multi-disciplinary work and collaboration. One must always remember that validation can never be separated from measurements in the real world whether experimental or observational. The experiment-simulation connection in validation is primal and non-negotiable.

There are three types of lies — lies, damn lies, and statistics.

― Benjamin Disraeli

A second part of the issue is the hot topic nature of UQ. UQ has become a buzzword and seems to be a hot issue in publishing and research. Saying you’re doing UQ seems to be a means to squeezing money out of funding agents. In addition UQ can be done relatively automatically and mechanically. Tools and techniques exist to enable UQ to be done without much deep thought even though it touches upon many deep technical topics. Actual validation is far harder and more holistic. The core to any work in validation is serious experimental expertise and hard-nosed comparison with simulations. The detailed nature of the experiment and its intrinsic errors and uncertainties is the key to any comparison. Without knowing the experimental uncertainty any computational uncertainty is context free. My grumpy intellectual would quip that validation requires thinking and that leads people to avoid it because thinking is so hard. The deeper issues are that validation is complex and mutli-disciplinary in nature making it collaborative and difficult. Experts in a single discipline can do UQ, so it is an easy out.

Five percent of the people think;

ten percent of the people think they think;

and the other eighty-five percent would rather die than think.

― Thomas A. Edison

One of the biggest issues is the stunning incompleteness of UQ in general. Most commonly UQ is done via an exploration of the variation of parameters in models. Complex models of reality have a lot of constants that are not known with great precision. Various techniques may be utilized to efficiently examine the variation in computational solutions due to changes in these parameters. Among the methods used are things like Markov Chain Monte Carlo (MCMC), polynomial chaos, and other sampling methods. The results from this work are useful and sound, but form a rather incomplete view of uncertainty. Even in these cases the sampling is often subject to lack of certainty with the assessment driven by the difficulty of determining uncertainty in high dimensional spaces. Modeling and simulation suffers from a host of other uncertainties not covered by these methodologies. For example most simulations have some degree of numerical error that may be quite large. Numerous techniques exist for exploring its magnitude and nature. Many systems being modeled have some stochastic or variability associated with them. Modeling assumptions are often made in simulating a system or experiment. The solution may change greatly on the basis of these assumptions or modeling approximations. A different computational modeler may make much different assumptions and produce a different solution.

Judge a man by his questions rather than by his answers.

― Voltaire

If validation is to be done properly a fairly complete accounting of modeling and simulation uncertainty is needed. One also needs to also understand the experimental error and uncertainty with equal completeness. One must be acutely aware of the intrinsic lack of certainty in the estimation of uncertainty. The combination of the solutions and the sizes of each uncertainty provides a modeling and simulation solution into proper context. Without knowledge of the uncertainties in each data source, the distance between solutions cannot be judged. For example if the experimental precision is very good and the uncertainty is quite small, the simulation needs to be equally precise to be judged well. Conversely a large experimental uncertainty would allow model to be much looser, and still be judged well. More critically the experiment wouldn’t provide actionable evidence on research needs, and expert judgment would reign.

The whole of the uncertainty provides an important source of scientific tension. If experimental uncertainty is small, it requires modeling and simulation to be equally precise to imply good results. It pushes the modeling to improve to meet the high standard of the experiment. If the modeling and simulation is very good, but the experiments have large uncertainty, it should push the experiments to improve because they fail to constrain and distinguish between models. By having a deep and complete understanding of uncertainty, we can define where we need to put resources to improve. We know what aspects of our current knowledge are the most in need of attention and limiting progress.

One must always be aware of the significant attraction of short changing uncertainty estimation. Doing a complete job of estimating uncertainty almost always results in an increase in the magnitude of uncertainty. This is where science as a fundamentally human enterprise comes into play. People would rather think uncertainties are small than large. Uncertainty is uncomfortable and people shy away from discomfort. By under-estimating uncertainty people unconsciously put themselves at ease by doing incomplete work. A more rigorous and complete approach almost always produces a discomforting result. When one combines discomfort with difficulty of accomplishment, the necessary factors for lack of effort and completeness becomes clear. With this temptation in mind the tendency to take the easy route must be acknowledged.

The bottom line is the necessity understanding uncertainty in a holistic manner can produce useful and defensible context for science. It can allow us to understand where we need to improve our knowledge or practice. Without this accounting the whole issue falls into the area of relying upon expert judgment or politics to make the decisions. We fail to understand where our knowledge is weak and potentially overlook experiments necessary for understanding. We may have the right experiments, but cannot make measurements of sufficient accuracy. We might have models of insufficient complexity, or numerical solutions with too much numerical error. All of these spell out different demands for resource allocation.

Much of the tension is captured in these two quotes although I hope Eddington was probably trying to be ironic!

Never trust an experimental result until it has been confirmed by theory

― Arthur Stanley Eddington

It doesn’t matter how beautiful your theory is … If it doesn’t agree with experiment, it’s wrong.

― Richard Feynman

It is not enough for code to work.

― Robert C. Martin

One of the big selling points for high performance computing is real world impact. A focus for this impact revolves around computer codes with the capability to produce answers for people working on real world problems. In organizations such as mine where modeling and simulation produces analysis used to assist decision-making, these codes are known as production codes, or the production codes. As these codes provide real value to our applied programs, and with this value provides generous financial support. This support is necessary for the codes to do their job and creates financial stability. With this support comes acute responsibility that needs to be acknowledged and serious effort needs to apply to meeting these.

Being a production code means producing results that are meaningful. The results are important by virtue of their utility in decision-making that impacts the real world. In the past those using production codes as expert users provided the credibility needed to make these codes important. In other words credibility was granted by the weight of the expertise of the users. This element in expertise-based credibility is still alive today, but increasingly being replaced by systematic approaches to bolster the purely human element. The appropriate and proper use of verification, validation, and uncertainty quantification along with software quality assurance provides a system for assessing credibility and constructing evidence. More and more this evidence is demanded to document credibility and reassure decision makers.

One of the main things production codes give is the ability to model important problems in the real world. This is the primary characteristic of production codes, modeling capability for real problems. While this character is primary in defining production codes everything else important in high performance computing is eclipsed by the modeling imperative. These codes are essential for the utility of high performance computing resources and often become the first codes to make their way and use high-end resources. They quite often are the explicit justification for the purchase of such computing hardware. This character usually dominates and demands certain maturity of software professionalism.

On the flipside there are significant detrimental aspects of such codes. For example the methods and algorithms in production codes are often crude and antiquated in comparison to state of the art. The same can be said for the models, the algorithms and often the computer code itself. The whole of the production code credibility is deeply impacted by these pedigrees and their impact on real World programs and things. This issue comes from several directions; the codes are often old and used for long periods of time. The experts who traditionally define the credibility drive this to some extent. It often takes a long time to develop the code to the level needed to solve the hard real world problems as well as the expertise to navigate the code’s capability into results that have real world meaning. Older methods are robust, proven and trusted (low order, and dissipative is usually how robust happens). Newer methods are more fragile, or simply can’t deal with all the special cases and issues that threaten the solution of real problems. Again, the same issues are present with models, algorithms and the nature or quality of the computer code itself.

Knowledge is something which you can use.
Belief is something which uses you.

― Idries Shah

In the final analysis the production code status must be earned and can not granted by fiat. Generally one might intend for a code to become a production code, but it only becomes a production code when it begins to produce. That production must be delivered with credibility and trust. It is an utterly organic process that cannot be forced. While significant support comes with production code status, it also comes with responsibilities as well. Increasingly in a modern context these responsibilities include software quality, verification and validation to be executed in a serious professional manner. Hopefully the “customers” for production code analysis will be more demanding and insistent on a more systematic pedigree. Even today this pull for systematic pedigree is poorly supported by the customers of production code results creating an environment where counter-productive practices and results continue to supported.

In far too many instances, the systematic pedigree defining steps are being skipped for the old system of person-centered credibility. The old person-centered system is simple and straightforward. You trust somebody and develop a relationship that supports credibility. This person’s skills include detailed technical analysis, but also inter-personal relationship building skills. If such a system is in place there is not a problem as long as the deeper modern credibility is also present. Too often the modern credibility is absent or shorted and effectively replaced by a cult of personality. If we put our trust in people who do not value the best technical work available in favor of their force of personality or personal relationships we probably deserve the substandard work that results.

Tell me what you pay attention to and I will tell you who you are.

― José Ortega y Gasset

 

Reality is that which, when you stop believing in it, doesn’t go away.

― Philip K. Dick

I applied the smoothed operators to the actual solution of a scalar advection law, and wanted to show how the methods impact the solution. This should put the discussion of the past couple of weeks into a bit sharper and more relevant focus. We can also explore the impact of the degree of regularization chosen in the smoothing. The good things below are finding out that my analysis seems to completely coincide with the results, and in that case the results are the lowest error and highest quality solution. More surprisingly, the best result is obtained with a smoothed function, not the original discontinuous ones!

We will solve the scalar advection equation for a square wave on a mesh of 200 cells in one dimension using a Sweby version of Fromm’s scheme. The solution will do one rotation on the grid using 400 time steps. I’ll plot the solution and the error plus display the L1, L2 and L-infinity norms of the error.

First with the standard unsmooth functions.

Error= 0.0314099 0.00649721 0.438233

Now with smooth functions all based on using the and .

Error= 0.0313124 0.00648103 0.4384

Now with smooth functions all based on using the and .

Error= 0.0350856 0.00694473 0.454124

Now with smooth functions all based on using the and .

Error= 0.0257311 0.00578931 0.420996

What we see is that the smoothed operators produce high quality and lower error solutions in most cases. The one case with the linear version of the smoothed sign function the larger error is supported by the analysis I gave last week. Generally as the regularization allows the function to come as close as we might like to the original functions.

To test a perfect theory with imperfect instruments did not impress the Greek philosophers as a valid way to gain knowledge.

― Isaac Asimov

Note: I got super annoyed with the ability of WordPress to parse LaTeX so there are a lot of math type in here that I gave up on. Apologies!

Last week I introduced a set of alternatives to discontinuous intrinsic functions providing logical functional capability in programming. In the process I outlined some of the issues that arise due to discontinuous aspects of computer code operation. The utility of these alternatives applies to common regression testing in codes and convergence of nonlinear solvers. The issue remains with respect to how useful these alternatives are. My intent is to address components of this aspect of the methods this week. Do these advantageous functions provide their benefit without undermining more fundamental properties of numerical methods like stability and convergence of numerical methods? Or do we need to modify the implementation of these smoothed functions in some structured manner to assure proper behavior.

The answer to both questions is an unqualified yes, they are both useful and they need some careful modification to assure correct behavior. The smoothed functions may be used, but the details do matter.

To this end we will introduce several analysis techniques to show these issues concretely. One thing to get out of the way immediately is how this analysis does not change some basic aspects of the functions. For all of the functions we have the property that the original function is recovered in an asymptotic limit that is becomes the original sign function, . Our goal is to understand the behavior of these functions within the context of a numerical method away from this limit where we have obviously deviated substantially from the classical functions. We fundamentally want to assure that the basic approximation properties of methods are not altered in some fatal manner by their use. A big part of the tool kit will be systematic use of Taylor series approximations to make certain that the consistency of the numerical method and order of accuracy are retained when switching from the classical functions to their smoothed versions. Consistency simply means that the approximations are valid approximations to the original differential equation (meaning the error is ordered).

There was one important detail that I misplaced during last week’s post. If one takes the definition of the sign function we can see an alternative that wasn’t explored. If we have . Thus we can easily rearrange this expression to give two very different regularized absolute value expressions, and . When we move to the softened sign function the behavior of the absolute value changes in substantive ways. In particular in the cases where the softened absolute value was everywhere less than the classical absolute value, it would be greater for the second interpretation and vice-versa. As a result functions like now can produce an absolute value, that doesn’t have issues with entropy conditions as the dissipation will be more than the minimum, not less. Next we will examine whether these different views have significance in truncation error.

Our starting point will be the replacement of the sign function or absolute value in upwind approximations to differential equations. For entropy satisfaction and generally stable approximation the upwind approximation is quite fundamental as the foundation of robust numerical methods for fluid dynamics. We can start with the basic upwind approximation with classical function, the absolute value in this case. We will base the analysis on the semi-discrete version of the scheme using a flux difference, $\frac{1}{h} \left[ f(j+1/2) – f(j-1/2) \right]  $. The basic upwind approximation is $ – \frac{1}{2} \left|a\right| \left[ u(j+1)  –  u(j)  \right]$ where is the characteristic velocity for the flux and provides the upwind dissipation. The Taylor series analysis gives the dissipation in this scheme as . We find a set of very interesting conclusions can be drawn almost immediately.

All of the smoothed functions introduced are workable as alternatives although some versions seem to be intrinsically better. In other words all produce a valid consistent first-order approximation. The functions based on analytical functions like or are valid approximations, but the amount of dissipation is always less than the classical function leading to potential entropy violation. They approach the classical absolute value as one would expect and the deviations similarly diminish. Functions such as or result in no change in the leading order truncation error although similarly the deviations are always produce less dissipation than classical upwind. We do find that for both functions we need to modify the form of the regularization to get good behavior to and .For the classical softmax function based on logarithms and exponentials is the same, but it always produces more dissipation than upwinding rather than less, . This may make this functional basis better for replacing the absolute value for the purpose of upwinding. The downside to this form of the absolute value is the regularized sign function’s passage through hard zero, which makes division problematic.

Let’s look at the functions useful for producing a more entropy satisfactory result for upwinding.  We find that these functions work differently than the original ones. For example the hyperbolic tangent does not as quickly become equivalent to the upwind scheme as . There is a lingering departure from linearity with proportional to the mesh spacing and . As a result the quadratic form of the softened sign is best because of the regularization. Perhaps this is a more widely applicable conclusion as will see as we develop the smoothed function more with limiters.

Where utility ends and decoration begins is perfection.

― Jack Gardner

Now we can transition to looking at a more complex and subtle subject, limiters. Briefly put, limiters are nonlinear functions applied to differencing schemes to produce non-oscillatory (or monotone solutions) with higher order accuracy. Generally in this context high-order is anything above first order. We have theory that confines non-oscillatory methods to first-order accuracy where upwind differencing is canonical. As a result the basic theory applies to second-order method where a linear basis is added to the piecewise constant basis the upwind method is based on. The result is the term “slope limiter” where the linear, slope, is modified by a nonlinear function. Peter Sweby produced a diagram to describe what successful limiters look like parametrically. The parameter is non-dimensionally described by the ratio of discrete gradients, $ r = \frac{(u(j+1) – u(j)}{u(j) – u(j-1)}$. The smoothed functions described here modify the adherence to this diagram. The classical diagram has a region where second-order accuracy can be expected. It is bounded by the function and twice the magnitude of this function.

We can now visualize the impact of the smoothed functions on this diagram. This produces systematic changes in the diagram that lead to deviations from the ideal behavior. Realize that the ideal diagram is always recovered in the limit as the functions recover the classical form. What we see is that the classical curves are converged upon from above or below, and produces wiggles in the overall functional evaluation. My illustrations all show the functions with the regularization chosen to be unrealistically small to exaggerate the impact of the smooth functions. A bigger and more important question is whether the functions impact the order of approximation.

To finish up this discussion I’m going to look at analyzing the truncation error of the methods. Our starting point is the classical scheme’s error, which provides a viewpoint on the nature of the nonlinearity associated with limiters. What is clear about a successful limiter is its ability to produce a valid approximation to a gradient with an ordered error of a least order . The minmod limiter produces a truncation error of $ \mbox{minmod}(u(j)-u(j-1), u(j+1) – u(j)) = u_x – \frac{h}{2} \left| \frac{u_{xx}}{u_x} \right| u_x $. The results with different sorts of recipes for the smoothed sign function and its extension to softabs, softmin and softmax are surprising to say the least and a bit unexpected.

Here is a structured summary of the options as applied to a minmod limiter, :

 

1. and . The gradient approximation is . The constant in front of the gradient approaches one very quickly as grows.
2. and . The gradient approximation is . The constant in front of the gradient approaches one very slowly as grows. This is smoothing is unworkable for limiters.
3. and putting a mesh dependence in the sign function results in inconsistent gradient approximations. The gradient approximation is . The leading constant goes slowly to one as .
4. and . The gradient approximation is .
5. and . putting a mesh dependence in the sign results in inconsistent gradient approximations. The gradient approximation is . This makes the approximation utterly useless in this context.
6. and . The gradient approximation is $ u_x \approx u_x – \left[ u_x\sqrt{n^2 + \left(\frac{(u_{xx}}{u_x}} \right)^2 \right]h+ \cal{O}(h^2)$.

Being useful to others is not the same thing as being equal.

― N.K. Jemisin

Sweby, Peter K. “High resolution schemes using flux limiters for hyperbolic conservation laws.” SIAM journal on numerical analysis 21, no. 5 (1984): 995-1011.

Everything that looks too perfect is too perfect to be perfect.

― Dejan Stojanovic

This post is going to delve directly into this blog’s name, how to regularize a singularity, but in this case we are talking about an artificial one. When one is writing a computer program to solve differential equations it is easy to introduce a discontinuity into how the program operates. This is particularly true when you’re implementing an elaborate model or numerical method. In the process of taking care of special cases or making a program robust for general use, logic is employed. Should a circumstance arise that causes the program to fail, it can be detected and avoided by making a logical change in the operation. The most common way to do this uses logic in the program through an “if” statement or some sort of switch. When the “if” triggers on a floating-point value, the impact on the solution can be subtle and creates a host of practical issues.

Ability to find the answers is more important than ability to know the answers.

― Amit Kalantri

As computer program development becomes more rigorous testing of various sorts becomes important and valuable to quality of the work. One form of the testing is regression testing. Here the program is run through a series of usually simple problems with well-defined answers. If the program’s solution changes in some way that is unexpected, the testing should pick it up and alert the development team. In addition this testing often runs across a bunch of different computers to make sure the answers are the same, and the program works properly on all of them. For basic quality assessment and control, regression testing is essential. It is one of the hallmarks of serious, professional code development. The logic and if statements introducing discontinuous behavior into the code based on floating point numbers can wreck havoc with the testing! We can end up with a situation where the tests produce much different results because of infinitesimal changes in the numbers at some point in the calculation.

You might be asking how this can happen? This all seems rather disturbing, and it is. It is simple matter whenever the logical decision is made on the basis of a floating-point number. Consider a simple, but common bit of (pseudo) computer code,

if (value > 0.0) then

newvalue = value1

else

newvalue = value2

endif

which is a very simple test that one might see with upwind finite differences. In some cases a logical switch like this might trigger an elaborate mathematical expression, or even call a very different function or subroutine. Consider the case where the “value” is very near zero, but not exactly. In this case small differences in the quantity “zero” will trigger completely different evaluations of the logic. For special values (especially zero) this happens all the time. If the solution is dependent upon a certain sequence any regression test depending on this test could differ based on inconsequential numerical differences. As programs become more complex these branches and differences explode. Code development teams relying upon regression testing end up chasing this sort of problem over and over. It becomes a huge drain on the productivity and quality.

These problems can be replicated with a set of standard functions. The logic above can be replaced by a single statement using a “sign” function,

newvalue = sign(value) * value1 + 0.5*(1.0 – sign(value)) * value2

which gives exactly the same result as the if test in the previous paragraph. It is also prone to exactly the same problems in practical testing. These issues are the tip of a proverbial iceberg. It isn’t just regression testing that suffers from these issues, if the method for solution involves solving a nonlinear equation that goes through the above logic, the solution can stall and stagnate causing solution accuracy to suffer. The same switches can produce breaks in symmetry or bifurcation of solutions near critical points. Next, I will describe ways of implementing the sign function to alleviate these problems. It turns out that there is a whole family of functions that can replace the discontinuous behavior with something continuous, and the sign function can be used to construct other functions with the same switching behavior built in. I’ve written about some of these functions before in a different context where discontinuous logical functions were replaced by differentiable functions for the purpose of conducting modified equation analysis that rely upon valid Taylor series expansions.

Here are a couple of times I’ve hit upon this topic before: https://williamjrider.wordpress.com/2016/06/07/the-marvelous-magical-median/, https://williamjrider.wordpress.com/2015/08/17/evolution-equations-for-developing-improved-high-resolution-schemes-part-2/, https://williamjrider.wordpress.com/2016/06/22/a-path-to-better-limiters/. This post might be a good postscript to these because the techniques here can cure some of the practical ills remaining for these rather powerful methods. We see issues with solving nonlinear equations where limiters are used in discretizations, various symmetry breaking effects, and extreme sensitivity to initial conditions. As Iwill touch upon at the very end of this post, Riemann solvers-numerical flux functions can also benefit from this, but some technicalities must be proactively dealt with.

Slow is smooth; smooth is fast.

― Jack Coughlin

Using the sign function we can systematically remove the switching behavior that plagues regression testing, nonlinear solutions, symmetry preservation, and extreme sensitivity to initial conditions.

For me, the first function to start this was the hyperbolic tangent function. The basic behavior of this function acts to create a switch between two states based on the argument and steepness of the transition, as becomes larger, the function approaches the idealized step function. It turns out that there are a number of smooth functional representations of the sign function including , , and . There are many others that can be derived as well as other more exotic functions. These functions are used in other fields to remove the discontinuity from a switching function (https://en.wikipedia.org/wiki/Sigmoid_function ).

These functions provide a flexible foundation to build upon. As an initial example take the definition of the absolute value, (https://en.wikipedia.org/wiki/Sign_function ). This can be rearranged in a number of useful forms, . We can see that a simple smoothed version of the absolute value is . We can now build an entire family of softened or smoothed functions that can be differentiated (they are ). Each of the classical versions of these functions cannot be differentiated everywhere and create a host of problems in practical programs. Another common switching function are “min” and “max”. We can rewrite both functions as , and . The modified smooth versions are relatively obvious, , and . From this basic set of function we can build the backbone of the limiters, the minmod function and the marvelous magical median function. What we have removed in the process is the discontinuous switching process that can wreck havoc with finite precision arithmetic.

We note that there is a separate version of the softmin and softmax functions used in some optimization solutions (https://www.johndcook.com/blog/2010/01/13/soft-maximum/, https://en.wikipedia.org/wiki/Softmax_function, ). This uses a combination of exponentials and logrithmns to provide a continuously differentiable way to take the maximum (or minimum) of a set of arguments. My naming convention “soft” comes from being introduced to the ideas in this blog post. This separates the idea from a “hard” max where the arguments switch based on the precision of the floating-point numbers as opposed to being continuous. For completeness the softmax uses the following expression , which may be expanded to additional arguments without complications. By the same token we can define a “softmin” as that can be similarly be expanded to more arguments. In both cases the parameter controls the sharpness of the smoothed version of the standard function, the larger the value the closer the function is to the standard function.

Using our previous definitions of “softmax” we can derive a new version of “softabs”.   We rearrange to derive a . We start with the observation that therefore . We find that this version of the absolute value has much different properties than the previous softened version in some important ways. The key thing about this version of absolute value is always greater in value than the classical absolute value function. This would turn out be useful for use with Riemann solvers by not violating the entropy condition. With appropriate wavespeed estimates the entropy condition will be satisfied (wavespeed estimates are out of scope for this post). By the same token this absolute value is not valuable for limiters because of the same property!

Ultimately we want to understand whether these functions alter the basic accuracy-consistency or stability properties of the numerical methods based on using the classical functions. The answer to this question is subtle, but can be answered via analysis and numerical experiment. Not to belabor the details, but we use series expansions and discover that with appropriate regularization of the smoothed functions, we can use them to replace the classical functions and not undermine the accuracy of the discretization. This has been confirmed for the softened version of the “minmod” limiter. A downside of the softened limiters are small deviations from idealistic monotonticity preserving behavior.

Finally as eluded to early in the post, we can also use these functions to modify Riemann solvers. The first code example can form the logical basis for upwind bias with a finite difference by choosing a one-sided difference based upon the sign of the characteristic velocity. When Riemann solvers are examined we see that either “if” statements are used or when full flux functions are used absolute values (the flux is in a general sense a characteristic quantity multiplied by the characteristic velocity), the absolute value of the characteristic velocity introduces the sign convention the “if” statement provides.

The lingering problem with this approach is the concept of entropy violating approximations. This issue can easily be explained by looking at the smooth sign function compared with the standard form. Since the dissipation in the Riemann solver is proportional to the characteristic velocity, we can see that the smoothed sign function is everywhere less than the standard function resulting in less dissipation. This is a stability issue analogous to concerns around limiters where these smoothed functions are slightly more permissive. Using the exponential version of “softabs” where the value is always greater than the standard absolute value can modulate this permissive nature.

Let us study things that are no more. It is necessary to understand them, if only to avoid them.

― Victor Hugo