The Danger of Purity Tests

“With four parameters I can fit an elephant, and with five I can make him wiggle his trunk.” – John von Neumann

I’ve noticed a strange parallel between the forces that hinder progress in the technical world and in politics. This essay explores how imperfection can be an ally of progress, while perfectionism leads to stagnation and decline. One of the things I hold most dear is the continual need for progress. This is true for society and for science alike. The enemy of progress is a demand for purity of thought. Purity tests are toxic to political and cultural progress, and they are just as toxic in science. Here, I’ll take one example from social progress and map it onto math and physics.

The key to progress is accepting imperfections in the current state of affairs. The progress of the past was made by imperfect people with imperfect ideas, and they moved things ahead anyway. Through the lens of today, those imperfections are obvious. What we fail to embrace is that the imperfections are essential to making progress. This was true in the past and is even more true today. Purity tests and “absolutism” are deadly to progress and almost guaranteed to lead to setbacks.

The lesson here is one of strategic compromise. I’m arguing that absolutism holds progress back because it demands absolute answers. It does not allow adaptive, incremental progress. It is all or nothing. I’ve watched this play out in the obvious dysfunctions of our political system, and it also operates within technical and scientific work. When you raise a question or a problem, you’re met with dogma. It’s true across the board, whether you’re a physicist or a mathematician. When we become absolutists, we create dogma that props up existing views at the expense of progress.

Progress happens in the gray areas between perspectives. It is discovered by acknowledging that the perfect solution does not exist. This is true for technical problems, and it’s true for our political system. The key is to keep striving for perfection while knowing it can never be achieved.

“We have found it of paramount importance that in order to progress we must recognize our ignorance and leave room for doubt. Scientific knowledge is a body of statements of varying degrees of certainty, some most unsure, some nearly sure, but none absolutely certain.” – Richard Feynman

A Non-Scientific Example

Let me dive into the deep end. This is admittedly a dangerous approach, but it’s also instructive for the sake of clarity. I’m going to critique a place where this mentality shows up and hampers progress. The irony is that these particular purity tests kill progress and come from the progressive left. In a very real sense, the left is its own worst enemy. After that, I will discuss how, because scientists do exactly the same thing.

People on the left are fond of criticizing society’s heroes. In a nutshell, the main issue is the misapplication of today’s standards to yesterday’s people. Worse yet, they apply the most progressive standards, ones that aren’t even broadly accepted today. They then demand that the heroes be canceled and removed from the pantheon for these crimes. It is hard to imagine a more self-defeating way to catalyze progress. It only enrages the average person and makes the progressives look nuts.

As the USA approaches its 250th birthday, the Founding Fathers offer an instructive example. By modern standards of behavior, almost all of them would be viewed as conservatives or worse. Most were slaveowners. They engaged in abusive, exploitative practices that are illegal today. Their behaviors and life practices would be familiar to the MAGA movement. So some progressives want them canceled for this. This is idiocy. Let’s look at which Founding Fathers they would actually accept.

To put it differently, if the Founding Fathers had expressed the ideals and ideas the current left supports, they would have made no progress at all. They would have been imprisoned or institutionalized as criminally insane. Yet if we travel back 250 years, these men were viewed as radical progressives. They envisioned a society radically different from any that existed. Within the norms of 1776, they were arguably even more progressive than the people who criticize them today.

Another way to think about this is the Overton Window: the range of policy or social change that’s considered acceptable at a given moment. That range moves. The Overton Window of 1776 was driven by the Founding Fathers and the revolution itself. The possible changes in that era were vastly different from those of today. A standard conservative policy of today would have been considered radically left-wing then. The condemnation coming from the left is misplaced and illogical. Worse yet, it is completely counterproductive. It actually pushes today’s Overton Window away from progress.

Applying today’s standards to people from the past is plain stupid and reflects no logical understanding of how progress is made. If we make progress as a society, the things today’s people do will eventually be viewed unfavorably by progressives in the future. Applied with logical consistency, this same empathy would result in today’s progressives being canceled themselves. They would not stand up to the scrutiny of tomorrow’s progressives. Maybe those future progressives will act more rationally. We can hope.

The other thing that purity tests generate is mistrust. The demands of purity of thought from various parts of the academic left have destroyed trust. Institutions such as Universities have lost trust. This is a true threat to them. It has created a backlash. This doubles the damage to progress. The purity is alienating. It undermines the very institutions necessary to support progress.

What Does This Have to Do with Science?

“Essentially, all models are wrong, but some are useful.” – George Box

The concept of the Overton Window applies to science as well. There are ideas ripe for acceptance because the spectrum of the acceptable has finally opened to them. The story of “limiters” that I’ve told fits this. Overcoming Godunov’s barrier theorem went from impossible to possible to common practice. This shifts over time. This is the nature of progress, but it is also the fact of changing perspective. A variety of the great ideas of today would have been flatly rejected decades ago. Things that are acceptable and even commonplace today would have been viewed as outlandish in the past. This is the nature of progress, and we need to work with this reality.

A wonderful nexus for these problems is turbulence and the incompressible Navier-Stokes equations. I’ve criticized that construct repeatedly. Most of my critique centers on the lack of physical causality and realizability embedded in the choice of incompressibility. The model seems ill-suited to describing turbulence. The infinitely fast linear sound waves are a fatal flaw. In compressible flow, sound waves form shocks, meaning discontinuities. These shocks produce entropy at the cube of the variation in normal velocity. In compressible flows, shocks are ubiquitous, and it is pathological for them not to form. In my opinion, the rational conclusion is that turbulence is actually a compressible phenomenon. Its canonical behavior is, on balance, approximately incompressible, but entropy is produced naturally.

I’ll make a second argument here that is, in some ways, more compelling, and it invokes Occam’s razor. The phenomenon of turbulence is ubiquitous. It is everywhere in the world and the universe. We see it all around us every day, from the kitchen sink to the atmosphere and clouds to the cosmos. It is almost impossible to suppress. Incompressibility seems simple, pure. Some manipulations of the governing equations are enabled by it. This simplicity and purity of form are an illusion.

“Since all models are wrong the scientist cannot obtain a ‘correct’ one by excessive elaboration. On the contrary following William of Occam he should seek an economical description of natural phenomena.” – George Box

With that as a background, which I believe is an unassailable observation about our physical world, my mathematical observation is this: if the incompressible Navier-Stokes equations were the true basis for understanding turbulence, it would not be so difficult to solve them in a manner consistent with turbulence. Instead, many of our greatest mathematicians have struggled to show that these equations produce the structures necessary for turbulence. Where they have nearly succeeded, they have had to construct solutions so structurally bizarre and unrealizable that they defy description. If incompressible Navier-Stokes were the correct basis, the structures in the solutions would be common. Instead, they are pathological.

This points to a further argument that the equations are the wrong basis for understanding turbulence. If they were indeed the right equations, producing solutions consistent with turbulence would be far easier. It would be commonplace and simple. Instead, we are looking under a very classical lamppost for the keys to turbulence, and the keys are actually somewhere off in the dark. There is a trail of breadcrumbs we don’t even notice. We have mistaken a practical engineering model of incompressibility for a deeper scientific model. It is still useful for many applications. A deep scientific understanding of the physics of turbulence is not likely to be one of them.

Mathematical Purity Tests

“The purpose of computing is insight, not numbers.” – Richard Hamming

The example I mentioned a couple of posts ago about the publication content of the SIAM Journal of Numerical Analysis illustrates the dangers of the purity of thought. The main reason results have become scarce or even discouraged is the purity of thought ideas. The journal is that it has become focused only on concerns relevant to people who work in this stripped-down version of numerical analysis. This is math that is divorced from its utility. It is all ego and no sense. We do numerical analysis to produce numerical results. We do it for confidence in our algorithms, methods, and codes. The changes are akin to mental masturbation. In my opinion, this is a disservice to an area that applied math has contributed greatly to and is counterproductive.

A dismissal of mathematical progress by physicists is a prime example of absolutism working against progress. I have seen it in the reactions to my writing about the Lax equivalence theorem. In absolute terms, the theorem can be criticized for applying only to linear problems. The same is true of theories like total variation theory for hyperbolic PDEs. These critiques are made as the theory is fundamentally one-dimensional. Everything revolves around what you can rigorously prove. They fail to reflect the power of this work in areas where rigor cannot be achieved.

“We absolutely must leave room for doubt or there is no progress and no learning.” – Richard Feynman

A prime example is TVD theory, which provided a fundamental mathematical approach to limiters. Those limiters have been incredibly powerful, and they were largely developed by physicists. Even though the TVD result applies only to linear equations, it created a rigorous theory that set bounds on limiters. This helped make limiters more broadly accepted, especially in the engineering community. Despite that, the physics community I’ve worked in at the national labs doesn’t hold this in high regard and widely criticizes it for its narrow, rigorous applicability. I firmly believe that critique is only a threat to progress, and that it fails to recognize the benefits of the theory even where its rigorous applicability is limited.

The Lax equivalence theorem is limited. More deeply, it expresses the value proposition of increased computing power for solving partial differential equations. It articulates the basic components of verification practice with clarity. Its rigor is limited, and it does not hold in many cases, which remain challenges for mathematics. Still, it represents a huge breakthrough and a clear, crisp articulation of how you design numerical methods. It tells you how you evaluate their success. It tells you the value proposition for computing, clearly and crisply. In large part, its value proposition is seen in practice.

It is the essence of verification, but absolutists reject it because of its limited applicability rather than celebrating it as the progress it represents. Holding onto it is the glass-half-full view of progress. Rejecting it is the glass-half-empty view that represents stagnation and a lack of progress. This pattern recurs throughout mathematics whenever details undermine rigor. I think this is an excuse. It is a bad excuse at that.

It should be seen as a first step and as a gauntlet thrown down for greater mathematical results in the future. In all likelihood, we will never get a fully rigorous nonlinear theory for these equations. It will always be out of reach, but that does not mean we should stop reaching for it. This pattern is repeated over and over. There is a theory with limits on rigor and caveats. Nonetheless, the theory provides a foundation to build on. The same mentality holds for physical modeling as well. There is a lack of precision, rigor, and caveats across physical models. They are not held to the same standard.

The thing that doesn’t fit is the thing that is most interesting.” – Richard Feynman

Examples abound in numerical methods. Linear stability theory is essential to constructing methods. It only applies linearly and often to periodic problems, yet provides essential feedback to methods. The Lax-Wendroff theorem is another. It has limits and caveats around entropy conditions needed to select proper weak solutions. The key is to realize that these theorems are not bulletproof. The limits and caveats are places for continued work. Nonetheless, the basic principles and guidance are essential building blocks. Time and time again, we have found that linear theories guide nonlinear methods.

“Science is the belief in the ignorance of experts.”– Richard Feynman

Scientific Snake Oil

“The first principle is that you must not fool yourself, and you are the easiest person to fool.” – Richard Feynman

One thing I’ve learned over my career is that certain kinds of purity of thought can be particularly dangerous. That’s especially true today, when bullshit has become one of the most effective marketing schemes available. There are some prime examples to point to.

I saw the bullshit factory at work during the Exascale program. The claims of performance gains were hardly matched by reality, because the computers’ efficiency was dropping over time. Meanwhile, the things that actually make a computer valuable were not being invested in, which has led to a hollowing out of computational science. The marketing worked, and money flowed. We have faster computers that are very hard to use. We also saw code, methods, and algorithms stagnate. The insanity of hardware focus is being repeated with AI.

The same kind of bullshit surrounds the fusion power boom we’ve seen recently. It starts with people seeing money and making outlandish claims, like the National Ignition Facility’s “break-even” claim. That was an outstanding accomplishment, but it was hardly break-even. They cooked the books in the accounting to make it look that way. Still, the fusion evangelists and those who believe it must be the power source of the future jump on the bandwagon, and the result is irrational optimism about how close fusion energy is to being a viable source of electricity. I think it’s still quite far off. There are massive engineering and physics challenges we haven’t solved, and won’t until we can produce fusion at the scale and in the form required to actually generate electricity.

Similar bullshit comes from quantum computing. It has massive potential, but its real utility is far more limited than they would have you believe. Quantum projects and scientists bullshit their way to continued funding. Over time, when promises are not kept and potential is hyped in outlandish ways, trust erodes. Trust is one of the things most lacking in society, and these well-intentioned people end up becoming charlatans, exploiting the trust deficit and deepening it in the minds of people who feel duped.

At this point, you should be thinking, “What about AI?” AI is definitely another one of these over-inflated promises. I believe AI is a huge advance, on the scale of the Internet, as a breakthrough. Still, the AI charlatans (Altman, Amodei, Musk, and the rest) oversell it. They are trying to engineer IPOs out of their bullshit. Their claims are over the top and poorly thought through. They’ve inflated a giant stock market bubble while destroying trust even faster than quantum and fusion did. AI evangelists are scaring people left and right. In my opinion, AI is huge, but we are approaching it wrong. To get it right, we need to get down in the mud. It is going to be harder and fuzzier than they describe.

To be clear, each of these is worth pursuing. Fusion is an important technology for future energy production. Quantum is an essential modality for future computing. AI is going to change our economy and our future. In each case, the story is far murkier than the evangelists would have you believe. Fusion and quantum both face massive engineering challenges that stand in the way of success. AI needs a far more difficult kind of societal engineering to succeed. It also needs to avoid the trap that social media fell into, where a new technology was allowed to prey on society in order to make short-term profit. If AI follows the same path (and it appears to be), we will fuck this up completely.

“I can live with doubt and uncertainty and not knowing. I think it’s much more interesting to live not knowing than to have answers which might be wrong.” – Richard Feynman

Imperfection Defines the Space for Progress

In science, there is rarely a simple, easy, pure theory. All theory is flawed. The real work, and the real path to success, is uneven and difficult. Success is found in letting go of the purity of thought. Things are not black and white. Reality sits somewhere between the two, in the gray. The engineering challenges above are all messy, wicked problems. If we want to succeed at these big ideas, the wicked problems have to be solved. The purists avoid all of this. It isn’t good marketing, and it doesn’t bring in money. It is hard. Until we do the hard stuff, the big promises will keep falling short.

In science, the turn away from rigorous verification and validation is rooted in an emphasis on purity of thought. This approach focuses on exposing flaws and pointing out where the gray area lies. When marketing and funding depend on the acceptance of the purity of thought, V&V is the enemy. It attacks the purity of thought, which props up almost everything in the world of funding. V&V is rejected because it unnecessarily complicates things. For example, with exascale computing, why verify when you can assume convergence? Without proof, you work under the presumption that a faster, bigger computer will yield better answers. This approach relies on faith rather than evidence.

When I conducted verification, it was an active effort to confirm theoretical expectations. In many cases, the theory holds even when its rigorous applicability is not available. That is notable because it points to areas where the theory’s rigor could perhaps be extended.

There are also cases, especially where instability and turbulence exist, where the linear theory falls apart. In those places, the theory does not offer answers or expectations, and something else needs to be defined to fill the gap more proactively. This is the scientific process in operation. The key is to collect evidence and assess whether it matches the available theory. If it does, great. If it does not, you know where you have work to do. This is the process we are failing to engage in.

Validation provides feedback to theory and experiment, while verification provides feedback to mathematics and numerics. In validation, you need to understand the uncertainty and limitations of theory and experiment. Similarly, the limitations of mathematics and methods serve as the foundation for verification. In both cases, the feedback can highlight areas where more scientific effort is needed. It also points to where breakthroughs have been achieved with evidence that provides proof that this can be claimed.

“The idea that no one really knew how to run a government led to the idea that we should arrange a system by which new ideas could be developed, tried out, and tossed out if necessary, with more new ideas brought in, a trial and error system.” – Richard Feynman

In my view, the same principle applies politically. The past and the present are imperfect, but they represent a struggle toward perfection. This is a project we should engage in and move toward. It is a struggle that represents the best of humanity, and we take it on knowing that perfection is impossible. The way forward is to reject absolutism and live in the gray, whether in political discourse or in scientific and technical discourse. In politics, this means embracing “a more perfect union.” This is the essence of positive patriotism.

“We the People of the United States, in Order to form a more perfect Union…” – Preamble, U.S. Constitution

In science, it means building progress on imperfect theory and imperfect results. You examine where you have uncertainty or a lack of rigor, and you look for opportunities to push back against either one. You push back uncertainty with new methods of measurement or new analysis of data. You push back the lack of rigor with new physical theories or new theorems. New tools, ideas, and approaches all contribute to progress. Sometimes an idea needs wicked engineering work before it will succeed. The key is to recognize and embrace the imperfections we begin with. Once that reality is accepted, success and progress become possible.