“The miracle of the appropriateness of the language of mathematics for the formulation of the laws of physics is a wonderful gift which we neither understand nor deserve.” — Eugene Wigner

I’ll start with a bit of subtext, both a boast and a complaint. All my degrees are in engineering, specifically nuclear engineering. I worked at Los Alamos for 20 years, ending up in the infamous X Division, also known as Applied Theoretical Physics. Working there gives you an honorary standing as a weapons physicist, and it becomes part of your identity. As such there is a heavy emphasis on the primacy of physical theory. By the same token math gets somewhat diminished in the Los Alamos pantheon. What is most important is recognizing when math supports the physics, impact is massive. One of the biggest problems for Los Alamos is ignoring math that matters. In its place they emphasize tradition.

One thing I’ve always appreciated is applied math. One of the greatest compliments I’ve ever received came from a coworker at Lawrence Livermore, who assumed I had degrees in applied math rather than engineering because of my knowledge of and taste for the subject. The fact that I could pass myself off as a mathematician to someone holding a Berkeley PhD in the field has always been a source of pride. I truly value applied math, which is why I see the tragedy in what has happened to it and in how it is not being used to solve society’s most difficult problems.

“Computers are useless. They can only give you answers.” — Pablo Picasso

The SIAM Journal on Numerical Analysis is emblematic of many of these issues. I was at work one day lamenting the demise of this journal. I may not have realized it at the time, but someone in the audience was an associate editor. I said something to the effect of, “That journal used to be good, and now it sucks.” I still stand by that. The utility of the journal has been replaced by academic purity, and that is extremely bad for applied mathematics. It has contributed to the field’s declining impact on the technical world.

To the point: this journal used to provide clear, evidence-based proofs, with numerical evidence that supported what could be observed in the real world. That was the essence of verification. When the journal published papers that offered practical demonstrations of the math, it made an impact. Now it has neither. It reflects an almost suicidal approach to managing a field: numerical analysis without numerical results. It can certainly be done, but it is certainly not advisable.

What has changed is that you are now left with a theorem and a proof, with any demonstration reduced to an exercise for the reader, or simply a matter of trusting the mathematics. For me, seeing that the math has real-world impact is what encourages me to engage with it: to read it, digest it, and understand it. It also gives me confidence that the mathematics is correct. This is a deeply disturbing shift in the field. It is as if they have mindfully chosen to be irrelevant.

We have seen applied math’s impact wane across broader technical fields. The example from this journal is more an indictment of the gatekeepers who have steered it toward uselessness. This is a self-inflicted injury, where academic purity has replaced the focus on being useful and impactful. I find this development truly sad and in need of deep repair. I stopped reading the journal closely. I began treating each article with suspicion: I would read the abstract, look for results, and move on if it wasn’t compelling. If I didn’t find results, it generally wasn’t worth my time.

“The purpose of computing is insight, not numbers.” — Richard Hamming

I give a talk on shock physics verification and on how many mathematical results bear directly on what you actually see. I’ve spent a significant part of my career doing verification work for various shockwave solutions. What people often don’t recognize is that there are a number of reasonably rigorous results in shock physics. Much of this is built on foundational theory. The Lax-Wendroff theorem states that putting a solution in conservation form ensures you obtain a weak solution to the hyperbolic conservation law. The catch is that a weak solution is not unique, and it can be physically incorrect, which is why an entropy condition is needed. This can come from upwinding, Riemann solvers, or artificial viscosity. Further work by Peter Lax, with Ami Harten and James Hyman, built on this. Accuracy of the solution is a key part of verification, and for shocks a few foundational works tell us what to expect.

One is my work with Jeff Banks and Tariq Aslam on the behavior of linear waves. Those waves converge at a sublinear rate, and they typically meet the pacing accuracy requirement under mesh refinement. This builds on work by Majda and Osher, who showed that you get first-order accuracy for results emanating from any discontinuity. This applies to the Riemann problem, except for the linearly discontinuous wave, where my results with Jeff and Tariq apply. Finally, there is a result by LeFloch and Hou on what happens when you do not use conservation: the result will be wrong, and the error will be proportional to entropy production. The caveat is that entropy production is itself necessary to get a physically correct solution. This is quite a bind.

What has always astounded me is how little these results are known and used at the national laboratories where I worked. Indeed, the ignorance of this body of work, set against my own knowledge of it, played a critical role in the unethical behavior that led me to retire rather than continue working in a place where ignorance is celebrated.

What I’ve observed is that these mathematical results consistently match what we see in practice. Even though the equations are nonlinear and the methods are numerical, the connection between the mathematical theory and the observed results is clear and repeatable.

“Young man, in mathematics you don’t understand things. You just get used to them.” – John Von Neumann

This gets at one of my conundrums. The national labs I worked at tend to ignore these results and even treat them with considerable animosity. The ignorance is especially profound when it comes to computing, which is reflected in how heavily they emphasize high-performance computing. In my opinion, that approach leads to enormous wasted effort relative to investment in methods and mathematics.

One of the ironies is that the people doing verification work often remain ignorant of the math that underpins what they should expect, and as a result they lose focus. This shows up most clearly where the solutions are discontinuous, even when exact solutions are available. The fact that a method does not deliver the expected order of accuracy at a discontinuity does not mean you can stop measuring its accuracy and convergence rate, especially since those circumstances are usually much closer to what you see when the method and code are applied to real problems.

This is captured by the relative lack of emphasis on the Lax equivalence theorem. Strictly speaking, it applies narrowly, and generally to equations and methods not in use today. Still, it captures the exact requirements we apply uniformly in verification work. Why do we expect precisely what the theorem spells out in cases where it does not formally apply? And why do we dismiss it when we are not rigorously applying it? It is exactly what we expect, and exactly what we demand from our methods. We define methods that fail to meet these expectations as incorrect or full of bugs.

The big message is that some degree of rigor is lacking everywhere, and yet we still take a leap of faith forward. People generally recognize that rigor and precision are lacking in physics, particularly in important areas. The same is true for mathematics.

Verification and validation are where these gaps can be exposed and addressed. By comparing actual results to mathematical derivations, we can demonstrate the level of rigor in the mathematics. Similarly, validation can reveal the precision of the physics. Solving the equations brings these two together into a unified exercise. To understand where rigor is lacking and where work is needed, we must separate which part of the problem lies in mathematics and which lies in physics. From my experience, both areas carry significant burdens that our current science programs do not adequately address.

“As far as the laws of mathematics refer to reality, they are not certain; and as far as they are certain, they do not refer to reality.” — Albert Einstein

The Navier-Stokes equations are amazing and largely predictive, but they are imperfect. They break down in situations like turbulence. These are the places where both physics and math fall short, and real classical science is needed to fill in the gaps. I will express clear doubts that AI is a significant path forward here. I’ve written about the problems with the incompressibility usually invoked for turbulence. By all accounts it removes the discontinuous behavior the observations point to. It removes thermodynamics. Turbulence is one of the most universal means for assuring the second law in our universe.

“(turbulence is ) the most important unsolved problem of classical physics.” – Richard Feynmann

Let me strike another blow against the incompressible Navier-Stokes equation’s ability to explain turbulence. This is, of course, the Clay Millennium Prize problem. Notable mathematicians, including Terence Tao, have brought their considerable talents and intellect to the task of showing that these equations produce a singularity. That singularity is necessary to explain the observations we have regarding turbulence. This points toward the problem being posed improperly in the first place.

The fact that it has not yielded to this assault is quite telling. Turbulence is such a universal phenomenon that I would submit that such singularities must be simple and common. They would naturally arise in the solutions. That they are so unyielding to analysis suggests that the equations themselves are the problem. The key point is that imposing incompressibility on these equations makes them unphysical. It blocks one from finding a reasonable solution that explains the phenomena. That’s what needs to be removed from the equation set. The problems with incompressibility from a physical perspective are mirrored by the mathematical challenges it creates. The incompressibility constraint makes the equations elliptic. That ellipticity is exactly what makes them so difficult. Removing it would yield conditions for a solution and the discontinuous behavior needed for the math to match physical reality.

The incompressibility constraint is the source of the mathematical difficulty. It is also the source of the physical difficulty. It renders the equation to have an elliptic character, which is contemptible both on physical grounds as well as mathematical grounds. It’s condemned from both sides. Removing it would allow the solution to include the discontinuities physical theory demands. To me, it’s obvious That it is the thing that is needed. Yet we persist in beating our heads against the proverbial wall.

“Far better an approximate answer to the right question, which is often vague, than an exact answer to the wrong question, which can always be made precise.”– John Tukey

Solid mechanics equations work well when things are well behaved, but they fail when you have fracture and spall. We need to be mindful of where the mathematics breaks down, which often happens at discontinuities. The dynamics of shocks and turbulence share significant dynamical similarity, and this might yield a path to progress. The places where the continuum breakdown are the miscrostructural details. The basic equations average over these and the small scale structure has a macroscopic impact. Models bootstrap this influence, but the models are ad hoc and unconvincing. The mysteries of physics and mathematics run through discontinuous phenomena. These are weak solutions, and much is already known about them.

One key lesson I took from my time at Los Alamos and Sandia was the importance of being a first mover in a field. This is reflected in the continued commitment to numerical methods based on von Neumann’s original vision, as refined by Richtmyer’s work on artificial viscosity. There are two elements to this, both worth criticizing in depth. The first criticism is the devotion to the Lagrangian frame of reference. It becomes increasingly absurd as virtually every physical system evolves over time. Effects we commonly attribute to turbulence and instabilities eventually undermine the Lagrangian description. They render it useless. The Lagrangian description is rooted in classical physics. The imperfections become exposed as the physics grows more complex.

Physicists are still eager to think in this Lagrangian frame because it is the core academic lineage. The numerical side of things may be even worse. We continue to use methods that have been shown to have critical flaws, as Peter Lax so keenly and ably demonstrated. Most acutely, this shows up in the failure to solve the equations in conservation form. The Lax-Wendroff theorem makes is crystal clear. I remain somewhat flummoxed by the lack of recognition of this critical flaw and the continued adherence to solving these equations in non-conservative form. The lack of progress due to this intransigence is perplexing. The right response would be to acknowledge and react to the mathematics..

The key to advancing science is recognizing the primacy of observation and theory. New technologies like computing and artificial intelligence do not displace these fundamentals. They augment them. The path forward is to make the best of this augmentation while preserving and supporting the basics. Instead we have allowed the basic to wither away.

What we have lost sight of is the importance of the fundamentals. The core aspects of understanding the universe, rooted in theoretical models. We should always remind ourselves There mathematics and its partnership with physics gains value. This partnership reaches its zenith in the practice of numerical methods for these models. The full power of AI will be most fully realized by pairing it with applied mathematics to a much greater degree than we do today.

In the future, we will see that diminishing applied mathematics in the face of these new technologies has been a serious mistake. That mistake is setting progress back. Continued emphasis on math is the way forward.

There are other places where things break down as well, such as black holes, where the continuum equations and relativity break down. Are these really just discontinuities, or does physics take over? At large and small scales, the separation disappears, and one intrudes into the other. All of these gaps are the places where we need to work. Over the past several decades we have stepped away from attack on those challenges. Instead looking toward silver bullets of exascale computing or AI. Both means are useful, but do not veer toward explanation.

AI can fill the gaps statistically and projecting observations into modeling that can mimic. The thing it does not do is explain and understand the gaps. This is useful, but not an endpoint. The same holds for computing. It has utility and provides a temporary relief, but not the science. The ultimate goal of science is to explain the Universe. The place to do this is constructive physics models. These are mathematical in nature. This is where applied math is essential. It provides rigor and deep structural knowledge of the equations paired with physics. Together this provides a springboard for computing to work. This includes AI, which operates in the gaps physics and math leaves behind. The smaller the gaps, the better the understanding. This path is what we should pursue.

References

Lax, Peter D., and Robert D. Richtmyer. 1956. “Survey of the Stability of Linear Finite Difference Equations.” Communications on Pure and Applied Mathematics 9 (2): 267–293. https://doi.org/10.1002/cpa.3160090206.

Lax, Peter D., and Burton Wendroff. 1960. “Systems of Conservation Laws.” Communications on Pure and Applied Mathematics 13 (2): 217–237. https://doi.org/10.1002/cpa.3160130205.

Majda, Andrew, and Stanley Osher. 1977. “Propagation of Error into Regions of Smoothness for Accurate Difference Approximations to Hyperbolic Equations.” Communications on Pure and Applied Mathematics 30 (6): 671–705. https://doi.org/10.1002/cpa.3160300602.

Harten, Amiram, James M. Hyman, and Peter D. Lax. 1976. “On Finite-Difference Approximations and Entropy Conditions for Shocks.” Communications on Pure and Applied Mathematics 29 (3): 297–322. https://doi.org/10.1002/cpa.3160290305.

Hou, Thomas Y., and Philippe G. LeFloch. 1994. “Why Nonconservative Schemes Converge to Wrong Solutions: Error Analysis.” Mathematics of Computation 62 (206): 497–530. https://doi.org/10.1090/S0025-5718-1994-1201068-0.

Banks, J. W., T. Aslam, and W. J. Rider. 2008. “On Sub-linear Convergence for Linearly Degenerate Waves in Capturing Schemes.” Journal of Computational Physics 227 (14): 6985–7002. https://doi.org/10.1016/j.jcp.2008.04.002.