I wrote this in a notebook a few weeks ago and it would be a good idea to explain myself. The lack of certainty and chaos in the Universe is obvious, yet so much of our modeling is based on deterministic thinking. This legacy of Newton pollutes so much of
our thinking that it is probably one of the greatest impediments to progress today. It is an “unknown known” that I wrote about a couple of days ago. I’m not going to go down the philosophical rabbit hold of “free will” or any of that. My point-of-view that this sort of discussion is fairly pointless, for all practical purposes we can never know our initial or boundary conditions well enough to treat our World as deterministic.

The assumption of an absolute determinism is the essential foundation of every scientific enquiry.

― Max Planck

It is impossible to trap modern physics into predicting anything with perfect determinism because it deals with probabilities from the outset.

― Arthur Stanley Eddington
Newton’s laws were revelation, but also woefully incomplete. They spawned a revolution in science that we benefit from today, but their power was not fully realized until many revisions were made. Nonetheless, many scientists are still enthralled by the beauty and purity of his perfects laws. Examples abound and perhaps most acutely with the Navier-Stokes equations. I regard the Clay prize,

http://www.claymath.org/millenium-problems/navier–stokes-equation ,

associated with the proof of existence for the 3-D Navier-Stokes equations to be outright foolishness. It is the enshrinement of mathematics as divorced from reality. The reason is several fold: the equations as given are patently unphysical, and singularities are a relic of the same Newtonian thinking that defines the Universe as a clock simply moving forward with complete determinism.

This isn’t to say that the Navier-Stokes equations aren’t useful as approximations or for modeling under some circumstances, they demonstrably are. The problem is that they have taken on all too great a meaning, and we are asking questions of them they shouldn’t be answering. The specific unphysical aspect of the equations is the divergence free velocity field, which implies infinite sound speeds and divorces the equations from thermodynamics. This is strike one.

Strike two is the notion of needing a singularity to define its characteristic dissipative nature. This is a bit of a physical Mobius strip where the equations are simply showing their limits of applicability. By removing thermodynamics and sound waves from the system, the natural nonlinear mechanism for singularities goes out too. The observations of flow in nature indicate the presence of dissipation without specific dependence on the value for the physical viscosity, i.e., evidence of a singularity. The result is an enormous waste of time and energy.

… Nature almost surely operates by combining chance with necessity, randomness with determinism…

― Eric Chaisson

This is only one instance where determinism is hurting science. In general the modeling of the World or Universe proceeds along lines that implicitly expect determinism despite all the evidence to the contrary. A lot of the time the variation in behavior of a system is relatively small, or the nature of variability is reliable and can be captured in the constitutive laws. A problem that we are increasingly facing is the solution of systems where length and time scales where the variability exists are coming into the resolution of our codes. Continuing to promote the fallacy that the system is deterministic is simply wrong on the face of it.

In some cases the lack of deterministic evolution is obvious like with weather. There some of the issues are being attacked in a fairly head on manner. In other cases like continuum mechanics, the deterministic mindset is so pervasive that the old ways continue unabated. A bit of multiscale thinking is creeping in, but itself is polluted by determinism. Our models are getting to the point where grain structure and material structure can be directly modeling, yet we persist in using models defined when length and time scales were vastly larger. Our approach needs to change for progress to be made.

The uncertainty principle signaled an end to Laplace’s dream of a theory of science, a model of the universe that would be completely deterministic. We certainly cannot predict future events exactly if we cannot even measure the present state of the universe precisely!

We could still imagine that there is a set of laws that determine events completely for some supernatural being who, unlike us, could observe the present state of the universe without disturbing it. However, such models of the universe are not of much interest to us ordinary mortals. It seems better to employ the principle of economy known as Occam’s razor and cut out all the features of the theory that cannot be observed.

― Stephen Hawking

When we change the way we communicate, we change society

― Clay Shirky

Over the weekend a good friend of my wife (her best friend from childhood) was going through a box of mementos and came across letters my wife wrote her over thirty years ago while they were in college. One of the letters was written shortly after my wife and I started dating. In the letter she expressed herself honestly to her friend, and the results were stunning. My reaction to reading it was the same as my wife, an outpouring of emotion in seeing her feelings about me so earnestly expressed. It was remarkable, and incredibly touching. It was like looking back in time and seeing yourself through today’s eyes.

This morning I began to wonder could this happen to a couple starting out today, thirty years from now. The sort of letter my wife wrote doesn’t get written today. We have Facebook, Twitter, instant messaging and e-mail. We don’t write letters with the expectation that no answer will come for days to weeks. Because communications like these can be so easily and cavalierly forwarded or seen by others, the honesty and emotion is usually lacking. As such, the communications are a bit less intense or memorable. Perhaps we aren’t as honest with others or ourselves about things.

Current communications also don’t have the staying power of the letter. They don’t get saved. I can’t lay my hands on emails much older than six or seven years back, and usually less. I wonder how much of history is getting lost in the ephemeral way we communicate. It is worth considering. Generally, I am unabashed in my enthusiasm about technology, but this episode gave me pause. Maybe we don’t even realize what we’ve lost. We have gained a lot, but we may have lost a bit of our soul.

Reports that say that something hasn’t happened are always interesting to me, because as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns — the ones we don’t know we don’t know. And if one looks throughout the history of our country and other free countries, it is the latter category that tends to be the difficult ones.

Donald Rumsfeld

http://en.wikipedia.org/wiki/There_are_known_knowns

Donald Rumsfeld made this marvelous quote in a remarkably cynical way, as a way to avoid answering a question. It has been a source of much discussion, a recent documentary

http://en.wikipedia.org/wiki/The_Unknown_Known

and the most obvious cultural touchstone for uncertainty quantification. He also forgot the most important category, the unknown known, and the axiomatic things that we don’t even question or think about. For Rumsfeld it was the unknown knowns that more than anything drove us to war and influenced decisions in ways that we never fully acknowledged. All of us have this sort of blind side, the things that drive our decisions that we don’t even think about.

So what our some of the unknown knowns for that I deal with?

For example at Sandia the Finite Element Method is an unknown known. It is applied without thought or debate, it is just assumed to be the method of choice. There is
no discussion or thought with the choice as it is applied axiomatically. Even when non-finite element methods are used, they are called a finite element discretization as a way to smooth the way to acceptance.

I’ve noticed that our viewpoint on modeling in the continuum hypothesis is similarly axiomatic. In many ways it is remarkable that we continue to solve problems in exactly the same philosophical way that Von Neumann envisioned even though computers have advanced by a factor of more than quadrillion in power. In some cases we don’t include details of physics that now come into view such as grain structure, texture, anisotropic properties, etc. In hydrocodes we simply “paint” materials into regions and apply homogeneous properties even though the materials are definitely heterogeneous at the scale of the mesh. Generally speaking, the changes necessary to model things correctly is barely on the technical agenda. These features are more prevalent in
weather and climate where the differences in the physics are more obvious. We know that the asymptotics differ and the equations need to change. In continuum mechanics this is beginning to dawn on people too.

I suppose it is tempting, if the only tool you have is a hammer, to treat everything as if it were a nail.

― Abraham Maslow

Our society has many of its own axioms, such as “American exceptionalism” which is either axiomatically good or bad depending on who you talk to. The view of socialism, free enterprise, and liberalism all fall prey to the unknown known. It has become the prevalent way of managing complexity and simply assuming the answer is easier than thinking about the inherent complexity of things. This thought process is the heart of racism, sexism, gay bashing and a host of societies greatest ills. By simply assuming certain things to be true without question makes life easy, but it also allows people to do terrible things with complete justification. This brings us back to Rumsfeld.

Your assumptions are your windows on the world. Scrub them off every once in a while, or the light won’t come in.

― Isaac Asimov

There is nothing quite so useless as doing with great efficiency something that should not be done at all.

– Peter Drucker

In today’s America it is axiomatic that the concept that applying corporate principles to organizational governance is good and appropriate. It is applied without question and used to justify all manner of mismanagement. The reality is that corporate governance principles are somewhere between inappropriate to completely incompetent for managing research institutions. I’d argue that they are ruining the economy itself because the current “principles” are oriented toward the benefit of the few people at the top of the food chain.

Doing the right thing is more important than doing the thing right.

– Peter Drucker

The cornerstone of corporate governance is the maximization of shareholder value. Almost every business decision flows from the application of this principle. This principle greatly benefits a select few and often costs other stakeholders greatly. Over time, the shareholder class and the management of businesses have merged into the same thing. Business is now managed to enrich its management. It is basic conflict of interest, and society as a whole is paying the price. The business principles being used are bad for the businesses as they are used as sources of money that is systematically siphoned off to compensate “shareholders”.

All of this is at the cost of the long-term health of the companies, the jobs for the employees and the communities they are in. The shareholders don’t care because they have already had their payday.

 Management is doing things right. Leadership is doing the right things

-Peter Drucker

The consequence is that we are over-managed and under-led. These principles are slowly destroying our National system of research laboratories. Whether it is the department of energy or defense, or NASA or NOAA, we are strangling our research through systematic misapplication of management principles that only benefit a few. Take community service like the United Way as an example. Institutions will manage an employee-giving program and then have the institution take credit for the good.

All the while they systematically suppress the employees wages and increase executive compensation. They benchmark themselves to corporations to justify everything, and no one even blinks.

I worry that this process will continue until we have a crisis. Our destruction of our research capability in the name of applying business principles may finally end with disaster. Our dominance of science and research is ending. Europe and China are overtaking the United States. Our government seems to be in denial of this, but it is more obvious all the time. A large part of the blame can be laid at the feet of the misapplication of corporate governance to research institutions.

It matters not how a man dies, but how he lives.

Insanity is everyone expecting you not to fall apart when you find out everything you believed in was a lie.

When you are doing a single calculation a couple of things can happen; the calculation can run to completion and give you an answer, or the calculation can bomb. In reality we have four distinct possibilities:

1. You will converge to the correct solution
2. You will converge to a solution, but not the correct one
3. You get a solution, but don’t converge
4. No solution, unstable (code bombs).

This is really a continuum of solutions with varying degrees of each of these four categories. For example there might be a big difference between solutions that are wandering in phase space as opposed to one that is bouncing between several possible equilibrium points.

One day everything will be well, that is our hope. Everything’s fine today, that is our illusion

Too often we simply see a single calculation be the end product. Determining what category your calculation is actually in has great bearing on the credibility of the calculation. In other words the credibility of a single isolated calculation is questionable a priori. You really need to be offering ensembles of calculations that together define the credibility rather than a single calculation.

The point is that if your calculation runs to completion it could be in one of three states. You should be quite interested in which one. Verification is the vehicle to help you figure this out. You can really only start to sleuth out what state the calculation is in by doing a sequence of nearby calculations using variations in the models and methods. If you run a family of problems you will generally find that the easier problems converge to the exact solution without problems (assuming you have an exact solution). You can only distinguish between 1 & 2 when you have an exact solution.

As you raise the problem difficulty either by varying the parameters or geometry or materials, you will gradually get solutions that depart of the exact one, but converge to some solution. As the conditions for the problem become more extreme, the code will stop converging to a single answer, and ultimately encounter stability problems. For more complicated problems especially in multiple dimensions one may always be looking at problems that don’t converge to a single solution, nonetheless you should be asking these questions all the time.

Yesterday’s adaptations are today’s routines.

Anything worth doing, is worth doing right.

I’m pretty sure that I invented the term “viewgraph norm”. It isn’t a pretty story since the invention of the term came in a fit of frustration and disgust at a co-worker, but history doesn’t have to be pretty. The viewgraph norm is the modern compliment to the “eyeball norm”. Comparing results in the eyeball norm is not precise and doesn’t constitute proof, it’s a rough check. Sometimes it is offered as evidence because the author doesn’t want to look to close either because they are lazy, or they know that the result won’t bear up under scrutiny.

It was back in 1997 or 1998 and I was working in Los Alamos’ Applied Theoretical Physics Division (i.e., the infamous X-Division). Every week or two we’d have a “work in progress” seminar given by one of the staff on an ongoing project. One of these talks was on comparing the solution on a number of standard test problems for shock physics. The talk looked at a number of the available computer codes on a number of test problems with analytical solutions. As the talk proceeded, it became crystal clear that the analytical solution would be used only for plotting.

The talk was simply a series of plots comparing the code solution to the plot of the analytical. The message was clear, everything is fine, and all the codes converge to the analytical. Never once was an actual numerical error computed, or convergence actually checked. I became disgusted and increasingly agitated because I knew the truth was far different, many, if not most of the codes were not convergent to the analytical solution because I had been running the same problems with the same codes.

Finally near the end of the hour and finding myself utterly revolted by the whole talk, I stormed out and exclaimed, “It converges in the viewgraph norm!”

The term lived on, and other used it to call out charlatans employing the same sort of proof in their talks, with comments like “nice viewgraph norm!” Over time, I developed a sort of metric for the actual quality of a viewgraph norm. The better a result actually is, the longer the speaker will leave the viewgraph up to be examined. If the result is poor, the viewgraph won’t be available for examination for more than a few seconds.

It also turns out that when it comes to shock tube solutions, the viewgraph norm is the standard. No one even today publishes the error magnitudes with solutions even though it is trivial, and the differences between methods (or codes) is quite substantial.

Do ordinary things extraordinarily well.

Late last week I tweeted that I thought high performance computing (HPC) was a zombie. I’ll admit that this is an intentionally provocative statement emblematic of the age of Twitter, but it is also worth some deeper thought too. So in my first daily blog post let me explain.

 The riskiest thing we can do is just maintain the status quo.

― Bob Iger

A zombie is a fearsome motif of modern horror, the product of post-modern fears of technology, as well as a metaphor for the dehumanizing effect of the modern world. The zombie lives the existence of the undead driven by an inhuman hunger for flesh (or brains!). It has no other purpose, but to mindlessly consume the living.

So, why am I calling supercomputing a zombie?

Because it’s mindless, just lurching forward consuming resources without any vision of where it is going or why. We are just trying to have the fastest computer, so that we have the fastest computer instead of China. Whether the computer is useful for anything is really secondary or tertiary to the purpose of having the fastest computer. The unfortunate side-effect of this approach is that the computers are increasingly difficult to solve problems on, and little or no thought is going into how to use them.

At some point in time around 1990 people got the idea that we should rate supercomputer utility by their speed instead of a holistic view of the computer’s problem-solving utility. At some deeper level a computer is simply a tool to be used. We don’t endeavor to have the World’s largest hammer or wrench; we recognize that these are tools whose function is essential. We don’t recognize this about computers, for some reason their status as tools has been lost. In this sense supercomputing research has become a bit of a fetish.

There are three classes of men; the retrograde, the stationary and the progressive.

― Johann Kaspar Lavater

A long time ago supercomputing was about solving problems, and it mattered whether the computer was actually useful for solving problems. Originally scientists envisioned solving problems through computation and then build computers to bring this vision to fruition. Speed was always welcome as long as the speed was useful. Along the way supercomputing underwent the transition to massively parallel computing as a matter of course.

Around that same time supercomputing was equivocated with National Security. The virus to create the zombie has infected the victim. It was just a matter of time before the zombie began to lurch forward hungrily devouring resources. The program to develop supercomputers had to be successful so they developed a new metric of success, weak scaling. Weak scaling is the ability to get ever-higher performance as the number of processors used to solve a problem grows as the problem size grows as well. It’s validity is predicated on the “bigger is better” point-of-view”.

What’s measured improves

― Peter F. Drucker

The problem with weak scaling is that it allows the computational intensity to drop (which it has) while yielding success. As a result weak scaling increasingly drives our computer development and the algorithms used on them. All the while their actual computational performance is plummeting. None of these measurements really have much to do with problem solving, just declarative success with building a faster computer. All of these advances only improve problem solving as a collateral benefit rather than the central purpose.

This is the zombie, a pursuit of a mindless goal without regard to any purpose beyond the pursuit itself.

Two reasons why people hate and/or fight change: (1) People fear the unknown; and (2) There are always people profiting from how things are.

― Mokokoma Mokhonoana

I been contemplating things over the weekend and decided that a change was needed. This blog is intended primarily as a place for me to focus on positive habits like writing every day, and establishing more of an online presence. I think its working well, but I need change it up. I’ve decided to move to a daily blog post instead of the weekly one. The writing is usually related to something I’m working on, or thinking about, or ne
ed to talk about.

So, look for a new post every day starting later today.

We become what we repeatedly do.

― Sean Covey

A man who can’t bear to share his habits is a man who needs to quit them.

― Stephen King

Good habits are worth being fanatical about.

― John Irving

The purpose of computing is insight, not pictures.

– Nick Trefethen channeling his modern Richard Hamming

One of the most impressive achievements of computing is the stunning progress in simulating physical phenomena. Fluid mechanics holds a special place in the pantheon of scientific computing being among the first fields to see the achievements, and remains an exemplar of computing’s impact on science. The discipline of computational fluid dynamics (CFD) defines this capability being responsible for many of the most iconic uses of computing and the visual power available today. Often when the results are given more to satisfy the eye rather than the mind CFD gets rechristened “Colorful Fluid Dynamics” as a slam. This is a sharp poke in the eye to the well-intentioned scientist seeking to impress someone through glitz rather than though convincing intellectual arguments.

Or at least I thought as much.

Many in the CFD world seem to accept the label of colorful fluid dynamics as a compliment and without a hint of irony. The eye poking isn’t even noticed. It is so common now for results to be taken as accurate simply because the graphics look cool that many rarely bother to do any real scientific work examining the validity of their calculations. Has the vapid and vacuous nature of the modern celebrity driven culture infected science? I think it has and certainly to a degree that I’m not comfortable with. Even worse it’s the extent to which color fluid dynamics has become a veritable currency of discourse in engineering (and related areas such as computational continuum mechanics).

Marketing is what you do when your product is no good.

– Edwin H. Land

When I first did CFD 25 year ago it was the idea of computing something that matched reality that grabbed my imagination. Cool, realistic looking visuals are certainly part of this, but not a replacement for really computing things that truly match reality they are a bonus. The “colorful fluid dynamics” is a slam, not unlike the perspective on the world of reality TV shows. Reality TV is a contrived and unrealistic view of people’s real lives with a great deal of flourish added for ratings because reality is actually grim, monotonous and boring most of the time. Simulation is similar in that most of the important stuff is boring, dull and utterly essential, cool visuals avoid all of the reality and replace it with an advertisement.

Fluid dynamics has the virtue of creating beautiful visuals through its natural nonlinear interactions. If simulated with even modest levels of fidelity fluid mechanics is stunning and beautiful. It grabs your attention and captures your imagination without the slightest difficulty. Nothing creates patterns that wow the senses like the vortical motions that drive turbulence. These features are prevalent and have exemplified scientific visuals for hundreds of years. Fluid phenomena have stirred the curiosity of the greats including De Vinci who famously sketched the turbulence in a fountain. Every year we have a contest to create the most stunning and scientific compelling visuals put on by the American Physical Society in their Gallery of Fluid Motion taking off from Van Dyke’s classic book of the same name.

It should come as no surprise that computations of fluid dynamics have similarly stirred the artistic juices within modern science. The visuals arising from computation are compelling and seemingly reproduce the features seen in nature. Visualization software has become so good at creating beautiful images from computed data it actually works against aspects of science. In the sense of marketing the work of CFD, the visualization is the best advertising imaginable. Why rock the boat by asking whether the calculation is really mesh converged (what ever that means!), or whether the experimental data is modeling correctly? It just looks so damn cool! Our scientific management is simply so overworked and under-curious they are primed to accept the meager evidence offered by glimmering graphics as all the quality they need to see.

This is the problem that the tag colorful fluid dynamics is designed to project, an interest in how the computation looks over how good it really is. The colorful fluid scientists often reject the notion of looking deeply at the quality of what they do. This shouldn’t happen, but very often it does. Far too often they are successful and rewarded for their lack of focus on scientific content by rewarding showmanship and sleight of hand. The quality of computations is very important especially as computation becomes a greater player in decision-making across a vast variety of fields. Looking good and realistic is important, but that is a standard for Hollywood, not science.

On the other hand computations that appear realistic provide an important element of confidence for those seeing it. In many cases the appearance of calculations is not congruent with the quantitative assessment. A prime example is turbulence. Turbulent computations do not look much like turbulence observed in nature. One of the largest elements in what does not appear turbulent in computation is the degree of intermittency, the large deviations from mean behavior. For visual purposes like Hollywood’s CGI, the methods need to add significant high-energy, small-scale content to get realistic-looking turbulent flow. These same mechanisms are not utilized in the scientific computation of turbulence although there might be virtue in examining the deeper meaning.

Turbulence experiments are focused on mean flow especially for engineering purposes. The extremes and more esoteric aspects of the statistics of the flow are not typically important. For this reason flows that appear non-turbulent can meet quantitative criteria, but fail to look turbulent to the eye. On the other hand this is the essence of the difference between the judgments of the senses compared with the quest for scientific understanding. At some level the aspiration should be cast toward satisfying both goals.

The bountiful success of CFD is do doubt in small part due to the appeal of visualizations. The problem comes when the use of computing becomes more about the appearance of the results than their scientific validity. In many cases the realistic appearance is all the proof needed to trust the results and little in depth analysis is done to establish the credibility of the calculations. This is a dangerous path to take especially as the role of computational simulation grows and the stake in its credibility similarly expands. The potential for misuse and disaster looms if the proper care and seriousness isn’t taken.

As technology advances, the ingenious ideas that make progress possible vanish into the inner workings of our machines, where only experts may be aware of their existence. Numerical algorithms, being exceptionally uninteresting and incomprehensible to the public, vanish exceptionally fast.

Nick Trefethen

While the beauty of the results has provided for easier and greater adoption of CFD as a stable of modern engineering, the lack of quantitative engagement with quality has left CFD stalled. The vast progress of the first couple of decades has been replaced with an acceptance of the status quo and a loss of innovation and improvement in capability. This process has in part allowed the unhealthy obsession with computing hardware to take root, while starving the algorithm and method development of support. The bigger, faster computer has become the lynchpin to any ability to compute while other means to improvement have languished.

This approach is leading to a hollowing out of the field, as more people are merely practitioners of codes with little or no understanding of how the calculations are done. In many cases the CFD codes are run by hacks that understand frightfully little about how the calculations are done, and the various pitfalls that await them. At the same time relatively primitive algorithms and methods are powering the CFD particularly in the case of commercial products. Powerful visualization, bigger computers, graphic user interfaces and a lack of real standards for computational quality allow all of this. We also have a management class who are susceptible to the shallow, deceitful
marketing practices and increasingly lack the technical knowledge and skill to even ask the right questions. We have allowed our scientific standards to slip dangerously low through the glossy seduction of colorful fluid dynamics.

Life is hard. It’s harder if you’re stupid.

John Wayne

At their core most commercial CFD codes are crap. I mean it; they are generally based on terrible methods. They have wonderful user interfaces, mesh generation and visualization capability, but their solvers are mostly ancient garbage. Most of them rely upon methods that are more than two decades old. Even then the methods were not the best available then, but instead chosen for “robustness” rather than accuracy. One is fortunate if they can find a reliable second-order discretization and the standards for solving nonlinear systems of equations are pathetic (the community accepts a nonlinear residual that is enormous!). Turbulence is modeled using old antiquated methods of similar vintage. Of course, the acceptance of this terrible technology is aided by the relative lack of progress over the last couple of decades in no small part due to the shift in emphasis toward hardware and away from algorithms and methods. The real core of the problem is the willingness of the community to accept this crap as state of the art. Stunning visuals seal the deal and help craft the path toward mediocrity.

Without change something sleeps inside us, and seldom awakens. The sleeper must awaken.

― Frank Herbert

Colorful fluid dynamics should be an insult; instead it has become the path to professional success and stunted progress.