Which of us can resist the temptation of being thought indispensable?

― Margaret Atwood,

In the past year I’ve fully embraced the online world. This is in a lot more ways than just writing a blog, tweeting etc. My most used programs have changed from the standard laptop, desktop applications like Microsoft office to a set of mobile apps: Evernote, Pocket, Zite and recently Wunderlist. I’m finding the apps to be indispensible, and hope they can make me more capable of working effectively. It also helps the ability to balance things between work and home.

Best of all these apps work together and run across multiple platforms. Evernote is the hub, but all of them add a lot of value to my day.

Be infinitely flexible and constantly amazed.

― Jason Kravitz

Evernote is the definitely the flagship app for me. If you’re not familiar with it, Evernote is an electronic filling cabinet. It ends up being the clearinghouse for information flowing from all these other apps and more. Evernote also runs on every platform (phone iPhone/Android, laptop, desktop, iPad, and web). You can always get your data. My wife uses it at her office as the common system for managing multiple accounts able to deal with data from many sources.

I use it to jot down ideas, plan writing assignments, take notes at meetings, save articles, and keep a food, exercise, and personal journal. It can accept info in almost any format and allows you to write notes, and draw graphical comments inside most. It also has a fantastic web clipper that allows me to save web pages directly plus add comments. Another great thing to do is use Evernote to save business cards, travel cards, and other documents that pile up in wallets, or get easily lost. I’ve also used it for saving details of medical appointments, and repair/remodel work at home. It’s the place where I store the details of my referee assignments for soccer as well as notes on the games for future reference.

It is really appropriate to use the overused word, awesome, to describe it. Evernote is awesome and utterly indispensible. If you haven’t tried it you should.

Next is the most recent addition to my staple of mobile apps, Wunderlist. Wunderlist is a to do and task manager. I’ve added it because it is multiplatform. I can have a uniform and synched list across mobile, laptop and desktop apps. I have a set of things that I want to accomplish every day, which repeat along with a set of single use items. All of them can be tracked and dealt with. It is still being tried out, but it’s almost sure that it will be a keeper.

The last two apps are Pocket and Zite, which work hand-in-hand. Pocket is a way of saving almost anything to read. I use it mostly for web articles to read later, mostly while exercising at the gym. It can also save files from multiple formats. The best thing is that it works automatically from web browsers, and other apps. I can also push the articles directly to Evernote if I really like it! Pocket also has an archive of its own to save things for the long term.

Zite is a way of reading online content. It takes the top articles from multiple sources and making it available for viewing. Often I’ll find interesting things and saving it for later reading in Pocket. I get to choose what I’m interested in and Zite collates the top articles in each area for me to see. It also puts everything together for a list of the top, top stories in my interest list. The only downside to Zite is its unavailability on laptops, desktops and web browsers. I do wish it were more available on all my platforms. On my mobile platforms it is right near the top of my use.

The only way to get what you’re worth is to stand out, to exert emotional labor, to be seen as indispensable, and to produce interactions that organizations and people care deeply about.

― Seth Godin

These apps have made me more productive and well informed. Maybe they could help you too.

This week I attended a conference in Los Alamos. It is difficult to not be nostalgic having worked there for 18 years. It harkens to the old saw that people tend to pull out, “those were the good ole days”. What if they were? Why aren’t things getting better?

I will submit that one of my observations this week is that I remember the good far more than the bad. I am not alone in that tendency.

Remembrance of things past is not necessarily the remembrance of things as they were.

― Marcel Proust

First, some things are getting better. I’m shamelessly progressive and love the benefits of discoveries in science; technology and medicine that make life better in many ways. Medical treatments make once fatal conditions, trivial to treat. For example I had a minor congenital defect that would have proved fatal a few decades before, and was treated by a simple surgery when I was a month old.The Internet, Google, smartphones, instant messaging and a host of other miracles make our lives better in many ways. I used to spend hours in the Los Alamos Lab library doing research; copying papers where today I could do even more from home or my office with almost everything being available as a PDF. Some of the collateral consequences are problematic, but I have faith that a happy equilibrium will be achieved.

What you end up remembering isn’t always the same as what you have witnessed.

― Julian Barnes,

In Los Alamos with dead certainty there were the good ole days. The past was dramatically better. The Lab was better, working conditions were better and the Lab’s work meant something to the Nation. With Los Alamos it is almost a fait accompli with its role in World history guaranteeing a downhill slide over time. Nonetheless, Los Alamos continued to make history for its first twenty years or so, before decay set in. Of course, its origin included a host of future Nobel Laureates, and a job of monumental gravity. The success played a key roll in shaping the remainder of the 20^th Century.

The issues dragging Los Alamos down go well beyond local conditions. Ultimately, with the end of the Cold War the Nation as well as Los Alamos has lost its bearings. We continue to move down a path where any sense of deeper meaning is hard to find in the lives we live. The almost systematic destruction of the middle class seems to go hand-in-hand. It is as if the powers that be were working to assure that most people’s lives are spent in pursuit of survival rather than the achievement of aspirational goals. Our systems are in deep decline and no one seems to be able to muster enough vision and leadership to see a way out.

In this context the old saw of the “good ole days” is appropriate and correct.

It is strange how we hold on to the pieces of the past while we wait for our futures.

― Ally Condie

Management is doing things right; leadership is doing the right things.

–Peter Drucker

Once upon a time the National Labs were paragons of excellence in science and engineering. No more. Over time the Nation has demanded that the Labs become paragons of accountability. The over-emphasis on accountability has ironically worked to drive excellence from the Labs. In an accountability-driven culture if no one is accountable for excellence, excellence dies. This is a direct consequence of our current over-managed and under-led status both locally and nationally.

Efficiency is doing things right; effectiveness is doing the right things.

–Peter Drucker

Today no one focuses on excellence except through a host of metrics that only give a shadow imprint of excellence. All the while a culture of excellence is not cultivated at all. Rather than do the things that lead to excellence, the culture of accountability acts to undermine it. Staff will avoid going to seminars or classes that would develop them professionally for the long-term because their current projects won’t pay for them. Continually the projects drive the staff to think only in a short-term tactical project-focused manner despite the damage done to their long-term development.

Consider attendance at a conference, which has become extensively scrutinized lately. Almost any conference I attend is a broad-brushed opportunity to develop professionally across a suite of projects present and future. The accountability culture only cares about what I am presenting, but nothing about what is presented to me. In other words attending a conference is all about what is the attendee is presenting.
The reality should be balanced between what is given and what is received. A large part of a conference is the exposure to new ideas, the current focus of a community and networking with other attendees. In fact most of the benefit has nothing to do with the reason given for attending. The culture of accountability misses the key points. A culture of excellence would have no problem is accommodating the proper perspective.

The explicit drive for excellence and professionalism has been destroyed by the “customer is always right” attitude. My experience is that the customer is almost always wrong, and could greatly benefit from treating the Labs as reservoirs of expertise that could greatly improve their judgment. Too often today the customers are simply taking knowledge and products from the Labs while doing little or nothing to support the foundation that created the expertise. As such the expertise is running dry, if the well of knowledge is not sustained it will die. Our current customer-focused accountability culture is hopelessly shortsighted. There is little or no focus on the long-term development of the Labs expertise. The research is becoming ever more customer-focused and tactical. The investments in long-term sustaining research are minimal in large part because the “customer” receives no perceived benefit in the short term. All the while the customers are happy to siphon off benefit from the expertise they do little to sustain.

Of course this entire philosophy gets the core of the problem. The lack of a broad-based societal imperative for supporting the development of societal expertise is troubling. This is in contra-distinction to the events following World War II when the system of National Laboratories came into existence. The benefits of the expertise were felt across the Nation and World and usually beyond the direct impact on the agency that sponsored the Lab. Quite often the Energy or Defense or NASA Lab produced breakthroughs that impact the work of the other Labs or industries. The benefit quite often spilled over into other activities such medicine or industrial production. Computing is the archetype of this cross-fertilization. Computing’s various breakthroughs came from numerous fields and ultimately spurred the creation of a massive industry. No system of accountability could have been responsible for what happened, it was the product of broad-based excellence in science. I would submit that the current short-term culture of accountability would have likely short-circuited the entire thing. I worry that our focus on accountability is probably undermining our future prosperity already.

A primary task of management in the developed countries in the decades ahead will be to make knowledge productive.

–Peter Drucker

The result is the nearly systematic destruction of an essential National resource. The true irony is that no one is accountable for this act violence against our National security. In fact it is hidden behind a veil of accountability standards that provide the façade that everything is being done well. We only assure that the terrible things are done efficiently. A large part of the devotion to accountability is couched in fear and suspicion. Excellence is founded on hope and trust guided by principle.

 There is nothing quite so useless as doing with great efficiency something that should not be done at all.

–Peter Drucker

 The same thing is happening at Universities across the country too. The educational aspect of a university is the epitome of excellence, and any observation of the value system in place shows unequivocally that teaching has less value than research. Our students are not simply burdened by student loans and the concomitant debt, but also by increasingly poor instruction. They are getting a worse education at a substantially higher price. At the core of the problem is money. Less societal support for education is driving universities to focus on research grants as a source for money along with the student loans. The grants drive emphasis from teaching and push a variety of inappropriate foci such as research associates, post-docs as labor, and adjunct professors as cheap teacher (adjuncts are another key measure of the value placed in teaching, which ain’t much!).

The combined effect of the erosion of excellence in Labs and Universities is hurting our Nation’s prospects for the future. No amount of accountability can fix this. Only by backing away from the current shortsighted philosophy can we recapture the excellence that once exemplified these institutions. It will require us to do a number of things we have lost sight of including long-term goals, trust for our fellow citizens, and the belief that we have things worth working together toward.

What gets measured gets improved.

–Peter Drucker

The purpose of computing is insight, not pictures.

–Nick Trefethen
This is a brief take on the intersection of a couple of previous posts: the recent one on the viewgraph norm (https://williamjrider.wordpress.com/2014/10/07/the-story-of-the-viewgraph-norm/), colorful fluid dynamics (https://williamjrider.wordpress.com/2014/10/03/colorful-fluid-dynamics/), the Millennium prize for the Navier-Stokes equations (https://williamjrider.wordpress.com/2014/03/07/the-clay-prize-and-the-reality-of-the-navier-stokes-equations/), and numerical viscosity (https://williamjrider.wordpress.com/2013/10/04/there-is-nothing-artificial-about-artificial-viscosity/).

The basic view of quality is predicated on the belief that more “energy (disorder)” and complexity in the computed flow is directly correlated to the quality of the computation. This is typically applied in an intrinsically ad hoc manner that may not actually provide an accurate assessment of quality. At some point the disorder in the computation is too great and the quality is judged to be lower. This is done purely by expert judgment, not based on any sort of clear definitive measure or feature. The real issue is whether the computation is swirlier due to incipient errors that are on the verge of losing stability. This may inadvertently favor instability in the numerical method point-of-view (in fact, almost certainly).

There are three great branches of science: theory, experiment, and computation.

–Nick Trefethen,

This topic involves deep-seated issues with each of these branches.

As soon as a fluid flow becomes unstable and vortically dominated the knowledge of the exact solution is absent. These flows are exceedingly important thus the quality of calculations is extremely interesting, but difficult-to-impossible to specifically determine. At the heart of the issue is the lack of theoretical grasp of turbulent flows. This is a fundamental limitation on our ability to reliably compute the behavior of real fluids and correspondingly determine the quality of computing methods. This in turn leaves us with the current state of affairs swirlier is better.

I became most troubled by this aspect of the determination of quality after seeing a standard applied, which amounts to “the more swirly the result, the better the method” (more swirly means more vorticity). An exemplar of this approach is the paper by Shi, Zhang and Shu in the Journal of Computational Physics, 186, pp. 690 (1993) http://www3.nd.edu/~yzhang10/euler-weno9.pdf. Several problems are studied using mesh refinement (good!) including shock-driven mixing and Kelvin-Helmholtz, and Rayleigh-Taylor instabilities with high-order methods. The conclusion is that the higher order methods are better because they produce more fine scale structure.

My concern about this issue is that the higher order methods also contain insidious and problematic numerical instabilities that could potentially contribute to physically incorrect solutions. The current “swirlier is better” standard yields little or no guidance towards improving the methods or uncovering their shortcomings. The problems with these methods can manifest themselves as entropy violating solutions, which are by definition unphysical. An unphysical solution will produce more vorticity, and hence be swirlier by the standard applied in the community; it would be viewed as better. In fact it would be worse and dangerously so.

In chaos, there is fertility.

― Anaïs Nin

Why does this standard exist?

The use of the first-order upwind method historically produced too much numerical dissipation. Upwind methods were robust enough to be used for applications, but also had large errors. These errors led to the destruction of vorticity, which made flows distinctly less swirly than reality. Modern methods provided the robustness of upwind methods with much smaller error, and much more realistic swirliness. The problem is that instabilities can lead to swirliness too and this standard leaves no room for determining the limits for methods. This is left for validation against experimental data. This is thoroughly unsatisfying because there is not a mathematical ground truth. Modeling and numerical effects are muddled together. Unfortunately, mathematics is not currently attacking this problem very aggressively (see my Applied Math critique https://williamjrider.wordpress.com/2014/10/16/what-is-the-point-of-applied-math/). In truth, the mathematics to address this issue is not presently sufficient.

What can be done to improve matters? One way would be to rely upon experimental comparison to decide quality. This leaves little guidance for improving the methods based on mathematical principles. Insofar as applied mathematics is concerned, a better theory for the development of these instabilities would enable guidance toward better methods. This is lacking today rather seriously. It would be useful to have a refined understanding of what unphysical solutions look like for these cases. Today such a characterization is not available to be applied. We are left with experimental comparison and/or expert judgment.

All that it is reasonable to ask for in a scientific calculation is stability, not accuracy.

–Nick Trefethen

Measure what can be measured, and make measurable what cannot be measured.

― Galileo Galilei

A modification of this famous quote was the title of an interesting paper I read on compressed sensing a couple of days ago, “Measure what should be measured”. In today’s world of data explosion it is a curious statement. The real thing that might apply is “measure everything, look at what is important”. This might have real application to control the expansion of data. Compressed sensing might have a great deal to say about how to do this.

I started thinking about computing. What do we do today, do we simply “compute what can be computed”? Shouldn’t we be “computing what should be computed”? or better yet compute what is important. Like data, we perhaps will “compute everything and look at what is important”. Again, the philosophy and methodology of compressed sensing might apply to getting there.

One route is efficient, but extremely difficult, while the other is wasteful and potentially tractable. Something needs to happen, the future of computing depends on the answer.

This week is going to be difficult. I have made a commitment to writing every day, but for today the blog post will be short and sweet. I have to drive from Albuquerque to Los Alamos and back. While in Los Alamos I am in a classified meeting, so no electronics. I also give a  talk and will chair a session.  With three to three-and-a-half hours of driving there isn’t much time for anything.

With luck the meetings this week will be inspiring and interesting. If nothing else I will learn some new things that will be worth writing about later. Keep looking here and find out. I intend to give this habit enough room to breathe.

We become what we repeatedly do.

― Sean Covey

I have always found it quaint and rather touching that there is a movement [Libertarians] in the US that thinks Americans are not yet selfish enough.

― Christopher Hitchens

The irrational fear of Ebola has thrust the competence of our government into the spotlight. While some conservative voices would point at the failing of government, I believe their aim is both spot on, and completely wrong. We don’t have a failure of government, we have a failing of governance both private and public. The problems with Ebola are exemplars of incompetence from both government and business with both contributing greatly to the debacle in Dallas.

The greater issue is the general crisis in governance in our country. No one seems to be able to do anything right. Government is ineffective and wasteful. Business is amoral and unethical. Neither should be acceptable. The only thing we are doing with any competence is directing more and more of our societal wealth into the hands of a very select few. This is being done in an intrinsically amoral and unethical manner despite its explicit legality since the laws are basically for sale.

It might be nice if the key issue in politics were associated with fixing our society-wide incompetence. We need competence and effective governance from both private and public entities. I would argue that the problem is an unhealthy focus on the individual rather than the overall society. The narrow definition of success associated with the combination of short-term gains and organizational locality are making every decision tactical. This tactical decision-making benefits very few and leads to outcomes that hurt society at the large scale.

In business this produces choices that give shareholders the option of cashing out while destroying jobs, and the future of companies. In government this looks like buck passing and the CYA culture. Together they equal the web of mistakes that made the Dallas Ebola case so much worse. Make no mistake this case is the combination of profit focused medicine coupled with a lack of proper government execution. For example the profit motive is one of the main reasons we don’t have more effective medicines for treating diseases like Ebola. There is little or no profit to made there despite its potential importance to society or its destructive potential.  The core problem is a lack of outrage about the overall lack of competence in governance. This is the thing we should be fixing and it is a completely bipartisan problem.

We should be demand competent thoughtful governance from both the private and public sectors. The outcomes need to balance the good of the individual and society as a whole. We need to explicitly reject the governance that only benefits a precious few. In the long run a more balanced approach will lead to a far better future for everyone including those few who take nearly all the benefits today.

Selfishness and greed, individual or national, cause most of our troubles.

― Harry S. Truman

 Simplicity is the ultimate sophistication.

― Leonardo da Vinci

Life is really simple, but we insist on making it complicated.

― Confucius

V&V and UQ are often described by detractors as being too complex. It certainly can be, but it doesn’t have to be. In keeping with all the brilliant advise quoted here, I’m going to offer simple definitions for the main components of the practice. Each can be posed as a question we seek to answer.

• Verification: (1) Do more computing resources yield better answers? (2) Is my model implemented correctly?
• Validation: Does my model represent reality?
• Uncertainty Quantification: How much different could my answers reasonably be?
• Sensitivity Analysis: What variables do my answers most depend upon?

The downside is the failure to address a host of important technical details, but the simple definitions provide the core of this important topic.  What do you thinK?

Our life is frittered away by detail. Simplify, simplify.

― Henry David Thoreau

Any darn fool can make something complex; it takes a genius to make something simple.

― Pete Seeger

I read a lot. Technical papers. Blogs. You name it. The other day I came across an article that didn’t grab me immediately, but I soldiered on, and deep into the article (http://pchiusano.github.io/2014-10-13/worseisworse.html, I mentioned it yesterday where it describes the issues with applied mathematics) a thought arose in me, “this is some good shit!” The concepts applied so much more broadly than the original and primary focus on software, but to broad swaths of science, technology, business and the human existence in general.

The recent blog post rightly focused on the problems with culture and a bit more commentary might be useful. The core of the problem is one of local optimization. One can choose a solution that is optimal for one’s self or a small group that has awful consequences more broadly. If the implicit or explicit reward system isn’t carefully constructed and the consideration of the global consequences are ignored, something comfortable, but ultimately awful will be allowed to persist as a solution. For software this is common because of its niche status even today. For most organizations a legacy system provides value today and they aren’t willing to pay or wait for a better solution. This is part of essence of “worse is better”. It is part of the reason that the status quo always has a “home field advantage”.

The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man.

George Bernard Shaw

At a deeper level the issue is associated with the notion of winner and losers in any resource battle. While the current economy is demonstrably worse for a vast majority of Americans (or Westerners), it is much better for a select cadre of the wealthiest. These wealthy are powerful and work to influence the gears of politics and power to maintain this status quo. It is difficult to move toward a different system when those with true power oppose it. This is in spite of a general conclusion that a more equitable system that favored a much larger segment of the population would actually result in more for most of the wealthy too. Decisions favor the incumbent. It is always more clear to defend what you already have rather than what you might have.

There are three classes of men; the retrograde, the stationary and the progressive.

― Johann Kaspar Lavater

These forces are always working against progress. Usually the thing that represents progress (a new language, new code, new theory…) is actually lower in performance than the incumbent, which has been optimized (or calibrated). Those who are not invested in the progress will usually choose the status quo for completely rational reasons including its performance, and the relative comfort of the known product. Thus for many, if not most, worse is really better (or easier). If the system is not carefully designed to blunt this outcome, progress will be especially painful. When the system is designed to stop progress, progress is almost impossible.

Better never means better for everyone… It always means worse, for some.

― Margaret Atwood

 There can be no progress without head-on confrontation.

― Christopher Hitchens

For reference there is a Wikipedia entry, and some reflections on Gabriel’s original essay:

http://en.wikipedia.org/wiki/Worse_is_better

http://dreamsongs.com/WorseIsBetter.html

I work with a lot of mathematicians, applied mathematicians to be precise. A lot of the time I ponder the point of their work. Is the importance of the work the beauty and knowledge of math itself, or the utility of the work for practical purposes? My sense is that the “applied” in applied math pushes the balance toward utility. Too often the utility in the work being sold as applied math
today is almost impossible to divine. This is the rub. It ends up being the same dynamic as pure versus applied research. How applied, does applied math need to be to be applied math?

 Physicists have come to realize that mathematics, when used with sufficient care, is a proven pathway to truth.

― Brian Greene,

We had a talk at work earlier in the week that brought these issues into focus. A relatively well-known and successful professor came for a visit and gave a research seminar on his work. On the face of it, the talk looked interesting and topical. This rapidly faded when the talk unfolded for a very simple reason. The professor was limiting discussion to where he could prove results. If the flow he was studying became too energetic (too high a Reynolds number, or its equivalent, the proofs couldn’t be constructed). As a result the work had limited applicability to investigations because results can’t be proven for most applied problems. Most applied problems
have too high a Reynolds number to be amenable to the techniques he was applying. Furthermore these higher Reynolds number flows are the challenge applications and computing is most paced by. Despite the importance of the applications, the applied math isn’t being applied. Arrrgggg!

 Mathematics is the art of explanation.

― Paul Lockhart

Is it really applied math, if I can’t apply to the results to things we care about?

My attitude is that I will roll up my sleeves and work to understand the math if the results can be shown to apply to situations I care about. If the math avoids the situations of interest, can’t be demonstrated, or simply doesn’t demonstrate itself, I won’t make the effort because the mathematician hasn’t done their part to meet me half way. What should happen when we have important applied cases where results can’t be proven? Should the effort in math be given to expand the grasp of mathematics to handle these cases? Or should mathematicians work on proving weaker bounds or results?

Deep in the human unconscious is a pervasive need for a logical universe that makes sense. But the real universe is always one step beyond logic.

― Frank Herbert

My opinion is that proving results on simple problems of little relevance is basically useless insofar as applied math is concerned. Nothing is wrong with providing a sliding scale where the strength of the guarantees changes with problem difficulty. The important thing is to provide the proper mathematical grounding for the problems people solve. If the math simply doesn’t exist for important problems, then say so and set about to improve the capacity of math to provide results.

The important problems will continue to be solved. The issue is that applied math won’t be participating. The retreat of applied math from relevance has been palpable for the past two decades. Once upon a time applied math was a key partner in computational, modeling and physics progress. This role has shrunk over time due to an unwillingness to get their hands dirty. There also seems to be a desire to look more like pure math, which leads to a lack of demonstration.

This leaves me with the question: if applied math can’t be applied? Is it really applied math?

I’m an easy sell for the community; I know that applied math can contribute mightily to progress. All that is needed is for the applied mathematicians to make an earnest effort to work on problems that matter. In other words solve the problems that are important, not the ones that are easy to solve. Demonstrate that your results actually mean something on real problems. Deal directly with problems that are “dirty” rather than simplify real problems until they lose connection with reality.

We all die. The goal isn’t to live forever, the goal is to create something that will.

― Chuck Palahniuk

Today applied math is optimized locally, but globally it is in crisis. This is yet another instance of “Worse is Better”: http://pchiusano.github.io/2014-10-13/worseisworse.html . We’ve allowed this to happen. The excuse that people need to publish for professional success is hurting the field, and is largely a self-imposed condition. What is the point of success if the publications mean little to the development of technology?

The question to ask is whether it is “the mathematics of applications” or the “using math on applications”. There is a difference. Today it is largely the later, instead it needs to be doing math that impacts applications.

Since the mathematicians have invaded the theory of relativity I do not understand it myself any more.

― Albert Einstein