• About The Regularized Singularity

The Regularized Singularity

~ The Eyes of a citizen; the voice of the silent

The Regularized Singularity

Monthly Archives: September 2016

The Success of Computing Depends on Mathematics More Than Computers

27 Tuesday Sep 2016

Posted by Bill Rider in Uncategorized

≈ 2 Comments

 The best dividends on the labor invested have invariably come from seeking more knowledge rather than more power.

– The Wright brothers

Some messages are so important that they need to be repeated over and over. This is one of those times. Computing is mostly not about computers. A computer is a tool, powerful, important and unrelentingly useful, but a tool. Using computing is a fundamentally human activity that uses a powerful tool to augment human’s capacity for calculation and monotony. Today we see attitudes expressing more interest in the computers themselves with little regard for how they are used. The computers are essential tools that enable a certain level of utility, but the holistic human activity is at the core of what they do. This holistic approach21SUPERCOMPUTERS1-master768 is exactly the spirit that has been utterly lost by the current high performance computer push. In a deep way the program lacks the appropriate humanity in its composition, which is absolutely necessary for progress.

Most clearly the computers themselves are not an end in themselves, but only useful insofar as they can provide benefit to the solution of problems for mankind. Taking human thinking and augmenting it derives the benefits for humanity. It is our imagination and inspiration automated as to enable solutions through primarily approximate means. The key to all of the true benefits of computing come from the fields of physics, engineering, medicine, biology, chemistry and mathematics. Subjects closer to the practice of computing do not necessarily push benefits forward to society at large. It is this break in the social contract that current high performance computing has entirely ignored. The societal end product is a mere after thought and little more than simply a marketing ploy for a seemingly unremitting focus on computer hardware.

Mathematics is the door and key to the sciences.

— Roger Bacon

This approach is destined to fail, or at the very most not reap the potential benefits the investment should yield. It is completely and utterly inconsistent with the vgesamthubschrauber-01enerated history of scientific computing. The key to the success and impact of scientific computing has been its ability to augment its foundational fields as a supplement to human’s innate intellect in an area that human’s ability is a bit diminished. While it supplements raw computational power, the impact of the field depends entirely on human’s natural talent as expressed in the base science and mathematics. One place of natural connection is the mathematical expression of the knowledge in basic science. Among the greatest sins of modern scientific computing is the diminished role of mathematics in the march toward progress.

Computing should never be an excuse to not think; the truth is that computing has become exactly that; it is an excuse to stop thinking, and simply automatically get “answers”. The importance of this connection cannot be underestimated. It is the complete and total foundation of computing. This is where the current programs become completely untethered from logic, common sense and the basic recipe of success. The mathematics program is virtually absent from the drive toward MorleyWangXuElementsgreater scientific computing. For example I work in an organization that is devoted to applied mathematics, yet virtually no mathematics actually takes place. Our applied mathematics programs have turned into software programs. Somehow the decision was made 20-30 years ago that software “weaponized” mathematics, and in the process the software became the entire enterprise, and the mathematics itself became lost, an afterthought of the process. Without the actual mathematical foundation for computing, important efficiencies, powerful insights and structural understanding is scarified.

The software has become the major product and end point of almost all research efforts in mathematics to the point of displacing actual math. The product of work needs to be expressed in software and the construction and maintenance of the software packages has become the major enterprise being conducted. In the process the centrality of the mathematical exploration and discovery has been submerged. Software is a difficult, valuable and important endeavor in itself, but distinct from mathematics. In many cases the software itself has become the3_code-matrix-944969 raison d’être for math programs. In the process of the emphasis on the software instantiating mathematical ideas, the production and assault on mathematics has stalled. It has lost its centrality to the enterprise. This is horrible because there is so much yet to do.

Worse yet, the mathematical software is horribly expensive to maintain and loses its modernity with a frightful path. We hear calls to preserve the code base because it was so expensive. A preserved code base loses its value more surely than a car depreciates. The software is only as good as the intellect of the people maintaining it. In the process we lose intellectual ownership of the code. This is beyond the horrible accumulation of technical debt in the software, which erodes its value like mold or dry rot. None of these problems are the worst of the myriad of issues around this emphasis; the worst issue is the opportunity cost of turning our mathematicians into software engineers and removing the attention from some of our most pressing issues.

A single discovery of a new concept, principle, algorithms or technique can render one of these software packages completely obsolete. We seem to be in an era where we believe that more computer power is all that is needed to bring reality to heel. These discoveries can allow results and efficiencies that were comtitan2pletely unthinkable to be achieved. Discoveries make the impossible, possible, and we are denying ourselves the possibility of these results through our inept management of mathematics proper role in scientific computing. What might be some of the important topics in need of refined and focused mathematical thinking?

The work of Peter Lax and others has brought great mathematical understanding, discipline and order to the world of shock physics. Amazingly this has all happened in one dimension plus time. In two or three dimensions where the real World happens, we know far less. As a result our knowledge and mastery over the equations of (compressible) fluid dynamics is limited and incomplete. Bringing order and understanding to the real World of fluids could have a massive impact on our ability to solve realistic problems. Today we largely exist on the faith that our limited one-dimensional knowledge gives us the key to multi-dimensional real World problems. A program to expand our knowledge and fill these gaps in knowledge would be a boon to analytical and numerical methods seeding a new renaissance for scientific computing, physics and engineering.

One of the key things to understanding the power of computing is the comprehension that the ability compute belies a deep understanding that enables analytical, physical and domain specific knowledge. A problem intimateldag006y related to the multi-dimensional issues with compressible fluids is the topic of one of the Clay prizes. This is a million dollar prize for proving the existence of solutions to the Navier-Stokes equations. There is a deep problem with the way this problem is posed that may make its solution both impossible and practically useless. The equations posed in the problem statement are fundamentally wrong. They are physically wrong, not mathematically although this wrongness has consequences. In a very deep practical way fluids are never truly incompressible; incompressible is an approximation, but not a fact. This makes the equations have an intrinsically elliptic character (because incompressibility implies infinite sound speeds, and lack of thermodynamic character).

Physically the infinite sound speeds remove causality from the equations, and the removal of thermodynamics takes them further outside the realm of reality. This also creates immense mathematical difficulties that make these equations almost intractable. So this problem touted as the route to mathematically contribute to understanding turbulence may be a waste of time for that endeavor as well. Again, we need a concerted effort to put this part of the mathematical physics World into better order. The benefits to computation through some order would be virtually boundless.

This gets at one of the greatest remaining unsolved problems in physics, turbulence. The ability to solve problems depends critically upon models and the mathematics that makeimagess such models tractable or not. The existence theory problems for the incompressible Navier-Stokes equations are essential for turbulence. For a century it has largely been assumed that the Navier-Stokes equations describe turbulent flow with an acute focus on incompressibility. More modern understanding should have highlighted that the very mechanism we depend upon for creating the sort of singularities turbulence observations imply has been removed in the process of the choice of incompressibility. The irony is absolutely tragic. Turbulence brings almost an endless amount of difficulty to its study whether experimental, theoretical, or computational. In every case the depth of the necessary contributions by mathematics is vast. It seems somewhat likely that we have compounded the difficulty of turbulence by choosing a model with terrible properties. If so, it is likely that the problem remains unsolved, not due to its difficulty, but rather our blindness to the shortcomings, and the almost religious faith many have followed in attacking turbulence with such a model.

Before I close I’ll touch on a few more areas where some progress could either bring great order to a disordered, but important area, or potentially unleash new approachimageses to problem solving. An area in need of fresh ideas, connections and better understanding is mechanics. This is a classical field with a rich and storied past, but suffering from a dire lack of connection between the classical mathematical rigor and the modern numerical world. Perhaps in no way is this more evident in the prevalent use of hypo-elastic models where hyper-elasticity would be far better. The hypo-elastic legacy comes from the simplicity of its numerical solution being the basis of methods and codes used around the World. It also only applies to very small incremental deformations. For the applications being studied, it should is invalid. In spite of this famous shortcoming, hypo-elasticity rules supreme, and hyper-elasticity sits in an almost purely academic role. Progress is needed here and mathematical rigor is part of the solution.

A couple of areas of classical numerical methods are in dire need of breakthroughs with the current technology simply being accepted as good enough. A key one is the solution of sparse linear systems of equations. The current methods are relatively fragile and it’s been 30-40 years since we had a big improvement. Furthermore these successes are somewhat hollowed by the lack of a robust solution path. Right now the gold standard of scaling comes from multigrid, invented in the mid-1970’s to mid-1980’s. Robust solvers use some sort of banded method with quadratic scaling or pre-conditioned Krylov methods (which a_12122_tex2html_wrap26re less reliable). This area needs new ideas and a fresh perspective in the worst way. The second classical area of investigation that has stalled is high-order methods. I’ve written about this a lot. Needless to say we need a combination of new ideas, and a somewhat more honest and pragmatic assessment of what is needed in practical terms. We have to thread the needle of accuracy, efficiency and robustness in both cases. Again without mathematics holding us to the level of rigor it demands progress seems unlikely.

Lastly we have broad swaths of application and innovation waiting to be discovered. We need to work to make optimization something that yields real results on a regular basis. The problem in making this work is similar to the problem with high-order methods; we need to combine the best technology with an unerring focus on the practical and pragmatic. Optimization today only applies to problems that are far too idealized. Other methodologies are laying in wait of great impact among these the generalization of statistical methods. There is an immense need for better and more robust statistical methods in a variety of fields (turbulence being a prime example). We need to unleash the forces of innovation to reshape how we apply statistics.

When you change the way you look at things, the things you look at change.

― Max Planck

The depth of the problem for mathematics does seem to be slightly self-imposed. In a drive for mathematical rigor and professional virtue in applied mathematics, the field has lost a great deal of connection to physics and engineering. If one looks to the past for guidance, the obvious truth is that the ties between physics, engineering and mathematics have been quite fruitful. There needs to be healthy dynamics of push and pull between these areas of emphasis. The wjohn-von-neumann-2orlds of physics and engineering need to seek mathematical rigor as a part of solidifying advances. Mathematics needs to seek inspiration from physics and engineering. Sometimes we need the pragmatic success in the ad hoc “seat of the pants” approach to provide the impetus for mathematical investigation. Finding out that something works tends to be a powerful driver to understanding why something works. For example the field of compressed sensing arose from a practical and pragmatic regularization method that worked without theoretical support. Far too much emphasis is placed on software and far too little on mathematical discovery and deep understand. We need a lot more discovery and understanding today, perhaps no place more than scientific computing!

Mathematics is as much an aspect of culture as it is a collection of algorithms.

—  Carl Boyer

Note: Sometimes my post is simply a way of working on narrative elements for a talk. I have a talk on Exascale computing and (applied) mathematics next Monday at the University of New Mexico. This post is serving to help collect my thoughts in advance.

A Requiem for Personal Integrity in Public Life

20 Tuesday Sep 2016

Posted by Bill Rider in Uncategorized

≈ Leave a comment

Above all, don’t lie to yourself. The man who lies to himself and listens to his own lie comes to a point that he cannot distinguish the truth within him, or around him, and so loses all respect for himself and for others. And having no respect he ceases to love.

― Fyodor Dostoyevsky

Over the past handful of years the capacity to maintain professional success and personal integrity has become increasingly strained. Simultaneously the whole concept of integrity within the public life has similarly become strained. I’ve been startled by the striking symmetry between my private, professional and public life as the incredibleimgres and terrifying shit show of the 2016 American Presidential election. It all seems to be coming together in a massive orgy of angst, lack of honesty and fundamental integrity across the full spectrum of life. As an active adult within this society I feel the forces tugging away at me, and I want to recoil from the carnage I see. A lot of days it seems safer to simply stay at home and hunker down and let this storm pass. It seems to be present at every level of life involving what is open and obvious to what is private and hidden.

How do we deal with all of the conflict, tension and danger when the rules of the game seem to have been thrown out? Is this confluence of effects common to everyone helping to explain the World, or is it personal? I think it’s worth pondering the breadth and scope of the challenges as we head forward toward a hopefully more optimistic future and 2017.

I’ve highlighted the concept of integrity as the focal point and the thing at more imminent risk. This is an expansion of my previous discussion of peer review that revolves around the same axis. It gets to the ability to care with some depth about the work I do, and whether that concern is actually appreciated. What does personal integrity mean to me? Like most things it is complex and combines multiple aspects of the content of my daily life. The greatest element of personal angst deals with the truth and any willingness for truths to be articulated openly. Research and progress depends upon good ideas being applied to areas of opportunity. Another less charitable way of sayingfight-club-poster the same thing is that progress depends on finding important, valuable problems and driving solutions. A second piece of integrity is hard work, persistence, and focus on the important valuable problems linked to great National or World concerns. Lastly, a powerful aspect of integrity is commitment to self. This includes a focus on self-improvement, and commitment to a full and well-rounded life. Every single bit of this is in rather precarious and constant tension, which is fine. What isn’t fine is the intrusion of outright bullshit into the mix undermining integrity at every turn.

When people don’t express themselves, they die one piece at a time.

― Laurie Halse Anderson

The core of the issue attacking integrity is the power and prevalence of bullshit in professional and public life. At a professional level bullshit has become the prevalent means of communication of results. Why create real work when fake work can be spun into results of equal and greater value. In fact bullshit is better because it can be whatever it needs to be for success. People continually produce results of mquick-fix-movie-to-watch-office-space-imageinimal value that get marketed as breakthroughs. The lack of integrity at the level of leadership simply takes this bullshit and passes it along. Eventually the bullshit gets to people who are incapable of recognizing the difference. Ultimately, the result of the acceptance of bullshit produces a lowering of standards, and undermines the reality of progress. Bullshit is the death of integrity in the professional world.

There exists an appalling symmetry within the broadest public sphere. We are witnessing a political movement of disturbing power founded on bullshit. We see outright lies produced every day and never actively challenged. This bullshit may actually elect a completely unqualified and dangerous person President of the United States. Why work with facts or truth when bullshit is so incredibly effective? When we look more deeply at this problem we start to see that our political dysfunction is built upon a virtual mountain of bullshit. We see reality television, Facebook, Fox News, CNN, online dating, and a host of other modern things all-operating dude_wtfwithin a vibrant and growing bullshit economy. Taken in this broad context, the dominance of bullshit in my professional life and the potential election of Donald Trump are closely connected.

A big part of the acceptance of bullshit as the medium of universal discourse is related to fear. Increasingly our professional and public lives are ruled by irrational fears. Fear of failure professionally is rampant. Fear of terrorism is also rampant. Fear of immigrants is yet another common fear tied to terrorism, racism and economic stress. Fear is a powerful emotion that overrules most rational responses to problems. It leads to people shrinking away from the sort of professional risk that research depends upon. Fear is also one of the most powerful tools of political despots. In each of these cases bullshit can be used to either quell or amplify fears. In the technical World bullshit can produce seeming success without regard to actual technical accomplishment. The acceptance of bullshit in place of actual results quells the risk of failure. In the political World bullshit can produce fear where little or none is warranted. We are seeing trillions of dollars and millions of votes being generimagexsated by fear mongering bullshit. Even worse it has paved the way for greater purveyors of bullshit like Trump. We may well see bullshit providing the vehicle to elect an unqualified con man as the President.

Here are two major paths to take in life, follow and be rewarded by the power structure, or confront power with the reality of their failing. So I’m confronted with guidance “don’t be a troublemaker” versus show integrity “speak truth to power”. Which is it? Which of these paths do our institutions support today? That’s easy. Be quiet and go quietly through life, and don’t make waves. Better yet, join the bullshit economy and contribute to its vibrant growth as our greatest export.

If you can’t speak truths and provide honest assessments in today’s World, or call out lies in public, can you have personal integrity? When actual bullshit has become the path to professional success, how does one come back? How much of yourself gets left behind in the process? And how does one keep ones self from being a mere shadow of one’s true self? I’ve been struggling with these questions at work with ever-greater regularity. My personal devotion to progress and research is continually undermined by bullshit replacing progress. Today it is easier to make shit up and pass it off as being completely equivalent to the result of honest good work. Moreover bullshit doesn’t have the downside of potentially not working, its “success” is virtually guaranteed. Just tell the people above you in the food chain what they want to hear, you’ll be rewarded in spades! Th1466041941244_947e beauty of it is the made up shit can conform to whatever narrative you desire, and completely fit whatever your message is. In such a world real problems can simply be ignored when the path to progress is problematic for those in power.

These same issues happen to energize a political dialog that becomes a shouting match. There is no truth and the whole thing degenerates into a shouting match. Beyond the shouting match you get the ability to ignore real problems that are inconvenient. A perfect example is climate change. For many traditional businesses climate change is really an inconvenient truth. When bullshit rules, it can be publically ignored without real risk to political success. We are seeing this play out right in front of all of us. We are in danger of losing all connection to facts, truth and science as guiding forces for determining optimal solutions to our very real problems. The road to this end is paved by allowing bullshit to be viewed as equivalent to solid facts.

Is there a path forward? Perhaps things simply need to devolve into the natural outcome from the current path. Nothing short of catastrophe will stop this orgy of bullshit choking public life. One might hope that we have the collective wisdom to avoid a calamity, one might hope.

One of the greatest regrets in life is being what others would want you to be, rather than being yourself.

― Shannon L. Alder

Is Coupled or Unsplit Always Better Than Operator Split?

16 Friday Sep 2016

Posted by Bill Rider in Uncategorized

≈ 1 Comment

No.

The ideal is the enemy of the real.

― Susan Block

Ijohn-von-neumann-2n the early days of computational science we were just happy to get things to work for simple physics in one spatial dimension. Over time, our grasp on more difficult coupled multi-dimensional physics became ever more bold and expansive. The quickest route to this goal was the use of operator splitting where the simple operators, single physics and one-dimensional were composed into complex operators. Most of our complex multiphysics codes operate in this operator split manner. Research into doing better almost always entails doing away with this composition of operators, or operator splitting and doing everything fully coupled. It is assumed that this is always superior. Reality is more difficult than this proposition, and most of the time the fully coupled or unsplit approach is actually worse with lower accuracy and greater expense with little identifiable benefits. So the question is should we keep trying to do this?

This is another example where the reality of simulating difficult problems gives a huge home field advantage to simple approaches. It is much the same as the issues with high-order methods for discretization. Real problems bring complexities and singularities (shocks, corners, turbulence, etc.), and this relegates results to first-order accuracy or less. Operator splitting is often first order accurate without extensive and difficult measures. We have the situation where reality collides with the simplest approach. The truth is that the simple operator split approach is really good and powerful in many, if not most cases. It is important to realize when this is not and something better really is needed.

The unsplit, fully coupled approach really yields an unambiguous benefit when the Supernove-Shocks-1solution involves a precise dynamic balance. This is when you have the situation of equal and opposite terms in the equations that produce solutions in near equilibrium. This produces critical points where solution make complete turns in outcomes based on the very detailed nature of the solution. These situations also produce substantial changes in the effective time scales of the solution. When very fast phenomena combine in this balanced form, the result is a slow time scale. It is most acute in the form of the steady-state solution where such balances are the full essence of the physical solution. This is where operator splitting is problematic and should be avoided.

Such balances are also rarely the entire problem, and often only present in a localized region in time and space. As such the benefit of coupling is not present everywhere and the cost of it should not be applied by the entire procedure. Unfortunately, this isn’t what people do, once they remove operator splitting and fully couple, they do it everywhere. A way forward is to only apply fully coupling where it has a favorable impact on the solution in the region of critical points, and use more effective, accurate and efficient operator splitting elsewhere.

The other reason for not applying coupled methStabilityods is their disadvantage for the fundamental approximations. When operators are discretized separately quite efficient and optimized approaches can be applied. For example if solving a hyperbolic equation it can be very effective and efficient to produce an extremely high-order approximation to the equations. For the fully coupled (unsplit) case such approximations are quite expensive, difficult and complex to produce. If the solution you are really interested in is first-order accurate, the benefit of the fully coupled case is mostly lost. This is with the distinct exception of small part of the solution domain where the dynamic balance is present and the benefits of coupling are undeniable.

This entire dialog is even stronger when considering multi-physics where procedures for solving single physics are highly optimized and powerful. The fully coupled methods tend to be clunky and horribly expensive often being defined by dropping the entire system into an implicit system without regard to the applicability and utility of such an approximation for the problem at hand. To make matters worse the implicitness often undermines accuracy in really pernicious ways in the very regions where the coupling is actually necessary. Moreover the cost of this less accurate approximation is vastly greater due to the nature of the full system, and the departure from all the tricks of the trade leading to efficiency.

A really great path forward is the encouragement to pursue fully coupled methods only where their benefit is greatest. This is another case where the solution 24-Figure17-1method should be adaptive and locally tailored to the nature of the solution. One size fits all is almost never the right answer (to anything). Unfortunately this whole line of attack is not favored by anyone these days, we seem to be stuck in the worst of both worlds where codes used for solving real problems are operator split, and research is focused on coupling without regard for the demands of reality. We need to break out of this stagnation! This is ironic because stagnation is one of the things that coupled methods excel at!

The secrets of evolution are death and time—the deaths of enormous numbers of lifeforms that were imperfectly adapted to the environment; and time for a long succession of small mutations.

― Carl Sagan

 

 

I’m Better When I Don’t Care

12 Monday Sep 2016

Posted by Bill Rider in Uncategorized

≈ 1 Comment

When we are no longer able to change a situation, we are challenged to change ourselves.

― Viktor E. Frankl

urlToday’s title is a conclusion that comes from my recent assessments and experiences at work. It has completely thrown me off stride as I struggle to come to terms with the evidence in front of me. The obvious and reasonable conclusions from the consideration of recent experiential evidence directly conflicts with most of my most deeply held values. As a result I find myself in a deep quandary about how to proceed with work. Somehow my performance is perceived to be better when I don’t care much about my work. One reasonable conclusion is that when I have little concern about outcomes of the work, I don’t show my displeasure when those outcomes are poor.

Do I continue to act naturally and care about my work despite the evidence that such concerns are completely unwelcome? Instead do I take my energy and concern elsewhere and turn work into nothing but a paycheck as feedback seems to directly say? Is there a middle path that preserves some personal integrity while avoiding the issues that seem to cause tension? Can I benefit by making work more impersonal and less important to me? Should I lose any sense of deeper meaning and importance to the outcomes at work?

Persoleadersnal integrity is important to pay attention to. Hard work, personal excellence and a devotion to progress has been the path to my success professionally. The only thing that the current environment seems to favor is hard work (and even that’s questionable). The issues causing tension are related to technical and scientific quality, or work that denotes any commitment to technical excellence. It’s everything I’ve written about recently, success with high performance computing, progress in computational science, and integrity in peer review. Attention to any and all of these topics is a source of tension that seems to be completely unwelcome. We seem to be managed to mostly pay attention to nothing but the very narrow and well-defined boundaries of work. Any thinking or work “outside the box” seems to invite ire, punishment and unhappiness. Basically, the evidence seems to indicate that my performance is perceived to be much better if I “stay in the box”. In other words I am managed to be predictable and well defined in my actions, don’t provide any surprises.

vyxvbzwxThe only way I can “stay in the box” is to turn my back on the same values that brought me success. Most of my professional success is based on doing “out of the box” thinking working to provide real progress on important issues. Recently it’s been pretty clear that this isn’t appreciated any more. To stop thinking out of the box I need to stop giving a shit. Every time I seem to care more deeply about work and do something extra not only is it not appreciated; it gets me into trouble. Just do the minimum seems to be the real directive, and extra effort is not welcome seem to be the modern mantra. Do exactly what you’re told to do, no more and no less. This is the path to success.

When a person is punished for their honesty they begin to learn to lie.

― Shannon L. Alder

I have evidence that my performance is perceived to be better when I don’t give a shit. I’ve quick-fix-movie-to-watch-office-space-imagedone the experiment and the evidence was absolutely clear. When I don’t care, don’t give a shit and have different priorities than work, I get awesome performance reviews. When I do give a shit, it creates problems. A big part of the problem is the whole “in the box” and “out of the box” issue. We are managed to provide predictable results and avoid surprises. It is all part of the low risk mindset that permeates the current World, and the workplace as well. Honesty and progress is a source of tension (i.e., risk) and as such it makes waves, and if you make waves you create problems. Management doesn’t like tension, waves or anything that isn’t completely predictable. Don’t cause trouble or make problems, just do stuff that makes us look good. The best way to provide this sort of outcome is just come to work and do what you’re expected to do, no more, no less. Don’t be creative and let someone else tell you what is important. In other words, don’t give a shit, or better yet don’t give fuck either.hqdefault

Why should I work so hard or put so much effort into something that isn’t appreciated? I have other things to do in my (finite) life where effort is appreciated. The conclusion is that I should do much more about things away from work, and less at work. In other words I need to stop giving a shit at work. Its what my feedback is telling me, and it’s a route to sanity. It is appalling that its come to this, but the evidence is crystal clear.

Some men are born mediocre, some men achieve mediocrity, and some men have mediocrity trust upon them.

― Joseph Heller

 

 

The Real Problem with Classified E-mail

09 Friday Sep 2016

Posted by Bill Rider in Uncategorized

≈ Leave a comment

635933172260783601-hillary-clinton-miami-rally-super-tuesday-27The news is full of stories and outrage at Hillary Clinton’s e-mail scandal. I don’t feel that anyone has remotely the right perspective on how this happened, and why it makes perfect sense in the current system. It epitomizes a system that is prone to complete breakdown because of the deep neglect of information systems both unclassified and classified within the federal system. We just don’t pay IT professionals enough to get good service. The issue also gets to the heart of the overall treatment of classified information by the United States that is completely out of control. The tendency to classify things is completely running amok far beyond anything that is in the actual best interests of society. Tobill-clinton-pardoned-john-deutch-rcompound things, further it highlights the utter and complete disparity in how laws and rules do not apply to the rich and powerful. All of this explains what happened, and why; yet it doesn’t make what she did right or justified. Instead it points out why this sort of thing is both inevitable and much more widespread (i.e., John Deutch, Condoleezza Rich, Colin Powell,imgres and what is surely a much longer list of violations of the same thing Clinton did).

Management cares about only one thing. Paperwork. They will forgive almost anything else – cost overruns, gross incompetence, criminal indictments – as long as the paperwork’s filled out properly. And in on time.

― Connie Willis

imagesLast week I had to take some new training at work. It was utter torture. The DoE managed to find the worst person possible to train me, and managed to further drain them of all personality then treated him with sedatives. I already take a massive amount of training, most of which utterly and completely useless. The training is largely compliance based, and generically a waste of time. Still by the already appalling standards, the new training was horrible. It is the Hillary-induced E-mail classification training where I now have the authority to mark my classified E-mails as an “E-mail derivative classifier”. We are constantly taking reactive action via training that only undermines the viability and productivity of my workplace. Like most my training, this current training is completely useless, and only serves the “cover your ass” purpose that most training serves. Taken as a whole our environment is corrosive and undermines any and all motivation to give a single fuck about work.

Let’s get to the point of why Hillary was compelled to use a private e-mail system in the first place? Why classified information appeared in the place? Why people in positions of power feel they don’t have to follow rules?

Most people watching the news have little or no idea about the classified computing or e-mail systems. So let’s explain a few things about the classified systems people work on that will get to the point of why all of this is so fucking stupid. For starters the classified computing systems are absolutely awful to use. Anyone trying to get real work done on these systems is confronted with the utter horror they are to use. No one interested in productively doing work would tolerate them. In many government places the unclassified computing systems are only marginally better. The biggest reasons are lack of appropriately skilled IT professionals and lack of investment in infrastructure. Fundamentally we don’t pay the IT professionals enough to get first-rate service, and anyone who is good enough to get a better private sector job does. Moreover these incompetencedemotivatorprofessionals work on old hardware with software restrictions that serve outlandish and obscene security regulations that in many cases are actually counter-productive. So, if Hillary were interested in getting anything done she would be quite compelled to leave the federal network for greener, more productive pastures.

The more you leave out, the more you highlight what you leave in.

― Henry Green

Where one might think that the government would give classified work the highest priority, the environment for working there is the worst. Keep in mind that it is worse than the already shitty and atrocious unclassified environment. The seeming purpose of evermistakesdemotivator_largeything is not my or anyone’s actual productivity, but rather the protection of information, or at least the appearance of protection. Our approach to everything is administrative compliance with directives. Actual performance on anything is completely secondary to the appearance of performance. The result of this pathetic approach to providing the taxpayer with benefit for money expended is a dysfunctional system that provides little in return. It is primed for mistakes and outright systematic failures. Nothing stresses the system more than a high-ranking person hell-bent on doing their job. The sort of people who ascend to high positions like Hillary Clinton find the sort of compliance demanded by the system awful (because it is), and have the power to ignore it.

peter_nanosOf course I’ve seen this abuse of power live and in the flesh. Take the former Los Alamos Lab Director, Admiral Pete Nanos who famously shut the Lab down an denounced the staff as “Butthead Cowboys!” He blurted out classified information in an unclassified meeting in front of hundreds if not thousands of people. If he had taken his training, and been compliant he should have known better. Instead of being issued a security infraction like any of the butthead cowboys in attendance would have gotten, he got a pass. The powers that be simply declassified the material and let him slide by. Why? Power comes with privileges. When you’re in a position of power you find the rules are different. This is a maxim repeated over and over in our World. Some of this looks like white privilege, or rich white privilege where you can get away with smoking pot, or raping unconscious girls with no penalty, or lightened penalties. If you’re not white or not rich you pay a much stiffer penalty including prison time.7597423806_3213679a80_b

I learned the lesson again at Los Alamos in another episode that will remain slightly vague in this post. I went to a meeting that honored a Lab scientist’s career. During the course of the meeting another Lab director read an account of this person’s work noting their monumental accomplishments and contributions to the national security. All of the account was good, true and correct except it was classified in its content. I took the written text to the classification office at the Lab and noted its issues. They agreed that it was indeed classified. Because the people who wrote the account (very high ranking DoE person) and the person who read it were so high ranking they would not touch this with the proverbial ten-foot pole. They knew a violation had occurred, but their experience also told them that it was foolish to pursue it. This pursuit would only hurt those who pointed out the problem and those committing the violations were immune.

Let me ask you, dear reader, how do you think someone would treat the Secretary of State of the United States. How much more untouchable would they be? It is certainly wrong in a perfect World, but we live in a very imperfect world.

A secret’s worth depends on the people from whom it must be kept.

― Carlos Ruiz Zafón

Castle_Union
bomb.jpg_1718483346

The core philosophy in all of this is that we have lots of secrets to protect because we are the biggest and baddest country on Earth. It was certainly true at one time, but every day I wonder less and less if we still are and gain assurance that we are not. So we have created a system that is predicated on our lead in science and technology, but completely and utterly undermines our ability to keep that lead. We have a system that is completely devoted to undermining our productivity at every turn in the service of protecting information that loses its real value every day. To put it differently our current approach and policy is utter and complete fucking madness!

I also want to be clear that classification of a lot of material is absolutely necessary. It is essential to the safety and security of the Nation and the World. The cavalier and abusive way that classification is applied today runs utterly counter to this. By classifying everything in sight, we reduce the value and importance of the things that must be classified. By using classification of documents to cover everything with a blanket, the real need and purpose of classification is obscured and harmed deeply. All of this said I have not discussed the most widely abused version of classification, “Official Use Only,” which is applied in an almost entirely unregulated manner. It is abused widely and casually. Among the areas regulated by this awful policy is the Export Controlled Information, which is easily one of the worst laws I’ve ever come in contact with. It is just simply put stupid and incompetent. It probably does much more harm than good to the national security of the nation.

Power does not corrupt. Fear corrupts… perhaps the fear of a loss of power.

― John Steinbeck

GOP 2016 Debate

Republican presidential candidate, businessman Donald Trump stands during the Fox Business Network Republican presidential debate at the North Charleston Coliseum, Thursday, Jan. 14, 2016, in North Charleston, S.C. (AP Photo/Chuck Burton)

Let’s be clear about the Country and World we live in. The rich and powerful are corrupt. The rich and powerful are governed by entirely different rules than everyone else. Mistakes, violations of the law, and morality itself for the rich and powerful are fundamentally different than the common man. So to be clear Hillary Clinton committed abuses of power. Donald Trump has committed abuses of power too. Barack Obama has as well. Either Hillary or Trump will continue to do so if elected President. Until the basic attitudes toward power and money change we should expect this to continue. The same set of abuses of power happen across the spectrum of society in every organization and business. The larger the organization or business, the worse the abuse of power can expect to be. As long as it is tolerated it can be expected to continue.

A man who has never gone to school may steal a freight car; but if he has a university education, he may steal the whole railroad.

― Theodore Roosevelt

Our societal approach to classification of documents is simply a tool of this sort of rampant abuse of power. We see any sense of a viable “whistleblower” protection to be imgres-1complete and utter bullshit. People who have highlighted huge systematic abuses of power involving murder and vast violation of constitutional law are thrown to the proverbial wolves. There is no protection, it is viewed as treason and these people are treated as harshly as possible (Snowden, Assange, and Manning come to mind). As I’ve noted above people in positions of authority can violate the law with utter impunity. At the same time classification is completely out of control. More and mocnt4_fr53-1re is being classified with less and less control. Such classification often only serves to hide information and serve the needs of the status quo power structure.

In the end, Hillary had really good reasons to do what she did, and believe that she had the right to do so. Everything in the system is going to provide her with the evidence that the rules for everyone else do not apply to her. Hillary wasn’t correct, but we have created an incompetent, unproductive computing system that virtually compelled her to choose the path she took. We have created a culture where the most powerful people do not have to follow the rules that the regular guy rules. The system has been structured by fear and lack of trust without any regard for productivity. If we want to remain the most powerful country, we need to change our priorities on productivity, secrecy and the corruption of power.

The whole issue of runaway classification, classified e-mails and our inability to produce a productive work environment in National Security is at the nexus of incompetence, lack of trust, corruption resulting in a systematic devotion to societal mediocrity.

Arguing that you don’t care about the right to privacy because you have nothing to hide is no different than saying you don’t care about free speech because you have nothing to say.

– Edward Snowden

 

How to strive for excellence in modeling & simulation

02 Friday Sep 2016

Posted by Bill Rider in Uncategorized

≈ Leave a comment

Where the frontier of science once was is now the centre.

― Georg Christoph Lichtenberg

mediocritydemotivatorI’ll just say up front that my contention is that there is precious little excellence to be found today in many fields. Modeling & simulation is no different. I will also contend that excellence is relatively easy to obtain, or at the very least a key change in mindset will move us in that direction. This change in mindset is relatively small, but essential. It deals with the general setting of satisfaction with the current state and whether restlessness exists that ends up allowing progress to be sought. Too often there seems to be an innate satisfaction with too much of the “ecosystem” for modeling & simulation, and not enough agitation for progress. We should continually seek the opportunity and need for progress in the full spectrum of work. Our obsession with planning, and micromanagement of research ends up choking the success from everything it touches by short-circuiting the entire natural process of progress, discovery and serendipity.

Iunderachievementdemotivatorn my view the desire for continual progress is the essence of excellence. When I see the broad field of modeling & simulation the need for progress seems pervasive and deep. When I hear our leaders talk such needs are muted and progress seems to only depend on a few simple areas of focus. Such a focus is always warranted if there is an opportunity to be taken advantage of. Instead we seem to be in an age where the technological opportunity being sought is arrayed against progress, computer hardware. In the process of trying to force progress where it is less available the true engines of progress are being shut down. This represents mismanagement of epic proportions and needs to be met with calls for sanity and intelligence in our future.

If you find that you’re spending almost all your time on theory, start turning some attention to practical things; it will improve your theories. If you find that you’re spending almost all your time on practice, start turning some attention to theoretical things; it will improve your practice.

–Donald Knuth

So how do we get better at modeling & simulation? The first thing is mindset; do we think going into our work, my goal is to make this thing good? Or state of the art? Or simply how can I make it better? The final question is the “right” one, you can always make it better, and in the process the other two questions will get answered. Too often today we never get into the fundamental mode of simply working toward continual improvement as our default mode of operation. The manner of energizing our work to do this is frightenly simple to pursue, but rarely in evidence today.

8The way toward excellence, innovation and improvement is to figure out how to break what you have. Always push your code to its breaking point; always know what reasonable (or even unreasonable) problems you can’t successfully solve. Lack of success can be defined in multiple ways including complete failure of a code, lack of convergence, lack of quality, or lack of accuracy. Generally people test their code where it works and if they are good code developers they continue to test the code all the time to make sure it still works. If you want to get better you push at the places where the code doesn’t work, or doesn’t work well. You make the problems where it didn’t work part of the ones that do work. This is the simple and straightforward way to progress, and it is stunning how few efforts follow this simple, and obvious path. It is the golden path that we deny ourselves of today.

The reasons for not engaging in this golden path are simple and completely and utterly pathological. The golden path is not easy to manage. This golden path is epitomized by out of the box thinking. Today we prize in the box thinking because it is suitable for management, and strict accountability. This strict accountability is the consequence of societal structures that lack trust and implicitly fear independent thought. Out of the box is unstructured and innovative eschewing management control. As such we introduce systems that push everything inside the proverbial box. Establishing results that are predictable has become tantamount to being trustworthy. Out of the box thinking is dangerous and the subject of fear because it cannot be predicted. This is the core of our current lack of innovation, and the malaise in modeling & simulation.

The element of thinking that is missing from how things currently progress is a sense of satisfaction about too much of what has driveimages-1n the success of modeling & simulation to date. We are too satisfied that the state of the art is fine and good enough. We lack a general sense that improvements, and progress are always possible. Instead of a continual striving to improve, the approach of focused and planned breakthroughs has beset the field. We have a distinct management approach that provides distinctly oriented improvements while ignoring important swaths of the technical basis for modeling & simulation excellence. The result of this ignorance is an increasingly stagnant status quo that embraces “good enough” implicitly through a lack of support for “better”.

There seems to be a belief that the current brand of goal-oriented micromanagement is good for technical achievement. Nothing could be further from the truth; the current goal based management philosophy is completely counter-productive and antithetical to good science and achievement. It leads to systematic goal reduction and lack of risk-taking on the part of organizations. A big part of this is the impact of the management style on the intrinsic motivations of the scientists. Scientists tend to be quite easily and intrinsically motivated by curiosity and achievement while the management system is focused on extrinsic motivation.

The test of a man isn’t what you think he’ll do. It’s what he actually does.

― Frank Herbert

We end up undermining all of the natural and simple aspects that lead to productive, innovative excellence in work, replacing these factors with a system that undermines what comes naturally. All of this has a single root, lack of trust and faith in the people doing the work. Without rebuilding the fundamental trust in providing intrinsically motivated and talented people a productive environment, I fear nothing can be done to improve our outcomes and grasp the excellence that is there for the taking. People would gravitate toward excellence naturally if the management would simply trust them and work to resonate with people’s natural inclinations.

computer-modeling-trainingModeling & simulation arose to utility in support of real things. It owes much of its prominence to the support of national defense during the cold war. Everything from fighter planes to nuclear weapons to bullets and bombs utilized modeling & simulation to strive toward the best possible weapon. Similarly modeling & simulation moved into the world of manufacturing aiding in the design and analysis of cars, planes and consumer products across the spectrum of the economy. The problem is that we have lost sight of the necessity of these real world products as the engine of improvement in modeling & simulation. Instead we have allowed computer hardware to become an end unto itself rather than simply a tool. Even in computing, hardware has little centrality to the field. In computing today, the “app” is king and the keys to the market hardware is simply a necessary detail.

Cielo rotatorTo address the proverbial “elephant in the room” the national exascale program is neither a good goal, nor bold in any way. It is the actual antithesis of what we need for excellence. The entire program will only power the continued decline in achievement in the field. It is a big project that is being managed the same way bridges are built. Nothing of any excellence will come of it. It is not inspirational or aspirational either. It is stale. It is following the same path that we have been on for the past 20 years, improvement in modeling & simulation by hardware. We have tremendous places we might harness modeling & simulation to help produce and even enable great outcomes. None of these greater societal goods is in the frame with exascale. It is a program lacking a soul.

The question always comes to what am I suggesting be done instead? We need to couch our overall efforts in modeling & simulation in supporting real world objectives. Something like additive manufacturing comes to mind as a modern example that would serve us far better than faster computers. We need to define a default attitude that progress is always possible and always something to be sought. Unfortunately, this more sensible and productive approach is politically untenable today. The real problem isn’t intellectual or bound in thoughtful dialog, but rather bound to a deep lack of faith and trust in science and scientists. We have poorly thought through programs focused on marketing and micromanagement as a direct result. Progress be damned.

There is a very real danger present when we suppress our feelings to act on inspiration in exchange for the “safety” of the status quo.

We risk sacrificing the opportunity to live a more fulfilling and purpose driven life. We risk sacrificing the opportunity to make a difference in the lives of others. We risk sacrificing the beautiful blessing of finding a greater sense of meaning in our own lives.

In short, we run the very real risk living a life of regret.

― Richie Norton

 

Subscribe

  • Entries (RSS)
  • Comments (RSS)

Archives

  • February 2026
  • August 2025
  • July 2025
  • June 2025
  • May 2025
  • April 2025
  • March 2025
  • February 2025
  • January 2025
  • December 2024
  • November 2024
  • October 2024
  • September 2024
  • August 2024
  • July 2024
  • June 2024
  • May 2018
  • April 2018
  • March 2018
  • February 2018
  • January 2018
  • December 2017
  • November 2017
  • October 2017
  • September 2017
  • August 2017
  • July 2017
  • June 2017
  • May 2017
  • April 2017
  • March 2017
  • February 2017
  • January 2017
  • December 2016
  • November 2016
  • October 2016
  • September 2016
  • August 2016
  • July 2016
  • June 2016
  • May 2016
  • April 2016
  • March 2016
  • February 2016
  • January 2016
  • December 2015
  • November 2015
  • October 2015
  • September 2015
  • August 2015
  • July 2015
  • June 2015
  • May 2015
  • April 2015
  • March 2015
  • February 2015
  • January 2015
  • December 2014
  • November 2014
  • October 2014
  • September 2014
  • August 2014
  • July 2014
  • June 2014
  • May 2014
  • April 2014
  • March 2014
  • February 2014
  • January 2014
  • December 2013
  • November 2013
  • October 2013
  • September 2013

Categories

  • Uncategorized

Meta

  • Create account
  • Log in

Blog at WordPress.com.

  • Subscribe Subscribed
    • The Regularized Singularity
    • Join 55 other subscribers
    • Already have a WordPress.com account? Log in now.
    • The Regularized Singularity
    • Subscribe Subscribed
    • Sign up
    • Log in
    • Report this content
    • View site in Reader
    • Manage subscriptions
    • Collapse this bar
 

Loading Comments...