Monthly Archives: January 2025

Watching Regression of Progress Live

30 Thursday Jan 2025

Posted by in Uncategorized

≈ Leave a comment

Tags

, , , ,

tl;dr

When I examine the arc of my career I see a repeated over-emphasis on focusing on computers instead of balanced progress. The first epoch happened in the mid-1990’s where buying fast computers was sold to replace nuclear testing. We are in the midst of a longer period of only focusing on computers (the Exascale project). This is a simple narrative and sells. It is also wrong. Progress depends on support for broader activities. The result is a diminished level of progress. We are witnessing this again with AI via DeepSeek’s revelations this week.

“Don’t let the noise of others’ opinions drown out your own inner voice.” ― Steve Jobs

Progress is not Guaranteed

I’ve devoted my life to science and its progress. The promise of doing exactly this was the reason I joined Los Alamos back in 1989. Their track record in science was stellar, powered by a pantheon of science superstars. I was honored and lucky to join them. At that time Los Alamos still had a marvelous spirit of discovery. I benefited from a fantastic sense of generosity from my fellow scientists who shared their knowledge with me. It made me who I am, and shaped my career. The other thing that has been a centerpiece of my career there and later is stockpile stewardship. This program is the care and understanding of the nuclear weapons stockpile using science and engineering. A big part of this program is the use of modeling and simulation as a tool so that testing the nukes is unnecessary.

A big part of modeling and simulation is computers. The truth is that the bigger the computer the better. Well, not necessarily big, but faster computers are “always” better. There is a caveat to the “always” that needs deep consideration. Faster and bigger is always better comes with caveats and those conditions are subtle and complex. Being subtle and complex they are ignored. Ignored at our peril. Even science has succumbed to the superficial nature of today’s society. In a nutshell, many technical fundamentals need to be in place for the bigger and faster is better to hold. Today those fundamentals are at risk. The risk comes from them being ignored with astounding regularity.

To review, one of the key aspects of stockpile stewardship when it was initiated in the mid-1990s was simulating nuclear weapons. The original approach was basically to put computer codes on the fastest computers in the World. The program happened exactly at the time that the basic approach to high-performance computing changed from Cray vector computers to massively parallel computing. This meant rewriting the codes to use this new type of computer. The process of replacing the old codes (deemed legacy codes) was difficult. This was because the legacy codes were used to design and analyze weapons during the test era. So the legacy codes were subjected to repeated testing against difficult experiments. Thus legacy codes were trusted and ably used. Therefore they were held onto and revered almost as sacred. It took more than a decade to replace them and it was a mighty struggle.

The program did not explicitly want to produce better codes with better methods or algorithms or physics. Nonetheless, some of this happened because methods, algorithms, and physics make a big difference in modeling quality. This almost happened in a subrosa fashion as modernized codes only really meant codes running on modern computers.

Simple Narratives Win

“Physics is to math what sex is to masturbation.” ― Richard Feynman

The reason for this is easy to see. A faster computer is obviously better than a slower computer. The speed makes for an easy narrative about improvement. We live in an age where simple narratives rule. The public and politicians alike seem to recoil from complex and subtle explanations or solutions. In the wake of this trend, we see a loss of effectiveness and a massive waste of resources. The constant din of a simple solution to progress is reliance on computing power to solely carry progress. This did not make sense in the era when Moore’s law was in effect. It makes even less sense with Moore’s law being dead.

Moore’s law was an empirical law about the growth of power in computing over a long period. It was first observed by Gordon Moore in 1965 and held until around 2015. It was a powerful exponential law that had computer power doubling every 18 months to 2 years. If one does the math over that 50-year period (a factor of about a million). This yields phenomenal speed-ups in computing. Physical limits of computing hardware basically led to the slowing down and end of Moore’s law. Computers are speeding up, but much more slowly now. The response to the government funding was to then focus on computing, which is mind-blowing. The money was applied to try to bring the dead patient to life. This produced the National Exascale program with its focus on computing hardware.

Why is this so dumb?

Over the long history of computational science other advances have been as beneficial as computing power. In a nutshell, algorithmic advances have led to more improvements than computing. In the modern era, the focus and support for algorithmic advances have slowed to a trickle. Even in the early years the algorithmic advances received less support but had an equal or greater impact on computing capability. Nonetheless, the support never reflected the value of the approach. The reasons will be explored next.

“Creativity is intelligence having fun.” ― Albert Einstein

Algorithmic advances never created the sort of steady improvement of Moore’s law. They tend to be episodic and unpredictable. Modern project management is not suitable for such things. Progress is often fallow for long periods with a sudden advance. In a low-trust environment, this is unacceptable. Algorithmic research is extremely risky. Again, the risk is something that low trust annihilates. The long-term impact of the failure to invest in algorithmic work is a profound and massive reduction in computing benefits. I would argue that we have lost orders of magnitude of computing ability through a lack of investment in algorithms alone.

Reality is Complex

As I was writing the news broke of DeepSeek, the Chinese AI Chatbot that shattered the narratives around LLMs. The assumptions that American companies and the government had about advancing AI were overturned.

Why did this happen?

Our actions as a nation forced the Chinese to adapt and innovate. American companies were following the path of brute force. This was exactly like the computational science’s exascale program. The warning signs have been brewing for a while. The LLMs are running out of data to scale. We are running out of computer power too. The energy demands were huge and excessive. We are starting to see AI as a threat to the environment. The training of LLM models is inefficient and very expensive. In a nutshell, the brute force approach was about to collapse as a source of progress. This was predictable.

This problem was ripe for disruption and a bolt from the blue. DeepSeek looks like that bolt.

I will return to this. The narrative applies to our approach to computational science more broadly. We have an overreliance on brute force while discounting and ignoring the power of innovation and efficiency. We can all see the lessons embedded in the DeepSeek episode unfolding have been present and obvious for years. Obvious does not mean that they are acknowledged. Obvious does not compel our leaders to action. The overreliance on computing power is a simple narrative that is hard to dislodge unless is smacks us in the face.

Here is the truth and a lesson worth holding close. Reality is a real motherfucker. Reality is complex and dangerous. Reality will eventually win every battle. Reality is undefeated. Reality will fuck you up. This is the maxim of “fuck around and find out.”

There are many paths to progress. As exemplified by the DeepSeek example when we are denied the obvious path, people innovate. In computational science in the USA, we have made computer power the obvious choice. At the same time, other paths are ignored and systematically divested from. In some cases, paths to progress are explicitly removed from possibility. This looks like a doubling down on the focus on the approach being taken even as evidence piles up that it’s stupid. The worst part of this is the outright ignorance and avoidance of learning from the past. We ought to know better because the evidence is overwhelming.

Balance and Opportunity

“The formulation of the problem is often more essential than its solution, which may be merely a matter of mathematical or experimental skill.” ― Albert Einstein

Over the long time computational science has progressed on many fronts. There is no doubt that raw computing power is part of the reason. Computers are tangible and obvious signposts to progress. Various eras of computational science are clearly marked by the computers used to do the science. Early computers are far different than vector crays, or massively parallel computers. Today’s massive GPU computers and data centers are emblematic of today. Models, algorithms, and computer codes are far more abstract and less obvious to the casual observer. Nonetheless, the abstract aspects of progress are essential, and perhaps more important.

The models produced by the codes are solved using algorithms that harness computer power. The computers are useless without them. A bad algorithm assures that the computer itself is used ineffectively and wastes time and energy. Almost the entire utility of modeling and simulation is bound to modeling, thus its importance. Computer code has become an important part of modern life with an entire discipline devoted to it. At least the code is somewhat paid attention to.

The problem with models and algorithms is twofold. Above I focused upon the abstract nature of them as a problem. Being abstract they are difficult to understand. They have a second more difficult issue surrounding their progress. This is the episodic nature of progress. Both models and algorithms require difficult theoretical work highly prone to failure for progress. Often improvements are many years apart with extensive failure. At the same time when modeling or algorithms do improve, the leap in performance is large and essentially discontinuous. It looks like a quantum leap as opposed to the incremental steady climb of Moore’s law. Risk-averse program managers wanting predictable outcomes recoil from this. As a result, the work in this area is not favored. Years of failure are punished rather than seen as laying the ground for glorious success. All of this equals the choking off of progress in these areas.

The damage to the potential progress is massive. Rather than seeing a balanced approach to progress, we put all our effort into incremental computing growth. The equal or greater source of progress is ignored because we don’t know how to manage it. Computer codes move along being adapted to new computers, but encoding old models and algorithms. These new codes would nominally be perfect vehicles for introducing new algorithms and models. More often than not the codes simply move along reimplementing old models and algorithms. In many cases, we simply get the same wrong answers with poor efficiency for a greater cost. This is nothing short of a tragedy.

On occasion, we get a peek at these things. The example of DeepSeek is one such view and it was a shock. Suddenly we saw that everything we thought and had been told about LLMs was suspect. The reason for this is the acceptance of the narrative that the quality of LLMs is built on massive data and computing. The breakthrough we saw a couple years ago was powered by an algorithm (ChatGPT and LLMs were enabled by the “Transformer” algorithm). After this, we were lulled into just seeing it deriving from raw computational power. Plus it was great for NVIDIA stock and our 401Ks. It did not spur and investment into what actually drove the progress.

The algorithm was the actual “secret sauce.”

“The measure of intelligence is the ability to change.” ― Albert Einstein

Why are we such idiots? Why do we make the same mistakes over and over? Seeing the rise of computing focus while everything else fades. We learn nothing from the past. Reality is coming for us again.

“We can only see a short distance ahead, but we can see plenty there that needs to be done.” ― Alan Turing

Fundamental American Decency

20 Monday Jan 2025

Posted by in Uncategorized

≈ Leave a comment

Tags

, , , ,

tl;dr

Recently, I took a trip with my elderly father. What stood out to me was the kindness, generosity and decency of everyone we encountered. This contrasts with the general discourse highlighted every day online, or in politics. This seems to say that in person we are better. Somehow we need to get our true selves engaged more and our online avatars less. If we don’t things are going to go to shit.

“I’m inspired by the people I meet in my travels–hearing their stories, seeing the hardships they overcome, their fundamental optimism and decency.” ― Barack Obama

Common Decency

Recently, I took a trip with my dad. We flew from Albuquerque to Minneapolis via a connection in Denver. The purpose was a visit to the Mayo Clinic for treatment. My brother and his wife both work at Mayo and have access to care there for my dad. It was an extremely difficult trip because of my dad’s condition. He is 87, and has multiple medical issues including near blindness. He is quite weak and needs a wheelchair in the airport. Just for reference, I am 61 and while I am fit, strong, and vibrant; I’m not a young man. This was one of the hardest flights I’ve ever taken. By the time I handed my dad off to my brother, I was exhausted emotionally and tired physically.

When I reflected upon the day traveling, one thing stood out to me. Everyone we encountered was great. People were helpful. People were generous. People were kind. At every juncture, the airport employees, the airline employees, and our fellow passengers treated us wonderfully. People observed the situation and gave my dad deference and care. People helped us and stepped aside. Flight attendants were so helpful, ingenious, and kind. I saw lots of extra effort to help us and make the best of a very difficult situation. What I saw was Americans being the best versions of themselves and it was phenomenal. It was a tonic after the recent months of horror.

With everything else going on in the USA, it also made me say “What the fuck?”

Uncommon Indecency

“A saint is a person who behaves decently in a shockingly indecent society.” ― Kurt Vonnegut

The entire experience of this flight is in direct conflict with what we see elsewhere in American society. All the evidence would point to Americans being mean, cruel, and thoughtless. We see anger and ignorance everywhere. We just elected a petty, cruel, and selfish man as President. The incoming President displays these characteristics all the time. Somehow Americans overlooked his obvious shortcomings, and appalling character when voting. We are about to be led by someone who is the worst of us.

In person, I saw Americans who were the complete opposite. I saw people who exemplified the care and love of their common man. I saw something that gave me pride and hope. Yet in the engagement and discourse we see every day in the news and online, Americans are horrendous to others. We can all ask why? and examine the causes for this dissonance. One would think we want to be our best selves rather than our worst.

So WTF?

I think the key difference is the prevalence of our online self and remote discourse. The online world seems to encourage a level of vitriol and negativity commonly called trolling. Social media platforms like X (Twitter) and FaceBook thrive on this sort of awful dialog. We all say and talk to people in ways that we’d never do in person. Somehow society has transformed into a reflection of this dynamic more broadly. Our politics has become like social media and unremittingly ugly. We have decided to elect the trolls to run the country. Instead of the common decency I saw in person, we see ugliness and hate. A government is the reflection of its people. Rather than good and decency like we are in person, we have chosen evil and indecency.

In every respect our lives would be better off if Americans treated each other better. Having seen what is possible on this trip this much is obvious. People can be good to each other. They can act with kindness, love, and respect to their fellow man. This stands in stark and genuine contrast to the dynamic seen every day in the news and online. People have it in them to be better. I fear that we need to be led to do good. Right now, we are being led to be the opposite. I hope we do not lose sight of what is possible.

“For the powerful, crimes are those that others commit.” ― Noam Chomsky,

Goodbye Jim with Love and Gratitude

12 Sunday Jan 2025

Posted by in Uncategorized

≈ Leave a comment

Tags

, , ,

tl;dr

I recently discovered that one of my best friends had died. Jim was one of the most important people in my life. But I only discovered his death 21 months after it happened. There are reasons for this. To put this in context, I’ll talk about the death of three people who have touched my life for good and ill. There are lessons to learn from each of them. Among these lessons are what people mean and how I should leave life myself when my time comes.

This essay will be about death and life. It will be a little raw. If that’s not what you’re prepared for don’t read on, or come back later when you are.

“A purpose of human life, no matter who is controlling it, is to love whoever is around to be loved.” ― Kurt Vonnegut

How to Say Goodbye

A few weeks ago, my week started off wondering about a friend, wondering if that friend was still alive. She wasn’t. It was the week after Thanksgiving, and work was spinning up anew after the holiday. The day before Thanksgiving, my friend, Sandy, sent me a brief text: “Thank you for being a good friend and lover.” Sandy had been sick for over a year, afflicted with cancer. Later in the morning, my worry was confirmed. Sandy had passed away that morning. Her kids posted the news on her Facebook profile. The message the previous Wednesday was goodbye, and a heartfelt thanks.

I hadn’t seen her since the previous February when she told me of being worried about the cancer. Her brother had died from cancer, and it seemed to run in the family. We kept in touch through texting, and I knew generally how she was doing. Her treatments worked for a bit until they didn’t. I knew she had a PET scan. I also know how that can work. I remember the moment of seeing my father-in-law’s PET scan and knowing then that the end was near. It is a test that can be a release or a death sentence. I suspect this was what happened to Sandy. She was a lovely lady who loved heavy metal. We shared an enjoyment of Alice in Chains quite often when we got together. She was a casual friend, what someone would call a “FWB”. Still, she said goodbye and left me a thank you for the time we had.

We had closure.

“The fear of death follows from the fear of life. A man who lives fully is prepared to die at any time.” ― Mark Twain

A photo of Jim from 2013 at a Mathematical Workshop he helped organize.

How Not to Say Goodbye

Sandy’s death filled me with a modest melancholy, but it was also expected. I had time to prepare and understand the context of our friendship. The very next day, I awoke to find a Facebook message that hit me hard. The previous evening, I posted the last of my series on my career, the Requiem series, with its focus this time on my time at Sandia. My friend Peter, who is a Mechanical Engineering Professor, asked about Jim. The three of us worked together in Los Alamos during Peter’s postdoc there.

About once every two or three months for the past eight years someone asks me about Jim: how can I get in touch with him? The presumption is that I will know how to contact him. I don’t, as I will explain shortly. When I woke the next morning another friend, Raphael, who is a Professor in France, notified me about Jim’s status.

My friend Jim was dead.

He had died the previous March (March 1, 2023, which I discovered via internet searches) and had managed to donate his math books to the University. He had time and knew he was going to die. He lived in a very small village in France with his wife. It was beautifully decorated in the fashion of New Mexico houses, too. That was it. I knew nothing else. Jim was gone. Worse yet, there was no closure, and there would be none.

For most of us who knew him, Jim disappeared in August 2016. I remember well our final conversation over lunch at Hot Rocks in Los Alamos a few months prior. I remember a somewhat contentious and heated discussion of the state of the Country and Lab. My own life was unsettled at that time. Jim was upset at the United States and the possibility of Trump being elected. Los Alamos had lost its luster and was disappointing him. Maybe he was disappointed with me, too. I’d been getting tattooed and had an open marriage. Maybe I wasn’t the person he thought I was. Who knows? It was a final conversation unfit for two people who had experienced so much life together. It was not a worthy goodbye to someone so important to me.

When I say Jim “disappeared”, I mean it. Aside from Raphael, no one had heard from him. Every friend I contacted since informing them that Jim had died knew nothing about his fate. I spent much of the next week contacting people who worked with Jim via e-mail and Facebook. In every case I got a note of sadness and surprise, but never anyone who said, “Yes, I had heard.” As this sank in, I felt a little bit of relief in the feeling that Jim left everyone behind. I wasn’t singled out either for good or ill. He ghosted everyone. A few friends talked about other people who disappeared suddenly, too. In every case, the disappearance of a friend is a source of pain.

“The opposite of love is not hate, it’s indifference. The opposite of art is not ugliness, it’s indifference. The opposite of faith is not heresy, it’s indifference. And the opposite of life is not death, it’s indifference.” ― Elie Wiesel

Closure and Perspective

One of the things that lingers with me with Jim’s death is the lack of closure. Closure as a process is a precarious and challenging concept. My wife has struggled with it as a relationship she had ended without any closure. Later over time, she got some closure, but it was deeply unsatisfying, too. It did not meet expectations at all. With Jim, nothing ever came, and he approached death without any attempt to close the door with me, or anyone else. So, it is left to the living to find a way to close this chapter of our continuing lives.

Someone else I knew died without giving me any closure. Unlike Jim, this person had a horrible influence on my life. Sam was one of the most toxic people I have ever met. The fact that he was placed in a position of leadership was an indictment of people’s judgment. He was disingenuous to his core; he was also manipulative and vindictive. He abused power. All of this is generally ignorable except the fact that I was the object of this abuse. He was behind one of the worst things in my adult life. With his death, the minute chance of apology was gone.

Any closure or forgiveness on my part was purely one-sided. I need this, too. I need to forgive Sam for his horrible behavior. I need to move on. It is a work in progress. Sam’s death was a genuine tragedy. In addition to the personal side of it, Sam never had a chance to be a better person. He never could heal from whatever demons drove him to such monstrous behavior. I can give myself some closure in that he was a victim of an environment that created his dysfunction, and a system that rewarded him for it. He hurt me badly and likely didn’t care at all. He didn’t care about the well-being of others in his charge. He acted with cowardice and dishonestly toward me. This is a sad way for someone to live. I can learn from this and work toward being a better human from the lesson.

I can also take this lack of closure forward to putting Jim’s life in perspective and how he impacted my life.

“Closure is just as delusive-it is the false hope that we can deaden our living grief.” ― Stephen Grosz

A Life Well-Lived

“Death ends a life, not a relationship.” ― Mitch Albom, Tuesdays with Morrie

Jim was a great influence on my life. He was a good man, and I am richer for knowing him. I have worked with many great and wonderful people during my career, but Jim stands out. We had a great bond of friendship and shared numerous battles and adventures in an exciting time. For this reason, the way Jim deserted me hurt especially deeply. Anyone who knew us would have assumed Jim would stay in touch with me. His abandonment of me and all things American was painful. It is worth some deeper consideration. Perhaps, for Jim, it was simply too painful to continue engaging with all of us.

I met Jim in 1996 as we joined the hydrodynamics group in X-Division. The burgeoning ASCI program was injecting life into a weapon’s program that had been in freefall since the end of the Cold War. Our group leader, Len, had the wisdom to introduce Jim and I, seeing we might work well together. It was a stroke of genius by Len. Jim and I shared basic ethics and goals in work but also complimented each other almost perfectly. I was creative and free-thinking but lacked attention to detail at times. Jim was more confined in thinking but had meticulous attention to detail. We helped each other with our differing strengths coupled with a common vision. Together we began to sketch out a collaboration that would stretch into the next 20 years. A fast friendship made the work even better.

I had a crisis that left me with a better workplace balance. Gone was my sense of imposter syndrome, replaced with confidence. I was now imbued with the sort of scientific superiority and spirit that made Los Alamos special. Both of us inserted ourselves into the sense of possibility that ASCI gave us. We had freedom and could explore modeling nuclear weapons with computers. Together, we understood that scientific credibility in the simulations relied upon evidence. That evidence was found in verification and validation. Verification is the proof that a simulation is mathematically correct. Validation is evidence that the simulation models something close to objective physical reality.

Jim was blessed with mathematical skill and precision. He also had attention to detail that powered him to a PhD from Caltech. My pedestrian education from the University of New Mexico felt like an anchor. I graduated with a doctorate from Los Alamos. It was far greater and broader than any university could have given me. I had creativity and big ideas with an ability to dream big. Together, we were far better than either of us could be, separately. Jim was also generous and connected well with people. Both of us grew as scientists and our statures grew. We were a great team.

We were a dynamic duo with an eager energy. Ideas would bounce from each other. Throughout our time working together, the friendship grew. We also pushed each other to new heights. We hosted the first TriLab V&V workshop, and Jim’s ideas gave my own extra bite and swag. He came up with the idea of the seven deadly sins slide with the imagery of Hieronymus Bosch to spice it up. We crafted proposals together to work on the most difficult validation problems—images of turbulent chaotic flows central to our mission. Together, we joined the trips to Russia in scientific diplomacy that were part of the hope for lasting peace after the Cold War.

These trips to Russia opened a new level of connection. Jim took the hardest part of the travel and built a level of trust with the Russians. His encouragement brought me along for trips there. I went on seven international trips for this program. Two of these trips were to Vienna for a conference we hosted that included the Russians. One trip was to Ekaterinburg, 12 time zones away in January. Temperatures were as cold as –10°F. The other four trips were to Moscow, and then a train ride to Sarov. Sarov is the place where the Soviet nuclear program was born.

These trips were long and intense. It was the hardest travel I’ve ever done. Jim was a consummate traveler, always ready for every problem. On one particularly difficult trip, we ended up with nicknames. Jim’s was “Candyman”, because of his perpetual supply of homeopathic remedies. He was like a little pharmacy away from home. I remember needing stool softeners halfway into a trip and Jim having them at the ready. My nickname was “Gutterball”, characterizing my own tendency to see the dirty in everything. I could propel any conversation into the gutter in short order.

I remember one of the funniest things Jim ever said. It was 2005 and we were walking past the new NSSB building at LANL. I asked, “When will it be completed?” being completely serious for once.

Jim replied in a completely deadpan way, “When the flaming eye of Sauron is placed on top of it!”

We erupted in gut-wrenching laughter. It also tells you how Jim felt about the new Los Alamos management. This was also a harbinger of disappointments to come.

Right before I left Los Alamos, I was a manager. Jim was one of my employees. Jim was a model employee, being the best in a group full of stars. I can’t think of someone easier to manage. When I left Los Alamos in 2007, Jim followed me to Sandia shortly thereafter. The changes in Los Alamos didn’t sit well with him either.

In retrospect, I think Jim’s movement to Sandia was a twofold break from his past. On the one hand, he was searching for work that felt good. Los Alamos’ decline was stark and heartbreaking. I was providing a naive sunny-side-up view of Sandia. I suspect he never forgave me for that. Jim’s time at Sandia was unhappy. He saw it far clearer than I did. I worry that he blamed me for it and the lack of disclosure of Sandia’s faults and shortcomings. We continued to work together at Sandia, doing some great work. Nothing we did at Sandia could hit the heights Los Alamos gave us.

“To say goodbye is to die a little.” ― Raymond Chandler

With time, Sandia wore out its welcome with Jim. He still lived near Santa Fe with his wife Celine. Celine was French and a nurse. It was clear that Jim’s plan upon retirement was to live in France with its public single-provider health care. France also had a better lifestyle and attitudes than America’s nasty dog-eat-dog culture. Gradually—and then suddenly—Sandia became harder for Jim to integrate his life with. Jim left Sandia and went back to Los Alamos. He and I stayed in touch, but a space had opened. Los Alamos had also declined and was disappointing. The United States felt increasingly foreign too. In August 2016, Jim left the United States without any notice, or information about where he was. I never saw him again.

“How lucky I am to have known somebody and something that saying goodbye to is so damned awful.” ― Evans G. Valens

I will never know the answers. I can just look at the evidence. Jim had lost faith in Los Alamos, Sandia, and the United States. I was seemingly included in his condemnations, or not. It was and remains heartbreaking to me. Jim was as close to a brother as I had at work. We fostered a deep friendship of immense value to me. I won’t ever lose that. I am eternally grateful for knowing him and having him as a friend. I hope Jim felt the same way. I simply don’t know the answers.

It tells me that I need to work on forgiveness and connection. I want to feel the love and gratitude for Jim. I hope others feel the same for me when the time comes.

“Time doesn’t heal all wounds, only distance can lessen the sting of them.” ― Shannon Alder

What is Expertise? How does one get it?

07 Tuesday Jan 2025

Posted by in Uncategorized

≈ Leave a comment

Tags

, , , ,

tl;dr

Most of us would be love to be recognized as an expert at something. One would think it is a way to be professionally successful. The path to expertise runs through skills and experience, but takes a bit more to actually achieve. An expert sees what can’t be taught, and has the ability to move past current knowledge and practice. The expert can solve novel problems and adapt the state of the art. Expertise is earned through hard lessons that include many mistakes and failures. It also needs to be valued and respected to be born. It is an uneven and long journey guided by grit, determination and talent. Today, the expert is also the subject of critique. Expertise is under attack. Thus, expertise today is a dangerous and perilous endeavor.

“An expert is someone who knows some of the worst mistakes that can be made in his subject, and how to avoid them.” ― Werner Heisenberg

The Path of Expertise

“We need to be willing to risk embarrassment, ask silly questions, surround ourselves with people who don’t know what we’re talking about. We need to leave behind the safety of our expertise.” ― Jonah Lehrer

The start of expertise is always the same. You learn the basics and fundamentals of a field. First, the foundational principles are imparted to the burgeoning expert. With the foundation in place, the student turns to a focus in a given area. This follows a similar path with the knowledge be found in textbooks, or the literature. Ultimately, the student needs to begin to start the process of reproducing the state of the art independently. This means known results are recreated and compared with the standard. In this process, the student begins to pick up and demonstrate competence. In that competence gradually confidence is established. At this point the student is still not an expert. The student is a skilled practitioner. Most stop there and go no further.

Along the way important milestones occur that begin to lay the groundwork for expertise. Key among these are beginning to make the same mistakes as the preceding experts. This gets to a feature of the existing literature and knowledge for a field, mistakes and traps are not reported. Success is usually the only thing published. Often a mentorship can be established with an existing expert who provides the growing expert with guidance. Through the mistakes, guidance and lessons learned, the skilled novice inches their way toward expertise. At this point the novice is on the precipice of expertise. There is one more critical step forward to complete.

“This is a fundamental truth about any sort of practice: If you never push yourself beyond your comfort zone, you will never improve.” ― Anders Ericsson

Raw talent and ability is part of the picture, but offers traps that many fall into. There are many very talented students who basically become skilled technicians. The example of the perfect student who’s perfectionism rules their life. The valedictorian from high school is often the epitome of this person. A great student to be sure, but trapped by perfectionism. To become an expert you need the talent and grit, but you have to step into the unknown to risk and experience failure. Often as I’ve seen the perfectionist can’t break from the mold that created the success as a student. They never become an expert.

Being a perfectionist is antithetical to expertise. Given that many gifted and excellent students are perfectionists this might be counter-intuitive. A perfectionism will push a person away from failure and failing is a key part of becoming an expert. Perfectionists often stay within the boundaries of the known, and the boundaries of the known do not contain expertise. Doing what is needed to be an expert requires courage. The perfectionist is skilled, but their excellence in tinged with mediocrity. If you see someone who never fails and always does great work that person is almost certainly not an expert. Expertise is born from pushing hard past limits into the unknown, which invariably leads to mistakes and failure. The perfectionist must cast off their tendencies and the courage to take a leap into the unknown.

“Enthusiasm is more important than innate ability, it turns out, because the single more important element in developing an expertise is your willingness to practice.” ― Gretchen Rubin

Experts Matter?

I’ve always operated under the assumption that expertise is both good and matters. Experts can produce results that mere technicians cannot produce. They solve problems that were unforeseen and unexpected. There is a distinct and substantial difference between competence and expertise.

It has been recognized that experts are treated with distrust and suspicion today. This is a consequence of the unfortunate value system in the current world. There is very little doubt that expertise is under attack from all quarters. There is an entire political movement that is devoted to ignoring expertise. They are in power and operate under the premise that reality can be messaged. We see business interests built on expertise that have shed experts because they are too expensive. Boeing is a prime example. The reality is that Boeing likely a reflection of the danger rather than being an outlier.

If you want results for the long term, experts are essential. In the short term experts are terrible for the balance sheet. This is where politics and business intersect. Current trends are focused intently on the short term. Quarterly results are all that matter. Experts are simply lots of difficult reality that is cheaper to ignore. Until it isn’t. Reality will eventually assert itself. Planes crash or doors fall off. Hurricanes happen and make landfall. Reality will eventually win, and the hedge that experts represent need to be present. Then the experts are be worth every penny spent on them. Today, I wonder, will they be present to step up when needed?

I see this at work. You would think that at a National Laboratory experts would rule. They do not. Experts are a pain in the ass. When reality bites, and it will, the expert will save your ass. In these days it would seem that the message matters and reality is at bay. It is simply a matter of time, reality cannot be denied. That said, we saw experts being repudiated during Covid. More than a million people died and experts were continually beaten down or ignored. One needs to wonder, what sort of disaster would it take for the experts to be valued?

When we look at the consequences of rejection of expertise, Boeing looks like an herald of the future. I remember 20 years ago at Los Alamos taking a meeting with a Boeing engineer. He told us that Boeing eviscerated its work on turbulence getting rid of almost everyone working in the field. Only the expert who “solved” the problem was retained (Spalart) , and no more progress was needed. They had declared the problem to be solved. An absolutely ridiculous notion on the face of it. It turns out that the repudiation of expertise was even broader at Boeing. Then starting with the building of the 787 then the 737 Max, the problems started to manifest in reality. Delays and quality control problems plagued the 787. Then the actual engineering work created flaws that crashed two 737 Max planes by foreign airlines. The problems continued with a door flying off a plane more recently.

All of this seeded by the removal and rejection of expertise by the company. All of this done to improve the bottom line and the short-term financial health of the Company. Reality hit hard and now Boeing is in free fall. A sterling reputation built over decades was destroyed by cheapness and greed. The same motivations and drives are present all across the business world, and replicated at places I work. I see financial factors treated as essential and primal to success. Expertise and technical quality are afterthoughts and simply assumed to be in place.

The result is mismanagement of technical work and a collapse of expertise. The lack of trust present across society results in a fear of failure. This in turn becomes management malpractice. We are graded on how we perform on key milestones. We are basically told that these milestones cannot fail. Thus we create milestones that are too easy and can’t fail. The result being a systematic dumbing down of the most important work we do to avoid the possibility of failure. It is also the highest profile work we do, which ironically is engineered to be mediocre.

This gets to some factors in the creation of experts which are cultural and emotional. The culture of the organization needs to support the expert in several key ways. First the activities needed to develop and maintain expertise must be encouraged and resources be provided. Secondly, the expert needs to be respected and valued. The novice can easily observe whether expertise is encouraged by the management. More importantly they can see whether being an expert matters and their views are respected.

We can ask some key questions about the culture. Is being an expert a path to professional success? Does the organization provide opportunities to experts? Is being an expert a path to being supported with ease? If these questions are answered affirmatively, experts are a natural outcome.

The answer to each of these questions is now in the negative. In the business world (e.g., Boeing) and the Labs we can see this. Its consequences are starting to become obvious.

When I look at my career the answers to these questions provide a guide. When the answers were affirmative, the expertise was built and grew. When the answers were negative, expertise retreated and languished. Experts are not free, nor does their quality and availability come without broader implications. If the evidence is that expertise is not valued, one won’t put effort into being one. Without experts we cannot meet our greatest challenges with solutions that work. In the long run we can expect reality to ultimately expose our short-term strategy as flawed. It will be a failure in the bad sense of it.

Expertise is Dangerous and Expensive

“The death of expertise is not just a rejection of existing knowledge. It is fundamentally a rejection of science and dispassionate rationality, which are the foundations of modern civilization.” ― Thomas M. Nichols

In an environment that prioritizes perfectionism and allows for few or no mistakes, an expert is seen as the problem. The expert sees past the trivial and looks deeper. The perfect rarely survives past the superficial observation. The short-term management solution is to get rid of and ignore the experts. We are seeing how that worked out for Boeing. Reality bites and bites hard. What I suspect is that Boeing is simply the most evident example of a broader war on experts. Reality will show itself and expose the gaps in our strategy.

The Covid Pandemic was another example of how experts are not listened to or respected today. If the expert provides something that is uncomfortable or difficult, the current response is to ignore them. Even worse, the response is to make them a villain. The best example is the vitriol directed toward Anthony Fauci. The same is directed toward experts far and wide in less obvious ways. We simply see managers penalize and punish experts for providing a preview of reality. Usually you get the feedback that you need to work on your messaging. Be more positive and stop being a “negative Nelly.” The only good news message is broad and clear across society at large. The National Labs are no different and its hollowing them out.

“Often a sign of expertise is noticing what doesn’t happen.” ― Malcolm Gladwell

One can see the retreat of verification and validation in this light. If one is focused on perfect success V&V is the enemy. V&V is all about finding the problems with a body of work. If one looks carefully at virtually any work with V&V, problems are found. These problems are a direct assault on perfectionism. Accepting V&V examinations and evidence usually chafes the perfectionist. The simplest way for the perfectionist to survive the examination is reject it, or not do it all. This explains the retreat of V&V and the decline in the quality of the work done.

The recent death Jimmy Carter offers a window into some of the systemic problems underlying the death of expertise. While so laudable as a former President, Carter is derided for his time in office. Front and center in this assessment is the infamous “malaise” speech. It is seen as the end of the success in office as Carter called out the public in ways that ring true then and today. He was replaced in office by Reagan who foreshadowed the feel good form of communication we see replicated today. He was also an actor and public figure who mastered media. This episode also coincides with the demise of expertise as essential to success. It does not seem that these events are independent, but rather part of the same problem.

“These are dangerous times. Never have so many people had so much access to so much knowledge and yet have been so resistant to learning anything” ― Thomas M. Nichols