“Knowledge has to be improved, challenged, and increased constantly, or it vanishes.” ― Peter Drucker

Back in the day I used to write up my thoughts on conferences I went to in this blog. It was a good practice and encouraged me to sit back and get perspective on what I saw. What I learned. What I felt. The workshop I attended this week was excellent with amazing researchers. Thoughtful and wise people who shared their knowledge and wisdom. I saw a great menu of super talks and I had phenomenal conversations. Some of these were one-on-one sidebars, but also panel discussions that were engaging and thought-provoking. I am left with numerous themes to write about for the foreseeable future. A good week indeed, but it left me with mourning too.

The workshop was called “Multiphysics Algorithms for the Post Moore’s Law Era.” It was organized by Brian O’Shea from Michigan State along with a group of illustrious scientists largely from Los Alamos. It was really well done and a huge breath of fresh air. Los Alamos Air is good for that too. I was there largely because I had an invited talk, which I really enjoyed giving. I had put a great deal of thought into my talk. It was some thoughts needed for this present moment. Invited talks are an honor and a good thing to accept. They look great on the resume or annual assessments. I quickly lost any sense of making the wrong decision and immediately felt grateful to attend.

I won’t and really can’t hit all the high points or talks, but will give a flavor of the meeting.

Moore’s law is the empirical observation about the growth of computing power. For about fifty-some-odd years computer power doubled about every 18 months. Over such a period of 60 years, this gives an advance of over a billion times (2 to the 30th power). Starting around 2010 people started to see the end of the road for the law. Physics itself is getting in the way and parallel computing or those magical GPUs that AMD and Nvidia produce aren’t enough. Plus those GPUs are a giant fucking pain in the ass to program. We now spend a vast amount of money to keep advancing computing, and we are not going to be able to keep up. This era is over and what the fuck are we going to do? The workshop was put together to answer this WTF question.

“Vulnerability is the birthplace of innovation, creativity and change.” ― Brene Brown

I will start by saying Los Alamos carries some significant meaning for me personally. I lived and worked there for almost 18 years. It shaped me as a scientist, if not made me the one I am today. It has (had) a culture of scientific achievement and open inquiry that I fully embrace and treasure. I had not spent time like this on the main town site for years. It was a stunning melange of things unchanged and radical change. I ate at new places, and old places running into old friends with regularity. I was left with mixed feelings and deep emotions at the end. Most of all my view of whether leaving there was the right professional move for me. It was probably a good idea. The Lab I knew and loved is almost gone. It has disappeared into the maw of our dysfunctional nation’s destruction of science. It is a real example of where greatness has gone, and the MAGA folks are not doing jack shit to fix it.

More later about the Lab and its directions since I left. Now for the good part of the week, the Workshop.

“The important thing is not to stop questioning. ― Albert Einstein

The first day of the workshop should have left me a bit cold, but it didn’t. The focus was what is the computing environment of the near future. It was all the stuff the high-performance computing people were doing to forestall the demise of Moore’s law. There are a bunch of ideas and zero of them are really appealing or exciting. The biggest message of the day is a focus on missed opportunities. The decade of focus on exascale computers has meant huge opportunity cost. This would unfold brilliantly as the week went along. The greatest take-home message was the cost of keeping up and the drop off of performance in the aggregate list of the fastest computers. We can’t do this anymore. The other big lesson is that quantum computing is no way out. It is cool and does some great shit, but it is limited. Plus its always attached to a regular computer, so that’s an intrinsic limit.

The second day was much more about software. We have made a bunch of amazing software to support all these leading-edge computers. This software is created on a shoestring budget and maintaining it is an increasing tax. The biggest point is that GPUs suck ass to program. We have largely wasted 10 years programming these motherfucking monstrosities. If we weren’t doing that what could we have done? Plus the GPUs have a limited future. There have been some great ideas for dealing with complexity like Sandia’s Kokkos, but there are dead ends. We are so attached to performance, why can’t we work with computers that are a joy to program? Maybe that would be a path we could all support.

At the end of each day, all the speakers formed a panel and we had a moderated conversation with the audience. The first day they asked Mike Norman to lead the conversation. Mike is a renowned astrophysicist and leader in the history of high-performance computing. It was cool to get to meet him. During the discussions, major perspectives came clearly into focus. An example is the above comment about whether we wasted time on GPUs for 10 years? Yes is the answer. Another issue is the problems and cost of software, which isn’t well-funded or supported. I can report from my job that the maintenance cost of code can quickly swallow all your resources. This grows as the code gets old and we make a lot of legacy codes in science. Another topic of repeated discussion every day of the meeting is the growing obsession with AI. There is a manic zeal for AI on the part of managers, and it puts all our science at serious risk. A bit more later about this.

Finally, at the end of day 2 we started in on algorithms and the science done with computing. Thank god! While appreciate learning all about software and computing, I need some science! I was introduced to tensor trains and I’ll admit to not quite grokking how they worked. It was one of several ideas for extremely compressed computing. A great thing is to leave a workshop with homework. After this, we heard about MFEM from Livermore. Lots of computing results and not nearly enough algorithms (which I know exist). They didn’t talk about results with the code, only how fucking great it runs. That said this talk was almost an exclamation point on what GPU-based computing has destroyed.

Wednesday was my talk. I was sandwiched between two phenomenal astrophysics talks with jaw-dropping results and incredible graphics. I felt honored and challenged. Jim Stone gave the first talk and wow! Cool methods and amazing studies of important astrophysical questions. He uses methods I know well and they produce magic. My physics brain left the talk wishing for more. I could watch a week of talks like that. Even better he teed up some topics my talk would attack head-on. After my talk, Bronson Messer from Oak Ridge talked about supernovae. It was sort of a topic I have an amateur taste for. Incredible physics again like Jim’s talk and gratifying uses of computing. I want more!

I gave my talk in a state where I was both inspired and a bit gobsmacked having to sit between these two masterpieces. I had trimmed my talk down to 30 minutes to allow 15 minutes for questions. Undaunted, I stepped into the task. My talk had three main pieces: a discussion of the power and nature of algorithms, how V&V is the scientific method, and how to use verification to embrace true computational efficiency. I sized the talk almost perfectly. I do wish I would move more during the talk and be more dynamic. I was too chained to my laptop. Also hated the hand mike (would have loved to drop it at the end, but that would be a total dick move).

“The only way of discovering the limits of the possible is to venture a little way past them into the impossible.” ― Arthur C. Clarke

I always believe that a good talk should generate questions. My talk generated a huge reaction and question after question. Some talked about making V&V more efficient and cheaper. I have a new idea about that after answering. No, V&V should not be cheap. It is the scientific method and a truly great human endeavor. It is labor intensive because it is hard and challenging. People don’t do V&V because they are lazy, and want it on the cheap. It is just like thinking and AI. We still need to think when we do math, code, or write. Nothing about AI should take that away. Science is about thinking and we need to think a lot more, not less. Computers, AI, algorithms, and code are all tools, and we need to be skilled and powerful at using them. We need to be encouraged to think more, question, and do the hard things. None of it should be done away with by these new tools. These new tools should augment productivity making us more efficient. They should have free time to really think more.

The big lasting thought from my talk is about the power of algorithms. Algorithms fall into a set of three rough categories worth paying attention to. This taxonomy is structured with the power of these algorithms too. I will write about this more. I have in the past, but now I have new clarity! Thanks workshop! What an amazing fucking gift!

This taxonomy has three parts:

1. Standard efficiency mapping to computers (parallel, vector, memory serving, …). This is the focus of things lately. They are the lowest rung of the ladder.

2. Algorithms that change the scaling of the method in terms of operations. The archetypical example is linear algebra where the scaling originally was the cube of the number of equations like Gaussian elimination. The best is multigrid which scales linearly with the number of equations. The difference in scaling is truly quantum and rivals or beats Moore’s law easily.

3. Next are the algorithms that are the game changers. These algorithms transform a field of science or the world. The archetype of this is the PageRank algorithm that made Google what it is. Google is now a verb. These algorithms are as close to magic as computers do.

The trick is that each of the rungs in the hierarchy of algorithms is harder, more failure-prone, and rare. These days the last two rungs are ignored and only happen with serendipity. We could do so much more if we were intentional about what we pursue. It also requires a taste for risk and tolerance of failure.

“Any sufficiently advanced technology is indistinguishable from magic.”― Arthur C. Clarke

I wanted this to be a brief post. I have failed. The workshop was a wonderful gift to my brain. So this is a core dump and only a partial one. I even had to clip off the last two days of it (shout out to Riley and Daniel for great talks plus the rest, even more homework). Having worked at Los Alamos I have friends and valued colleagues there. To say that conversations left me troubled is an understatement. I am fairly sure that the Los Alamos I knew and loved as a staff member is dead. I’m always struck by how many of my friends are Lab Fellows, and how dismal my recognition is at Sandia. At Los Alamos, I would have been so much more at least technically. That said, I’m not sure my heart could take what was reported to me. The Lab is something else now and has lost its identity as something special.

The Lab was somewhere special and wonderful. It was a place that I owe my scientific identity to. That place no longer exists, You can still make it out in the shadows and echoes of the past. Those are dimming with each passing day. You may recall that last month, Peter Lax died. A friend shared the Lab’s obituary with me. It wasn’t anything horrible or awful, but it was full of outright errors and a lack of attention to detail. Here is one of the greats of the Lab and a member of the few remaining scientists from the Manhattan Project. He was someone whose contributions to science via applied math define what is missing today. The work in applied math that Peter did is missing today. It is what AI and machine learning need. It is absent. Worse yet, the current leaders of the Lab and nation are oblivious. They botched his obituary and I suppose that’s a minor crime compared to the scientific malpractice.

One cool moment happened at Starbucks on Thursday morning. It was a total “only in Los Alamos” moment. I was sitting down enjoying coffee, and a man came up to me. He asked, “Are you Bill Rider?” He was a fan of this blog. I invited him to sit and talk. We had a great conversation although it did little to calm my fears about the Lab. I can’t decide if I should feel disgusted, a nod of submission, or deep sadness. A beacon of science in the USA and the world is flickering out. At the very least this is a tragedy. The tragedy is born of a lack of vision, trust, and stewardship. It’s not like the Lab does anything essential; it’s just nuclear weapons.

“The present changes the past. Looking back you do not find what you left behind.” ― Kiran Desai,

Rather than close on this truly troubling note, I’ll send on a bit of gratitude. First, I would like to give much appreciation to Brian who did much of the operation and management of the workshop. He did an outstanding job. Chris Fryer and CNLS hosted the workshop under its auspices. It was joyful to be back in the CNLS fold once again. I have so many great memories of attending seminars there along with a few that I gave. Chris and his wife Aimee host wonderful parties at their home. They are truly epic and wonderful with a tremendous smorgasbord of culinary delights and even more stimulating conversations with a plethora of brilliant people. Always a delight to visit them and enjoy their generous hospitality.

“Every revolutionary idea seems to evoke three stages of reaction. They may be summed up by the phrases: (1) It’s completely impossible. (2) It’s possible, but it’s not worth doing. (3) I said it was a good idea all along.” ― Arthur C Clarke