On May 28 I opened a talk to a group of portfolio-company CEOs with “First off, happy new model day.” A few weeks earlier I had moved my own work entirely to one lab’s tools, because its newest model had pulled ahead of its main rival’s. That morning the rival shipped a new one. I hadn’t tried it yet, and I said so. Then I spent the next hour giving advice about tools whose ranking had just changed.
Between May and the end of September my preferred model changed at least four times. In mid-September I told a workshop of executives that one lab’s models were “significantly better” than another’s, and added in the same breath that a third lab’s release, expected the following week, “may change the game completely.” Two weeks later I opened a different session by asking how everyone liked the second lab’s latest model, which I described as “changing my life.” One participant said they were “fully in the honeymoon phase” but keeping their eyes open about “what’s it going to be like in 2 weeks from now.”
That advice was accurate when I gave it, and it had a short half-life. In physics, a half-life is the time it takes for half of a radioactive sample to decay. The idea carries over to knowledge reasonably well, because the things an executive knows go bad at very different rates, and the rates are steadier than the things themselves. A ranking of AI models is good for a few weeks. A clear understanding of which customers your business depends on is good for years.
My argument is that the central executive skill for a technology moving this fast is sorting what you know by how quickly it decays, and then setting a review schedule for each class. In my experience most executive attention on AI goes to the fastest-decaying class: who is ahead, what shipped, which technique everyone is passing around. Much of that attention is wasted. The slow-decaying things get looked at rarely, if ever. I mean a written understanding of how the business actually runs, the company’s own data, the judgment of its best people, the organization’s trust in new tools and how quickly it makes decisions. Those are the assets that compound.
Short shelf life
Start with the rankings, since I’ve already confessed. A model ranking answers a real question: which tool will do this job best today? That matters to the person doing the job. It matters much less to the person setting strategy, because by the time a ranking reaches a board deck it has usually changed. The executive version of the question is narrower. Can we switch quickly when the ranking moves? I told that September workshop that “as the executive, as the CEO, as leadership, you will do better if you can switch quickly to the newest and greatest thing.” An IT team’s life is simpler when everyone uses one tool. The company’s is better when nobody is stuck with one.
The instructions companies wrap around models decay almost as fast. Any firm that uses AI seriously builds up a layer of rules: files that tell an agent what to do and what never to do, prompts tuned to one model’s habits, workarounds for things the model used to get wrong. In August I warned my own team about this layer. “If you create a bunch of agentic rules around do this, don’t do that,” I said, “that actually has to be refreshed every model.” A rule written to stop one model from doing something dumb becomes dead weight once the next model stops doing it on its own. Sometimes it gets in the way.
At the May talk, a CEO in the audience pushed back on much of what we were showing. “I don’t want to overinvest,” he said, “when six months from now a lot of this infrastructure may not even exist.” About the scaffolding, he was right, and I’d tell him so today. Matthew answered him with what I still think is the better question: “what are you not going to regret” spending time and money on, knowing that the models will make some of it easier and some of it obsolete. A good share of what we showed that day was scaffolding. Some of it was getting a company’s data into one place and building on top of it, which I don’t expect anyone to regret.
How-to writing goes stale too, which I say with some embarrassment. Our first magazine, printed last winter, included a short piece on why executives should use a particular AI coding tool and how to start. It was accurate when we printed it. It also described an interface, and the interface kept moving. By May I was telling Matthew that articles like it were now “a dime a dozen,” and that “nobody’s gonna read the thing about my agentic workflow.” Matthew has argued since that we should stop publishing how-to content altogether because it goes stale. You won’t find any in this issue.
Vendor feature comparisons live and die by release notes. A matrix of who offers what, built in July, is mostly history by October. Some features also deserve suspicion for reasons beyond their shelf life. When one lab announced new places to keep a company’s working context inside its own product this fall, a colleague’s reaction in our debrief was blunt: “I think the benefit is vendor lock-in for them.”
News has a half-life you can almost measure. In February I said on our podcast, “Six months ago, we were talking about the MIT article that said 95% of these projects are failing… That feels like such a distant memory.” The report behind that article, from an MIT group called NANDA, claimed that only about 5 percent of enterprise AI pilots were producing rapid revenue gains, and it was everywhere in August 2025.11 Sheryl Estrada, “MIT report: 95% of generative AI pilots at companies are failing,” Fortune, August 18, 2025. The study drew on 150 interviews, a 350-person survey and 300 public deployments; Critics quickly faulted its sample, its short window and its definition of failure, which counted any pilot without a measurable effect on profit and loss. It did not hold up for long. Within weeks, critics had taken apart the method, and by February the people who had passed the number around had stopped citing it. A statistic can go viral and be discredited inside a single quarter. Anyone who rewrote a plan around it in August had rewritten it around a number that didn’t survive the fall.
The lag also runs the other way, from the people closest to the tools out to everyone else. In September an AI technologist I coach told me he had started using a new tool after seeing me post about it. He then texted friends who work in AI, though not as engineers, and they had no idea what he was talking about. His line to them was “you’ll see it on LinkedIn in about 3 to 6 months from now.” As a rule of thumb for when something reaches the mainstream business feed, that sounds about right to me.
Put those two lags together and the problem with tracking AI news gets clear. By the time a tactic is all over your feed, a good part of its useful life is already gone. The executive who reads everything is mostly reading obituaries.
Weather and climate
The fast variables still matter. The trick is to treat them the way a farmer treats weather. Nobody plans a season around Tuesday’s forecast, and every farmer plans around the climate. For AI, the climate is a small number of trends that have held for years while the particular model on top changed every few weeks. You can model them, write them into your plans and stop reacting to each release.
The clearest of these trends concerns how long a task AI can complete on its own. In March 2025, researchers at METR, a nonprofit that evaluates AI systems, published a way of measuring it. They timed skilled people on a set of mostly software tasks, then asked how long a task a model could finish half the time. By that measure, the length of task frontier models could handle had been doubling roughly every seven months since 2019.22 Thomas Kwa et al., “Measuring AI Ability to Complete Long Tasks,” arXiv:2503.14499, March 2025. The paper notes the trend “may have accelerated in 2024.” In January 2026, using a larger task set, METR estimated that since 2024 the doubling time had shortened to about three months.33 METR, “Time Horizon 1.1,” January 29, 2026. On the expanded suite, the fit for 2024 onward gives a doubling time of 89 days. By May the most capable systems had “essentially saturated” the benchmark. The ruler wasn’t long enough anymore.44 METR, “Frontier Risk Report,” May 19, 2026. The report notes its suite “can’t reliably measure time horizons above 16 hours.”
The second trend is price. The Stanford AI Index calculated that the cost of querying a model about as capable as the one behind the original ChatGPT fell from $20 per million tokens in November 2022 to $0.07 by October 2024, a drop of more than 280-fold.55 Stanford Institute for Human-Centered Artificial Intelligence, Artificial Intelligence Index Report 2025, April 2025, ch. 1. The benchmark is GPT-3.5-level performance on MMLU; prices are a weighted average of input and output tokens. A token is a fragment of a word; a million of them comes to roughly three-quarters of a million words. Epoch AI, a research group that tracks these prices, found declines ranging from 9 to 900 times per year depending on the task, and cautioned that the fastest drops were recent and might not last.66 Ben Cottier, Ben Snodin, David Owen and Tom Adamczewski, “LLM inference prices have fallen rapidly but unequally across tasks,” Epoch AI, March 12, 2025.
What do you do with trends like these? Write them into your planning assumptions with a date attached. If the price of a given level of capability keeps falling at anything like these rates, a workflow that’s too expensive to automate this quarter may be cheap next quarter, so last quarter’s “no” deserves an expiration date. If the length of task an agent can handle keeps doubling every few months, the question of what to hand to software has to be asked again every quarter, because the honest answer keeps changing. Trends also break. METR’s own ruler ran out. That is the reason to date each assumption, so that someone checks it.
The trend lines have a limit. They describe what the models can do. They say very little about what your company will get out of them. When clients ask me how fast this will change their business, I tell them I expect a slow takeoff. The models can already do far more than most companies get from them, and the difference is made of people, habits and approvals. Somebody still has to get the person in accounting to explain what she actually does.
There’s an old adage for this gap, usually credited to the futurist Roy Amara: we overestimate the effect of a technology in the short run and underestimate it in the long run. The attribution is shaky. The earliest version anyone has found appears in J. C. R. Licklider’s 1965 book Libraries of the Future, where he presents it as “a modern maxim”: “People tend to overestimate what can be done in one year and to underestimate what can be done in five or ten years.”77 J. C. R. Licklider, Libraries of the Future (MIT Press, 1965). The trail from Licklider to Amara is traced in “Quote Origin: People Tend To Overestimate What Can Be Done In One Year…,” Quote Investigator, January 3, 2019. Whoever said it first, the shape holds. It explains why model news feels so loud and changes so little inside a company in any given quarter. Electrification is the classic case, and the history piece in this issue covers the forty years it took to reach factory productivity.
That gives three speeds. The tools change every few weeks. The trends underneath them move slowly enough to model. Slowest of all is the company’s ability to absorb any of it, and that layer is where I’d put most of an executive’s attention.
Some verities
In November 1957, Dwight Eisenhower spoke to a conference of civilian executives who had volunteered to help run the government in a national emergency. He told them a story about the Army’s staff college at Leavenworth before the First World War. The college trained its officers on maps of Alsace-Lorraine and the Champagne region of France. A group of reformers decided it was silly to train on foreign terrain and replaced them with maps of Leavenworth and Gettysburg. About two years later, the Army was fighting in Alsace-Lorraine and the Champagne. Then Eisenhower said this:
I tell this story to illustrate the truth of the statement I heard long ago in the Army: Plans are worthless, but planning is everything. There is a very great distinction because when you are planning for an emergency you must start with this one thing: the very definition of “emergency” is that it is unexpected, therefore it is not going to happen the way you are planning. So, the first thing you do is to take all the plans off the top shelf and throw them out the window and start once more. But if you haven’t been planning you can’t start to work, intelligently at least.
The middle line usually gets quoted alone and credited to him, though he was careful to credit the Army.88 Dwight D. Eisenhower, “Remarks at the National Defense Executive Reserve Conference,” Washington, DC, November 14, 1957, The American Presidency Project, University of California, Santa Barbara. What he said next gets quoted less. He allowed that there were still “some verities, some unchanging truths,” and named one, human nature, which was “exactly what it was, apparently, in the time of Pericles and Alexander.” Everything else, he said, “even terrain, even weather, seems to change.” He was sorting by half-life in 1957.
So what keeps inside a company? The following is my opinion, built from what has held up across a year of client work in which nearly everything else changed.
The first is a written, agreed model of how the business actually runs. Most mid-market companies run on tribal knowledge: the reasons behind approvals, the exceptions to the standard process, the customer who always gets special handling and the person who knows why. Getting that knowledge out of people’s heads and onto paper is most of what our projects actually consist of. In July, talking with an engineer who was considering work like ours, I put it another way. “The easy part of the job is making the software and architecting that well. The hard part is you have to learn how this solar construction company works and why there’s 20 exceptions.” Those twenty exceptions don’t change when a new model ships. If anything, the document that captures them gains value with each release, because each new model makes better use of it.
The second is your own data, kept in forms you control. In September a participant in one of my workshops asked how to avoid getting boxed into one model. I described what keeps happening when I try to move someone from one lab’s tools to another’s. They push back and stall, sometimes for months; I’ve watched it happen a dozen times. “Every single time they actually do it, they say, oh yeah, that took an hour,” I said. “Everything lives in files and folders, so there is no transition.” The model was the fast-decaying part. The files were the slow part, and because they were plain and portable, switching cost an hour.
A technology executive who evaluates companies for investors made the bigger version of this point to us in March. Old technology can be fixed quickly now, he argued, so a company valued down for its tech debt may be a bargain. “Well, the thing you cannot create quickly,” he said, “is amassing 40 years of data.”
The third is the judgment of your best people. It is harder to write down, and other pieces in this issue take it on directly, so I’ll keep this short. The models now produce a passable first draft of almost anything. They don’t supply the sense of which draft is right for this customer in this quarter. The stage worth aiming for is one where people set the taste and the direction and the model produces the draft. Taste and direction belong to people, and they survive every model change. A company that has identified whose judgment it runs on, and has given those people the best tools available, holds an asset that next month’s release makes more valuable.
The fourth is trust, which has an odd decay curve. It builds slowly and can collapse in an afternoon. This spring I told an investor who advises our firm that AI projects “fall flat because they biff the rollout. They deploy something that has some bugs, and you immediately lose the trust of the entire organization. And they say, AI doesn’t work.” I added that “almost all the executives who’ve touched AI have experienced that.” Trust is a slow asset that a fast mistake can wipe out, and so the rollout deserves more care than the choice of model.
The fifth is how fast the company decides. When I look at where our projects lose time, very little of it goes to building. Most of it goes to waiting for someone to look at a prototype and say yes or no. Consultants know this problem better than anyone, because the client’s silence is the one thing their skills can’t fix. Inside a company the same lag shows up as approval chains, legal queues and meetings that only happen monthly. A company that decides quickly gets to try more things per quarter. The habit takes years to build and, once built, outlasts any particular model.
The review
How would a leadership team actually use this? What follows is what I’d suggest, and roughly what we are trying at Runpoint, with the caveat that we are still working it out.
Once a quarter, the executive team spends a few hours on what I’ve started calling a half-life review. It sits apart from the operating review. Nobody presents a dashboard. The job is to go through what the team currently believes about AI and the business, and to sort each belief by how long it deserves to be trusted.
The first part of the meeting is deciding what to stop tracking, and for most executives this comes as a relief. Which model leads this month and which vendor shipped what don’t need to reach the executive team at all. They need to reach one or two people whose job includes trying new tools, with a standing instruction to raise something only when it would change a decision the company has already made. I’ve argued that a company’s strongest AI users should be free to “change month to month on what they’re using as new models come out.” That cadence is right for them and wrong for the people setting direction.
The second part is re-examining the assumptions underneath the plan, written down with dates. As of this October, what does it cost to have a model read and sort a thousand documents? How long a task can an agent handle before a person needs to check its work? Which workflows did we decide last quarter weren’t worth automating, and has the price trend changed the math? What rules and instructions have we written for our agents, and has anyone rewritten them since the last major model release? Each of these has a half-life of roughly a quarter. Some will hold up and some won’t, and the review is where you find out which.
The written form matters more than it sounds. An entry might read: “October 2026. An agent drafts first replies to routine customer questions, and a person approves every one before it goes out. Revisit in January.” In January someone has to say whether that line still holds or needs rewriting. Most companies make assumptions like this all the time. Few write them down, so nobody notices when one expires, and the company keeps paying a person to approve drafts that stopped needing approval months ago, or keeps trusting drafts that never earned it. Our own version is modest. The plan for the operating manual I mention in the letter at the front of this issue is to refresh it each quarter from the transcripts of our offsite.
The third part is naming what the company will hold for years, out loud, so the team stops relitigating it every time a new model ships. For most mid-market companies, that list includes getting their data into one place in formats they control, writing down how the business really runs with its exceptions, making sure the people whose judgment the company depends on have the best tools, and rolling new things out carefully enough that trust survives. None of these depend on which lab is ahead this month. At the review you check whether you’re still doing them, and you leave the question of whether to do them alone.
These clocks map onto the ones we stumbled into internally this year. Our two-week commitments are the fast layer, small and binary and easy to drop. Bets get checked often and killed when they stop making sense. Direction, meaning our eighteen-month goals and the few numbers we track, stays put. A quarterly review of assumptions belongs between those layers, and it is the piece we are still building.
One caution: things move between piles. In May, Matthew argued that the basic building blocks of an AI agent, the standing instructions, the context it reads and the tools it can call, were “all understood and decided now.” If he’s right, the concepts moved from the fast pile to the slow one this year.
The move I watch most nervously goes the other way. I’ve argued that a written model of how your business runs is among the slowest-decaying things a company can own. But much of the work of producing one is extraction, sitting with the person in accounting until she explains what she actually does. If the models get good at that interview, the written model becomes something a competitor can assemble in weeks, and the slow pile gets shorter. I don’t think that has happened yet. It’s on the list for our next review, dated October 2026, a few lines below the name of the model I’m using this month.