So you want to predict the future? Be careful not to simply project a graph outward.
I have a few habits that have been very helpful to my own success in life (both personal and professional). One of them is the ritual of continuously reflecting on my past predictions and whether they succeeded or failed. As a result, I’ll rarely make a prediction out loud without being quite confident. Of all my public predictions, I’d wager my success rate is upwards of 80%.
This is more or less impressive depending on how many of these I’ve made and what public opinion was relative to mine, but it’s a useful skill. I’m often asked to make predictions at work.
How long will this take? Should we do A or B? Can this scale?
Even basic life choices require some predictive power. Which career should you choose? Is this job a sinking ship? Is this a health fad? Will this new habit stick?
There is a list of factors that help me decide whether to make a broad prediction or hold back. One of the most common has been: Is this foundational?
If you’ve got a background in philosophy, this is similar to the analytic vs synthetic classification. Are we discussing something that is obviously true because of a definitional property of the system?
The classic example is claiming that a bachelor is unmarried. This is basically a tautology. It is true by definition. My version of this might be claiming that the set of all bachelors will continue to be unmarried into the future.
Nothing is ever this clean, and I’m not a robot free from imperfect statements. My predictions look a little different.
I’ve noticed that “being good at computers” is less about intellect and more about emotional response. If you’re uncomfortable with uncertainty or under deep stress, you’ll probably be worse at navigating a novel interface.
As such, elderly people who are good at computers seem to exhibit very specific personalities. The kind of person who can remain engaged with a constantly changing interface will have other related qualities.
I am the kind of person who likes computers. I love engaging with and tinkering with unfamiliar things. I am nervous about how I might change as a person into my 60s and 70s, but I’ll make a prediction: I will continue to exhibit these qualities into old age, because they are relatively foundational.
This is not a great example, but it’s one of the best I have. Most predictions that rely on foundational properties are so obvious or shared as to not be worth saying. On average, this framing is more useful for spotting predictions that aren’t justified.
Absent a massive revolt, we can assume capitalism and “market forces” will play a role for at least the next 50-odd years. This system rewards a set of behaviors that can range from compliance to ruthlessness, depending on the context.
From this, I’ve made some (I think) pretty accurate predictions on LLMs.
In the very early days of the technology, many people derided the software for its mistakes. It was so easy to break the systems that you’d do it accidentally on every other use. I saw lots of posts on social media talking of these faults as fundamental limitations of the tools.
At the time, I was inclined to believe them. If what we were looking at was autocorrect, then it made sense for it to be limited by the corpus of human speech. It couldn’t really think. If there was no answer within the text it was trained on, it obviously couldn’t come up with an original thought.
We started preparing for these limitations to persist over the long term. I still come back to my article How I Use LLMs to get a sense of my takes on this. My use has shifted significantly since, but the article was a correct take for a while. I don’t make any concrete predictions in it, but you can assume I was preparing for a long term in which the technology didn’t improve so significantly.
The mistake was seeing these problems as foundational.
Once it was obvious that LLMs were improving, was there any reason to believe that prose was one of the things they could not do? It sure seemed like no matter how “smart” these machines got, they could not write well.
Then GPT 6 models improved on this, and then Opus 5.5 improved on it again. I think we will still see improvement. Prose is clearly not a foundational problem.
At the time, it wasn’t targeted. It wasn’t the highest-value goal, and evals for it are not nearly as straightforward as those for verifiable programs.
The same is true of media. The videos, images, and sounds we got out of earlier models were not great, but there was no foundational limitation, so they got better.
You cannot assume active problems are foundational problems. Given enough demand, such problems tend to resolve themselves.
In my area, the next big barrier I hear a lot about is “taste.” Unlike us humans, the models supposedly don’t have taste. This is the big dichotomy, I’m told.
But this is frankly nonsense. Taste is a proxy for science and a tool we use to reach a result despite our lack of understanding. Nothing in this leads me to believe we’re looking at a foundational problem.
That LLMs today might have bad opinions on how to lay out a UI or decide what to build does not dictate their future.
We are looking only at an active problem.