🎇 Mark Zuckerberg's vision of 'personal superintelligence'
The Meta CEO makes the case for powerful AI that enhances rather than replaces humans
“Over the last few months we have begun to see glimpses of our AI systems improving themselves. The improvement is slow for now, but undeniable. Developing superintelligence is now in sight. … I am extremely optimistic that superintelligence will help humanity accelerate our pace of progress.” - “Personal Superintelligence” by Mark Zuckerberg
My fellow pro-growth/progress/abundance Up Wingers,
Before a quick analysis of what Meta boss Mark Zuckerberg is offering in his new 616-word essay, let’s spend a moment on what he means by “superintelligence.” He doesn’t give any sort of technical definition, that’s for sure. The following, for comparison, are the criteria for the Metaculus forecasting question, “When will the first weakly general AI system be devised, tested, and publicly announced?”:
For these purposes we will thus define "AI system" as a single unified software system that can satisfy the following criteria, all easily completable by a typical college-educated human.
- Able to reliably pass a Turing test of the type that would win the Loebner Silver Prize.
- Able to score 90% or more on a robust version of the Winograd Schema Challenge, e.g. the "Winogrande" challenge or comparable data set for which human performance is at 90+%
- Be able to score 75th percentile (as compared to the corresponding year's human students; this was a score of 600 in 2016) on all the full mathematics section of a circa-2015-2020 standard SAT exam, using just images of the exam pages.
Be able to learn the classic Atari game "Montezuma's revenge" (based on just visual inputs and standard controls) and explore all 24 rooms based on the equivalent of less than 100 hours of real-time play.
That’s not how Zuck is rolling here. Now as it happens, Alexandr Wang, the founder of Scale AI and now Meta’s new chief AI officer, recently coauthored a paper on the geopolitical implications of advanced AI, “Superintelligence Strategy” that offers this definition of the title term:
Many people use the term AGI in many different ways, which can lead to confusion in discussions of risk and policy. We find it more productive to focus on specific capabilities, since these provide clearer metrics for progress and risk. … Although the term AGI is not very useful, the term superintelligence represents systems that are vastly more capable than humans at virtually all tasks. Such systems would likely emerge through an intelligence recursion.
So it that what Zuckerberg is talking about? Not quite.
He writes, as I note above, that “we have begun to see glimpses of our AI systems improving themselves,” calling this a reason why “developing superintelligence is now in sight.” That nod to self-improvement gestures toward at least a mild form of intelligence recursion — a key ingredient in classic definitions of superintelligence, as seen in the aforementioned “Superintelligence” paper, as well as a lot of science fiction.
He also describes future AI assistants that can help you “achieve your goals, create what you want … experience any adventure … and grow to become the person you aspire to be.” That sounds a lot like general-purpose cognitive capability that potentially exceeds human ability across a broad range of domains.
Finally, Zuckerberg frames the technology as transformative, likening it to the advent of the Agricultural or Industrial Revolutions. So pretty important!
In short, while Zuckerberg avoids the science fictional language of runaway hyperintelligence beyond human comprehension, he clearly envisions systems of sweeping power and near-universal utility. Indeed, it’s how the CEO sees superintelligence operating in the real world that gives the clearest idea of a) what he means by the word and b) what his main message here is.
As to the latter point, Zuckerberg is presenting a blueprint for an Age of AI grounded in individual empowerment. The goal, as he see it, should not be centralized AI that replaces human labor, but AI that enhances human agency by becoming a deeply personalized tool for creativity and exploration — not just economic growth for growth’s sake. This is version of the future saturated with AI-enabled glasses and virtual assistants that help us navigate life.
I think my friend Neil Chilson, head of AI policy at the Abundance Institute, is directly over the target here:
In the Stephenson book that’s a favorite in Silicon Valley, the author describes the interactive, AI-powered book thusly:
In your Primer you have a resource that will make you highly educated, but it will never make you intelligent. That comes from life. Your life up to this point has given you all the experience you need to be intelligent, but you have to think about those experiences. If you don't think about them, you'll be psychologically unwell. If you do think about them, you will become not merely educated but intelligent.
As such, Zuckerberg is seemingly trying to contrast his vision with those of some other AI companies and CEOs — Sam Altman, Dario Admodei, and Elon Musk might be on that list — who foresee a future in which powerful AI systems do all economically valuable work, while humans subsist on the spoils via government-supplied incomes of some sort. Whatever the marketing impact of this vision, I think it’s a far healthier and Up Wing way of thinking about AI and presenting it to a public that has been marinating on dystopian visions — AI kills us or merely renders us useless and purposeless — for decades.
Oh, and the timeline for all this? Well, Zuckerberg is strongly hinting at big changes over the rest of this decade, which syncs with many prediction markets that give good odds of at least some form of AGI emerging by 2030.
Faster, please!
As Congress Does Less, The Courts Are Doing More
Breaking news: The Dispatch has acquired SCOTUSblog, the gold standard in Supreme Court analysis.
Reliable coverage of the Supreme Court has never been more important. With an unbridled executive branch and a Congress eager to surrender its constitutional prerogatives, the federal judiciary is playing an increasingly important role in shaping the country’s direction.
Join 600,000 loyal readers and check out The Dispatch today. No insulting clickbait, no false outrage, no annoying auto-play videos—just reliable journalism that helps you understand the big decisions that will shape our nation’s future.
Faster, Please! readers: Take 25% off a Dispatch membership today
Micro Reads
▶ Economics
Billionaires' Baby Boom Has Lessons for Our Bust - Bberg Opinion
What Does Consulting Do? - SSRN
The Weirdest GDP Report Ever - WSJ Opinion
▶ Business
▶ Policy/Politics
▶ AI/Digital
The Real Demon Inside ChatGPT - Wired
▶ Biotech/Health
▶ Clean Energy/Climate
The $87 Trillion Bill That Comes From Denying Reality - Bberg Opinion
▶ Space/Transportation
▶ Up Wing/Down Wing
You Can’t Quit America - NYT Opinion
What’s Behind the Great American Murder Decline? - Bberg Opinion
Even Nuclear Experts Are at a Loss Right Now - Bberg Opinion
▶ Substacks/Newsletters
How to use AI to Optimize your Personal Life and Free Time - AI Supremacy
Tax the tourists - Slow Boring
Ads are inevitable in AI, and that's okay - Strange Loop Canon
A Note on the Closing of the Path to Future U.S. Hegemony—Economic, Political, Geostrategic, Cultural - Grasping Reality
Who's Going to Succeed With AI? - The Geek Way