Article
A 27-billion-parameter model now runs on an iPhone. Here's why small, efficient AI models, not bigger ones, are becoming the frontier worth watching.
For years, the assumption behind every major AI release was the same: bigger model, bigger data center, better result. Two releases this summer challenge that assumption directly, and neither one is a bigger model.
PrismML announced Bonsai 27B in July, a 27-billion-parameter model compressed down to as little as 3.9 gigabytes, small enough to run natively on an iPhone. At full precision, a model that size would need roughly 54 gigabytes, well beyond what any phone or most laptops can hold. PrismML got there using extreme quantization, compressing each weight in the model down to close to a single bit rather than the usual 16, and open-sourced the result under an Apache 2.0 license.
Thinking Machines Lab took a different approach to the same goal. Its new Inkling-Small model matches the performance of its larger Inkling model, a system four times its size, while using a quarter of the compute. Inkling-Small is a mixture-of-experts model with 276 billion total parameters but only 12 billion active at any given time, and on several reasoning and coding benchmarks it actually outperforms its bigger sibling.
Why smaller is the harder engineering problem
Making a model bigger and more capable has been the industry's default lever for two years. Making a model dramatically smaller while keeping most of its capability is a different, arguably harder problem, and it matters for a very practical reason: it changes where AI can run and who can afford to run it.
A model that needs a data center only works where you have a data center connection, a data plan, and a willingness to send your data somewhere else to process it. A model that runs on the device in someone's pocket works in a hospital exam room with unreliable connectivity, on a factory floor where sending sensitive data offsite isn't an option, or inside a mobile app where every network round-trip costs the user time and the company money.
The catch worth knowing about
Independent analysis of Bonsai 27B found something worth flagging before treating the compression story as a free lunch: the model's headline claim of retaining 90 percent of the original's intelligence is an average that hides real variation. The compressed version holds up well on math but loses more ground on tool-calling and instruction-following, which happen to be two of the capabilities a phone-based assistant needs most. Compression isn't uniform, and the marketing number rarely tells you which capabilities took the hit.
That doesn't undercut the broader trend. It's a reminder that "smaller and almost as good" needs the same scrutiny "bigger and better" always should have gotten: which benchmarks moved, which didn't, and whether the ones that didn't matter for your specific use case.
What this means for choosing AI tools
Most businesses evaluating AI tools default to whichever model tops the general leaderboard. That's the wrong question increasingly often. The better question is whether the task actually needs a frontier-scale model in the cloud, or whether a smaller, open-weight model running locally would do the job with less latency, lower cost, and none of the data residency concerns that come with sending information to a third-party API.
Open-weight releases like Inkling-Small make that evaluation easier to run, since the model, its benchmarks, and its licensing terms are public rather than locked behind an API you have to test blind.