When the model is free, the chip is the moat
As Hugging Face crosses 2M+ models, Nvidia's new hybrid GPU/LPU strategy quietly answers the question every team will face: who pays for all those tokens, and on what hardware?
The number that matters today isn't a benchmark — it's 2 million. That's how many models now sit on Hugging Face as of 2026, alongside 500k+ datasets and a growing Spaces and Inference Endpoints layer. The catalog isn't the story. The story is that running any of those models cheaply is now harder than picking one, and the inference-hardware market is finally starting to reflect it.
The Register reported on May 3, 2026 that inference is giving AI chip startups a second chance, and the technical detail buried in the piece is the more interesting part. Groq's SRAM-heavy LPU architecture could, with enough chips, churn out tokens faster than any GPU — but limited compute capacity and aging silicon meant they couldn't scale efficiently. Nvidia's response, according to that same reporting, is to split the inference pipeline: keep the compute-heavy prefill on GPUs, hand the bandwidth-constrained decode to its own LPU-class silicon. That's the incumbent absorbing an upstart's architectural insight rather than competing with it.
For builders, the practical signal is that "which model" is becoming a separable question from "which substrate." The growing open catalog means a swap from one 8B model to another no longer involves a procurement cycle — it's a config change. But the cost-per-token math depends entirely on whether your inference provider is running pure-GPU, pure-LPU, or the hybrid Nvidia is now pitching. Different substrates produce different latency curves at different batch sizes, and that gap will widen.
For buyers, the implication is that vendor lock-in is migrating one layer down. 2 years ago the lock-in lived at the model API level; today the model is portable but the inference economics aren't. Asking your AI vendor what hardware sits behind the endpoint is no longer an esoteric question.
For creators and product teams, the meta-signal is that the open-source ecosystem and the proprietary hardware ecosystem are now co-evolving in public. When the model is free but the chip strategy is the moat, the chip strategy is what to watch.
The bottom line: Treat your inference substrate as a first-class architectural decision, not a billing detail — the hardware your tokens run on will define your unit economics for the next 12 months.