Flipping the Chip: How AI Will Make Games Gorgeous and GPUs Unrecognizable
What happens when the “graphics” in graphics card is just a thin shell, and nearly the entire chip is an AI engine? A look at the coming shift from rasterization to real-time neural rendering – and why it will change game visuals forever.
Last week, an Israeli AI startup called Decart raised $100 million at a $3.1 billion valuation. Their pitch? Replace Netflix, YouTube, TikTok… and gaming. What caught my attention wasn’t the consumer-tech bravado – it was the underlying technology: real-time video generation. That’s exactly the kind of capability that could turn a certain thought experiment I floated years ago into reality.
Double-Secret Meetings
I first raised this idea at an industry advisory board for a major chip company (who will remain anonymous). Graphics advisory boards are a kind of behind-closed-doors meetings where companies bring in a select group of developers to help chart the next few years of GPU architecture. These aren’t marketing focus groups. They’re working sessions with some of the sharpest engineers in the field, hashing out what the hardware should look like a generation or two from now.
One of the perennial debates in these rooms is how to spend precious die area. Back in my Microsoft days, when I MC’d the DirectX advisory board, we’d have Epic, Valve, Crytek, EA, NVIDIA, AMD, Intel – all in one room – arguing about how much silicon to devote to performance, memory, precision, and so on. More recently, the debate has shifted: how much should you dedicate to AI?
My knee-jerk reaction in that meeting: Do we really need to go through this again? In the past, we’ve had long arguments about carving out fixed-function blocks for specific workloads, only to have them replaced in the next generation by more general-purpose compute units. Dedicated vertex and pixel processing units became unified shaders. Why not just fold AI into the same compute pipeline?
We walked through the details together – the compute characteristics of AI inference are different enough that dedicated hardware offers big performance gains. We’re not a generation away from merging them. So I suggested a more extreme exercise: what if we flipped the whole thing?
Flipping the Chip
Imagine a GPU where the “traditional GPU” is a tiny sliver of the chip – basically a DirectX-7-era processor. Think World of Warcraft circa 2004: fixed-function lighting, simple shaders, low-poly geometry. All that little GPU does is block out the scene: where objects, characters, and effects go. Camera placement, basic motion, nothing fancy.
The rest of the die? All inference. You’d feed that DX7-style frame – along with a series of previous frames for temporal coherence, so motion, lighting, and reflections remain consistent – into neural networks trained and tuned by an artist, an LLM, or both. The runtime AI model would fill in everything: reflections, global illumination, vegetation, hair, weather effects, atmospheric scattering.
The creative workflow would split in two:
- Build phase: human artists, AI artists, or a hybrid use tooling to train, configure, and fine-tune models for the game’s style and scenarios – then “bake” them for inference (think shader baking in the DX9 days, but vastly more flexible).
- Play phase: the GPU spends most of its silicon running those models in real time.
The styles could be swapped at will: photorealistic, Pixar, anime, comic book, cell-shaded. The same MMO could be rendered as a gritty dystopia or a lush fantasy world, with no change to gameplay. The same FPS could instantly shift from ultra-realistic bloodbath to Saturday morning cartoon. Want to play Command & Conquer: Red Alert 2 in a lush jungle? A neon cyberpunk cityscape? Just load the model.
We’re already nibbling at the edges of this idea. AI upscalers, denoisers, and DLSS run in-frame today on current-gen GPUs. This just scales it up — radically. And with companies like Decart proving that real-time, interactive video generation is practical, the hardware will inevitably follow.
The Tragedy of Nanite
And then there’s the tragedy of Nanite. For decades in these advisory board meetings, Tim Sweeney railed against fixed-function hardware, championing fully programmable, general-purpose compute — slower, but vastly more flexible. Intel’s Larrabee was supposed to be that: so flexible you could literally write your own rasterizer and texture sampler in software, even for operations so performance-critical that doing them outside fixed-function hardware felt insane. And then he finally did it. Unreal Engine’s Nanite system delivered near-infinite geometry detail by — among other breakthroughs — implementing rasterization in software. Software! It not only beat fixed-function rasterization in their scenario, it opened up an entirely new world of geometric complexity. The biggest shake-up in game rendering architectures since… maybe Quake.
And the tragedy? Just as Sweeney finally pulled it off, AI is arriving to leapfrog the whole thing. Nanite-style visualization — and much, much more — will soon be achievable without the complexity (or mathematical beauty) of Nanite. The entire rasterization pipeline, fixed or programmable, is about to be replaced by AI.
Next Steps
When I first proposed this architecture in that meeting, it felt like a moonshot — possible in theory, but impractical in the near term. Now, watching both hardware and AI models evolve, it’s looking less like a moonshot and more like the next stop on the track.