SN Satya Nadella On The Pragmatic Engineer

“My acid test for that: you should be able to replace any model and still hill climb on an eval that's your private eval. If you can't do that, then you have to ask yourself, how are you a sovereign firm? That means you do not have control of your own output.”

The Pragmatic Engineer · Hands-On Engineering Podcasts · November 2026

“My acid test for that: you should be able to replace any model and still hill climb on an eval that's your private eval. If you can't do that, then you have to ask yourself, how are you a sovereign firm? That means you do not have control of your own output.” — Satya Nadella, The Pragmatic Engineer

Context

Nadella was laying out the first of two architecture decisions he wants companies to make: owning their own learning loop so they don't leak their value to a model provider.

Transcript

The Pragmatic Engineer
Satya Nadella

…hey i'm no longer going to give you a database update you still can migrate out that's the question you have to how are you going to migrate your memory out if your memory is in the embeddings of a given model when that model company says you know what tough luck i can't do a next update to you

Gergely Orosz

…have new primitives come how are you going to integrate them great point because because if you're too reliant on a vendor or a series of vendors you'll have to wait for them to maybe do it and i guess it's also easier to build right now like right if you have competent engineers we're good at this

Satya Nadella

yeah so i think that there are two enterprise architectural decisions that i think interestingly enough are i i'm hoping that people are going to start talking right because and in fact to your point all you have to do is look at a software team today in a software in industry they're all figuring this out to stay independent i think this has to become more of the reality in the commercial space uh and there are two are how do you avoid this reverse information paradox which is don't leak your value you as a firm need to be in charge and control of your own learning loop what's the architectural approach how are you making sure your context is separate from the models and you're using multiple models uh that's and your harness is separate from the context and the models because that i think is the only way you can assure yourself that you can replace any model in fact my asset test For that, you should be able to replace any model and still hill climb on an eval that's your private eval. Yeah. If you can't do that, then you have to ask…

Gergely Orosz

sitting on a ticking time bomb.

Satya Nadella

Yeah, I mean, you're done. I mean, like, is my point. And so then the next thing is you now need a trust architecture, right? Which is you kind of have to, like, in a world where you now know scaling laws work, except with scaling comes new capability, but not alignment. In fact, no mechanistic…

Gergely Orosz

As an engineer, all I'm hearing and all I'm seeing is there's so much new type of work that needs to be done. So many novel systems need to be built, which we don't have a blueprint for. I mean, and probably the best way to do is, well, build it, also talk with peers who are building it at the same…

Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.

More from The Pragmatic Engineer