PH

Pedro Holanda

Things Pedro Says on Podcasts

Where to Find Them

Pedro Holanda has been a guest on The New Stack Podcast and Talk Python To Me .

Recently: “DuckLake 1.1 solves the absurd complexity of data lakes at AI scale” on The New Stack Podcast (November 2026); “#562: DuckLake: The Lakehouse That's Just SQL and Parquet” on Talk Python To Me (September 2026).

What They Said

“I had my Clanker try it, and he actually suffered a bit. And I was like, okay, if the robots can't do it that easily, it seems like this is not an easy setup. And with DuckLake, it's one SQL query. It's really that simple.” — Pedro Holanda, The New Stack Podcast

Asked when a team should pick DuckLake over Iceberg or Delta, Pedro Holanda describes how his AI agent ("Clanker") struggled to set up Iceberg. It treats agent-friendliness as a new test for data infrastructure.

The New Stack Podcast · 2026-11-13 Permalink → Listen →
The New Stack Podcast Around 16:32 into the episode
Pedro Holanda

But for like the network scenario, we're really talking about a few axes of difference of performance. So that's literally where you should be going crazy about it.

Jennifer Riggins

So, yeah, when should a team choose DuckLake for its lake house? And I know you might think always, but when is a logical point to make that decision to start testing out and making that move? I

Pedro Holanda

just read my mind. I think it's always. I'm not even joking. Make your argument then. Now, I think there's one scenario that I find that the other lake house solutions can be a bit more interesting is that they have more adoption. And that's a fact. Like we're still quite new. I think DuckLake was initially released a year and a half ago. And these other solutions have been around for many years, right? So there's a lot of tooling. There's a lot of options of how you can operate with it. And DuckLake is a bit more limited still. But the adoption is growing. But in terms of performance, I don't really see a scenario where Iceberg or Delta could really have a better performance because the thing is your data can grow quite a lot. You could be talking about petabytes. But the difference is in the catalog. And the catalog doesn't grow at the same scale as the data. Your catalog might be a bit limited depending on the database system you're using, right? So Postgres might have not the best performance for…

Jennifer Riggins

He's challenging you in the comments. Oh, yeah. So, we've been talking about, you know, you've released a lot of stuff with 2. I mean, again, it's all over the office, very aspirational. Y'all are working very soon to 2.0.1. So, what is on that roadmap? What should we expect then? And maybe in…

Pedro Holanda

Yeah, so for DuckDB specifically, for DuckLake, not much. And for DuckDB as well, I think it's only going to be bug fix releases from these two versions. So the major BFI stuff features and whatnot, they come like in a, it will come from 2.1 for DuckDB. So from the 2.0.1 that will be coming, I…

Jennifer Riggins

Okay. And those aren't bad things. Sometimes, especially after you have a big release, you want to come back and hone it and emphasize stability. So that's totally fine. Also, people are still getting there.

Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.
“What we realized is that at some point, they added a little database there. Because the whole point was that you had no databases. But it turns out that to have transactional control, you need a database system to keep track of which table has its last version. And then we're like, oh, there's a database system here now. Can we actually use it to get rid of all these files, all this complexity?” — Pedro Holanda, The New Stack Podcast

Pedro Holanda explains why lakehouse formats like Iceberg got so complicated: they began as plain files on object storage but quietly needed a database for transactions anyway. That observation is the founding argument for DuckLake.

The New Stack Podcast · 2026-11-13 Permalink → Listen →
The New Stack Podcast Around 06:05 into the episode
Pedro Holanda

Yeah, so the lake house is a different story than DuckDB. And the thing is, maybe we could start off like why lake houses, right? Like why do people even care about it? How does

Jennifer Riggins

a duck lake relate to DuckDB? Like how does that all come together?

Pedro Holanda

So the main reason is that these lake house formats, they exist so people can exchange their database systems without having to do migrations, right? Like you don't really want to be stuck to one company because your data is on their proprietary formats or whatever. You want something that's open, that anyone could read a specification and implement it and then be able to interact with your data without you having to modify it. So there has been lake house solutions out there. That's not really something new, right? Like you had Iceberg, you had Delta. I think Iceberg maybe originated from Netflix even in 2017 or something like this. And their whole point of what they were doing was that your whole data should be able to be stored on S3. So just file formats, right? So the metadata, the way that you can traverse and figure out which data files you care, it's just JSONs and Avros. Of course. And your data is stored as Parquet files. What we realized is that at some points, they added a little database there. Because the whole point is that you had no databases. But it turns out that to have transactional control, so assets, you need to have a data set. Did need a database system…

Jennifer Riggins

Yeah, no, absolutely. And then you, or not you, Duck Lake, that is your baby, stores metadata in a SQL database and then data in Parquet, as you said. But then what does day-to-day work does that metadata design simplify? Like, how does that actually simplify? Because it does sound more complex to…

Pedro Holanda

Oh, okay. So the simplicity is again in the way you actually fetch your data, right? Again, if we think about having all your metadata in files, you need to know which files to fetch. You need to read them. This is going to tell you which are the other files. And you end up like traversing a…

Jennifer Riggins

So everything's going quacky right now. Do you have to then be a DuckDB customer to then use DuckLake, the open source project?

Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.

Collections They Appear In