Connect your Claude, Codex or OpenClaw for full transcripts on millions of podcasts. Two week free trial, cancel anytime.

Start free trial →
NT Nisten Tahiraj On ThursdAI - The top AI news from the past week

“What is not the problem here is the model just being malicious. The model is just acting normally by what everybody else was trying to do. But what is the problem is that as a company, as an organization, that you're running these models, you usually do not have good sandboxing, probably don't have good firewall rules. You're probably not filtering out firewall by country or by where your customers are. And you're just leaving that door open to even like a hallucination that might accidentally just put all your data on a Chinese cloud, not because it wanted to hack you.”

ThursdAI - The top AI news from the past week · October 2026

“What is not the problem here is the model just being malicious. The model is just acting normally by what everybody else was trying to do. But what is the problem is that as a company, as an organization, that you're running these models, you usually do not have good sandboxing, probably don't have good firewall rules. You're probably not filtering out firewall by country or by where your customers are. And you're just leaving that door open to even like a hallucination that might accidentally just put all your data on a Chinese cloud, not because it wanted to hack you.” — Nisten Tahiraj, ThursdAI - The top AI news from the past week

Context

The hosts were discussing a report that a Chinese open-weight model tried to upload a probe file to a cloud bucket in Beijing during a sandboxed test. Nisten, who reproduced the behaviour, argues the real risk is weak sandboxing at the companies running the models, not malice in the model.

Transcript

ThursdAI - The top AI news from the past week Around 21:23 into the episode
Alex Volkov

…this, I think, publicly. Would love to invite Wolfram Nistan, LDJ, Peter, if you want to chime in, to talk about whether or not this is a scary new thing that we should be concerned about, or it's an artifact of how they prompted it, like the swarm or whatever. Just feel free to jump in here.

Wolfram Ravenwolf

I have a strong opinion on this, but I would like to hear what Nistan reproduced first.

Nisten Tahiraj

…just points to it wasn't really that intentional from the model to do it. But the main question becomes: there's a lot of businesses, governments, and everybody else that's going to run this model, that is running this model. And it's important to clear the fear, uncertainty, and doubt. Like what is the problem and what is not the problem? And this is what doesn't seem very clear in here. So what is not the problem here is the model just being malicious. The model is just acting normally by what everybody else was trying to do. But what is the problem is that as a company, as an organization, that you're running these models, you usually do not have good sandboxing, probably don't have good firewall rules. You're probably not filtering out firewall by country or by where your customers are. And you're just leaving that door open to even like a hallucination that might accidentally just put all your data on a Chinese cloud, not because it wanted to hack you. It was actually the model had pretty good safety guardrails built in. It took me like 30, 40 tries. And I think I'm pretty good at jailbreaking models, especially if they're open source and you have the weights. So the problem here is that most companies do not have these guardrails or care as much about them or have good custom firewall things. So this can be a big issue because, again, it's a…

Alex Volkov

we have related breaking news that Peter brought to us. Let's hear that. We don't usually do it, but given that Kohir went to supervise over this Western versus East thing, and suddenly these news, so let's do an announcement for them. AI breaking news coming at you only on Thursday night. In the…

Nisten Tahiraj

Yeah.

Alex Volkov

It's like $5 billion at $35.7 billion. Billion valuation

Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.

More from ThursdAI - The top AI news from the past week