Things Sharadh Says on Podcasts

I'm going to do two things. I'm going to show skills and I'm going to show projects, right? So when I come to say skills, we have this thing. It's great. Everyone loves the dashboard, but I don't want to be creating this dashboard and paying a bunch of tokens and time every time. So the thing that I think Kai did really well and one reason for its product market fit was I can create a skill that basically takes what I've done this session and packages it up so that it can become a load-bearing, repeatable workflow, right? And that's when the AI goes from here's something I'm just like iterating with on the site, like a chat interface, to here's something I can trust to run my, to sort of like run my business or run my workflows or help me. Help me do that. And a little bit after, you're going to see this kick off this like skill creator skill. It's a skill that the harness has that's going to go in and create, like take all the things it's learned from the session from its interaction with me and package that up nicely into something that I can just like pull up at any time. And we'll talk a little bit more about how we do these skill retrieval. I think that's a really cool part of the system as well. But while that's cooking, right, let's actually look at this other thing I love about how we built Kai, which is this notion of projects, right? So the thing about projects is there's so much stuff happening at Stripe. We have like 2,000 skills, right? Projects are this really nice packaging mechanism where we can draw a boundary around those skills and say these are what most people who are doing this workflow must be using. We have projects that are created for projects, like short-lived things. We have projects created for teams, like the people team has a super secure version of Kai in a different project that's backed by a totally secure backend and stuff. The thing about this is, again, it lets one person or a few people who are DRIs of the space to figure out how to get the agent to perform well for everyone. The really interesting thing I have on projects, there's this thing called settings that, like I said, you can use it, you can use a custom agent to power your project. It doesn't have to be Archai, which is pretty good and general purpose. But let's say you have something really bespoke. You can use all the same features we have, but just backed by a different API in the back end and a different harness, right? So really, again, when we talk about AI at the enterprise, there's going to be heterogeneity. It's going to be a lot of different cases. And building this in layers so we can give maximum leverage and customizability. One of the cool things about projects that can be very concrete for people is the idea of tool policies. Now, I mentioned the people team. Let's say you're a person on the HR team who's dealing with a bunch of sensitive information, right? You really don't want the agent to send up a Go Rogue and put that sensitive data into some public Google document that all Stripes can access. That seems like an accident waiting to happen. We don't like that. But you also don't want to tell them, oh, you can't use any tools because your workloads are too sensitive, right? So what projects let us do is to say, for this workflow, I'm going to set up a tool policy that says, in this case, I said the run Hubble tool so that I don't inadvertently put some confidential information into the demo, right? But we could test this out with some other tool. And it's going to kick off what is a human-in-the-loop workflow. So create a calendar invite for me and Wang tomorrow at 11 a.m. Pacific, right? And I set this up ahead of time just to show what a human-in-the-loop flow would look like. But you can see it extends to any other kind of tool. What this is going to do is going to tell me, hey, should be familiar to most people who've used like Cursor or the other, you know, big products out there. But I want to do this, right? The interesting part, and why this isn't the fun part. The fun part is what I showed before, which is that someone who is the DRI of a space can decide that certain tools are kind of sensitive for the workloads that these people are going to be using. So we need a human in the loop to confirm if that action can be taken by the agent. Agents are really good. They're very creative. So we've got to put some restrictions on them so they don't go rogue, right? And projects help us decide. I'd also don't want this to be happening for every person at Stripe. That would be kind of frictionful. So projects are again drawing the boundary around it.

Yeah, I love this because I think a lot of the existing tools let you maybe configure some of this at the individual level, but then it applies to every session. It's not contexted to what you're working on, and you can't share that permission set across different users. And so what I think is interesting about CHI is the permission and context boundaries are very purpose-built for how your company works on things. And I think, you know, when people are asking themselves either, should I build something myself? And does that make a lot of sense? Or do I need to pluck something off the shelf for my enterprise use case? Again, you need to ask yourself how much appropriate or inappropriate friction will this put in everybody's day-to-day work? Because at the end of the day, what you want to do is make everybody's life easier without causing chaos or trouble. And you want the management requirements to go down really low, right? You don't want everybody to have to think every task, like, do I need to turn on this connector, off this connector, connect to this data? And so I do think one of the benefits right now of teams building their own thing is. They can really think about bespoke agents for bespoke use cases, but kind of hide all that complexity from the end employee, the end teammate, and just let them get to work.

again.

This is what happens if you prepare too well for demos. This

is when you do it live. I mean, we believe you. I think what you're showing here, great. You have kind of a skill creator skill or tool. You have a specific spec that you're using to ensure that it's both written well generally for agents, but also written well very specifically for the Kai harness. And then I love this idea of a draft kind of skill editor that you can test and edit and optimize and manage. It's quite nice. I know people just love fussing around in Markdown and Python files, but just a little quality of life UI here can go a long way.

100%. That is such a great question. So, like I said, luckily at Stripe, we care about our data so much that we've invested a lot into both the data querying layer. We use Trino as our data sort of querying layer and our warehouse in that perspective. We've invested a lot into making that super resilient, right? And those investments have helped agents slam it like crazy and not bring it down, right? We've invested a lot our data, the data platform side of things. We've invested a lot in a catalog of data and tiering of data. So we have access to schema that can quickly tell us, oh, these are the relevant data sets that you might want to find and use and how would you use it, right? But even there are higher-level investments as well. There is a blessed analytics layer where like the really key metrics go in, right? And like there's a tiering system where there's an analytics layer. If you fail that, you go look at all the standard data dashboards that we have and you use the queries from there. And if you fail that, then you use the data catalog and search through for the high quality data sets and figure out how to use it. Agents are incredibly good at figuring this out. However, the key part, and you asked about the ask data skill, the key part is we have some really smart data scientists as well who sort of said, hey, this is probably the right way that most data queries should be handled. And what the skill does, if we dig into the ask data skill itself, what it's going to be saying here is route to direct artifacts first, use the analytics layer first, and if that fails and fall back and fall back and fall back until you like hit, you actually hit the data catalog directly, right? So these investments were made for humans, but have held up really well for agents because terms are the reasoning through it, agents have the same problem. They can answer the question, but they have no idea if it was the right query or the right table. And these investments have paid off in helping guard that.

I want people that are listening to hear a couple things. And, you know, I'm going to make the Stripe team blush. I say this specifically about Stripe a lot, which is, I think one of the reasons why Stripe has been able to benefit so much from AI is prior to AI, there's been a commitment to developer experience, developer platform, data platform, analytics layers, like all these things that made humans really efficient at the company pre-AI are foundational investments that now give you extreme leverage when you throw agents at it. And so, you know, when people ask me, like, Claire, what can I do to ship more product with AI? They think I'm going to say something about product development. And I say, double the size of your DevX team, double the size of your data team. Like work on platform investments, good for humans, good for agents. And that's what will let you run. The other thing you said, and I don't want people to miss, because I love this idea of this like three-layer triage that a data agent can go through. And that's really smart. Like find existing reports, please. Then use the analytics layer to find the right query. And then if like you really have to, fall down to the data catalog and write your own query. The thing that I also heard you say is your data warehouse has to be very resilient to high volume queries because when in doubt, an agent will just brute force it. And so again, this is like infrastructure hardening investment, performance investment, not sexy, not what people are thinking about when you're building these data agents, but actually allow agents to do a really effective job because you don't worry about like, you know, turning over your data warehouse because an agent is hammering it.

I love it. Okay, so we've yapped while Kai ran. Let's show what Kai actually generated using these skills and tools in Sandbox.

Yeah, of course. So here's what you see. You see that, you know, Kai adoption is looking good. And this is something I'm personally super happy about. Like pretty much everyone at Strike uses Kai, like 86 plus percent of the company now. So really AI for everyone, which is how we started out this process. And you see this ramp that's gone from a fairly low number. I think if we had done this a couple of weeks ago, it would have been in the hundreds up to a very high number. So happy to talk more if you're interested, if viewers are interested into how we manage that. But okay, we have a dashboard. Dashboard looks good. It also looks like vaguely stripey. So I need to go back and see how the agent figured out that it needs to make things blurbable. So I got to go figure that out. But it has a bunch of things here. It's an interactive dashboard and it has links to a bunch of things, right? That's fine. This is great. We can already see how this can be useful for like, I now generate a dashboard every meeting I go to because it's so easy and it helps me drive the meeting a lot better. But the real power here starts to come in when you talk about multi-turn conversations, right? So great. We have a dashboard. Awesome. But let's do something more. Let's sort of like get Kara to iterate on this for us, right? So, hey, I love this dashboard, but let's do some more here and use this query, get a breakdown, yada, yada, yada. And it's going to do some interesting things here. So I'm going to kick this off, but I'm going to talk through what I'm doing, right? A, the dashboard isn't like, the artifact isn't like created and like it's not fire and forget, right? We give a chance for people to do deep work by iterating on their artifacts, and that's really powerful. It's better for token efficiency. You don't want to be throwing away your HTML dashboard every turn, but it's also really moving into this idea where the agent and you are collaborating on a task, right? And we have turns that are like super deep, like hundreds of turns over multiple weeks. So the idea here is you have sort of like a somewhat like a pretty smart collaborator who has some artifacts and you can iterate with them on it. I'm going to add this query and I'm going to do some really interesting things. And this is something that I think it's worth getting into. I'm telling it, okay, it's not just pulling the data. It's not about pulling the data and displaying it. I want you to do things with the data. I want you to like munch the data in some way or form so I get what I want. And the reason why I'm touching upon this is a lot of the data sort of things that people want to do end up being last mile data. You think about people's workflows, it's so different. It's so hard to build a dashboard for everyone to do every part of their job. And then you have like a gazillion dashboards and how do you manage them? You can't keep the right dashboards at the right level of quality. Using an AI like Kyle to do this means that you can create like light apps almost like the whole like the lovable style thing where people are creating apps to just hyper optimize for their workflow. And the fact that they have a sandbox that anybody, regardless of whether they're an engineer or not, can get the agent to write code for them and do whatever the heck they want with the data. It's really powerful. And I'm pretty sure that, again, as expected, it's gone in. It's sort of like said, okay, here's the actual data. And I want you to go do some summation somewhere to do the other tab and out at K. So super interesting. And if I open up the updated dashboard, it's the exact same dashboard. And you should now see this really cool little segment below that. So I could keep yapping about this dashboard. I love the fact that our marketing team is like 100% all in. They need

it, right? I don't know a single marketing person that doesn't either want some sort of app built or some sort of dashboard. You have product market fit. That leads me to my next question, which is: how do you roll out? I'm just curious, kind of, you know, inside inside the doors of Stripe, how do you roll something like this out? Is it really organic adoption? How did it get built? How did it get shared to the team? Was this like 20 engineers? Like, how did this come to be?