VR Vivek Ravisankar On Software Engineering Daily

“I think the difference between any AI product that is a slop and a magical experience is in the quality of your evals. That's what it comes down to.”

Software Engineering Daily · Hands-On Engineering Podcasts · November 2026

“I think the difference between any AI product that is a slop and a magical experience is in the quality of your evals. That's what it comes down to.” — Vivek Ravisankar, Software Engineering Daily

Context

Asked how HackerRank keeps its AI interviewer from being biased or running on rote paths, Ravisankar explains why his team built an applied research group focused on evaluation.

Transcript

Software Engineering Daily Around 29:37 into the episode
Vivek Ravisankar

…That's what we have actually built in a collaborative environment for both of them to get a sense of what it is like to work together with the person as well as on a real-world task. So, we call it the AI-led screen and the human-led interview. So, that is the combination for the next-gen process.

Josh Goldberg

…are a lot of common biases or repeat points that don't work well. So, given the very sensitive nature of an interview and the mixed non-deterministic results you get from AI, how do you make sure that this thing is actually interviewing properly and not just kind of going down rote original paths?

Vivek Ravisankar

…people speak to AI, it's usually in a voice form that people are used to. You've spoken to Alexa, you've spoken to Siri, spoken to ChatGPT voice. It doesn't have a video of TAR. So people are comfortable talking to AI and voice. So don't put a video on that. So the second one is like the form factor of how it actually appears. The third one, and by the way, this is all in no particular order. I think the difference between any AI product that is a slop and a magical experience is in the quality of your evals. That's what it comes down to. So we have a real applied AI research team that actually works on evals where we run chakra across many different scenarios, how much it can probe, how much it can pull back, how much it's able to ask follow-up questions so that it is Meaningful and it's really good. And we continuously make sure that we can improve our evals. And the fourth one is it needs to be like a combination of work and…

Josh Goldberg

Are you worried that if everyone uses something like Chakra, there won't be any strong differentiation between how different companies interview? Like, let's say that you build chakra and then Lee Code builds chakra. Like, how do you differentiate the hacker rank version of this versus your…

Vivek Ravisankar

A lot of it comes down, I think. Maybe I'll speak at the core aspect and then we can talk about like the specific features. I think there are two things that underpins success in this field: how strong is your evaluation and how strong is your integrity. So, you need to be able to make sure that…

Josh Goldberg

First of all, I'm excited to hear you. You seem genuinely excited about the product quality. But I do want to hone in on one little point you made there about bias because there's been a lot of research showing that traditional AI models are in fact biased against, let's say, folks who sound or…

Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.

More from Software Engineering Daily