Asked where AI had burned his team at Chime, Goldstein named intent-versus-behavior mismatch and gave this as the case in point. The detail that matters is that every automated signal passed: tests green, PR clean, screenshot attached. He had said a few turns earlier that Chime's rule is a human must test the change before it ships, and this is why.
…make the human in the loop as easy as possible. So when there's a PR, even if AI generated it and the tests and the review looks good, you are still expected to check out that branch's changes in the dog food app, and that is handled all automatically now with the infrastructure that we've built.
Awesome. Where has AI fallen short or burned you?
There are definitely a few places where AI doesn't really deliver on the full promise. One I may have hinted at already is intent versus behavior mismatch. Just because an agent makes a change, writes some unit tests, they all pass, opens a PR, or even has the ability to take a screenshot and attach to it, the behavior might not match. One example is an agent implemented some feature to copy some text. It wrote beautiful tests. The PR looked great. But when I went to check out the changes on the dog fooding app, when I copied it, it copied the whole markdown. So lots of additional data that I wouldn't want there. There have been numerous other examples where this has come up where a feature doesn't really work. One example is a list of items where only one item is pressable, so on and so forth. So the behavior versus the intent versus behavior mismatch. There are other areas that you may have already that more commonly you may have seen that aren't…
Absolutely. Yes. Sometimes AI loves to write more than you ask it to. I get it.
And then when you tell it to go another way, it tells you you're absolutely right.
We're always right. What advice would you give companies considering React Native?