You’ve probably heard how tools like Cursor and GitHub Copilot are changing the game for software engineers — automatically writing code, fixing bugs, running tests, basically being the intern every dev dreams of.
But here’s the twist: a fresh study from nonprofit research group METR just threw a little cold water on that party.
The study brought in 16 experienced open-source developers, gave them real tasks on actual large codebases, and split the work into two camps:
One with AI tools allowed (like Cursor Pro), and
One with zero AI support.
Now here’s where it gets interesting (and a little awkward): The devs expected AI to make them 24% faster. But in reality? It made them 19% slower. 😬
Turns out, using AI wasn’t the productivity cheat code everyone thought it’d be. Devs ended up spending way more time:
Trying to figure out the right prompts
Waiting around for the AI to respond
Double-checking the AI’s output — especially in large, complex codebases (which this test deliberately used)
Oh, and only 56% of the devs had previous experience using Cursor. So while they got trained for it, it wasn’t second nature. And in dev world, unfamiliar tools can slow you way down.
Now, to be fair, METR isn’t saying AI tools are totally useless. They’re just pointing out that right now, the productivity boost isn’t as universal as the hype suggests — especially for experienced devs working on serious, sprawling codebases.
Also, AI is evolving fast. The researchers themselves said results might look very different in just three months from now. And yes — other studies have shown AI tools can speed things up. So this isn’t the final word.
Still, this is a much-needed reminder that not all AI tools are instant magic. Not every dev needs a “vibe coder” in their workflow. And sometimes… old-school hands-on coding still gets it done faster.
So yeah — maybe don’t fire your senior dev just yet because your AI tool swore it could ship that next feature solo.
Or hey — if you’re gonna lean on AI, at least make sure you actually know how to use it well. That alone could make all the difference.
Here’s the full report if you’re ready to dive in.
