Matt Birchler:

I would bet at least 80% of people who say they read every line of code their AI agents write, don’t.

I’ve been thinking about this too. I blogged years ago in February that I read or skim everything. Still mostly true, but there’s more skimming, and I no longer read unit tests.

@manton the danger and curse of good enough.

@manton yep. proper software engineering is key but then again, it takes a lot more effort prompting chatbots for that than actually doing it yourself. LOL

@manton Interesting. I just started ramping up my unit test review on purpose, because I’m noticing a lot more unit tests being written around our company, and I commonly find them to be brittle and/or testing the wrong thing

@manton This is pretty obviously true of the people who use it to “help” write their prose, and even though writing code is quite different in some respects the very human urge to shortcut remains the same

@sameoldstory Yeah, and one similarity between code and prose is that AI tends to be verbose. Both probably benefit from editing.

@danielma I should probably give tests more attention… I have so many more tests now than I ever did before AI.

@manton the differentiator appears to be personal experiments versus professional services. The former’s goal is short term creation and the latter is long term delivery cadence.

@manton.org Same.

@manton Same.

@manton
Isn’t the old wisdom also that “reading and debugging code is harder than writing code”?

@manton there's your slippery slope right there - and the intentional unshakable confidence expressed by these systems, assures that "users" will pay less and less attention to the details -- with disastrous results, virtually certain

@manton @film_girl Based on what I see from clients, this is correct.

@manton
My honest answer is more like: I skim it, make sure I understand the general ideas (usually putting the comments more in my words along the way), and then test that the results are correct. I don't necessarily feel the need to look up the exact documentation for every function it invoked, though.

@manton I would also bet that most developers peer reviewing human written code also don’t read it all. And even if they do, few are good at reviewing it. And most developers on teams have no idea how 95% of the code in the project works either.

Many AI sceptics seem to live in a perfect dev world that I’ve never seen!

@richard5mith Ha, it’s true! “AI might introduce bugs or write messy code…” Yep, just like humans. 🙂

@richard5mith @manton But what did the good reviewers do?

If you’re not refactoring AI code (even if you’re just making the AI do it) to be DRYer and to keep concerns modular/separated, your AI is a lot better than what Copilot is feeding me.