Even using Fable (while it was briefly available), having it refine a plan, and directing it to make only small incremental changes, I still found reasons to reject its first pass at a lot of work. There was a lot of “You’re right to push back” responses. A lot of incidents where it would creat some giant complex set of abstractions to accomplish something that I could find ways to do much more elegantly and in a more maintainable manner.
It’s really eye opening to work with these tools on a codebase you know deeply because these problems are everywhere.
However if I opened an unfamiliar project in another language and I wanted to add a little feature with no intention of maintaining it, I’d happily accept the changes and loop until it worked well enough for my temporary needs.
The scary middle is when you’re dealing with coworkers who don’t care about anything other than closing tickets and collecting credit. With enough of a token budget you can now wrap loops around an LLM and have it try things until the program appears to work. Ask it to do a code review and then submit the PR without having understood what it was doing. There are a lot of workplaces where there isn’t a good mechanism to push back on this and the tech debt just keeps growing.
These "You're right to push back" scenarios are scary for me. I mostly code ML implementations, and some of the errors Claude Code (CC - have only used Opus 4.7) makes are very sneaky, and if you don't have sufficient experience in the area (I see this with people entering ML and writing their implementations with CC), you wouldn't know when to question CC and will let errors or future pitfalls silently slip into your code. A recent example was when there was data leakage in a model calibration step, which it refused to see as an error, till I wrote a detailed reason, and then it agreed that there was a "subtle leakage".
The leakage problem is so pervasive. None of the frontier models seem to have any idea how to actually hold out rows. God help you if you decide to change the data mix.
I was working on creating a next-n-actions predictor for one of our use cases and not paying much attention for a PoC. I was fairly happy with the progress for a few days, before actually reading the eval code and seeing that we leaked the final state in every eval.
It's nice to let claude run loose on porting from framework to framework (port my code from TRL to NemoRL to Tinker to VeRL) but looking at what it does in the intermediate steps makes me want to claw my eyes out. And getting it to adhere to our domain model (e.g. we have an SFTConfig and a .to_trl(), or a Row and a .to_harmony()) is impossible.
All Claude models are huge suck ups. The "you're absolutely right" meme is real even if that exact phrase doesn't show up as much anymore.
I don't want to start a fight or anything but IME Codex has a bit more of a spine. If you point out something weird, it sometimes gives a good reason for it. Whereas Claude will always say "whoopsie you're right as always sir" even when it's me who missed something.
I only use free AI chats to help me with my learning, but often I direct its responses neutral and to refrain from providing any encouraging language, or value judgements. It tends to get rid of these 'you're absolutely right' comments when I point out a mistake.
But your comment just made me think whether this tendency for LLMs to resort to flattery when found out is a built in strategy to distract the user from the error prone fragility of much of the output? It's perhaps a stretch to think these canned responses were put in strategically, but the result is that the user's attention may be deflected to contemplating their own superior knowledge and insight, and bask in the glory of all that, but then forgot to appreciate that 'Hey, chatLLM is just making all this stuff up/doesn't know which way is up/or down!'
IME it's Claude that pushes back, and Codex that just does the thing. It's happened once or twice where I've told Claude bluntly and directly "do this" and it responded "no, here's why that's a bad idea..." Maybe it's just my CLAUDE.md.
Not sure if there are sycophancy benchmarks for coding agents
> There are a lot of workplaces where there isn’t a good mechanism to push back on this and the tech debt just keeps growing.
If the "big ball of spaghetti" theory holds, where software companies who can't manage the debt stumble over themselves as they continue to add to the big ball of spaghetti code, I guess we'll see a row of companies declaring "software bankruptcy" or something in some/many months, depending on how well these workspaces learn to care slightly more and get better at pushing back against slop.
What concerns me the most is that improvements in software design are at an end. The “big ball of mud”, which really is a problem of modularity and dependencies, will never improve through innovation because the way it is done now is all there will ever be.
Coding agents have been better than the average "enterprise" programmer for a while now and nobody wants to admit it or talk about it. I have never seen an agent output an implementation called FooImpl that's tens of thousands of LOC in a single file, but I have seen plenty of human code like this.
People call coding agents bad because they don't know the asinine meaningless conventions at their particular company while they themselves write awful abstractions and brittle tightly coupled systems, but hey, at least they know how to write a for loop how their particular company likes.
> I have never seen an agent output an implementation called FooImpl that's tens of thousands of LOC in a single file, but I have seen plenty of human code like this.
And how long does it take a coding agent to output a thousand lines of code versus a human? The worst human at any company was rate limited by themselves. Those 'average enterprise' programmers aren't going away, they're the ones now spending tens of thousands on coding agents and filling your codebase with even more garbage without bothering to review an iota of it.
Which is why one of the big problems for the field right now is that a) most code bases still need someone more skilled than a mere robot driver, and b) many developers are not better than that.
In the past, a team of five mid devs and one good one would be fine, because that good one would ride herd on the mid ones. But now those mid ones are slamming out robot code that they're incapable of meaningfully reviewing (because it's better than they are already), and they're just overwhelming the good dev's capacity.
The solution, of course, is to fire them all -- they're worthless now -- but this is not going to happen quickly, and it's probably for the best that it doesn't.
> With enough of a token budget you can now wrap loops around an LLM and have it try things until the program appears to work. Ask it to do a code review and then submit the PR without having understood what it was doing. There are a lot of workplaces where there isn’t a good mechanism to push back on this and the tech debt just keeps growing.
I'm not making an argument in favor of people using LLMs for this, but people were doing this before we had LLMs it was just usually a bit slower. I can't even say it usually doesn't work out long term because I worked with a lot of guys who did this and took a ton of Adderall while working practically around the clock. Every incentive structure in the organizations rewarded it along with social credibility from more junior engineers. (The last cowboy I worked with who pulled this shit ended up becoming the most senior engineer in the company, a multi-millionaire and worshipped like a god by 90% of the mostly fresh grads we were hiring).
The problem is when invariably these people burn out eventually and leave, they leave a massive vacuum in their stead. Not from load they were carrying but creating.
I think the larger the organization I've been at, the more they reward the people making huge commits on nights and weekends. Worse, they could get away with TBRing their shit and merging it without review.
LLMs are often all of the bad habits and organizational problems that we already carryied just being speedrun. There are some places doing it right, but they already were.
> There are some places doing it right, but they already were.
Could you be more specific what "right" is?
> I can't even say it usually doesn't work out long term because I worked with a lot of guys who did this and took a ton of Adderall while working practically around the clock. Every incentive structure in the organizations rewarded it along with social credibility from more junior engineers. (The last cowboy I worked with who pulled this shit ended up becoming the most senior engineer in the company, a multi-millionaire and worshipped like a god by 90% of the mostly fresh grads we were hiring).
I'm having a tough time believing this, it sounds like you're trying to backwards rationalize more productive engineers were "on drugs" and they delivered but "did it wrong"
In fairness, you could throw the most senior engineer into a brand new codebase, and they would probably make a dozen mistakes if you immediately had them pick up invasive and risky work.
No, that's not "in fairness", that's misunderstanding the entire problem.
Having worked 20 years in this field and managed a few projects, no, I wouldn't make a dozen mistakes, because I would refuse to take on work I can't responsibly do.
Invasive and risky work IS the thing I want to be working on because it's the place where I can be most valuable, but part of my value comes from asking the right people the right questions. If I'm working on something invasive and risky, I'm going to work directly with the people who wrote it, and only when THEY think I understand it well enough am I venturing in alone.
Absent access to the people who wrote the code, I'm going to start by writing tests around the code and spend a lot of time checking my initial assumptions upon reading the code, because I know that I don't know what I don't know.
Yeah, if I did foolishly just started making changes, I'd make mistakes but that's missing the point: a good senior engineer knows not to do that.
That's the failure point of AI: it's arrogant. It will provide you statements without any idea if they're true and make changes without any idea if they're correct. It will never tell you "I don't know how to do that" or even "I am not sure if this is correct". It just does the work with infinite confidence even when that confidence is not justified and often it will be just as hard to figure out if the AI's work is correct as it would be to do the work yourself.
If we rephrased this to "When I reject my coworkers code even if it works" and give the same reasons there would be zero dissent. There is this weird idea that seems to come up with AI that any solution must be good and adequate. Software Engineering is all about rejecting code that works for the right code that works.
Yeah, but I think there's a difference here: If your coworker puts up code that you don't understand quickly, in most environments people give it an approval, as withholding approval is meant to indicate that there's a problem with the code. It's very rare that you'd actually force them to wait to merge until they've explained the code to your satisfaction.
(There are workplaces where that's the norm, I know -- it tends to be a thing with smaller teams with codebases that everyone understands fully, and much less a thing with larger teams where different people have areas of the code they understand more than others.)
With AI code, though, it's _your code_ and you can't give it a lgtm, you actually need to dig at it until you do fully understand it, fully agree with it, and could justify it to a hostile reviewer. It's a different level of rigor.
Not all engineers apply that rigor, though, which becomes a problem.
Coding with AI eventually comes down to two paths, I've realized. One is using AI exclusively for everything. The other is not using it at all. There is almost no middle ground. The reason is that as the complexity and depth of the problem increase, the code AI generates increasingly follows enterprise level patterns. The deeper the meaning of what I input, the more AI tends to produce code that goes beyond my own area of expertise. For example, a human expert's code is very powerful and deep within their own domain, but when you look at the entire codebase, it's often shallow and uneven outside that domain. But the moment you write code with AI, once you go deep in one part, AI tries to standardize the rest accordingly. This means the entire codebase converges toward enterprise level standard code, which essentially reflects the average patterns of senior programmers who built large scale systems.
The problem is this. Human cognitive resources are finite, so we inevitably become shallow outside our own expertise. There is no programmer who can do everything well. And as systems grow in scale, they become more modularized and fragmented, making it impossible to understand the whole system. So what should we do about this? That's always the question.
In the end, do I choose not to use AI, finish the project with uneven code outside my domain, and deliver it? Or do I use AI and deliver a program that is uniform and consistent, but not in my own style? I still don't know. I haven't found the answer yet.
As you know, the boundary ultimately depends on code quality. The problem is that AI generates code that looks high quality even outside my area of expertise, at least from my perspective. So now the boundary has to be redrawn. Refactoring usually ends up redefining those boundaries. At that point, the question becomes: do I rewrite my own code, or do I reject the AI code? Those are the two choices left.
In the end, an exceptionally skilled programmer might be able to keep their core domain intact, but I think the vast majority would find that very difficult. So it might be possible once you cross a certain threshold, but considering the sheer amount of code required to deliver a single modern program, it's hard to know which parts to focus on. However, my perspective might be different because I'm coming from the point of view of delivering a working program, not from the perspective of open source development
Yesterday I started working on an agent harness that tries to address some of the issues here.
What I'm hoping to build ultimately is something that works more like a pair-programming partner than existing harnesses do. I want the user to be an engaged part of the development process all the way through, I don't want the agent disappearing to work on its own. I even want to make it possible for users to swap into the driver role and have the LLM automatically assume the role of navigator when that happens.
There's more info in the readme (actually the readme is all that exists so far, I wanted to get the idea straight in my head first):
Even if nobody else uses it, I hope it will be a useful tool for myself and help me find a way to work with LLMs that doesn't harm my mental models, which is what I feel current harnesses do.
My personal rule of thumb: I am usually okay with agents driving e2e implementations if this won't make life noticeably worse when it does not work. Some analytical code? Perfectly fine. Hobby projects? Fine, though I prefer doing a fun part myself. Refactoring production code generating 10x more revenue than my salary? You'd better be at least understanding what it does.
I use 3 AI's (Claude, GPT and Gemini) to review each other's design plans and implementation on the same code base. Each often catches problems the others miss.
I try to make sure the architecture docs of the code base are refreshed regularly based on recent changes, so it's easier for humans and AI agents to make sense of the code.
I also regularly stop all other developments and just focus on auditing the code base with these AI's to make sure they are secure, robust, clean, and well structured and well tested -- some refactoring would be needed most of the time, and it's well worth it.
With this approach, nowadays I often merge code from AI without completely understanding what it's doing, but seems the code has been working so far. :)
I do sometimes have to steer the discussions between the AI's to the right direction, if they deviate too far away from the real problem, either because they miss some context, or because my original description of the problem was misleading.
To do that formally, I have a mechanism built-in the review loop where if a comment on a github issue or PR is signed as "-- Human Reviewer", then all AI agents have to treat the comment as the highest priority item to address.
I'm always curious when I see these stories. How long have you been doing this, for what sort of work, and was the codebase mature before you began working like this?
Yeah, this one is easy: I have been doing this for half a year. I have a couple of projects worked out this way, all green-field projects, code base grew from 0 to tens of thousand of lines each.
This is the way. I use gh copilot and have opus interrogate me and write the plan, then gpt review the plan and provide feedback; repeat this multiple times until gpt is either satisfied or starts to nitpick on unimportant stuff. Then sanity check the plan myself and have gpt implement it.
Each implementation is also reviewed by me before merging to master. I complete PRs only when I'm satisfied with the implementation, my feedback is addressed, and I fully understand what is going on. Agents are the replacement for typing and productivity multipliers.
I have big picture view of the product, each plan implements only a part of it, scoped to avoid merging unreviwed slop. Probably slower, but result is much better.
Most code they write is obviously fine. Much of the rest isn't obviously fine, but is in fact fine once you've gone through understanding it. But yes, there's some that still benefits from a human eye.
(For as long as that's true, "software developer" is still a job. It's not clear for how long it will be true.)
I mean, the reality is a ton of folks in the industry, myself included, are writing glorified CRUD apps in their day jobs. We're building into existing an codebase with established infrastructure and ways of working. What we're building isn't inherently complex or very interesting.
Meanwhile, those codebases often require a ton of boilerplate and drudgery to get anything done.
In these spaces it's very easy to read and comprehend AI generated output and review it fairly quickly. So the time savings from dealing with all that boilerplate and conforming with all that existing infrastructure are potentially substantial.
Depends on what it’s writing. There are times an LLM saves me a lot of time researching library functionality. Especially with testing frameworks. So many strange and arcane features out there beyond the basics, but not hard to understand what they do once you see the code. On that topic I should say I am careful when reviewing the actual test cases.
However if you’re highly familiar with a domain then LLMs are much less useful.
I feel the same way, reading AI built feature entire output makes me cognitively overloaded as well - I can only do so many throughout the day.
What I found myself doing is operating in two modes:
1. For projects that require my attention, I plan and instruct LLM, when needed will draft some code and ask agent to make it better or finish the mundane part (write code and leave gaps with comments asking agent to finish)
2. Full automode where I use spec driven development and TDD - I only ask for changes based on existing PRD, which agent also have to update. Here I do not look at the code at all.
If you reject AI code that works then your mindset is still too hands on. Put another way - you still have some loops to work on taking yourself out of. The agent should’ve delivered code that was acceptable as a first pass.
Agents respond really well to feedback! They have no ego and they’ll happily improve code if told where and how. But you need to provide the tools that provide that feedback without your involvement - otherwise you can’t scale.
All the linting and autoformatting you can put in, is a good start. Next, create custom scripts that check for every single dumb AI-ism you can think of, tell the agent about them, tell it to use them to check its work, and put them in hooks so the harness refuses to let the agent stop until all your linters show no errors.
Then, keep iterating basically forever. Any dumb AI-ism you see, make a linter for it, give it to the agent, and enforce it using the harness.
I’ve spent months doing this. When I review a PR - which was built by the agent with TDD so it definitely works - I’m no longer asking if it did dumb stuff or confirming it conformed to the architecture or duplicated code or missed opportunities for reuse. That’s all linted for. I don’t worry about duplication or outdated docstrings/comments because the self review caught all that. I mostly read it to look for opportunities to make the feature even better & more useful.
If this makes no sense or you disagree it’s possible, my contact details are on my profile and I’ll be happy to give a demo.
I am very curious what some of your lint rules look like in practice. In my mind a lot of the AI-isms in my code that I hate are stylistic or a matter of taste, not necessarily something I could write a deterministic rule to check. But I want to hear more. Like, what kind of linters did you create and which were highest impact?
The script can exit 2 to block the agent, and whatever it prints to stderr is shown to the agent. That’s a pretty darn flexible way to enforce whatever you like.
Despite this being in the codebase I still have no idea what python’s ast stuff is or does - I just let the agent rip, ensured it did TDD and reviewed it all to make sure the tests & code looked reasonable. I didn’t write this code and don’t want to. But I’ve watched it catch hundreds of dumb AI-isms, and watched the agent go “okay” and fix them ;) it’s been paying for itself over and over for months :)
The problem I have with this kind of approach is 1) it emphasizes scaling up a much as possible, which I don't believe is necessarily the most valuable thing, and 2) I really don't want my job to be band aiding agent problems, because it's like herding cats and there will never be an end to it. I'd rather just...get hands on and be involved in the code I am working to create.
Kinda fascinating watching a fairly reasonable response get downvoted. The AI psychosis really is catching...
Incidentally I also don't understand the drive to scale up. Show me a successful tech company and I'll show you a company that won, not by delivering code the fastest, but by delivering the right product with the right features at the right time.
Hell, Anthropic itself is the perfect example: they're doing well because unlike their competitors they realized the real revenues come from enterprise not consumer. They're winning by identifying the right market and giving them the right product.
Frankly, if that's truly your flow, then you cannot possibly know if the code really does what you expect it to do.
"TDD" isn't some magic trick. The tests codify the expected behavior. But if you don't review them for correctness, if you let the LLM build them blindly, then you have no idea what those tests assert and can make no claims about whether the code then does what you expect.
That's fine. That's your choice.
But you have to acknowledge you've chosen to accept that you personally cannot vouch for the quality or correctness of that code.
I fully expect this to be the direction the industry goes, where increasingly complex systems exist that no human actually understands or can reason about.
I think it's bad for the industry. Very bad.
But I'm not making those decisions, so... it is what it is, I guess.
I design everything with plan mode and review every line. Nothing happens to my codebase that I don’t decide should happen. With my way of working, tech debt doesn’t exist because I never have to create it.
You’ve made a bunch of assumptions you’re not conscious of. And now you’re blaming me for that.
Open your mind, you never know what you might (un)learn.
So then your response has nothing to do with the post.
The thesis of the post is (paraphrasing): "if an AI wrote it, and I don't immediately grok it or if the code quality is low, I throw it away, even if on the surface it seems to work, because simply 'working' isn't enough to say a piece of code is acceptable."
I'd add as a corollary "and therefore I would never want to be accountable for that code."
If you're reviewing every line then it sounds like you have no argument with the writer and I don't understand what your point is.
Your very first paragraph says:
> If you reject AI code that works then your mindset is still too hands on. Put another way - you still have some loops to work on taking yourself out of.
But if you do indeed "review every line" then you seem pretty damn in the loop yourself and I don't understand what you think taking oneself out of the loop is.
I think a particular failing with developers embracing AI is fighting the sunk cost fallacy. While you might not have spent as much time putting together a non-working solution, you still did spend time working with the agent to slap together a non-working solution.
Being able to step back and say "this was a failure and we need to discard the day's work and start over" is still hard with LLMs.
Completely disagree. I think this is one of the big wins of agentic engineering. When you look back at your own completed change and realize that you made it too complicated because your initial abstraction was wrong, you have to debate long and hard about whether it's worth going back and redoing the work -- is the abstraction actually that bad? Would you really get a huge win by changing it, enough to justify spending another day on the task?
But with the agent, you know that the change will be relatively quick and easy, so the bar to tell it to shift approaches is much, much lower.
"The reality is that code that runs and makes the CI green can still be a bad solution, and engineering has always been about implementing adequate, scalable, and extensible solutions."
It really, REALLY depends what you're working on. If you're throwing together an internal tool or simple dashboard, it doesn't really matter what the code looks like. But if you're writing software that other programs will depend on, bad design choices ripple out and affect another generation of software. Imagine slop in the linux kernel, in google chrome, or in your compiler or runtime. Its not acceptable.
I know a lot of people spend their careers writing end user software and web UIs. AI is increasingly a good choice for this sort of code. But that's not all of us. And its not all of the software being written.
Disagree, adequate means adequate. Done and cheap is what you call it when a solution is adequate. If the solution isn't adequate, it doesn't matter if it's cheap, because it isn't done.
I was just watching a video about system engineering and the following stucks:
Stakeholder needs: What people wants to get done with the product
Management needs: How to manage the spending of resources (time, money,…) to create the product
Engineering needs: What is the product
You have to balance the three. Sometimes it’s simple and easy to get right. Sometimes it’s complex enough, you’re never truly sure until the product is out in the wild.
Software is malleable and we can do easily do iterations which is not possible with hardware. But today, we have a skew towards engineering, where the whole focus is to create a solution, whatever that is. No understanding of the problem, no proper allocation of resources, just do something. Even if it is plastering over the crack for the eleventh time.
The stakeholders just want to send emails and excels around,
someone in management has a budget for a productivity enhancing tool to replace that and
the engineers have a half-baked solution that
some sales guys are saying is the second coming.
LLMs diverge, not converge. They slightly increase entropy if not controlled. While you can have DRY skills and use AI to organize AI (in loops(tm) like Boris does) but eventually if you don’t understand the code, you are taking yourself out of the loop. And not just the job security that’s on the line, it’s the increasing cost for AI to babysit AI. If you or your “loops” (or paperclip, Hermes, gastown, or next in class agents of agents that runs your entire company) let it gradually sneak in slop-debt, the cost to fix it later will become prohibitive. (You can always just rewrite it, but as the race for “feature complete” and “zero backlog” continues, rewriting an ever growing set of new daily table stakes will become an economical moat)
TLDR: Keeping your codebase human readable and reason-about-able is not just helping humans to stay relevant. It will save costs for LLMs to maintain it.
> Before coding agents, when given a task, I would explore the codebase, think of different solutions, experiment, and only then implement. That could take days of consolidating all that context. When I finally submitted that PR, confidence was higher, and explaining each of my changes to my coworkers was easier.
Now we are getting to the point where we are speed-running the deskilling of engineers into comprehension debt and they themselves rapidly losing confidence in reviewing code they did not write.
I think this blog post [0] is the best example of what could go entirely wrong and even worse when you do not know the technology.
If you cannot explain a change even when "the CI is green" or "all tests passing", I will immediately reject it.
Maybe great for vibe coding prototypes, but it all changes when that code is deployed onto mission critical systems. Just ask Amazon with Kiro. [1]
According to the author's intention, it is the code that he cannot understand or control. Even if the solution provided by the AI works, he will not adopt it. This is unless he can understand or control it. This should be an assumption.
However, if AI provides a solution, as the person using AI, one should conduct research before making a decision. This is not in conflict with or hindered by the use of the ideas provided by AI.
I will say--as someone who has fielded late night troubleshooting calls--I totally understand OP's point of view. It's reasonable to expect that you will be able to answer questions about something that you ship, or brainstorm ways to solve a problem a customer is encountering while using something you provided them.
The obvious counterargument is "well, just ask the AI for those answers," but the AI lacks the context and experience that you have. Sometimes, genuinely, the user really is just "holding it wrong," but none of the current AI models would ever admit that, and you'd spend hours trying to solve an unsolvable problem.
I think this policy is probably more prescriptive than I would go with myself. I like to think of my risk tolerance first to help make that determination.
For example, I use a vibecoded internal tool written in Go. I don’t even know how to write Go. Haven’t read a single line of the code. I just wanted to move from bash scripts to using cloud SDKs for performance reasons.
But the internal tool is a convenience tool, and you can do everything it does using alternative methods. So if it break, there is no real negative impact besides personal convenience of anyone using it. There’s some documentation on how to do everything manually if needed.
Here’s another example: you’re making a static website. No JavaScript, no interactivity. Truly, what could go wrong? And while I do understand HTML a lot better than Go, it wouldn’t really matter if I didn’t.
> Here’s another example: you’re making a static website. No JavaScript, no interactivity.
Linking a huge file consuming clients’s bandwith for no reason. Embedding PII in the html source? And if setting up your own server, misconfiguring it?…
If I'm on call solving a problem another engineer caused and I reach out to them for clarification and they say 'I don't know, the AI wrote it' I am going to advocate for them being fired tomorrow.
It’s really eye opening to work with these tools on a codebase you know deeply because these problems are everywhere.
However if I opened an unfamiliar project in another language and I wanted to add a little feature with no intention of maintaining it, I’d happily accept the changes and loop until it worked well enough for my temporary needs.
The scary middle is when you’re dealing with coworkers who don’t care about anything other than closing tickets and collecting credit. With enough of a token budget you can now wrap loops around an LLM and have it try things until the program appears to work. Ask it to do a code review and then submit the PR without having understood what it was doing. There are a lot of workplaces where there isn’t a good mechanism to push back on this and the tech debt just keeps growing.
It sounds like you've not conditioned your Claude to stop being a sycophant yet?
I was working on creating a next-n-actions predictor for one of our use cases and not paying much attention for a PoC. I was fairly happy with the progress for a few days, before actually reading the eval code and seeing that we leaked the final state in every eval.
It's nice to let claude run loose on porting from framework to framework (port my code from TRL to NemoRL to Tinker to VeRL) but looking at what it does in the intermediate steps makes me want to claw my eyes out. And getting it to adhere to our domain model (e.g. we have an SFTConfig and a .to_trl(), or a Row and a .to_harmony()) is impossible.
I don't want to start a fight or anything but IME Codex has a bit more of a spine. If you point out something weird, it sometimes gives a good reason for it. Whereas Claude will always say "whoopsie you're right as always sir" even when it's me who missed something.
But your comment just made me think whether this tendency for LLMs to resort to flattery when found out is a built in strategy to distract the user from the error prone fragility of much of the output? It's perhaps a stretch to think these canned responses were put in strategically, but the result is that the user's attention may be deflected to contemplating their own superior knowledge and insight, and bask in the glory of all that, but then forgot to appreciate that 'Hey, chatLLM is just making all this stuff up/doesn't know which way is up/or down!'
Not sure if there are sycophancy benchmarks for coding agents
If the "big ball of spaghetti" theory holds, where software companies who can't manage the debt stumble over themselves as they continue to add to the big ball of spaghetti code, I guess we'll see a row of companies declaring "software bankruptcy" or something in some/many months, depending on how well these workspaces learn to care slightly more and get better at pushing back against slop.
People call coding agents bad because they don't know the asinine meaningless conventions at their particular company while they themselves write awful abstractions and brittle tightly coupled systems, but hey, at least they know how to write a for loop how their particular company likes.
An average enterprise developer would never add bloat like that up-front, unless if the ability to change the order was a requirement.
Obviously a stable order can be easily derived from the ID or a creation time (if available).
Setting a position however requires extra steps to ensure the integrity of the sequence.
I see things like that all the time, and it's always stuff that grows the code base and adds unnecessary complexity.
And how long does it take a coding agent to output a thousand lines of code versus a human? The worst human at any company was rate limited by themselves. Those 'average enterprise' programmers aren't going away, they're the ones now spending tens of thousands on coding agents and filling your codebase with even more garbage without bothering to review an iota of it.
In the past, a team of five mid devs and one good one would be fine, because that good one would ride herd on the mid ones. But now those mid ones are slamming out robot code that they're incapable of meaningfully reviewing (because it's better than they are already), and they're just overwhelming the good dev's capacity.
The solution, of course, is to fire them all -- they're worthless now -- but this is not going to happen quickly, and it's probably for the best that it doesn't.
Why is this worse than splitting it across 1k files?
I'm not making an argument in favor of people using LLMs for this, but people were doing this before we had LLMs it was just usually a bit slower. I can't even say it usually doesn't work out long term because I worked with a lot of guys who did this and took a ton of Adderall while working practically around the clock. Every incentive structure in the organizations rewarded it along with social credibility from more junior engineers. (The last cowboy I worked with who pulled this shit ended up becoming the most senior engineer in the company, a multi-millionaire and worshipped like a god by 90% of the mostly fresh grads we were hiring).
The problem is when invariably these people burn out eventually and leave, they leave a massive vacuum in their stead. Not from load they were carrying but creating.
I think the larger the organization I've been at, the more they reward the people making huge commits on nights and weekends. Worse, they could get away with TBRing their shit and merging it without review.
LLMs are often all of the bad habits and organizational problems that we already carryied just being speedrun. There are some places doing it right, but they already were.
Could you be more specific what "right" is?
> I can't even say it usually doesn't work out long term because I worked with a lot of guys who did this and took a ton of Adderall while working practically around the clock. Every incentive structure in the organizations rewarded it along with social credibility from more junior engineers. (The last cowboy I worked with who pulled this shit ended up becoming the most senior engineer in the company, a multi-millionaire and worshipped like a god by 90% of the mostly fresh grads we were hiring).
I'm having a tough time believing this, it sounds like you're trying to backwards rationalize more productive engineers were "on drugs" and they delivered but "did it wrong"
Having worked 20 years in this field and managed a few projects, no, I wouldn't make a dozen mistakes, because I would refuse to take on work I can't responsibly do.
Invasive and risky work IS the thing I want to be working on because it's the place where I can be most valuable, but part of my value comes from asking the right people the right questions. If I'm working on something invasive and risky, I'm going to work directly with the people who wrote it, and only when THEY think I understand it well enough am I venturing in alone.
Absent access to the people who wrote the code, I'm going to start by writing tests around the code and spend a lot of time checking my initial assumptions upon reading the code, because I know that I don't know what I don't know.
Yeah, if I did foolishly just started making changes, I'd make mistakes but that's missing the point: a good senior engineer knows not to do that.
That's the failure point of AI: it's arrogant. It will provide you statements without any idea if they're true and make changes without any idea if they're correct. It will never tell you "I don't know how to do that" or even "I am not sure if this is correct". It just does the work with infinite confidence even when that confidence is not justified and often it will be just as hard to figure out if the AI's work is correct as it would be to do the work yourself.
I agree with your take, but AI is exactly as arrogant as the human driving it.
(There are workplaces where that's the norm, I know -- it tends to be a thing with smaller teams with codebases that everyone understands fully, and much less a thing with larger teams where different people have areas of the code they understand more than others.)
With AI code, though, it's _your code_ and you can't give it a lgtm, you actually need to dig at it until you do fully understand it, fully agree with it, and could justify it to a hostile reviewer. It's a different level of rigor.
Not all engineers apply that rigor, though, which becomes a problem.
If it’s not good it’s not good.
The problem is this. Human cognitive resources are finite, so we inevitably become shallow outside our own expertise. There is no programmer who can do everything well. And as systems grow in scale, they become more modularized and fragmented, making it impossible to understand the whole system. So what should we do about this? That's always the question.
In the end, do I choose not to use AI, finish the project with uneven code outside my domain, and deliver it? Or do I use AI and deliver a program that is uniform and consistent, but not in my own style? I still don't know. I haven't found the answer yet.
In the end, an exceptionally skilled programmer might be able to keep their core domain intact, but I think the vast majority would find that very difficult. So it might be possible once you cross a certain threshold, but considering the sheer amount of code required to deliver a single modern program, it's hard to know which parts to focus on. However, my perspective might be different because I'm coming from the point of view of delivering a working program, not from the perspective of open source development
What I'm hoping to build ultimately is something that works more like a pair-programming partner than existing harnesses do. I want the user to be an engaged part of the development process all the way through, I don't want the agent disappearing to work on its own. I even want to make it possible for users to swap into the driver role and have the LLM automatically assume the role of navigator when that happens.
There's more info in the readme (actually the readme is all that exists so far, I wanted to get the idea straight in my head first):
https://gitlab.com/philbooth/opair
Even if nobody else uses it, I hope it will be a useful tool for myself and help me find a way to work with LLMs that doesn't harm my mental models, which is what I feel current harnesses do.
Good ol' software architecture tricks can also help you slot "vibe coded" components into a larger system safely.
I try to make sure the architecture docs of the code base are refreshed regularly based on recent changes, so it's easier for humans and AI agents to make sense of the code.
I also regularly stop all other developments and just focus on auditing the code base with these AI's to make sure they are secure, robust, clean, and well structured and well tested -- some refactoring would be needed most of the time, and it's well worth it.
With this approach, nowadays I often merge code from AI without completely understanding what it's doing, but seems the code has been working so far. :)
I do sometimes have to steer the discussions between the AI's to the right direction, if they deviate too far away from the real problem, either because they miss some context, or because my original description of the problem was misleading.
To do that formally, I have a mechanism built-in the review loop where if a comment on a github issue or PR is signed as "-- Human Reviewer", then all AI agents have to treat the comment as the highest priority item to address.
Each implementation is also reviewed by me before merging to master. I complete PRs only when I'm satisfied with the implementation, my feedback is addressed, and I fully understand what is going on. Agents are the replacement for typing and productivity multipliers.
I have big picture view of the product, each plan implements only a part of it, scoped to avoid merging unreviwed slop. Probably slower, but result is much better.
(For as long as that's true, "software developer" is still a job. It's not clear for how long it will be true.)
Meanwhile, those codebases often require a ton of boilerplate and drudgery to get anything done.
In these spaces it's very easy to read and comprehend AI generated output and review it fairly quickly. So the time savings from dealing with all that boilerplate and conforming with all that existing infrastructure are potentially substantial.
However if you’re highly familiar with a domain then LLMs are much less useful.
What I found myself doing is operating in two modes: 1. For projects that require my attention, I plan and instruct LLM, when needed will draft some code and ask agent to make it better or finish the mundane part (write code and leave gaps with comments asking agent to finish) 2. Full automode where I use spec driven development and TDD - I only ask for changes based on existing PRD, which agent also have to update. Here I do not look at the code at all.
Seems to be working just fine.
Agents respond really well to feedback! They have no ego and they’ll happily improve code if told where and how. But you need to provide the tools that provide that feedback without your involvement - otherwise you can’t scale.
All the linting and autoformatting you can put in, is a good start. Next, create custom scripts that check for every single dumb AI-ism you can think of, tell the agent about them, tell it to use them to check its work, and put them in hooks so the harness refuses to let the agent stop until all your linters show no errors.
Then, keep iterating basically forever. Any dumb AI-ism you see, make a linter for it, give it to the agent, and enforce it using the harness.
I’ve spent months doing this. When I review a PR - which was built by the agent with TDD so it definitely works - I’m no longer asking if it did dumb stuff or confirming it conformed to the architecture or duplicated code or missed opportunities for reuse. That’s all linted for. I don’t worry about duplication or outdated docstrings/comments because the self review caught all that. I mostly read it to look for opportunities to make the feature even better & more useful.
If this makes no sense or you disagree it’s possible, my contact details are on my profile and I’ll be happy to give a demo.
Then have a look at https://github.com/cadamsdotcom/CodeLeash/blob/main/scripts/... (which was test-driven alongside https://github.com/cadamsdotcom/CodeLeash/blob/main/tests/un...)
The script can exit 2 to block the agent, and whatever it prints to stderr is shown to the agent. That’s a pretty darn flexible way to enforce whatever you like.
Despite this being in the codebase I still have no idea what python’s ast stuff is or does - I just let the agent rip, ensured it did TDD and reviewed it all to make sure the tests & code looked reasonable. I didn’t write this code and don’t want to. But I’ve watched it catch hundreds of dumb AI-isms, and watched the agent go “okay” and fix them ;) it’s been paying for itself over and over for months :)
Incidentally I also don't understand the drive to scale up. Show me a successful tech company and I'll show you a company that won, not by delivering code the fastest, but by delivering the right product with the right features at the right time.
Hell, Anthropic itself is the perfect example: they're doing well because unlike their competitors they realized the real revenues come from enterprise not consumer. They're winning by identifying the right market and giving them the right product.
"TDD" isn't some magic trick. The tests codify the expected behavior. But if you don't review them for correctness, if you let the LLM build them blindly, then you have no idea what those tests assert and can make no claims about whether the code then does what you expect.
That's fine. That's your choice.
But you have to acknowledge you've chosen to accept that you personally cannot vouch for the quality or correctness of that code.
I fully expect this to be the direction the industry goes, where increasingly complex systems exist that no human actually understands or can reason about.
I think it's bad for the industry. Very bad.
But I'm not making those decisions, so... it is what it is, I guess.
I design everything with plan mode and review every line. Nothing happens to my codebase that I don’t decide should happen. With my way of working, tech debt doesn’t exist because I never have to create it.
You’ve made a bunch of assumptions you’re not conscious of. And now you’re blaming me for that.
Open your mind, you never know what you might (un)learn.
The thesis of the post is (paraphrasing): "if an AI wrote it, and I don't immediately grok it or if the code quality is low, I throw it away, even if on the surface it seems to work, because simply 'working' isn't enough to say a piece of code is acceptable."
I'd add as a corollary "and therefore I would never want to be accountable for that code."
If you're reviewing every line then it sounds like you have no argument with the writer and I don't understand what your point is.
Your very first paragraph says:
> If you reject AI code that works then your mindset is still too hands on. Put another way - you still have some loops to work on taking yourself out of.
But if you do indeed "review every line" then you seem pretty damn in the loop yourself and I don't understand what you think taking oneself out of the loop is.
Being able to step back and say "this was a failure and we need to discard the day's work and start over" is still hard with LLMs.
But with the agent, you know that the change will be relatively quick and easy, so the bar to tell it to shift approaches is much, much lower.
Adequate often means done and cheap
It really, REALLY depends what you're working on. If you're throwing together an internal tool or simple dashboard, it doesn't really matter what the code looks like. But if you're writing software that other programs will depend on, bad design choices ripple out and affect another generation of software. Imagine slop in the linux kernel, in google chrome, or in your compiler or runtime. Its not acceptable.
I know a lot of people spend their careers writing end user software and web UIs. AI is increasingly a good choice for this sort of code. But that's not all of us. And its not all of the software being written.
Stakeholder needs: What people wants to get done with the product
Management needs: How to manage the spending of resources (time, money,…) to create the product
Engineering needs: What is the product
You have to balance the three. Sometimes it’s simple and easy to get right. Sometimes it’s complex enough, you’re never truly sure until the product is out in the wild.
Software is malleable and we can do easily do iterations which is not possible with hardware. But today, we have a skew towards engineering, where the whole focus is to create a solution, whatever that is. No understanding of the problem, no proper allocation of resources, just do something. Even if it is plastering over the crack for the eleventh time.
TLDR: Keeping your codebase human readable and reason-about-able is not just helping humans to stay relevant. It will save costs for LLMs to maintain it.
Now we are getting to the point where we are speed-running the deskilling of engineers into comprehension debt and they themselves rapidly losing confidence in reviewing code they did not write.
I think this blog post [0] is the best example of what could go entirely wrong and even worse when you do not know the technology.
If you cannot explain a change even when "the CI is green" or "all tests passing", I will immediately reject it.
Maybe great for vibe coding prototypes, but it all changes when that code is deployed onto mission critical systems. Just ask Amazon with Kiro. [1]
[0] https://sketch.dev/blog/our-first-outage-from-llm-written-co...
[1] https://www.reuters.com/business/retail-consumer/amazons-clo...
How do you verify that it works?
However, if AI provides a solution, as the person using AI, one should conduct research before making a decision. This is not in conflict with or hindered by the use of the ideas provided by AI.
The obvious counterargument is "well, just ask the AI for those answers," but the AI lacks the context and experience that you have. Sometimes, genuinely, the user really is just "holding it wrong," but none of the current AI models would ever admit that, and you'd spend hours trying to solve an unsolvable problem.
For example, I use a vibecoded internal tool written in Go. I don’t even know how to write Go. Haven’t read a single line of the code. I just wanted to move from bash scripts to using cloud SDKs for performance reasons.
But the internal tool is a convenience tool, and you can do everything it does using alternative methods. So if it break, there is no real negative impact besides personal convenience of anyone using it. There’s some documentation on how to do everything manually if needed.
Here’s another example: you’re making a static website. No JavaScript, no interactivity. Truly, what could go wrong? And while I do understand HTML a lot better than Go, it wouldn’t really matter if I didn’t.
What is this supposed to mean? How is a “cloud sdk” more performant than a shell script?
Linking a huge file consuming clients’s bandwith for no reason. Embedding PII in the html source? And if setting up your own server, misconfiguring it?…