34 comments

  • synalx 1 day ago
    I have this problem even in codebases. Claude will work on something, and add detailed comments in which it extrapolates the from the design and confidently states intentions and decisions which aren't actually grounded in reality. Then, later sessions suffer when it reads back those hallucinations and treats them as canonical.
    • t-writescode 1 day ago
      So, I admit that I'm finally getting into this "let Claude help you write code" stuff, and I'm enjoying it a bit for the stuff I, honestly, just don't want to write; but ... I still read it. I still review it. I still look at the code like I'm peer-reviewing it and go, "Uh ... I don't like this area at all. That's going to be really hard to debug at 3am" and so on.

      I guess my thought / question is: this is your code. Why are you submitting the code to the git repo like that, with all the wrong text? I get making mistakes here or there; but ... everywhere? Enough that it's causing huge problems you can't quickly correct?

      And ... aren't you lucky you still *have* the context in your mind? What if you hadn't caught it, or someone else had done that and you inherited that code with those wrong comments that you let through the PR and into the main branch?

      • ghoul2 23 hours ago
        The problem, in my experience is:

        1. It writes SO MUCH code, in minutes, that theres no way I as a human can review it.

        2. I am not very motivated to read/review this code anyway: it was cheaply written, by some _thing_ that is not going to improve from my feedback.

        3. If you DO review it conscientiously , it becomes a never ending thing: you keep finding issue upon issue.

        4. If you report the issues for fixes, the fixes normally fix that immediate issue, and typically add another parallel path/another option/another 100-odd lines of code, instead of a structural fix.

        5. If the code base is large, and if you let AI write a meaningful amount of code in it, its no longer _your_ code. You lose the depth+width of understanding needed to reason thru things mentally, cause you no longer KNOW enough about the code.

        6. If you do review seriously, and either fix things yourself, or have the ai/harness fix the issues for you, the rounds of fixes take so long, if you look back you realize it would have been better to just do it yourself in the first place.

        My personal opinion there are really only two choices:

        A. If you want to build FAST, using an agent, just let the agent write tests, validations, extensively, have it keep running them (it likes to call them gates), and just let it loose. Give up on the idea that its your code, and that you understand it. The bottom line becomes: does it work, and WHEN it breaks in weird ways, just use the agent to find and fix the issue (probably breaking something else in the process).

        B. limit AI use for non trivial, prod-quality projects to limited research, very tiny targeted changes, write most of the code yourself still, and review every line. You won't get much speedup, maybe 20-30%, but it will still be YOUR code, and you will still be able to reason about it.

        • t-writescode 18 hours ago
          Route A seems genuinely horrifying to me - maybe not for small and/or helper scripts; but for anything substantial. For anything that bites you or someone else hard when things go bad.

          I don’t want a vibe-coded, mass-produced diabetes tracking app, or banking tool, or tax management software. I don’t want a vibe-coded power grid analysis software.

          I don’t care if a person’s little, local scripting thing is vibe-coded. I don’t care if an artist’s 0->1 game code is vibe-coded; but that’s not what a lot of the things we’re talking about here really are.

          • nektro 7 hours ago
            Route A is what concretely what is behind those "a select few are seeing 100x productivity increases" posts. agents bring enterprise-level engineering to the IC. your average enterprise PM has no idea how the code works either but knows who to call when it breaks or needs a new feature added. the industry split on agentic use lies in how much and to what degree you believe "but AI is different!" in this analogy
          • anon373839 10 hours ago
            Route A is good for rapid, disposable prototyping. If you ever have a stray thought, “I wonder how this would work if the whole paradigm were turned sideways”, you now have a chance to preview a “working” version of your idea. If you like it, discard the code and reimplement it correctly. In this way, I think it can be a good adjunct to sketches and other lofi prototyping techniques. Just don’t outsource the creative ideation to the LLM, because all you’ll get are the same solutions as everyone else.
            • t-writescode 10 hours ago
              Have you seen many of the replies in this thread and others? They’re not doing that. They’re putting it into production code. Their company’s thought leaders are trying to say that review itself is a waste of time, etc.

              That’s not “disposable prototyping” that’s whole versions of the codebase written with barely a human in sight.

              • anon373839 8 hours ago
                Yes; my point was not related to any of that.
        • othmanosx 13 hours ago
          You should be the one building the gates then, but those gates should be mechanical and deterministic so the AI doesn’t wprk around them. Lint rules, type errors, commit lint, anything that tells the AI to stop instead of allowing workarounds.
          • ghoul2 5 hours ago
            > Lint rules, type errors, commit lint, anything

            These are basic: they work for human coders and are typically quick to setup. But gates at this level are far from enough for agentic coding. You need an extensive test suite/e2e suite/benchmark suite. Typically much larger than the code base itself: This is what enables those "i ported bun to rust in a weekend" headlines. Where ever such extensive 'gates', high quality ones, already exist, the agent can do a good job of making things work, cause it can automatically iterate.

            But: every hole, every gap in coverage, will be eventually found by the coding agent and 'exploited'. So you need full, extensive coverage.

            This suite itself is a LOT of work, much more than what it would be if you only had to worry about human coders. So if YOU are writing the gates yourself, you are going to fall between A and B: you still have a convoluted, unreliable, impossible-to-reason-about prod app, AND it still took you eons to build it cause you spent that time writing the gates. This is no win at all, on either front.

            Where gates, extensive ones, help in route A is they can significantly reduce the churn/spinning, fix-x-break-y, to just the parts that are gaps in coverage (which the agent will keep finding at a ridiculous rate). And your work would be to have it ALSO keep filing in those gaps with more tests/etc.

            One of the reasons go coding with agents works better than other langs is cause I even include AST based enforcements in my governance suite. Its still route A, but with somewhat more confidence, somewhat less frustration.

            • t-writescode 1 hour ago
              I mean, from my training long ago, a good test suite is usually / always a few multiples in size of the codebase.

              100k lines of app code? 300-500k lines of test, sort of thing.

              • ghoul2 7 minutes ago
                Exactly.

                I am now looking at 9 to 10x the app code, for non trivial stuff. And this is not just the usual unit/integration/e2e suites too. Extensive deadcode, field use, code shape tests (ast walking) - god objects, 15-param helpers, wrapper-piled-upon-wrapper, badly named (a skill) functions/methods/objects, path-dependent artifacts, etc. There is no way I could have written all this manually.

                It doesn't guarentee high quality OR reliability or readability or maintainability, but does reduce churn and makes me feel a bit more confident about deploying route A work product in prod.

      • cs02rm0 1 day ago
        > So, I admit that I'm finally getting into this "let Claude help you write code" stuff, and I'm enjoying it a bit for the stuff I, honestly, just don't want to write; but ... I still read it. I still review it. I still look at the code like I'm peer-reviewing it and go, "Uh ... I don't like this area at all. That's going to be really hard to debug at 3am" and so on.

        I used to. I've stopped in recent months, only because I just can't keep up with the pace it's churning out the code. If I was to read it, it would take multiples longer to develop anything, maybe into orders of magnitude longer. Occasionally I'll dip in just to get a sense of things, especially if it's struggling with something, or at the opposite end, if it's completely trivial. But I'm hardly reading anything now.

        • t-writescode 23 hours ago
          Why? You own the pager, I assume. You own the mental hit when something you let go live (especially with your name on the PR) leaks customer data or deleted someone’s work.

          And if you’re not reading your own work, is someone else during the PR? Are you reading other people’s PRs?

          How do you know what your product does and how it does it?

          • Izkata 10 hours ago
            From what I understand for most people it isn't a conscious choice, it happens slowly over time. People naturally don't want to waste time, the code is there and if you don't look too closely it works, so they start skimming instead of going in-depth, expecting anything majorly wrong to pop out. But those things probably won't because generated code is very good at looking extremely high quality at a surface level no matter how bad it actually is. So eventually even the skimming starts feeling like a waste of time so it tapers off too.
          • menaerus 23 hours ago
            When the combustion engine was invented, and we got the means by which we could accelerate our trip by 10x, and at the same time scale it to multiple people, we didn't tell the humans to keep pushing the vehicles by their hands, didn't we?
            • fortzi 15 hours ago
              As a developer you are more akin to a car mechanic, who’s still expected to know how the engine works even a century after it was invented, rather than a driver, who is just the user in this parallel.
              • menaerus 3 hours ago
                Mechanics I've been to generally have no idea about how things actually in the car work - from what I could understand, area is full of recurring problems, and this exactly benefits mechanics to solve them through trial and error approach rather than understanding much how underlying things work. Pretty much close to what I would say SWE will turn into, there's no other choice IMO in foreseeable future.
                • t-writescode 1 hour ago
                  I think you underestimate the skill of more experienced mechanics. Tractor (as in 18 wheeler) and master mechanics tend to know what’s going on under the hood in detail.
            • nickmonad 23 hours ago
              Speed doesn't mean anything if you end up driving off a cliff.
              • menaerus 22 hours ago
                Evidence so far does not suggest people driving off a cliff so while a valid concern it's not a likely one. We also acknowledged that the benefit of using a vehicle is such that it outnumbers the risks it may introduce. We never said there will be no risks attached.
                • t-writescode 18 hours ago
                  We’ve only been on this train for 3-4 years, and in that period, we’ve already invented terms for this kind of vibe-coded, highly-breakable crud. We’ve also had a few, relatively high-profile in their spheres bugs come out and hurt people’s real, lived experiences.

                  And that’s just the stuff where the house of cards failed quickly.

                  • menaerus 17 hours ago
                    Software by definition has always been and will remain broken in one or another way. AI makes no difference or whatsoever. Arguably, it will help raising the quality of software.
            • defrost 23 hours ago
              No, but the US and UK did enact Red Flag laws.
        • menaerus 23 hours ago
          Exactly. That would mean going back to square one, and I also personally don't review the code anymore but I am more focused on asking the model to demonstrate the value it created through benchmarks, workload-generators, and e2e tests.

          Mostly it proves as a valid approach, barring the bugs the model can introduce to value-demonstrating benchmarks which can of course skew the evidence on hypotheses, and thus code trajectory the model opts to go with.

          The problem I see with this is really that I am not anymore under the control but I am not sure I see other alternative. I am becoming more and more like a system observer with surface-level understanding of the system rather than the engineer with zoomed-in level of understanding of how the code actually behaves. Perhaps we're transitioning into a QA roles present.

          • fortzi 15 hours ago
            Are you working on complex systems that serve many users in production? Do you actually save time?
            • menaerus 4 hours ago
              Yes, I can't say exactly what but the backbone of cloud computing and/or infra, think databases, distributed storage, filesystems, ...
              • t-writescode 1 hour ago
                I mean, AWS did just recently go down due to a vibe-coding / agent-run-amuck incident, didn’t it?
                • menaerus 1 hour ago
                  I don't know which one exactly but can we now count how many incidents there were in the pre-agent era?
          • rasz 1 hour ago
            >asking the model to demonstrate the value it created through benchmarks, workload-generators, and e2e tests.

            LLMs are fantastic at faking those

      • bojan 1 day ago
        > Why are you submitting the code to the git repo like that, with all the wrong text?

        According to my enterprise architect I shouldn't be reviewing code, it's a waste of time in this new reality.

        I'm still doing it because it's going to be me answering that 3 AM call. But I don't know for how long I'll be allowed to swim upstream like that.

        • qsera 1 day ago
          I have found manual reviewing of LLM generated code to be an uphill battle. LLMs does not believe in abstraction. So complexity spills everywhere. Some details in the lowest level might be handled separately in more than one place, at the higher levels. In a short while it is a copiously documented unreadable mess (both code and documentation).

          LLM written PR descriptions and comments are a sight to behold. I am not sure how stuff can be written so cryptically. It seems that LLMs just make up what ever terminology so that it can cram as much details into a single sentence as possible! I generally just paste it to chatgpt and ask it to decrypt it.

          • sfn42 23 hours ago
            Yeah, that happens when you vibe code and just let the LLM be in control of everything. If you take responsibility for the architecture and instruct it to do things properly it will do them properly. You can tell it exactly how to do it or you can ask it to handle it in a way that avoids duplication, you can tell it to design a reusable abstraction for this usecase etc.
            • qsera 21 hours ago
              >If you take responsibility for the architecture and instruct it to do things properly it will do them properly.

              The problem here is that after a while it is impossible to detect when such potential abstractions is overlooked in the generated code. Because it has become hard to reason about the existing code.

              Not all work is green field.

              • Izkata 10 hours ago
                My personal suspicion is that a lot of preexisting codebases are running on inertia. The abstractions exist and are used well so the generated code also uses it most of the time, with only occasional breaks to the abstraction that look like an acceptable tradeoff in isolation. But those would keep piling up to where the existing abstraction is no longer so identifiable and that's where the major spaghetti code issues start happening.
              • voakbasda 19 hours ago
                I have wondered about this. Can’t you ask the AI to find the patterns than need abstracting?
                • fortzi 15 hours ago
                  Yes you can. Sometimes it’ll find them, sometimes it won’t.

                  I find myself having to use Claude to untangle its own mess piece by piece, a as it charged full steam ahead with the design we made together. However, not everything can be foreseen in a design, unless you go full waterfall. Sometimes I find myself in front of a mountain of bad abstractions stemming from a subtle oversight in the design. Before AI coding, I would find details while coding and catch them in time before they became sinful abominations

                • sfn42 3 hours ago
                  You can do that, with varying results. Or you can just do actual software engineering like we used to do, and tell it what patterns to abstract and how.

                  You choose your level of effort and involvement. The LLM can write whatever you tell it to write. If you send a drawing of an app and say "make this" then you'll get whatever it comes up with. If you tell it how to make it then it will make it the way you tell it to.

                  If your AI code is trash that's because you're trash at directing it.

      • stuaxo 21 hours ago
        Yeah - the comments it writes are often "waypoint comments" [1] - great for the LLM during creation but just crap for us - I ask it to remove those before committing.

        [1] Not sure if I read that or came up with it.

        • vitorsr 21 hours ago
          I believe the general phenomenon is metadiscourse and those in specific are signposting constructions.
      • killix 1 day ago
        [flagged]
    • bartread 1 day ago
      Delete the comments: I’ve gone on a tear with these recently because they’re nothing but trouble.

      The absolute least worst outcome is they chew up your token budget. But what tends to happen, and this is much more serious, is they poison future work and make further modification of the codebase more burdensome and error prone.

      • zahlman 1 day ago
        > But what tends to happen, and this is much more serious, is they poison future work and make further modification of the codebase more burdensome and error prone.

        Agreed.

        Probably better to add whatever instructions it takes so that the agent doesn't write comments, at all, ever. If you need comments to understand the agent's code, the necessary information should already be in a conversation somewhere; and you should summarize it yourself, because the comment will be for your own benefit. Otherwise you are letting past-agent steer future-agent more or less at random.

      • ThunderSizzle 1 day ago
        I use comments for external domain constraints (e.g. this table has three types of records in them that we use for different purposes - I might have the LLM agent be a bit thorough and clearer with the explanation, but I found those type of comments very much helped future iterations)
      • acedTrex 1 day ago
        I completely block all LLM comments via pi extension, it makes using them significantly more enjoyable. If the LLM wants to add a comment it must ASK me explicitly to do so.

        Seeing LLM comments in other peoples code is very upsetting because theres just so much meaningless noise.

        • zahlman 1 day ago
          What happens when the LLM tries to defy this? Is the file-write rejected, or does the extension just strip comments from what gets written, or just what?
          • acedTrex 20 hours ago
            I have it set up to pop up an approval dialogue that shows me the comments its trying to write and if i reject it the model is told essentially "no comments allowed, retry the patch without them."

            It usually only takes one of those in the context for the model to reorient its behavior for future edits.

        • furst-blumier 1 day ago
          Which extension is it?
          • acedTrex 20 hours ago
            Its a personal extension i wrote myself, maybe I should put it up somewhere.. Its so simple though i dont know if its worth it vs just "go build it yourself"
        • copperx 1 day ago
          What's the extension, please?
          • gessha 22 hours ago
            Author probably cobbler it together themselves.
      • LoganDark 1 day ago
        Absolutely. IMO, comments can explain historical reasoning for the code, but refactorings can benefit heavily from rethinking ideas from scratch, as opposed to trying to follow the same reasons. LLMs tend to be misguided by comments, probably by treating them as instructions. Ergo, get rid of them.
      • trenchgun 1 day ago
        Why delete comments! Constrained grammar! Make invalid state unrepresentable.
        • vaylian 1 day ago
          Comments explain why the design is the way it is. If you later need to refactor the code, you probably won't remember the reasoning behind it and you don't know if it will be safe to remove or change parts of the design.
    • post-it 1 day ago
      I've been whittling my way towards having Claude write zero comments in code. They're pretty much never helpful.
      • sdeframond 1 day ago
        How do you do it?
        • justbees 23 hours ago
          I include a rule in the initial PR prep that it runs. So there's a hard limit to the length of the comments and what type of comments it can write and they get flagged in the review. It also reads the comments and checks for consistency with the following code so they don't drift. And the reviewer is a completely different agent/different model.

          So all of that happens before the manual review and usually catches a lot of the 3-5 line comments it inevitably adds.

        • ch4s3 1 day ago
          You could probably add linting rules to your tool of choice and tell claude that your lint tool has to pass.
          • demibabs 1 day ago
            but then no human generated comments allowed either, right?
            • ch4s3 1 day ago
              I'd do this by only running the linter locally and using this as a custom rule for the LLM, but I need to think about it a bit harder.
              • devmor 1 day ago
                There is probably something you can do to only apply some rules to the actual changed lines in a git diff.

                You may have to write your own linter for that specifically.

                • notsirius 1 day ago
                  you'd probs want a commit hook for that
                  • unleaded 1 day ago
                    Does "don't write comments" not work?
                    • jaggederest 1 day ago
                      about as effective as don't make mistakes... For anthropic's models at least
                    • devmor 15 hours ago
                      It’ll work sometimes.

                      You’re using a non-deterministic algorithm to generate output. If you want deterministic rules applied to it, you have to use deterministic systems to do it.

            • oblio 1 day ago
              Most lint tools allow exceptions if you add a special marker. I think even allow custom user exceptions.

              E999: human generated comment :-)

              • gmueckl 1 day ago
                But then the LLMs sees the syntax and is likely able to mimic it.
                • oblio 1 day ago
                  Shouldn't the "human generated comment" part be a strong hint not to?
                  • gmueckl 5 hours ago
                    As if LLMs would ever not ignore explicit instructions if they stand in the way of a goal...
        • nottorp 1 day ago
          I just delete them most of the time. But I keep my LLMs on a short leash. No 40000 line PRs.
    • Utilera 1 day ago
      This is almost worse in code because the hallucination starts looking like documentation
    • gmueckl 1 day ago
      Even worse to me is the tendency to not properly rewrite when prompted to change something in a text. It will always add to the document instead of replacing any obsolete information. The lengths it goes to to turn every document into a (useles) decision log is astounding and annoying.
    • rco8786 1 day ago
      We have this in our harness markdown:

      ```

      ## Comments

      Use comments extremely sparingly. Most comments should be at the request of the user. When something warrants a comment, keep it to one or two lines: what the code does and why it's necessary. No background narrative, no replaying the investigation or failure mode, nothing a test name or the commit message already says. Applies to specs too. If a comment needs a paragraph, make the code clearer instead.

      ```

      The comments Claude was leaving got absolutely out of control. Just lines and lines of LLM drivel that was barely intelligible and not remotely relevant to what a code comment should be used for.

    • preommr 1 day ago
      I've had codex delete useful (albeit not directly relevant or perhaps messy wip notes) comments, even though I explicitly have it in my agents.md not to delete comments, and ask for permission if it thinks it should.

      It deleted the comments, and when I asked why it did that even though I expressedly asked it not to, it responded that me prompting it in the first place explicit permission. I have no idea if that's the actual reason or just some post-hoc explanation.

      But I genuinely don't think it's possible to just have these things be completely, 100%, indpenedent and also solve deep problems that need to also be understood by people in a people-based organization context.

      • debugnik 1 day ago
        > and when I asked why it did that even though I expressedly asked it not to

        Just be clear, it can't know, and by asking you're just making it roleplay as someone excusing themselves.

        It's very unlikely that the choice to remove the comment was driven by an internal monologue based on learned criteria that it can refer to. The sampler most likely picked tokens to remove it while writing the patch, because that's what the statistics modelled, and that's it.

      • recursive 1 day ago
        There might not be an "actual" reason that's expressible in human-comprehensible language.
        • wafflemaker 1 day ago
          Moreover, even if there was a reason, it was there "in the model" at the moment of generation.

          The new model can only guess/hallucinate when ordered to give reasons.

          It's like having an actor play a role while wearing a hat. Then the next actor comes, puts on the same hat, and we're asking them to explain why "they" did something while there was someone else playing the role.

    • coder-pm 1 day ago
      My approach is to keep the comments only with a references. My LLM based projects always have the decision log where are my decisions while working on the features are stored with the date. I found this useful to actually trace why something is in the codebase. It's much easier for both me and the LLM to navigate through the big projects where I spent months and dozens of full night sessions executing my plans with --dangerously-skip-permissions. In the morning I was answering all the model questions and iterating like that. Honestly, try that.
    • p0w3n3d 23 hours ago
      In numerical methods this is called error propagation. And since the neural networks are basically numerical approximations of a certain function, this shows apparent similarity of the propagation on a macro scale
    • andai 1 day ago
      A computer cannot be held accountable. Therefore, a computer must never make a decision. -IBM, 1979 (paraphrased)
    • mrtesthah 1 day ago
      Probably good to establish canonical spec documents up front that it can use and maintain as an independent reference?
      • rendaw 1 day ago
        That doesn't solve the problem of it hallucinating more design details and putting them in comments. The solution to too much authoritative documentation isn't more authoritative documentation.
        • pc86 1 day ago
          Comments exist for humans to understand the why of the code, including perhaps external dependencies/assumptions/requirements.

          LLMs inserting code comments makes zero sense by definition.

          • pavlov 1 day ago
            That’s not true because LLMs clearly benefit from having multiple statements representing aspects of the same thing.

            That’s why they’re always doing the “it’s not X, it’s Y” kind of patterns. It’s reinforcement.

          • selcuka 1 day ago
            > LLMs inserting code comments makes zero sense by definition.

            Not completely, because anyone who reviews the code in the future does not have access to your original prompt, so theoretically leaving a comment that explains the "why" portion of the prompt would be useful. LLMs rarely do that, though.

          • hombre_fatal 1 day ago
            LLMs benefit from knowing the why, the intent, and the reason/justification for code to exist, too.

            Catching code desyncing from it is a valuable place to reconsider assumptions and maybe even invariants.

      • lupire 1 day ago
        Doesn't need to be "up front", but you do need to curate the context.

        AI can't read your mind, and it will read what you (or anyone else) wrote.

        • Sharlin 1 day ago
          Humans can’t read minds either, but most humans aren’t so dumb as to forget what is a normative spec and what is just their own current, evolving model of the problem space. (Though some humans certainly are…)
        • pydry 1 day ago
          It can't but it will try and getting it to stop is often infeasible.
    • bsjdhdudj 1 day ago
      why would you not delete the comments? why are they saved or checked in? don't you read the code? and if you don't: ISN'T THIS WHY YOU NEED TO?

      I feel like I'm losing my mind, what the fuck is wrong with all of you?

      • dan_gggggg 1 day ago
        > what the fuck is wrong with all of you?

        They spent 3 years pretending that their magic robot made them 10x harder/better/faster/stronger than the lazy peons rubbing shoulders with them and now the bill has come due.

    • throwaway613746 1 day ago
      Anthropic hopes you just eventually give up and become completely dependent on Claude. Toss in 10 (or more) other developers with way less discipline and you will become dissillusioned with Claude's overengineered, incomprehensible, technosalad and your demise is pretty much guaranteed.
      • recursive 1 day ago
        Retirement feels farther away than ever.
    • dan_gggggg 1 day ago
      [dead]
    • abjhn 19 hours ago
      [dead]
  • mccoyb 1 day ago
    I don't get the second brain thing. I don't keep a second brain, I just read and think a lot. I once did a physics masters and, when asked how to make steady progress, one of the professors (who had published with Paul Dirac no less) made the comment "well, my secret is that I think about physics all the time"

    I started practicing this for the stuff I'm interested in ... and the result is that I generate a lot of ideas. 99% of them are garbage.

    Before AI, a conversation with colleagues would help me weed out the garbage, and possibly give me a new perspective ("here's something surprising about what you just said!")

    Now, AI plays a helpful role in helping me weed out the garbage for the subset of ideas that I can test empirically -- and sometimes it does give me a new perspective, but with significantly more noise than my colleagues, who would take me right to the point that would be useful for me to see.

    I've never felt like I needed a second brain ... because if I had such a thing, I think I would be keeping the 99% garbage around -- instead I just let biology GC it.

    • VBprogrammer 1 day ago
      I don't think I do the second brain thing well but keeping some detailed notes (using a daily note at a scratch pad for one off queries, small scripts etc) and occasionally branching them off into dedicated markdown files like "Database debugging" or "MongoDB" has saved me a lot of time on aggregate.
    • jhartikainen 1 day ago
      There seems to be two schools of thought for "second brain" stuff:

      1. Those who kind of semi-obsessively write down every little thought, semi-obsessively catalogue everything, etc.

      2. Those who use the idea of "writing helps you think", where the second brain functions in support of this

      Personally I find 2 quite valuable, and although I could "think about programming all the time" (or one of my other interests), I find it helpful to have f.ex. exact quotes from some book at hand (and various other Obsidian-related things)

      • threecheese 17 hours ago
        I’m in a third school: memory recall ability does not scale to meet the needs of interests and responsibilities. I’ll take all the help I can get.

        I keep obsidian vaults mostly to augment recall, and often all it takes is a small cue to reveal that thing I couldn’t remember. I’ve found LLMs helpful for translating my question into search criteria - “agentic search” is magic.

      • tristor 1 day ago
        Writing does help you think... but only if you write. Dumping a bunch of stuff into a "second brain" is not you writing. Keeping a journal is you writing. Those serve similar functions, but keeping a journal is more intentional and requires you to think critically about your inputs and decide what to record and how, which is the thinking part that comes with writing. That summarization that LLMs are doing for people is removing a significant portion of the thinking, which results in a loss of capability and soundness of mind.
        • jhartikainen 1 day ago
          100%. The term "second brain" is getting kind of a bad rep because of people using the dump+AI method. I don't think Tiago Forte who, as far as I know, coined the term meant doing that, at least based on his book on the subject.
    • CoolestBeans 1 day ago
      So I operate more in line with how you do, but having tried some organizational systems I think I understand the "second brain" people.

      One, it forces you to do a crude approximation of critical thinking. It allows you to examine and organize your thoughts, either physically or digitally. It isn't the same as being able to sit and break something down in your mind, but for those who struggle with that, a "second brain" allows them to reap some of those benefits.

      Two, it allows you to turnover things quickly in your head. By training yourself to immediately dump out an idea into an organizational system, you don't have to hold so many things in your head. Furthermore, words are immutable and the thoughts in your head aren't.

      But for me, the good thoughts usually rise to the top. And there are a lot of things I'm thinking about that are so vague, it's just a feeling. And trying to write those down robs those thoughts of the long term, "on the backburner" mental processing they need to become actual ideas. They're thoughts that are still in the embryonic phase. I can't write down the qualia of them, and when I try to I move them into a "second brain" I lose it.

      I think organizational systems are work dependent. Running a business lends itself well to an organizational system. Where it's good to try a lot of shallow things. But something like physics, for example, requires playing with ideas in your mind over a long period of time until they become solid enough to chase more seriously.

      I think programming probably straddles that line and it depends on what you're doing at any given time.

    • thorum 1 day ago
      When I have some important problem I don’t know how to solve, I of course think about it all the time. Every now and again I’ll think of something that seems extra useful and write it down.

      Eventually I’m looking at a whole page of good ideas, and some pattern jumps out at me. I attribute many of my best ideas to this process. Writing lets you work beyond the limits of your own brain, especially the constraints of working memory.

      I think that is the primary value in a “second brain” - externalizing your thoughts and reasoning makes them stronger - though the productivity community may use the term to mean some other thing.

    • chermi 1 day ago
      I don't know if that's what people mean when they say second brain? My "second brain" is basically a memory expansion/thought recorder. With the understanding that the memory isn't truly integrated like regular memories, which has plusses and minuses. A lot of value for me is just getting an idea out of my head if it's in the way, even if I never revisit it. Helpful for adhd and otherwise distracted minds as long the process of recording is very cheap. Which, as you say, yields mostly garbage.

      Where it works better is as a place to store more careful thoughts, and as a place to force you to express those thoughts in writing.

      That is, it works best if you don't try to use it as a literal second brain, don't rely on it doing any thinking for you.

      All that being said, confession time: using obsidian for 6 months as second brain among other things, i don't believe it has really helped me think other than as being a place where I also create more polished notes where I must crystallize ideas.

      What it has done is enable my idea hoarding/belief there's gold in them there old thoughts, so I spend too much time checking if I already had a thought rather than just thinking. So the verdict is still out for me.

      • Izkata 1 day ago
        > I don't know if that's what people mean when they say second brain?

        Note taking app, personal wiki, etc, broadly any central thing you store information in solely for yourself that's searchable. It's an artsy term that doesn't seem to add much of anything but confusion.

    • raincole 1 day ago
      The whole idea of second brain is simply cringe. I know using this word itself is cringe but it's the most accurate one for how I feel.

      While it doesn't necessarily have anything to do with AI, the trend of glue AI onto everything certainly didn't help.

      For you guys' 5 min of entertainment: https://youtu.be/6NukGtwJb7Y?t=256

      • _345 1 day ago
        I love Eric. He is up next for sure.
    • mmargenot 1 day ago
      A key part of maintaining a second brain is revisiting and pruning it. Much like a todo list, without that practice it absolutely just accumulates bunk.
    • ostros 1 day ago
      I like to to think while talking. Nowadays I don’t have always somebody to talk to so I talk to myself and record on the phone; then voice to text to obsidian.

      I have a lot of interesting notes and I can also plug them to AI to get some reminders, insights etc.

      But I agree with your general message - thinking is still superior

    • pibaker 1 day ago
      I suspect for many people productivity tools are more about feeling more productive than actually being productive. And then there is the kind of people who spend time on productivity tools to avoid spending time on real, hard work.
      • thom 1 day ago
        Yes. Note taking apps were very much the precursors of modern agentic coding.
    • hadlock 1 day ago
      >I don't get the second brain thing. I don't keep a second brain, I just read and think a lot.

      Yes, but writing also helps you lens your thoughts in a completely different way. This is why so many people journal, or blog, or vlog, even though only a handful of people might actually audit their output. Obsidian sort of abdicates a lot of the value of journaling, but I think there's still some (although much less, perhaps 5% as much) value in the actual act of directing the journaling, even if the rest of the thinking is outsourced to the clanker.

    • aprilthird2021 1 day ago
      Just wanted to say you're not alone. I've also never understood the appeal of a second brain. Most of our ideas are garbage, and we're often way too precious about our ideas
    • UltraSane 1 day ago
      the second brain is very useful if it have instant search.
    • jpardilla 1 day ago
      [flagged]
    • Dependance 1 day ago
      Hey, love the hivemind comment to mine. Maybe we do have kind of a second brain...? /s

      How do you take notes, generally speaking ?

      • mccoyb 1 day ago
        I've never been a good note taker. In the things I think about, when I'm really invested in something, I have to reverse engineer it to experience the why.

        Reverse engineering can take several forms for me: it started with writing parts of systems from scratch, and now (due to academic training) it oftens looks like "thinking about how to explain the system to a group of colleagues", which is one degree of separation from writing the system from scratch, but is much faster - and seems to work well when working with LM agents on code (because you have to confront your ignorance)

        part of the training in academia is how to know when you don't understood the thing, so this ends up working well for me now.

        One trick: you can force yourself to understand something better by making a presentation and imagining you're going to be grilled by someone smarter than you. This is kind of the feeling of pressure that academic paper reviews give (in an ideal world) and it translates when you think about explaining something to someone else.

  • altairprime 1 day ago
    Ironically, this is why one of the best ways to manage upwards is to prompt managers into coming up with your idea as if it were their own: they’re dramatically more likely to execute and deliver on it. Of course, this also obviously leads to undue credit given for coming up the idea, which is certainly a core problem with AI as well, but it’s still charming to watch managerial psych being rediscovered by this new generation of managers.
    • Cameri 23 hours ago
      Nothing beats the feeling of having a proposed idea dismissed just to have the higher ups “come up” with it some time later. Don’t even dream about getting a footnote mention.
      • altairprime 19 hours ago
        That’s the passive form of the experience, sure — and it often comes as a shock to people that this is a thing in human relations. If you set out to make it happen, and you don’t mind letting someone else execute on the idea, it can be quite fun to manipulate people into an epiphany you arranged for them. (See also teaching, coaching, et al. but with a flipped power differential.)
  • Dependance 1 day ago
    Could anyone enlighten me as to why to use Obsidian in one's daily life ? I have a dozen ideas popping a day and nothing comes close in terms of ease than a paper notebook for capturing and some apple notes equivalent for storage. Used to have a Notion but all the "Second Brain" movement seems...fishy to me ?
    • _def 1 day ago
      Taking the notes (and being able to read them again if I want) is all I need from my "second Brain". This perfectly can be pieces of paper, as long as i can carry them around with me. More realistically some digital notes software fulfills that requirement.
    • nelsonfigueroa 1 day ago
      I use Obsidian but I don't follow the whole "second brain" thing, I just organize my notes by directories. You can use Obsidian however you want and adapt to the way your actual brain works. The "second brain" thing doesn't work for everyone. Even taking digital notes doesn't work for everyone (for example, it looks like you prefer paper notes).

      If you haven't tried Obsidian at all I think you should give it a shot and write notes without following any system in particular. If you've tried it and you still prefer paper notes and the occasional apple note, then perhaps Obsidian isn't necessary for the way you take and organize notes.

    • pragma_x 1 day ago
      > Could anyone enlighten me as to why to use Obsidian in one's daily life ?

      My handwriting has degraded from consistent keyboard use over the last 30 years. Obisidan has replaced my paper notebooks, mostly because of this. However, it is handy to be able to mesh those notes with hyperlinks, LLM output snippets, quotes from other sources, embedded images, etc. Doing all of those other things would be laborious to impossible with paper.

    • spacechild1 1 day ago
      I just use Obsidian as a glorified notebook. I've been using paper notebooks, but I tend to lose them... With Obsidian I just push my files to git and never worry again. Since the files are just markdown, there's no lock-in.

      What I also really like about notetaking apps is that I can organize my notes hierarchically, move things around and make cross references.

      I still regularly sketch ideas on paper, but once I have something that's worth keeping, I write it down in Obsidian.

      I'm also skeptical about this "Second Brain" thing. People make videos on YT about how they have been enlightened by Obsidian and then try to sell you their online course. To me it seems like notetaking for its own sake. You could call it "idea maxxing" :)

    • beart 1 day ago
      I just want something that I can use to record notes, with good backups, and a format that doesn't lock me in. Obsidian offers that. It has a pretty decent mobile app and desktop app, in terms of the actual editing. And if I do want to do something advanced, like automatically generate a daily note for recording what I worked on today, it can do that. You can do all these things with other programs, or even just raw files, but Obsidian adds just enough to make it easy.
    • huurtehoog 23 hours ago
      Self soothing behavior. Helps quell the anxiety derived from the dopamine threadmill of modern digital media.

      We're all becoming mental hoarders and discussing our piles of though junk in our digital storage units.

    • Jtarii 1 day ago
      Obsidian and apple notes are basically the same thing. Obsidian just has some more features.
    • fsnovask 1 day ago
      There's also an article from this same site about that: https://www.ssp.sh/brain/digital-vs-paper

      IMO it's way better to do what you think is good, whether that's Obsidian, notebooks, or something else

  • helsinkiandrew 1 day ago
    There was a related study posted awhile ago:

    > When people think AI did the creative work, task meaning and effort decline

    https://www.brookings.edu/articles/when-people-think-ai-did-...

  • nixonaddiction 1 day ago
    ive never been able to use zettelkattsen. i like having daily notes instead of different notes for different ideas. its easier this way. my obsidian vault is a hybrid of diary and research and i am trying to separate the diary and research but its not easy tbh. (i suffer from a crushing urge to note that i "just ate a sandwich, was good" while liveblogging research and debugging.)

    that being said, i got a big kick out of feeding my daily notes into my local model. i had some more diary-like entries that i would never share with anyone, but it was extremely funny to have my local model take a 3000 word recap of what i did in a day and summarize it like "writer reflects on their job: they worry about their performance. meals: the writer ate oatmeal, and a sandwich. family: the author contemplates their fathers absence growing up."

    ive developed a really large corpus of written work over the years and am hoping to fine tune a model using my diary as the training set. i think this would yield hilarious results. keeping it all local though, i drop my passwords in there at times and i dont want that leaked.

  • ssivark 1 day ago
    > It’s so hard to finish an idea that is not yours and is just suggested by AI

    Yes! And that's because you can't really delegate ownership/authorship to AI (at today's capability levels). Coding agents might have been RL'd out the wazoo to write code to solve well defined problems, but given a situation the onus of making sense of the ambiguity and identifying the problem -- and even crafting a skeleton for an acceptable solution keeping in mind tradeoffs -- needs to be done by a human taking ownership.

    I wrote about this recently from a slightly different angle [1], but the core idea is the same -- which is why it is hard to take over an idea mooted by AI. Before even writing/implementing, the AI has internally resolved ambiguity with several decisions which are arbitrary, and match neither your mental model nor the actual ground reality (which you need to spend time understanding in order to productively interpret).

    [1] https://woventhought.substack.com/p/ai-assistants-need-adapt...

    • jesse_ash 1 day ago
      I like your idea of conversational tempo, it feels a little like a 'thinking' slider applied to a plan mode (how much back and forth vs. "rigor" you want at a given moment).

      Might be a good candidate for a pi extension.

  • gdulli 1 day ago
    What am I even here for if not to have my own ideas?
    • datakan 1 day ago
      “After we started thinking for you it became our society” The Matrix 1999
    • a2ff6eeb0 1 day ago
      That's for you to find out. When we automated muscles in the industrial revolution, people used to value strength. Today they value thought. They may value something else in the future.
      • hdhdhsjsbdh 1 day ago
        Seems like the only thing left to value will be social capital. If all forms of labor are free, the only thing that matters is how many people you know, what they think of you, whether they listen to you.
        • _def 1 day ago
          Nightmare fuel. This sounds like totally worthless to me.
        • CM30 1 day ago
          Sounds like society at the moment even without the free labour thing. The world generally loves celebrities and influencers and other famous people, and I hear in some fields (like publishing) they're actively selecting for online fame/popularity so they don't have to spend as much on marketing.
        • ModernMech 1 day ago
          The ultimate goal of project capitalism is to make ownership the only thing in the world that’s valued or valuable. The only thing that matters is how much and who you own.
          • parineum 1 day ago
            The actual goal of capitalism is the direct the impulses of the capitalist to align with the interests of the public.
      • theossuary 1 day ago
        We didn't automate muscles generally, only specifically. The leverage those specific situations have is so high it's worth it. Same with thinking, it's very far from automated generally. Maybe one day we'll have humanoid robots as generally physically capable, and AI as generally mentally capable, but that's a long way off. Until then there's still emmense value for humans in between the high-leverage areas, and even more value in adding some leverage throughout (like the first attempts at exoskeletons, or using Claude to help manage your daily tasks, etc.)
        • a2ff6eeb0 1 day ago
          When you say a long way off, how far are you thinking, especially with self improvement on the horizon?

          My bet is 3 or 4 years before the rocket really takes off, and then we start seeing capabilities improve incredibly quickly.

          • bigstrat2003 1 day ago
            There's zero reason to believe that self improvement is on the horizon. What we have now is far too stupid and unreliable to be plausibly capable of improving itself.
      • abjhn 19 hours ago
        [dead]
    • Zambyte 1 day ago
      I think the temptation for most people is not to offload your idea generation to AI, but to offload your storage and/or retrieval processes to AI.
    • brightstep 1 day ago
      I’ll have ideas on my own time. On the clock, I’m happy to delegate that when appropriate.
    • megous 1 day ago
      You can become a harness, like a lot of us.
  • t0bia_s 1 day ago
    Letting AI to read private thoughs is like giving public entities ability to read mind. In global scale, this is ideal tool for social engineering.
  • r0b05 1 day ago
    My second brain is 500 open tabs of Notepad++
  • yipinwong 1 day ago
    As the author does, i just have a separate vault for AI to manage.

    It used to be a mess, but with OKF (Open Knowledge Format), things are more manageable and navigatable (for me as well).

    Consider AI vault as a scratch pad that AIs use, not as something you put your own thought into it.

  • WhyNotHugo 1 day ago
    One of my main reasons for writing notes is that I learn while writing: the act of dumping information into textual format lets me re-think though all of it. Sometimes I pause and realise that I need to better understand something in order to put it into words. To explain a certain amount, I need to understand a wee bit more than that.

    Having an AI write notes would only produce text, but the actual notes are mostly a side effect — an important one, but not the most important one in note taking.

    I think hard about related notes, and peek at them when linking, sometimes finding interesting one-off connections. This makes me think further and sometimes expand the topics. Automated linking would not make me think, it wouldn't make me learn. It would just produce yet another pile of information that's already available online anyway.

  • luxshan_t 1 day ago
    In research, we can use AI efficiently to identify gaps and then validate them, but the final idea should be our own. I’ve faced an issue where, when AI suggests something unfamiliar, the research may not become successful because we start relying on it too much. Instead of actually learning and doing research, we become good prompters, and our brains start thinking, "How should I ask Claude to do this?" rather than thinking deeply about the research problem itself.
  • butlike 20 hours ago
    Well, yeah. It used to be you traded money for your time building someone else's dream. Now you do it for free. What kind of value proposition is that?

    I love you guys, but number 1 is numero uno, and if the chips are down and I'm working for free, I'm going to be working on myself.

  • Utilera 1 day ago
    AI is great as a query layer over my notes, but I don't really want it contributing to them. The whole value of old notes is that they preserve what I noticed at the time. If half the vault becomes competent-sounding generated text, searching it five years later seems much less useful...
  • chessucation 1 day ago
    i've been dabbling with various assistant models and instances and find that there's an interesting "idea" that pops up from their synthesis and relevancy analysis at a decent frequency. some kind of "inverse prompting" in which the machine has an angle, perspective, opportunity (based on the context it's working with/in) and the human turns those seedlings into something useful... or else decides its weed to be discarded.
    • HPsquared 1 day ago
      The prompter becomes the prompted!
  • layer8 1 day ago
    > I’m a strong proponent of avoiding adding lots of AI-generated summaries or other on-the-fly-generated text to my vault. The reason is simple: over time, I don’t know anymore whether the content was written by me or by an AI

    The reason to not add it is that the quality of summarization will likely only in prove in the future, so it’s better to do it on demand whenever needed.

  • sva_ 1 day ago
    The title is editorialized. It is "Keep AI Out of Your (Obsidian) Vault".

    For the Original title, I wish they added the qualifier "generative" for AI. You may well use e.g. embeddings to semantically search through your notes, and there are only gains in that.

    Having an AI write notes for you is however of course completely pointless. The act of writing notes is 80% of the reward.

    • copperx 1 day ago
      > Having an AI write notes for you is however of course completely pointless. The act of writing notes is 80% of the reward.

      The exception is transcriptions and summaries of transcriptions for things like lectures; I find those useful even when I know writing them myself would be better.

      • Fomite 1 day ago
        Yeah - I don't want AI writing "notes" in the academic sense, but minutes.

        Of course the problem with this for me (I'm a state employee) is that said transcript is a public record.

  • rootsudo 1 day ago
    For me it’s the opposite, AI has helped me do the final 20% that I could never do, making execution more painless and more successful.
    • avgDev 1 day ago
      I've found it very helpful with flushing out bugs, reviewing my code. I have a hard time trusting it to generate 100+ lines of code because in the end I am responsible for it.
  • GPerson 1 day ago
    I have separate directories for human only files and ai files. I’m not doing anything that matters but it works for me.
  • zazuke 1 day ago
    While I use Claude a lot for my Omarchy plugins and Linux optimization/failure detection, I don’t use it for writing and note-taking, especially for coming up with unique ideas and driving them home by writing them through to the end.

    It's so hard to finish an idea that is not yours and is just suggested by AI.

    • lupire 1 day ago
      It's also hard to finish an idea that is mine.
  • zahlman 1 day ago
    > an idea that is not yours

    LLMs don't have sense experience, so it doesn't have real reasons to prefer one option over another when nothing has been specified in the prompt. It can only guess based on what's most represented in its training. "Ideas" are only being simulated by an RNG; the AI can "suggest" (output) them, but they don't "have" the idea either.

    If you couldn't make the decision yourself in the first place, pulling a slip of paper out of the hat only helps in the cases where the choice really didn't matter at all. Otherwise you'll look at it, frown, and still feel stuck.

  • potato66 1 day ago
    I keep two separate vaults. It's a pain but no ai writing should mix with my own. Otherwise how can I differentiate my discoveries from its
    • pizzathyme 1 day ago
      I have claude just label at the top of all the notes it's generated "Generated by Claude". I also put them into specific claude folders. I agree with the downsides of not being able to distinguish your own writing from an AI's in the future, but don't see why it's a unsolvable problem.
  • SomeonesAccount 1 day ago
    This is why I never use AI for ideas. Always my own ideas.
  • chadcmulligan 1 day ago
    what happened to ideas are cheap - execution is everything.

    As Claude said to me when I said the same thing - it's not really its idea you had to ask the right questions. I find this true in some ways - asking the right question and finding out what the AI's have hidden inside them is like panning for gold imho.

    • zahlman 1 day ago
      > what happened to ideas are cheap - execution is everything.

      It used to be that we only recognized a tiny fraction of the actual ideas involved, because most of them were intermingled with execution. We would sit there and bang out code at single-digit WPM while trying to keep block diagrams of process memory (or UML diagrams, or whatever else) in our heads, and then run the compiler and/or the tests and realize we'd gone wrong anyway.

      Now we have lightning-fast "execution" devoid of real insight (just making choices more or less arbitrarily as they come up) because the LLM can do more and more without human intervention, but for some tasks it goes off the rails because the insight actually is necessary.

      • chadcmulligan 1 day ago
        I never really bought into it, I still think ideas are hard, and execution may be a little easier, but still hard to. The lack of thousands of amazing new startups since AI sort of validates this.
  • nicman23 23 hours ago
    why would you want ai to suggest high level ideas ?
  • andai 1 day ago
    >I don’t know anymore whether the content was written by me or by an AI

    Seriously? If so, which model produces actually human sounding output?

  • mohamedkoubaa 1 day ago
    Brian, CTO complains that it's so hard for him to execute against the technical strategy communicated by that smart intern on the third day of his internship two years ago. Poor Brian.
  • empath75 1 day ago
    I think having AI keep it's _own_ notes in Obsidian is a good idea because it makes them convenient for you to read, but yeah you need to quarantine agent ideas from your own ideas because otherwise you get bad cito-genesis problems. https://xkcd.com/978/
    • mrtesthah 1 day ago
      This might work to keep work separate:

      https://github.com/eighttrigrams/us-vs-them

    • lupire 1 day ago
      I don't understand. Are you saying that AI should write notes but then they should be excluded from the AI's context?
      • swiftcoder 1 day ago
        Two different knowledge bases. One to store AI's "learned" context, the other for you to manually edit.
      • empath75 1 day ago
        You need provenance for the notes, the AI can't tell the difference between its own code comments in code and team decisions.
  • Tim_yanxi 1 day ago
    [flagged]
  • jpardilla 1 day ago
    [flagged]
  • megous 1 day ago
    [dead]
  • jheriko 1 day ago
    [dead]