By day I’m the CTO at a 40+ year old software company that makes an on-premises ERP built on Microsoft technology. Millions of lines of code, different generations of technology, lots of implicit functionality rather than external requirements. This is a common state for historic successful projects to end in, but it still makes extending and supporting the product feel like a complex endeavour.

Some time last year I parked as many of my day-to-day responsibilities as I could and went deep into LLM coding. I had kept an eye on the growth of LLMs from novelty to psychedelic hallucinator to surprisingly competent fabricator of words and images. In the early 2020s I was doing Advent of Code problems with ChatGPT and then Cursor, and tracking their progress that way.

Something felt Good Enough last year. It wasn’t perfect, but I smelled a There there. In the second half of the year I found Jesse Vincent and Harper Reed, and found that a lot of my Gen X friends who had been doing frontier things in 2005 Open Source were now doing frontier things in 2025 AI. That accelerated me enormously – a peer group! I’m not the only Gay in the Village!

There’s a progression you go through and things you learn as you play with these tools. I feel like most everyone has the same first experiences and lessons, and we might diverge after that but there are some bases that are relatively universal.

We all start by “Old Man Shouts At Claude” prompting – you fire up an interactive tool and have a big Chat session to build and tweak and “no no, not like that”. From then on you’re learning:

  1. You can go further if you do more work up front. Write a big plan and then give THAT to the agent instead of “build me a word processor”.

  2. Use Superpowers [https://github.com/obra/superpowers] or your agent’s Plan mode equivalent to help you build that big prompt. Now you’re doing multiphase work: you prompt to have the agent build a problem description prompt, which another agent will use to build the implementation prompt. Eventually you even realise it’s worth reading the plan, not just agreeing to it, because it’s so much easier to have the agent build the right thing than to work with the agent to refocus/rewrite something that is Not Quite Right.

  3. You can go further if you give the agent the ability to compile its work, run its work, and see the output of its work/interact with it. It’s magic when you get Playwight or the Chrome Superpowers running and Claude controls your browser to test and debug a webapp it wrote.

  4. You start to pay attention to context window because you notice your agent making mistakes as you stuff more into the chat. Compaction is like a lobotomy. Write handoffs! Delegate to subagents to make this session go further!

  5. You build skills to prevent repeated mistakes in process or knowledge, and tell Claude to “remember!”.

  6. You start to really internalise “LLMs are non-deterministic” not just as “ignorant person uses this as a catchphrase to hold onto their anti-AI beliefs” but also as “a prompt is not a program: some of the times that it’s read, it will be misunderstood or not followed.” So you write scripts to put processes into.

  7. Then you experiment with tools that try to enforce workflows. Three I’ve used:

    • workgraph/wg where the process the agent should follow is in data-structures and actual code, and wg calls your coding agent for each step. This inversion of control brings determinism to the game! Very flexible workflows.
    • metaswarm gives you an opinionated development process and set of agent skills to support it. Workflows are text, supported by code. There’s a self-learning system to improve things over time.
    • spacedock gives you a Captain to talk with, who then fires off Ensigns to do the work. Workflows are text, supported by code.

  8. You have so many sessions going on that you need to tooling to support the volume of work:

    • Something like agentsview to search and display past agent sessions.
    • Worktrees to support multiple agents in the same codebase.
    • Something Claude Remote Control to let you leave your computer and still prompt your agents.
    • Multiple terminal windows is a crappy way to keep track of what’s currently running. Some people have built their own multi-agent managers.
    • I also use kata to make notes of future work (bugs and features); and roborev to provide continuous review and feedback as dev happens.

The teams at my company are all on this journey. We didn’t have a “THOU MUST!” because I’m not a dick. AI gives amazing results for greenfields work (no previous code to worry about compatibility / patterns / implications). For small projects it can do really good things. For mature repos like ours, it’s a mixed bag.

So we’ve been finding ways to get more good work from Claude in our codebase, with mixed success. When it all goes right, it’s great. But to catch it going wrong before there’s hours of wasted time requires constant attention (which is hard to sustain, not very interesting, and a challenge to parallelism).

We’ve got skills that impart knowledge about our codebase and practices. We had a lot of good documentation, and wisdom from Sharepoint and new developer training and all that. But still … it’s a challenge to get it to write code that uses our internal framework (which it wasn’t trained on) without reinventing things that are already provided.

Getting Claude to build and run and test old school Windows apps is harder and more involved than building and testing web apps. The on-prem legacy tooling makes it harder for Claude to see and do the things that would let it go further and deliver correct code. I’ve been enjoying running the agent tools on my Mac and they can do the builds on the Windows VM running in Parallels. That has been a key enabler for me.

One of the big challenges is “what’s the right thing to do?”. That’s definitely not something we’re ready to surrender to the LLM. We absorb the complexity of retail businesses, and our software has to support that complexity and support it reliably and accurately. “Right” is often not “right according to what the LLM was trained on” but “right according to what our retailers do out of sight of the LLM”. That’s context that isn’t in context.