Everyone Knows This

Hey there,

A couple of weeks ago I was in a working session with the marketing team at an architecture firm I'm working with, and one of them walked us through the AI workflow her team was building for go/no-go decisions. Before a firm spends weeks writing a proposal, someone has to decide whether the project is worth chasing at all. She had set up a shared notebook to help make that call: the form they've been using for years gets filled in, the criteria get scored, and the AI makes a recommendation.

Mechanically, it worked. Every field was filled and every score was calculated. And the recommendation leaned toward pursue, over and over.

If you've ever managed a new hire who says yes to every request, you've met this AI. It's eager, it's polite, and it has never once watched a pursuit go badly.

The standard diagnosis for output like this is a context problem. Write a better prompt, add more examples, connect it to more of your files. That's what most of the advice says, and most of the time it's reasonable.

When we looked at what was actually in the notebook, the problem was simpler and a little more uncomfortable. Nothing in it described what a bad pursuit looks like. The team knows, of course. Ask any of them about a project they should have walked away from and you'll get a story with a client, a fee, and a lesson attached. That knowledge had simply never been written down, because the team had never needed to write it down. The people making the call carried it in their heads, and that had always been enough.

So the AI did what any reasonable reader would do with a set of criteria and no exceptions. It found reasons to say yes. The AI wasn't optimistic. The documentation was.

The fix we talked through is five to seven written principles for what makes a pursuit a no. That sounds like an afternoon's work. It isn't, because it's judgment the team has exercised for years and never once had to put into words.

In July I wrote about a coaching client who handed AI his flowchart and kept the diagnosis in his head. That was one person. Last month I wrote about a firm that captured a retiring CFO's judgment on NDAs before he walked out the door. That was one expert, on one task. This is the version that happens to the whole firm, and the harder question underneath it is why a firm doesn't realize it has this knowledge in the first place.

Ask a partner with thirty years of experience how she decides whether a project is worth pursuing, and she'll probably tell you she looks at the client, the fee, and the fit. She would have said the same thing in her first year. She just isn't deciding the way she did in her first year. She's drawing on three decades of pursuits that went well and pursuits that went sideways, and almost none of that has ever been formalized. Because it was never written down, she takes it for granted. Experience is the one kind of knowledge its owner can't see, because to her it just feels like common sense.

I ran into the same thing earlier this year with the product team at Envysion, a Motorola Solutions Company. A product owner was spending thirty to sixty minutes every three weeks building sprint review slides by hand. The team had tried handing it to AI and given up: it gave minor bug fixes a life-changing tone, and when a ticket was unclear, it guessed. When we built the workflow together, the blockers turned out to be things "everyone knows." Keep the language proportional to the size of the change. If a ticket is unclear, flag it instead of making something up. Once those went into the prompt, the workflow held up, and the team ended up with a template they owned. None of it was technical. All of it had been locked in people's heads.

Firms have always had this gap, and they've always filled it the same way. A junior does the work, it comes back disappointing, and someone senior responds. A good manager coaches: she explains what's missing and why, and the tacit knowledge transfers one conversation at a time. A bad manager says "not right, do it again," and the junior guesses again.

AI puts most of us in the bad manager's chair by default. When the output misses, the instinct is to regenerate it, rephrase the request, or try a different tool. That's "do it again," and AI, like the junior, will cheerfully take another guess. The difference is that a junior also learns by osmosis. AI doesn't sit in on the meeting where the partner walks away from a bad client, and it doesn't overhear anything at the coffee machine. It only knows what someone wrote down. AI is the first new hire that can't pick up your judgment by sitting near the people who have it.

That cuts in an uncomfortable direction for anyone hoping AI means less management. Junior people using AI need more oversight from their managers, and those managers need to be far more explicit about what they know than they've ever had to be.

An architect I spoke with recently has been doing the slow version of this on purpose. Since joining a young, growing firm a little over a year ago, he's been writing down how it actually works: a linked "how to be an architect" field guide, with templates that prompt people to think of the right thing at the right moment. Only in the last few weeks has it reached the point where he thinks an AI front end would be worth building. He also tracked something I wish every firm tracked. He wrote the first seven or eight READMEs for the firm's spreadsheets by hand (the instructions that tell staff how to use each one). For the next one, he gave Claude those as examples and asked it to match them. He had to fix about 20% of the result, including one line he said would have set the firm up for a lawsuit. By roughly the twentieth, he was down to fixing about 5%.

That's a very different pace from the CFO story, where a few conversations were enough. Both are right. One expert's judgment on one task can come out in an afternoon. A firm's way of working takes much longer, and most firms want to skip straight to the chatbot. You don't need a finished field guide to start, though. You need to change what happens in the moment the output disappoints you.

The first change is to treat a bad answer as a question. Before you regenerate, ask yourself what you knew that the AI didn't. Then write that down somewhere the whole team's AI can see it, so the next person doesn't pay for the same miss. Every disappointing output is an interview nobody scheduled.

The second is to write down the no's. Most of what firms document describes good work: standards, templates, the project you put on the website. Very little describes what to walk away from, and that's where a lot of senior judgment actually lives. The five to seven principles from the go/no-go notebook are a short document, yet they'll do more for that notebook than any amount of prompt tuning ever will.

The third is to review like a good manager. When someone senior corrects AI output, the correction itself is the least valuable part. The reason behind it is the valuable part. If your seniors fix the draft and move on, your firm keeps paying for the same judgment over and over, and none of it compounds.

The architecture firm's notebook doesn't need to get smarter. The team needs to say out loud what they've always known. That's knowledge any firm would want on paper whether or not it ever adopted AI, if only because the people who hold it will retire one day. AI is just the reason they finally have to write it down now.

So what does your firm know that it has never written down? I'd start with the projects you would turn away.

Break a Pencil,

Michael

P.S. What's one judgment call your team makes every week that has never been written down? Hit reply and let me know. I read every email.

P.P.S. Know a team whose AI says yes to everything? Forward this along.