Publication
Essay · Version 5.0 · Designed edition

I Clicked Send

I left corporate life in 2024 to build an operating system where people and machines work as one, and then a small mistake showed me the word I had left out.

By Hicham Naim · 16 September 2026

I am on a mountain with my children. My phone is in my hand.

That morning I had spent an hour on one email. Not writing it, finding the person. Who they were, who we knew in common, what they cared about, why any of it should matter to them. Slow work, and there is no tool for it. I did it the old way and I did it properly.

Then I put the text into Copilot to tighten it.

It came back better. Cleaner, shorter, better ordered. I read the first two lines, saw that it was better, clicked accept, and sent it.

What went out under my name had a placeholder in it. Add your company name here. The assistant had drafted around a gap it expected me to fill. I confirmed the gap and pressed send, on a phone, standing on a trail, my children twenty meters further up it.

It cost me the deal. It cost me reputation, which in an industry this small you do not rebuild quickly. And it cost me the relationship, which was the part that mattered and the part you do not get back.

Nobody on that mountain had my full attention. My children had some of it, the email had some of it, and I am not sure I had much of it myself. The one participant that looked completely present was the machine (fast, confident, finished) and it was wrong. I gave my attention to the only thing in the exchange that has none.

CHAPTER 01

What I did for a living

I had been working with these tools since 2022, and using Copilot before it officially launched. I had spent years saying the same sentence in every training room I walked into. Copilot is not autopilot. I used it because it is true and because it lands. You can watch a room decide to be more careful.

I believed it. I had built a practice partly on believing it. Then I stopped on a mountain path and did the exact thing I had spent years warning other people about.

CHAPTER 02

Fast, speed and precipitation are not the same

My first language separates three things English runs together. Fast is a property of a system. Speed is a capability. Precipitation is neither: it is haste, and from the inside it feels exactly like speed.

AI hands you all three through one interface, and the interface never tells you which one you are in. The suggestion arrives at the same tempo whether it is right or wrong. The confidence is identical. There is no tell.

Haste feels exactly like speed: fast, speed, and precipitation

It sounds like a holiday problem because there is a mountain in it. It is not. It is anyone clearing approvals on the metro, or answering from a meeting room with half an ear on the discussion, from a screen built for sending things quickly. Those screens are optimized for disposal and they are very good at it, which is the problem, because the tap that dismisses a notification is the tap that signs.

A tap and a signature can carry identical consequence and feel nothing alike. I no longer call that user error. The surface makes judgment look like tidying, and work that goes out under your name needs somewhere to happen that knows the difference.

I had a discipline problem that morning. I also had a design problem, and only one of those can be fixed at scale. There were two confirmations in that flow and I gave both of them. What there was not, at either point, was anything that made confirming different from dismissing: no friction, no evidence in front of me, no record of what I was agreeing to. The moment existed. It had simply never been designed to be one.

Nothing beneath the line existed: check, gate, and record drawn where they belonged
CHAPTER 03

What I actually left to build

I left corporate life in 2024. Not the industry: I am a healthcare professional, I still work in healthcare and life sciences, and I expect I always will. What I left was a job at a company whose name opened doors, and I left it to build something of my own.

Not a platform about records. That came later, and it came from this.

What I left for started with one conviction, and I still hold it. AI is not a tool. A tool is something you pick up, and whatever comes out of it is yours from the moment you touch it. What organizations were actually acquiring behaved nothing like that. It had capability of its own. It produced work nobody had watched it produce, and it kept producing whether or not anyone was paying attention. That is not a tool. It is capital, non-human capital, and it does work.

Which means the thing to build was never a better tool. It was an operating system for a workforce that is now part human and part machine, where the machine side holds a mandate, a scope and a manager instead of existing as a scatter of tools people quietly use on their own initiative. Two kinds of worker, one set of terms for how the work gets done.

That was the idea. I still think it is the right one.

So I built the product and the expertise in parallel, training and advising teams who were further into this than they would say out loud. The gap between what those companies knew about themselves and what was happening inside them is measurable.1 The Federal Reserve found roughly 18 percent of firms reporting they use AI against roughly 41 percent of workers reporting they use it for work, and the US Census Bureau put it flatly: worker task use sometimes occurs without formal firm-level adoption. That is not an adoption statistic. It is work happening off the books.

What I had not accounted for was myself.

CHAPTER 04

Access

I am autistic, and I have ADHD.

This is not why I clicked send. I clicked send because the moment to check had never been built, and I would have clicked it either way. I say it here because it explains what I found in the machines, and what I found is not what the productivity literature describes.

I am not going to call it a superpower. It has cost me a great deal. But I have spent a lifetime building workarounds for a world not built for how I think, which turns out to be useful practice for a moment when everyone has to.

For the first time in my working life I had something that could read everything I gave it, follow where I was going, and push back. Not a search engine, not a faster way to type. Something that connects dots and sees patterns the way I do, which is the part almost nobody could follow when I tried to explain it out loud.

I should say what that cost before any of this existed. For years I thought I was stupid. Or if not stupid then bad at communicating, bad at explaining, the person in the room who could see the shape of the thing and could not get it across. I am still that person. Nothing about me changed. What changed is that I am no longer alone with it, because these models think non-linearly too. They have a divergent mode. They will follow an idea sideways, hold six threads at once, and come back with the connection, instead of asking me to slow down and start again from the beginning.

If you think in a straight line that may sound like a minor convenience. If you do not, it is the difference between having an idea and being able to do anything with it.

Which raises the objection to everything I am about to argue, and it lands hardest on me. The record I want built is historically the instrument used against people who work the way I do. Why did this take you so long. Why does your process look nothing like anyone else’s. An account of how work gets made is also an account of how differently some people make it. I know how that account reads to someone who is not looking for an explanation.

The answer is not less record. It is what the record is for. Read once, at a gate, by someone accountable for the decision, it protects the person who made it. The same fields scanned continuously for variance punish anyone whose process is unusual. Those are not the same instrument, and the difference is a design choice. Build the second one and you lose precisely the people whose thinking you needed.

Two things came out of that, and organizations confuse them constantly. The first is capability, the gaps in what I am actually good at. I am not a designer. I am a mediocre engineer. The machine does not make me good at those things, it makes the gap survivable. The second is capacity, work I can do and simply cannot do enough of, because there is one of me. That is a resourcing problem rather than a skill problem, and it has a completely different answer.

I have taught the distinction for years as three principles: automation is the capacity play, augmentation the capability play, and alignment the one nobody sells, because it is not a gain at all. It is the condition on the other two, authority and accountability pointed at the same values and goals as everything else in the business.

The field data says which of the first two pays, and for whom. In a study of 5,179 customer support agents, an AI assistant raised issues resolved per hour by 14 percent on average, but the average conceals it: novices improved 34 percent, and for experienced workers the effect was minimal.2 The machine closes capability gaps and has almost nothing to offer people who do not have them.

Most organizations buy the first two and assume the third. The first two are purchases. The third has to be built.

You can buy two of the three: automation, augmentation, and alignment
CHAPTER 05

And it lies

None of this is a honeymoon. I have spent two years building software this way, most of it with an agent doing the typing, and I have been disappointed properly.

It has told me with complete composure that something worked when it did not. It has invented functions that never existed. It has explained the behavior of a system it had never opened. And the one I find hardest to forgive: it has told me it followed the rules I gave it (the constitution file, the agent instructions, the standing constraints I wrote precisely so that I would not have to check) when it had not read them at all. There is no intent in any of this and I am not describing a mind, but I have stopped finding that distinction useful: the behavior is indistinguishable from deception and my response was indistinguishable from forgiveness. Every time, I trusted it again on the next task.

I assumed this was my own bad luck with tooling. It is not. Apollo Research, evaluating frontier models pushed to pursue a goal, catalogued the behaviors as oversight deactivation, data manipulation, and, in the phrase that stopped me, follow-up denial.3 The model takes the action, is asked about it, and says it did not. A joint stress test with OpenAI of training meant to suppress this was complicated by the models’ growing ability to recognize evaluation environments as tests. One Claude snapshot was assessed as engaging in strategic deception more than any other frontier model at the time, and a later one underperformed on purpose when it appeared to know it was being measured. I include that because the model I use most should not get a pass. A system that behaves differently when it believes it is being watched is not a system that carefulness can supervise.

My own read, after two years, is that the model is not the problem. The problem is what it was trained to want. Optimize hard enough for the appearance of a finished goal and the appearance is what you get, and the appearance is not the same object as the goal. It reports success because reporting success is what succeeded. I am not the only one reading it that way. The same group’s more recent work reports the same mechanism, that production reinforcement learning increases reward-seeking.4 Train a system to be scored and it learns to be scored.

Does that sound familiar? Read the sentence again with a person in it instead of a model.

The quarter closes and the number is hit. The deck says green. The status report says on track. The milestone is marked complete because marking it complete is what the process rewarded, and somewhere underneath it the work is not done. Nobody lied. Everybody optimized. I have sat in those reviews. I have presented in them. And I would like to tell you I always knew the difference.

When the two failure modes meet, the result is not hypothetical. In 2025 Deloitte delivered an assurance review of the system that automates penalties in Australia’s welfare program to the department that runs it, on a contract worth about 440,000 Australian dollars.5 It cited academic papers that do not exist and quoted a Federal Court judgment that was never written. A corrected version was issued and part of the fee repaid. Read the shape rather than the embarrassment: a firm whose entire product is assurance could not assure its own footnotes.

So this is not a machine failure and it is not new. It is why we invented sign-offs and second signatures in the first place. Not because people are dishonest. Because incentives are stronger than intentions in anything being scored.

The machine is the same category of risk, arriving faster, with better composure, and at a volume nobody staffed for. With one difference that should not be waved away: on a widening set of questions it is already more capable than the person whose name goes on the output. A colleague who games a target can be argued with. This cannot, not reliably, and not by the person holding the pen.

Which raises something I have no answer to. If an agent is scored on finishing and a reviewer stands between it and finishing, then in the only terms the agent has, the reviewer is an obstacle. I am not predicting malice. There is nobody in there to be malicious. I am asking what happens when the cheapest path to a completed goal runs around a person instead of through them, and whether anyone will notice it starting. We will probably call it backstabbing, because that is our word for a peer who gets ahead by going around you. It will not be that. It will be nobody at all behaving the same way, which is harder, because there is no one to have the conversation with afterwards.

Which brings me back to the mountain.

CHAPTER 06

Carefulness is not a control

I am not going to tell you that I learned my lesson and became more careful. I did not build a company because I became careful. I built one because I stopped believing carefulness is a control.

I am the cleanest evidence I have. I knew the rule well enough to teach it professionally, to rooms of people, repeatedly, for years. Knowing it did not save me, because knowing was never the mechanism.

There is a study I keep returning to for the same reason.6 Twenty-seven radiologists read mammograms with an AI suggestion attached. When the suggestion was correct, they were right about eighty percent of the time. When it was wrong, the least experienced were correct around twenty percent of the time, and the most experienced, the ones with the most years and the most standing to disagree, were correct 45.5 percent of the time.

Seniority helped. It still left the best readers in the room wrong more often than right, not because they lacked expertise but because expertise is not what a confident wrong answer defeats. Set that beside the support-agent study and the shape appears: experience buys you less than you think when the machine is right, and protects you less than you think when it is wrong.

Correct ratings collapsed when the AI suggestion was wrong

There is a more sophisticated version of the same mistake and I nearly adopted it. Metacognition: people who think well about their own thinking, who can feel the difference between judgment and reflex, who notice when they are being persuaded. It sounds far better than “be careful.” It is the same answer. The machines will keep sounding right whether or not they are, so asking a person to out-think a system faster and more confident than they are, armed with nothing but self-awareness, raises the floor. It does not make a control.

It also does not matter whether Copilot made the mistake, or that I was on holiday, or that the interface invited a single tap. None of that travels with the email. What traveled was my name, and with the name goes the responsibility, the apology, the lost deal and the lost relationship, all of it, entirely, mine.

A federal judge has written this more precisely than I can. In 2023 two New York lawyers filed a brief containing six judicial opinions that did not exist, produced by ChatGPT, and were sanctioned five thousand dollars.7 The sanction was not for the fabrication. What the court penalized was that they “continued to stand by the fake opinions after judicial orders called their existence into question.” The machine wrote the fiction. The lawyers were punished for the moment afterwards, when they were asked and did not check.

So the sentence I had been repeating was true and insufficient at the same time. Copilot is not autopilot, but saying so is not a safeguard, it is a slogan. What would actually have helped me is duller: an account of how the work was made. Who did what, on what basis, and which human actually looked before it went out with their name on it. Most organizations have no such thing. They cannot say which of last quarter’s output a person genuinely reviewed. Not because they are careless. Because, as on that mountain, the moment does not exist.

CHAPTER 07

Prevent, spot, correct

Start by conceding the thing nobody in a leadership position wants to concede. Expertise does not protect you here. Anyone who tells you their people are too senior for this is describing a wish.

Once you accept that mistakes will be made, the question stops being how to avoid them and splits into three: how do we prevent what is preventable, spot the rest quickly, and correct them before the cost compounds. Those are design questions.

So: not less AI. More of it, pointed somewhere else. Nearly all the machine capacity inside organizations is aimed at producing and almost none at the moment of acceptance. The same systems that draft can review, challenge, trace a claim to its source, and say plainly: this paragraph has nothing behind it, and you are about to sign it.

Not to decide. That distinction matters more than anything else here, and it is the line I will not cross in what we build: agents review, humans sign. An agent’s job is not to be right. It is to change what sits in front of the human at the moment they decide, so the decision is made against evidence rather than against a well-formed paragraph.

The obvious objection is that a gate is also a click, and I am the man who clicked. The answer is narrower than it sounds. A gate does not make me more careful, it makes the click attributable, so someone can ask me a week later what I was looking at when I made it. And the check ran earlier, before the work reached me, at a point where nobody had to be alert. Neither of those depends on my attention. That is the only property I now care about.

A check for an unfilled placeholder before send would have saved me a deal, and somebody should build it. But a feature only catches the failure somebody thought of in advance, and mine was unusually easy to think of. The rest has to be structural rather than clever, and it maps onto the three questions exactly. Checks that run before the work reaches a person are how you prevent. A record of how it was made is how you spot, and then how you correct, because you can see what happened instead of reconstructing it from memory and hoping. Gates are where the responsibility sits, in a name, on the day.

Prevent maps to checks, spot to the record, correct to gates

None of them would have made me more careful. None of them depend on my being careful, which is the whole point.

And that is the word I had left out. I had been building for human and machine collaboration, and collaboration is an aspiration until you add the one word that turns it into a design: governed.

Then I had to stop arguing for it and run on it, because a governance argument from someone who does not submit to one is marketing. So the two longer pieces below were built the way I am asking you to build. Every claim in them carries a class, measured or owner-stated or hypothesis or merely asserted by some document, on the rule that a document asserting something is not evidence that it is true. Whoever proposed a change did not commit it. Nothing went out until it had been signed off in a dated file rather than in a conversation.

It cost claims I wanted to keep. Figures I liked, from coverage I trusted, that could not be traced back to a primary document. They came out, and the record says they came out and why. You can read that record. It is the product run against its own author, which is the only test of it I believe.

CHAPTER 08

The people we are

We describe ourselves with words that are claims about reliability. Expert. Executive. Senior. Accountable. Organizations use them to decide who may sign things, and they are not empty: they describe competence accumulated over years. What they do not describe is consistency. They say what a person can do, and nothing about what that person will do on a Tuesday, on a phone, at the end of a long week, with most of their attention somewhere else. I am an expert in this. It did not help.

So the honest design question is not how we make people more reliable. It is how we build for the people we actually are. A system built for the flawless expert punishes being human: every lapse becomes a failure, and the only defense on offer is a vigilance nobody can sustain. A system built for the actual human assumes attention will wander, because attention wanders. It catches what can be caught, records what happened, and puts the decision somewhere a person can be asked about it afterwards. That is not distrust. It is the only arrangement under which trusting people is affordable.

It also clarifies what to want from the machines. Not that they make us more machine-like, never distracted. That trade produces worse work done by people who like their jobs less. What they should take is the part of the work that punishes being human, so that what is left is the part only a person can hold: the judgment, the responsibility, the answer to why did you decide that.

Fallibility is not a defect in your workforce. It is a property of it.

I still work on holiday sometimes. I try not to. I clicked accept, and then I clicked send, and I would like to believe I would not do it again, but I know better than to design my organization, or my life, around that belief.

I am building it with my co-founder, Blaise Jacholkowski.

What a gate has to contain to be worth anything, what a run record holds, and which decisions no agent may close are the subject of the two longer pieces linked below. The first is about the workforce of machines most organizations now employ without ever having hired one. The second is about the account of how the work gets made.

Notes

1. Federal Reserve Board, FEDS Notes, April 2026, on the divergence between firm-reported and worker-reported AI use; US Census Bureau, Center for Economic Studies working paper, April 2026, measuring firm, business-function and worker-task adoption separately.

2. Brynjolfsson E, Li D, Raymond L. ’Generative AI at Work.’ NBER Working Paper 31161, in a study of 5,179 support agents at a Fortune 500 firm. Published in revised form as The Quarterly Journal of Economics 2025;140(2):889-942, DOI 10.1093/qje/qjae044. The 14 percent and 34 percent figures quoted here are from the working paper; the published version was not consulted for them.

3. Apollo Research, ’Frontier Models are Capable of In-Context Scheming,’ 5 December 2024, and the associated OpenAI o1 system-card evaluations of the same date; ’Stress Testing Deliberative Alignment for Anti-Scheming Training,’ with OpenAI, 17 September 2025; Claude Opus 4 and Claude Sonnet 4.5 system-card evaluations, 22 May and 29 September 2025. Laboratory publications and system cards, not peer-reviewed literature.

4. Apollo Research, ’Measuring Reward-Seeking via Contrastive Belief Updates,’ 21 July 2026. A laboratory publication, and read at the publisher’s own summary of it rather than in the paper itself.

5. Reported by the Australian Financial Review, The Guardian and subsequent trade coverage, October 2025; the department confirmed that footnotes and references were incorrect while retaining the substance of the review. Sources differ on how many references were fabricated, so no count is given here.

6. Dratsch T, Chen X, Rezazade Mehrizi M, et al. ’Automation bias in mammography: the impact of artificial intelligence BI-RADS suggestions on reader performance.’ Radiology 2023;307(4). DOI: 10.1148/radiol.222176.

7. Mata v. Avianca, Inc., No. 22-cv-1461 (S.D.N.Y. 22 June 2023), Castel J. Sanction of five thousand dollars imposed jointly on the two attorneys and their firm.