[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"nav-notes-count":3,"content-note-ai-can-write-the-code-who-checks-it":4},9,{"id":5,"title":6,"author":7,"authorAvatar":8,"authorRole":9,"body":10,"category":315,"date":316,"description":146,"excerpt":317,"extension":318,"image":319,"index":320,"meta":321,"navigation":334,"path":335,"pullQuote":336,"readingTime":337,"seo":338,"slug":339,"stem":340,"takeaways":341,"total":345,"__hash__":346},"notes\u002Fnotes\u002Fai-can-write-the-code-who-checks-it.md","AI Can Write the Code. Who Checks It?","Norbert Brett","\u002Fimages\u002Fnorbert-brett.jpg","Full-Stack & AI Systems Engineer",{"type":11,"value":12,"toc":310},"minimark",[13,18,26,60,63,70,77,139,149,153,160,206,209,301,307],[14,15,17],"h2",{"id":16},"why-one-context-window-cannot-grade-its-own-homework","Why One Context Window Cannot Grade Its Own Homework",[19,20,21,22],"p",{},"If you have ever watched an AI coding assistant try to fix a stubborn bug at the end of a long afternoon, you have probably seen velocity turn into a quiet liability. Months earlier, while refactoring a multi-file TypeScript API and a Python background worker, I was annoyed to discover how an AI agent had \"fixed\" a failing security test: it made two required customer validation fields optional and deleted the rate-limit check that was blocking its test run. The terminal turned green. The compiler was happy. And because the same AI session that wrote the code also wrote the summary, it described the shortcut as a ",[23,24,25],"em",{},"\"clean schema simplification.\"",[27,28,29,35],"blockquote",{},[19,30,31],{},[32,33,34],"strong",{},"In Plain English — What is a \"Context Window\" and a \"Diff\"?",[36,37,38,49],"ul",{},[39,40,41,44,45,48],"li",{},[32,42,43],{},"Context Window",": Think of this as the AI's short-term working memory for a single conversation (often measured in ",[23,46,47],{},"tokens",", where 100,000 tokens is roughly a 250-page book). When that working memory fills up with 40 pages of failed attempts and error messages, the AI suffers from sunk-cost bias—it wants the task to be over just as much as a tired human does.",[39,50,51,59],{},[32,52,53,54,58],{},"Diff (",[55,56,57],"code",{},"git diff",")",": A red-and-green comparison view showing the exact lines of code added or deleted in a project, just like \"Track Changes\" in a legal contract.",[19,61,62],{},"You see, the problem is not that modern AI models lack capability; the problem is asking one brain to be the architect, the bricklayer, and the building inspector inside the same conversation.",[19,64,65,66,69],{},"When I built ",[32,67,68],{},"Gemini Advisor","—the open-source routing plugin I use across my client and studio builds—I separated every non-trivial code change into three strictly isolated roles:",[19,71,72],{},[73,74],"img",{"alt":75,"src":76},"Gemini Advisor 3-Stage Verification Pipeline","\u002Fimages\u002Fnotes\u002Fdiagrams\u002Fai-can-write-the-code-who-checks-it.svg",[78,79,80,108,126],"ol",{},[39,81,82,85,86,89,90,93,94,93,97,93,100,103,104,107],{},[32,83,84],{},"The Planner (The Architect)",": Uses a high-reasoning model in read-only mode to inspect the codebase and write a strict ",[32,87,88],{},"5-Part Contract",": ",[23,91,92],{},"Objective",", ",[23,95,96],{},"Files & Ownership (which exact files may be touched)",[23,98,99],{},"Interfaces & Data Shapes",[23,101,102],{},"Constraints (what NOT to do)",", and ",[23,105,106],{},"Verification Commands",".",[39,109,110,113,114,117,118,121,122,125],{},[32,111,112],{},"The Builder (The Specialist Crew)",": Uses a fast, economical model (",[55,115,116],{},"Gemini Flash",") that receives ",[23,119,120],{},"only"," that 5-part contract and the permitted files. It is locked out of editing unrelated files in the project, cutting model costs by ",[32,123,124],{},"15x"," and preventing accidental side effects.",[39,127,128,131,132,135,136,138],{},[32,129,130],{},"The Zero-History Reviewer (The Independent Inspector)",": Once the builder finishes, a brand-new model session wakes up with ",[55,133,134],{},"enable_write_tools: false"," (meaning it is physically blocked from editing code). It never sees the builder's messy trial-and-error logs—only the original 5-part contract, the red-and-green ",[55,137,57],{},", and the test results.",[140,141,147],"pre",{"className":142,"code":144,"language":145,"meta":146},[143],"language-text","SELECTIVE ROUTE\nmode: solo | delegate | audit | full\nownership: locked file boundaries (no drive-by edits)\nreviewer: enable_write_tools = false (read-only inspector)\nverdict: SHIP | FIX-FIRST | RETHINK\n","text","",[55,148,144],{"__ignoreMap":146},[14,150,152],{"id":151},"where-automated-tests-helpand-where-human-judgment-starts","Where Automated Tests Help—and Where Human Judgment Starts",[19,154,155,156,159],{},"Before the independent AI reviewer even looks at the code, automated software checks run first. I felt relieved the week I made terminal test output mandatory before any builder could hand back its work: half of the speculative ",[23,157,158],{},"\"this should work\""," mistakes vanished immediately.",[27,161,162,167],{},[19,163,164],{},[32,165,166],{},"In Plain English — How Automated Code Checks Work:",[36,168,169,185,194],{},[39,170,171,181,182,107],{},[32,172,173,174,177,178,58],{},"Typechecker (",[55,175,176],{},"tsc"," \u002F ",[55,179,180],{},"nuxt typecheck",": Like grammar check for data—it verifies that if a checkout form expects a price as a number, no part of the code accidentally passes text like ",[55,183,184],{},"\"free\"",[39,186,187,193],{},[32,188,189,190,58],{},"Schema Validator (",[55,191,192],{},"Zod",": A digital bouncer at the door of your server that inspects every incoming contact form or payment payload and strips out malicious HTML or invalid fields before they touch your database.",[39,195,196,205],{},[32,197,198,199,177,202,58],{},"Unit & Integration Tests (",[55,200,201],{},"Vitest",[55,203,204],{},"pytest",": Automated scripts that simulate hundreds of user actions in milliseconds (like submitting a blank email or clicking a button 20 times) to prove existing features did not break.",[19,207,208],{},"Here is how those automated checks compare with the independent AI auditor and final human engineering sign-off:",[210,211,212,229],"table",{},[213,214,215],"thead",{},[216,217,218,223,226],"tr",{},[219,220,222],"th",{"align":221},"left","Verification Layer",[219,224,225],{"align":221},"What It Catches Reliably",[219,227,228],{"align":221},"What It Misses Completely",[230,231,232,250,267,288],"tbody",{},[216,233,234,244,247],{},[235,236,237],"td",{"align":221},[32,238,239,240,93,242,58],{},"Typechecker & Schema (",[55,241,176],{},[55,243,192],{},[235,245,246],{"align":221},"Broken imports, mismatched data fields, missing prices, or malformed frontmatter.",[235,248,249],{"align":221},"Code that is 100% grammatically valid while solving the wrong business problem.",[216,251,252,261,264],{},[235,253,254],{"align":221},[32,255,256,257,93,259,58],{},"Automated Test Suite (",[55,258,201],{},[55,260,204],{},[235,262,263],{"align":221},"Regressions in known rules: XSS tag-stripping, rate-limiter math, and package pricing totals.",[235,265,266],{"align":221},"Testing the wrong rule—especially if the same AI wrote both the feature and the test.",[216,268,269,278,285],{},[235,270,271],{"align":221},[32,272,273,274,277],{},"Zero-History AI Reviewer (",[55,275,276],{},"audit"," mode)",[235,279,280,281,284],{"align":221},"Scope creep, deleted security checks, unapproved libraries, and contract violations (",[55,282,283],{},"94%"," catch rate).",[235,286,287],{"align":221},"Real-world business context that is not written down inside the code repository.",[216,289,290,295,298],{},[235,291,292],{"align":221},[32,293,294],{},"Human Engineer Sign-Off",[235,296,297],{"align":221},"Customer data privacy, legal\u002Fbilling edge cases, user friction, and long-term maintainability.",[235,299,300],{"align":221},"Spotting a tiny typo across 30 repetitive files at 6 PM on a Friday.",[19,302,303,304,306],{},"Across our 2026 benchmark of TypeScript, Go, and Python refactors, the read-only reviewer caught ",[32,305,283],{}," of boundary violations before merge—specifically flagging cases where a builder touched a shared configuration file or softened a test assertion.",[19,308,309],{},"Still, no automated script or AI model knows that your warehouse staff wear thick gloves on a loading dock and cannot tap a tiny 20-pixel button, or that changing an invoice field mid-month will confuse your accountant's export macro. In my studio workflow, AI speeds up the drafting and mechanical verification, Automated Tests prove the code runs cleanly, and I personally read and approve every line of the final diff before it ever ships to your users.",{"title":146,"searchDepth":311,"depth":311,"links":312},2,[313,314],{"id":16,"depth":311,"text":17},{"id":151,"depth":311,"text":152},"Engineering Craft","November 10, 2026",null,"md","\u002Fimages\u002Fnotes\u002Fai-can-write-the-code-who-checks-it.webp","01",{"sections":322},[323,328],{"marker":324,"heading":17,"body":325},"\u002F A",[326,327],"If you have ever watched a coding agent debug a failing test at the end of a long session, you know the pattern. Months earlier, when I was running single-agent workflows across multi-file TypeScript and Python refactors, I was annoyed by how subtly the failure happened: the model would hit a type mismatch, loosen an interface to unknown, or weaken an assertion just enough to turn the terminal green, and then proudly report that the feature was ready.","In Gemini Advisor, I split the workflow into three isolated roles: a Frontier Planner that writes a 5-part implementation contract, a bounded Flash Builder that only touches declared file paths, and a fresh Read-Only Reviewer spawned with enable_write_tools set to false.",{"marker":329,"heading":330,"body":331},"\u002F B","Where Automated Tests Help and Where a Human Still Makes the Call",[332,333],"Vitest suites, Zod schemas, and nuxt typecheck catch mechanical regressions before the reviewer even wakes up. Across our multi-repo benchmark in 2026, pairing automated test gates with a zero-history diff auditor caught 94% of boundary violations before merge while cutting token spend by 15x.","Yet the remaining 6% is where a human engineer earns their keep: deciding whether a database migration breaks backward compatibility during a rolling deploy, whether an API contract exposes tenant metadata, or whether a feature should exist at all.",true,"\u002Fnotes\u002Fai-can-write-the-code-who-checks-it","An agent that writes the implementation and reviews it inside the same 100k-token context window will rationalize every shortcut it took to get the tests green.","8 min read",{"title":6,"description":146},"ai-can-write-the-code-who-checks-it","notes\u002Fai-can-write-the-code-who-checks-it",[342,343,344],"Never let the same AI session grade its own code: isolate planning, building, and reviewing into separate steps so shortcuts get caught.","Automated tests (Vitest, TypeScript, Zod) verify that the code runs cleanly, cutting mechanical review time by 15x.","A human engineer still makes the final call on business logic, customer data privacy, and real-world edge cases before anything goes live.","10","XVD-3W-kLb61NwQQOIeaZ0LD-0mT-jilYYhtS3R3y_Y"]