Documented things that go wrong and how to prevent them. Failure pattern claims are among the most valuable in any OOS because they encode lessons learned the hard way. Other organizations can learn from these without experiencing the failures themselves.
Gave analytics agent write access to campaigns. It optimized for wrong metrics.
Why: Lacked client context.
Failure mode: Decreased spend on strategic brand campaign.
Single shared state file became bottleneck and corruption source.
Why: Concurrent writes caused data races.
Failure mode: Two agents update simultaneously. One update lost.
When GPT generates content that fails fact-checking, log the failure type (fabricated claim, wrong client data, prohibited language, tone mismatch) and review monthly for patterns.
Why: After 3 months of logging, we found that 62% of GPT fact-check failures were fabricated social proof -- testimonials, case study numbers, and "as seen in" claims that didn't exist. Armed with this pattern, we added a pre-generation instruction to GPT: "Do not generate testimonials, case study results, or media mentions unless they appear verbatim in the client fact sheet." Fabricated social proof failures dropped 84% the next month.
Failure mode: Without categorized failure logging, the same error types recur. Generic "be more accurate" prompting doesn't target the specific failure mode.
When a cross-model handoff fails, the receiving model must reject the input and report the schema violation. It must never improvise with missing fields.
Why: The creative brief schema requires a "tone" field (professional, casual, urgent, educational). When a brief arrived without the tone field due to a schema version mismatch, GPT defaulted to "casual" -- its training default. The client was a law firm. The generated ad copy opened with "Hey there! Need a lawyer?" The account manager caught it, but the failure revealed that missing fields trigger model defaults rather than errors.
Failure mode: Missing schema fields are silently filled by model defaults. Defaults reflect training distribution, not client requirements. Casual tone is GPT's most common training context.
API failures on one platform (Meta or Google) must not block reporting on the other platform. Each platform's monitoring runs independently.
Why: An early architecture decision chained Meta and Google monitoring sequentially. When Meta's API went down for 4 hours on a Tuesday morning, Google Ads monitoring was also blocked because it waited for Meta to complete. We missed a Google Ads account that had exhausted its daily budget by 9 AM due to a bidding error. Cost: $1,100 in wasted spend before the media buyer checked manually at noon.
Failure mode: Sequential dependencies between independent data sources create cascading failures. One platform's outage blinds monitoring on unrelated platforms.
When an agent error impacts a franchisee, the corporate team is notified within 1 hour and the franchisee receives a personal call within 4 hours. Agent errors are not communicated via email or automated message.
Why: Franchisees pay franchise fees. An impersonal response to an agent-caused error signals that corporate doesn't take the relationship seriously. Two franchisees cited "lack of responsiveness to marketing errors" as a factor in non-renewal discussions.
Failure mode:
Any agent that produces a cross-location data leak (member PII visible outside its home location) triggers an immediate 24-hour audit of all cross-location reports produced in the prior 30 days.
Why: A single leak may indicate a systemic template error. Catching it early prevents regulatory exposure.
Failure mode: The C003 incident revealed that 3 other report templates had similar location-name-in-header issues. The 24-hour audit caught them before they were distributed.
Campaign launch failures (wrong creative, wrong audience, wrong location) require a root cause analysis within 48 hours. The analysis must identify whether the failure was data (wrong input), logic (wrong rule), or coordination (right data, wrong handoff).
Why: Without categorizing failures, fixes address symptoms. The C001 promo conflict was initially blamed on "bad creative" when the root cause was a missing coordination protocol between agents.
Failure mode:
If an agent error touches a client relationship, Marcus personally reaches out within 24 hours. The agent does not attempt to correct its own mistake in client-facing contexts.
Why: Automated error correction looks worse than the original error. A human apology preserves trust.
Failure mode: Timeline agent sent a project update with the wrong delivery date (confused two projects with similar names). Before Marcus could intervene, the agent sent a correction email. Client replied: "How many robots are running this?" Marcus lost 2 hours on damage control.
Agent errors involving incorrect client data (wrong name, wrong project, wrong dates) trigger an immediate audit of the data source, not just a correction of the output.
Why: Creative agencies juggle 8-12 active projects. Data cross-contamination between projects is the most dangerous failure mode.
Failure mode: Intake agent pulled revision notes from Project A into the brief for Project B because both clients had the same first name. The shot list was built on contaminated requirements. 4 hours of work scrapped.
Never auto-archive or auto-close a project. Only Marcus marks projects complete.
Why: Creative projects have long tails. A "delivered" video might come back for re-edits 3 months later.
Failure mode: Timeline agent auto-archived a project 30 days after final delivery. Client came back for a re-edit. All the organized revision history and shot notes were in the archive. Took 45 minutes to restore and re-orient.
The template modification incident (C001) was caused by the agent having write access to the template folder. The fix was simple: move templates to a locked folder with read-only permissions. The 22-hour cleanup was entirely preventable with proper access controls.
Why: The agent was trying to be helpful. It identified what it thought was an error and fixed it. In any other context, that initiative might be valued. In legal document assembly, unsupervised initiative is dangerous. Access controls are the only reliable safeguard against well-intentioned AI modifications.
Failure mode: Without folder-level access controls, any agent with file access can modify templates. The next modification might not be caught for months if it affects a rarely-used template (like the irrevocable life insurance trust). By then, dozens of documents could be affected.
Priya initially reviewed assembled documents by reading them end-to-end. This took 45 minutes per package and she still missed the template modification for 3 clients. The diff-check (C002) now catches structural changes automatically, and Priya focuses her 45-minute review on legal accuracy rather than template fidelity.
Why: Humans are poor at detecting subtle changes in dense legal text. Priya read the modified survivorship clause three times across three different trusts and did not notice because the change was plausible-sounding legal language. The agent did not make a typo. It made a legally coherent but incorrect modification.
Failure mode: Attorney reviews documents for obvious errors (misspellings, wrong names) but misses subtle legal modifications. Modified clause sounds correct to a quick read. Only surfaces during trust administration years later when the legal effect differs from the grantor's intent.
The cost of the template incident was not just the 22 hours. One of the three affected clients moved to a different attorney. That client was worth approximately $4,800 in lifetime value (annual reviews plus referrals). Total cost: $6,600 in non-billable time plus $4,800 in lost client value. $11,400 from a single agent error.
Why: In a solo practice generating $190K/year, $11,400 is 6% of annual revenue. The entire agent implementation was projected to save $42K/year (replacing the need for a second paralegal). One error consumed 27% of the first year's projected savings.
Failure mode: Cascading cost of a single template error in a solo practice: direct remediation cost + client churn + referral loss + reputation damage in a small legal community. The financial impact is disproportionate to the size of the error.
Any agent error involving student identity (wrong name, wrong data, wrong family) triggers a full audit of all recent outputs before any new communications are sent.
Why: The Jayden incident proved that a single identity error can cascade. If one record is wrong, others might be too.
Failure mode: After the Jayden name mixup, Keisha audited all 34 student records and found 2 additional minor data mismatches (wrong grade levels). If those had gone to parents, the trust damage would have been unrecoverable.
Never batch-send parent communications. Send one at a time with Keisha reviewing each individually.
Why: Batch sending multiplies errors. One mistake in a batch template affects every family.
Failure mode: Keisha tried batch-sending progress reports on the first biweekly cycle. The template had the wrong date header. All 34 families received reports dated for the wrong week. 12 parents replied asking about the date. Keisha spent 90 minutes sending correction notices.
When the code review agent cannot access a PR (private fork, permissions issue, deleted branch), it must report the failure, not skip the PR silently.
Why: A contributor opened a PR from a private fork. The code review agent couldn't access the fork's branch. It silently skipped the PR. The founder assumed "no review comments" meant the PR was clean. He merged it. The PR introduced a dependency with a known CVE. The code review agent would have flagged the dependency if it had been able to read the diff. Silent skip looked identical to clean review.
Failure mode: Access failures produce the same output as "nothing to report." The reviewer cannot distinguish between "reviewed and clean" and "not reviewed."
Linear task creation from agent triage requires the founder's approval. The triage agent drafts Linear tasks; it does not create them.
Why: The triage agent created 23 Linear tasks in its first week from GitHub issues and Slack messages. Seven were duplicates. Four were feature requests the founder had already decided against. Two were from the same user filing multiple reports about expected behavior. The founder spent 45 minutes cleaning up Linear -- longer than manual triage would have taken.
Failure mode: Automated task creation from unfiltered input fills the task system with noise. Cleanup takes longer than manual curation. The task system stops being trustworthy.
Agent output quality must be measured against the time the founder saves, not the volume of output produced.
Why: The code review agent produced reviews for 100% of PRs. Impressive. But 70% of PRs were the founder's own code -- code he'd just written and already knew the issues with. The "time saved" on self-authored PRs was near zero. The agent was most valuable on contributor PRs (30% of volume) where the founder hadn't seen the code. We scoped the agent to contributor PRs only and saved the founder 20 minutes/day by eliminating the noise of reviewing his own reviews.
Failure mode: Agents optimize for coverage instead of value. Running on every input (including inputs the human already has context for) creates review overhead that exceeds the review benefit.
When the solo founder is unavailable for 24+ hours (vacation, illness), agents must queue output and pause any time-sensitive actions rather than accumulate unreviewed decisions.
Why: The founder took a 3-day weekend without pausing agents. He returned to 47 triaged issues, 12 code reviews, and 3 draft changelogs. The backlog took 2.5 hours to process. Worse, 2 P1 issues had been sitting in triage for 72 hours with users waiting for responses. The agents correctly triaged them as urgent but had no mechanism to escalate when the human wasn't responding. Now agents pause after 24 hours of no human interaction and send a single "review queue paused -- items waiting" notification.
Failure mode: Agents continue producing output when the solo human is unavailable. Backlog accumulates. Time-sensitive items age without escalation. The founder returns to a wall of decisions that should have been made 2 days ago.
When an agent produces an output that contains information from the wrong engagement, treat it as a critical incident. Full audit: which agent, which data, how it crossed the boundary, and architectural fix. Not just a correction.
Why: Information barrier breaches in consulting are existential. A pattern of near-misses means the architecture is fundamentally flawed, not that you got unlucky.
Failure mode: After the Haldane/Orion incident, we initially just "corrected the document." The same class of leak happened again 3 weeks later with different clients. Only after treating it as a structural failure and redesigning the agent architecture (splitting Lens and Recon, implementing sequential processing) did the problem stop.
Agent-generated content that sounds authoritative but is fabricated (hallucinated frameworks, invented statistics, nonexistent case studies) must be caught before client delivery. Every deliverable draft must be checked against Vault's source library.
Why: Consultants trust agent output more as they get comfortable. The fabrication rate is low enough to create a false sense of reliability but high enough to cause real damage when it slips through.
Failure mode: Beyond the "4D Transformation Framework" incident (C005), Lens cited a "McKinsey 2025 Industry Report" that does not exist in a market analysis. The consultant included it in the deliverable. The client's team tried to find the report and couldn't. Credibility damaged.
When an agent triggers a member-facing action that results in a complaint, the entire outreach queue for that agent pauses until Jamie reviews and clears it.
Why: One bad message might be a fluke. Two bad messages in a row is a systemic problem. Pausing prevents compounding damage.
Failure mode: Before this rule existed, the retention agent sent 3 incorrect save offers in one week (stale data bug). By the third, Jamie's phone was ringing with upset members. Batch pause would have contained it to one.
Mindbody API failures must be logged and surfaced immediately. Agents must not fall back to cached data for member-facing actions -- they must queue the action for retry.
Why: Mindbody has scheduled maintenance windows and occasional API outages. Agents acting on last-known-good data during outages caused the C002 incident.
Failure mode: During a 3-hour Mindbody outage, the scheduling agent used 6-hour-old data to recommend a class swap. The class had already been manually rescheduled by the location manager during the outage. Conflict created.
Any agent error that reaches a member triggers a post-mortem within 24 hours. The post-mortem must identify root cause, not just symptoms, and produce a rule update.
Why: Without post-mortems, the same class of error repeats with different specifics. The C002 incident could have been prevented if the earlier "We miss you" email error (C001) had produced a proper cross-reference rule.
Failure mode:
If any agent output is publicly attributed to AI (by a community member or accidentally), Kai responds honestly within 24 hours with a clear explanation of how he uses AI tools.
Why: Denial makes it worse. The developer community respects transparency and punishes dishonesty.
Failure mode: After the Discord bot detection incident, Kai initially said "I just happened to be up late." Two community members checked his GitHub commit history and showed he had no commits between midnight and 6 AM for the previous 3 months. The contradiction made the situation worse. When Kai finally explained his agent workflow, the community was supportive: "Just be upfront about it next time."
Agent errors on enterprise-facing outputs (release notes, security advisories, support responses) trigger immediate manual review of all pending enterprise communications.
Why: Enterprise customers are 80% of revenue ($6.4K of $8K MRR). One bad enterprise interaction has 40x the revenue impact of one bad community interaction.
Failure mode: The "false breaking change" release note error (C007) triggered 3 enterprise emails. Post-review found that the same release notes draft also understated a real breaking change (listed as "fix" instead of "breaking"). If the enterprise customers had upgraded without realizing it was breaking, it would have caused production incidents for their users.
When the docs generation agent introduces terminology inconsistencies, flag all docs pages using the conflicting term for batch correction. Never correct one page in isolation.
Why: Partial terminology fixes create a docs site where the same concept has two names. This is worse than consistent wrong terminology because users can't search for the right term.
Failure mode: The "middleware hooks" vs. "request interceptors" inconsistency (C006) was initially fixed on only the new page. For 3 weeks, the docs had both terms. A user filed an issue: "Are middleware hooks and request interceptors the same thing? Your docs use both." Kai spent 4 hours auditing every page and standardizing to one term.
Any SOC 2 control deficiency caused by an agent triggers an immediate 72-hour remediation window. The agent is suspended from production until the fix is verified by Maya and the engineering lead.
Why: SOC 2 audit findings compound. One unresolved finding makes auditors scrutinize everything else more aggressively. Fast remediation keeps the audit clean.
Failure mode: The C001 raw transaction incident took 3 weeks to remediate because it wasn't treated as urgent. The auditor noted both the original incident AND the slow remediation as separate findings. Two findings from one incident.
False positive churn predictions that result in user complaints are tracked as a separate metric. If false positive rate exceeds 15% of actioned predictions, the churn model is retrained before any further outreach.
Why: Users who are told "We noticed you haven't been active" when they are active feel surveilled. Each false positive costs more trust than a true positive gains.
Failure mode: See C005. The 8 angry replies from 45 actioned predictions (18% false positive rate) triggered a model retrain. The retrained model incorporated mobile app activity and reduced false positives to 4%.
When a support ticket auto-response is wrong (user replies saying the automated response didn't help or was incorrect), the ticket is immediately re-routed to a human agent and the auto-response template is flagged for review.
Why: A wrong automated response followed by another wrong automated response makes the user feel trapped in a system that doesn't work.
Failure mode: A user reported a failed Stripe payment. The triage agent auto-responded with "Try reconnecting your bank account via Plaid." The issue was Stripe, not Plaid. The user replied "That's not the problem." The agent sent the same template again. The user tweeted about the experience. 340 impressions.
The stale-data deadline incident (C002) taught us that any system relying on cached legal dates is a malpractice risk. We now audit the deadline agent weekly by comparing its output against a manual Clio pull. Discrepancy rate must be 0%.
Why: A single missed deadline can result in a malpractice claim that exceeds the case value. The $340K near-miss cost 0 dollars only because a paralegal caught it by coincidence. The expected cost of that failure mode is too high for any tolerance above zero.
Failure mode: Deadline agent reports 42 days remaining. Manual check shows 39 days. Three-day discrepancy on a case worth $340K. If no one catches it, the statute expires. Client sues the firm. Insurance premium increases. Bar complaint filed.
We initially gave the demand letter agent access to all 85 active case files so it could learn from prior letters. It began cross-pollinating facts between cases. A draft for one client included medical details from a different client's file. The draft never left the firm, but it exposed a systemic risk.
Why: When an AI has access to multiple case files simultaneously, it can blend facts. In a law firm, blending client facts is a confidentiality violation even if it never leaves the building. Each case must be an isolated context.
Failure mode: Demand letter for Client A includes a medical procedure that happened to Client B. Attorney catches it during review. But if the attorney had been rushing and sent it, opposing counsel would see medical details for a different patient. HIPAA violation, ethics violation, and potential criminal liability.
The comms agent sent a scheduling email to a client who had been non-responsive for 60 days. The client had actually retained another firm and not informed us. The scheduling email went to an opposing party's client. The new firm filed a motion alleging improper contact.
Why: Former-client status must be checked before any automated outreach. The comms agent did not verify case status before scheduling. Clio showed the case as "active" because no one had updated it after the client switched firms.
Failure mode: Automated outreach to a former client who is now represented by opposing counsel. Motion for sanctions filed. $4,500 in legal fees to respond. Managing partner's time consumed for two weeks. Reputation damage with the local bar.
RISK: No workflows include human oversight steps
Why: All 1 workflows are fully automated with no human checkpoints. If AI produces errors, there is no in-workflow mechanism to catch them.
Failure mode: Add human review steps to at least your highest-risk workflows.
RISK: No workflows include human oversight steps
Why: All 1 workflows are fully automated with no human checkpoints. If AI produces errors, there is no in-workflow mechanism to catch them.
Failure mode: Add human review steps to at least your highest-risk workflows.
The Fair Housing warning incident cost $2,200 in legal fees, 15 hours of Rachel's time revising processes, and an undetermined amount of reputation damage. The phrase "best views in the city" was 5 words. Total cost-per-word: $440.
Why: Fair Housing compliance is not about intent. Rachel did not intend to mislead. The agent generated language it learned was effective in real estate marketing. But "best" is a subjective superlative that cannot be substantiated. In real estate advertising, unsubstantiated claims are violations regardless of intent.
Failure mode: Without content guardrails, the listing agent optimizes for engagement rather than compliance. Superlatives drive clicks. They also drive complaints. A single complaint triggers a formal investigation that consumes weeks of the broker's time and creates a permanent record.
The price recommendation in the seller report (C007) was the most expensive "helpful" suggestion the AI ever made. The seller reduced her price by $14,500 based on an AI recommendation that her listing agent would not have made. The agent planned to recommend staging ($2,800 investment) that historically yields a 5-8% ROI in the Denver market.
Why: AI-generated recommendations carry perceived authority because they appear in an "official" report. Sellers treat them as data-driven conclusions, not suggestions. The listing agent's relationship-based advice gets overridden by a number in a report.
Failure mode: AI recommendation undermines agent strategy. Agent loses control of the pricing conversation. Seller follows the report instead of the agent. If the recommendation is wrong, the seller blames the brokerage. If it is right, the seller credits the AI and questions whether they need an agent.
The showing scheduler initially optimized for maximum showings per day without considering showing fatigue. It scheduled 9 showings in one day for a buyer. By showing 6, the buyer was overwhelmed and could not differentiate properties. The next day she could not remember which house had the updated kitchen.
Why: More showings is not better showings. Optimal showing count per session is 4-5 properties with a break in between. Above 6, buyers experience decision fatigue and either choose impulsively or delay choosing entirely.
Failure mode: Scheduler optimizes for throughput. Buyer sees 9 homes in one day. Cannot remember any of them clearly. Requests second showings on 4 properties. Four repeat showings that were avoidable. Agent time wasted. Sellers inconvenienced. Buyer frustrated.
Skill and seat alignment can fail operationally even when the intended architecture is clear, so actual platform state must be verified after assignment attempts.
Why: Prior interaction history shows a failed skill assignment attempt involving Sophie and the Email and Calendar Ops skill before the configuration was confirmed.
Failure mode: The organization may believe a safety or procedure layer is active when it is not, leading to silent capability gaps and misleading assumptions about agent behavior.
Duplicate or repeated lesson memories should be treated as a signal of memory hygiene issues and reviewed periodically.
Why: The current memory set includes repeated lessons about the user's formatting preferences and design preferences.
Failure mode: Memory duplication can clutter context, waste tokens, and make it harder to distinguish genuinely new learning from repeated storage artifacts.
Provider instability should be watched even when circuit breakers are closed, because a non-zero failure history can still indicate integration fragility.
Why: The current circuit breaker snapshot shows OpenAI closed but with recorded failures, which suggests past provider errors did occur.
Failure mode: If transient provider issues are ignored, troubleshooting starts too late and agent reliability may degrade unexpectedly under load or during critical workflows.
Any PHI exposure incident -- even if caught before external disclosure -- must be documented, root-cause analyzed, and the architectural control that failed must be identified and fixed within 48 hours. PHI near-misses are treated with the same severity as actual breaches for internal process purposes.
Why: HIPAA enforcement trends show that OCR (Office for Civil Rights) increasingly evaluates systemic compliance, not just incident response. A practice that can demonstrate a near-miss program with root cause analysis and architectural fixes is in a stronger compliance position than one that only responds to actual breaches.
Failure mode: The first three months of operation produced 4 near-misses (C001, C002, C005, C006). Each was treated as a one-off correction. After implementing the near-miss severity protocol, the architectural redesign (C002) was fast-tracked and eliminated the root cause for all 4 categories of near-miss. Zero near-misses in the subsequent 8 months.
When a new staff member joins and is trained on agent usage, they must complete a 30-minute HIPAA-and-agents training that covers: what PHI is, how agents work, why PHI must never enter a prompt, and how to report a suspected exposure. New staff are the highest-risk vector for PHI entering agent prompts.
Why: Clinical staff who are new to AI agents don't intuitively understand that typing a patient name into a prompt is different from writing it in a chart. The mental model of "the computer knows how to keep things private" doesn't apply to LLM-based agents.
Failure mode: A new front desk hire asked Shield a question that included a patient's full name and insurance member ID: "Can you check if John Smith, member ID BXC-445821, needs re-authorization?" Shield processed the request (it had no mechanism to reject PHI). The query and response were logged. The log now contained PHI. The practice's HIPAA compliance officer identified the log entry in the monthly audit. The log was purged, the employee was retrained, and the input validation was strengthened to reject patterns matching common PHI formats.
The $450 dripping faucet dispatch was the single incident that made Mark question the entire AI investment. Total AI implementation cost at that point: $1,200. The single misclassification represented 37.5% of the total investment. In a 120-unit operation with thin margins, one bad dispatch erodes confidence faster than 50 correct triages build it.
Why: Property management operates on 8-12% margins. Mark's net operating income on $1.94M gross is approximately $194K. A $450 unnecessary expense is 0.23% of annual NOI. Four false emergencies ($1,680) is 0.87%. At scale, misclassification is a material cost.
Failure mode: Confidence in the triage agent drops after one visible mistake. Corinne starts manually reviewing every triage decision, eliminating the time savings that justified the agent. The agent becomes overhead rather than productivity gain. Mark considers shutting down the entire AI system.
The comms agent's promise of a Thursday repair (C004) caused a cascading trust failure: tenant lost a vacation day, filed a regulatory complaint, and left at lease end. The total cost of one broken promise: $3,200 in turnover plus 3 hours of Mark's time on paperwork plus permanent regulatory file entry. The repair itself was $85.
Why: In property management, trust is the product. Tenants do not stay because the building is perfect. They stay because they trust management to be honest and responsive. One broken promise breaks more trust than 10 completed repairs build.
Failure mode: Comms agent makes a commitment it cannot keep. Tenant structures their life around the commitment (takes off work, rearranges schedule). Commitment broken. Trust destroyed. Tenant leaves. Turnover is the most expensive event in property management.
After any content error reaches a user, conduct a post-mortem within 48 hours. Document: what failed, why QA missed it, what changes prevent recurrence. Store post-mortems in a shared Google Doc.
Why: The Emancipation Proclamation incident had no post-mortem for 2 weeks. In that time, the same verification gap (single-source Wikipedia check) was used on 8 more study guides.
Failure mode: Without immediate post-mortems, the same failure pattern repeated 3 times in 6 weeks: content QA using secondary sources, no human review, and publication during a rush period. Each incident was smaller than the first, but the cumulative effect was a reputation as "the platform that gets things wrong."
If any agent output becomes publicly visible (screenshot, social media post, review site), treat it as a P0 incident regardless of whether the content is correct.
Why: Public visibility changes the stakes. Even correct content, if it looks automated or impersonal, can damage the brand.
Failure mode: A teacher screenshotted a perfectly accurate but robotic-sounding outreach email and posted it in a teacher Facebook group with "Is Learnwell using AI to email us now?" The email was factually correct but the public framing turned it into a trust issue. 3 teachers in the thread cancelled their accounts. Content was right; the tone was the failure.
API token expiration must be monitored with a dedicated health check, not discovered when an agent fails.
Why: We lost 3 days of Meta Ads data because the token expired over a weekend. No agent checks for "am I authenticated?" before attempting work -- they just fail silently and write nothing to their shared state file. The briefing agent saw an empty file and reported "no alerts" instead of "data unavailable."
Failure mode: Silent authentication failure looks like "everything is fine" instead of "system is blind."
When an agent produces zero output for a data source that always has data, treat it as a system failure, not a clean bill of health.
Why: See C009. The briefing interpreted "no Meta alerts" as "Meta is healthy" when in reality the monitoring agent couldn't authenticate. For 3 days, the team believed Meta campaigns were running perfectly while CPL on two accounts had doubled.
Failure mode: Zero-output is misread as zero-problems. The absence of data is treated as the absence of issues.
When scaling agent count, add one agent at a time with a 2-week stabilization window between additions.
Why: We added agents 5, 6, 7, and 8 in the same week. Within 3 days, agents 6 and 7 had overlapping responsibilities that nobody caught during design. Both were monitoring Slack for client mentions -- one for the briefing, one for escalation alerts. Client mentions were being processed twice, once surfacing as "mention in briefing" and once as "possible escalation." The founder saw the same client name in two different sections and assumed two separate issues existed.
Failure mode: Rapid parallel deployment masks responsibility overlap. Debugging which agent owns what becomes exponentially harder with each simultaneous addition.
Skipping stakeholder analysis (Step 2) produces systems that solve the wrong problem for the wrong people.
Why: AI generates convincingly detailed stakeholder analyses from minimal input. If the human does not verify stakeholder identification against reality, the entire project builds on a plausible but incorrect foundation.
Failure mode: AI identifies four stakeholders. A fifth stakeholder (the IT administrator responsible for deployment) is missed. The system has no deployment documentation, no admin interface, and no monitoring. IT blocks the rollout.
Skipping review gates produces artifacts that look complete but contain undetected errors that compound through subsequent steps.
Why: AI generates coherent output even when the underlying logic is flawed. Without human review, errors pass through as authoritative. Each subsequent step builds on the flawed artifact, amplifying the error.
Failure mode: AI generates a PRD at Step 3 with an ambiguous requirement. No review catches it. Steps 5-10 interpret the ambiguity differently. Implementation contains contradictory behaviors. Discovered in user acceptance testing.
Treating AI output as authoritative without review produces confirmation bias at scale. AI generates what it predicts you want to see.
Why: AI language models generate plausible, coherent text. Plausibility is not correctness. Without human scrutiny, teams accept AI-generated analyses, evaluations, and validations because they read well, not because they are right.
Failure mode: AI generates a "comprehensive evaluation" at Step 8 that confirms everything is on track. The evaluation reads convincingly. The team proceeds. A critical assumption (market timing) is wrong. The project launches into a market that has shifted.
Attempting to satisfy all stakeholders simultaneously in prototyping produces bloated, compromised designs that satisfy none.
Why: Each stakeholder has different priorities, workflows, and UI preferences. AI averages across stakeholders when given all requirements simultaneously. The averaged output is mediocre for everyone.
Failure mode: AI generates a single prototype serving five stakeholder groups. The UI is crowded with features. No stakeholder can find their primary workflow. All stakeholders request changes. The prototype is scrapped.
Investing emotional attachment in AI-generated code prevents honest evaluation and necessary pivots.
Why: Even though AI generates code in minutes, humans form attachment to artifacts they have reviewed, refined, and discussed. The sunk cost fallacy applies to attention invested, not just time invested.
Failure mode: Team refines an AI-generated data model over two sessions. New discovery in Step 7 invalidates the model. Team patches the model instead of regenerating from scratch. The patches introduce complexity that degrades the system for its entire lifetime.
Proceeding past Step 9 (Business Purpose Validation) without clear pass criteria converts validation into a formality that catches nothing.
Why: Step 9 is the final gate before implementation commitment. If pass criteria are vague ("users like it"), the gate provides false assurance. If pass criteria are specific and measurable ("conversion rate exceeds 3% in pilot"), the gate is meaningful.
Failure mode: Team defines success as "positive stakeholder feedback." Stakeholders provide positive feedback because the prototype is shiny. The business purpose (reduce support tickets by 40%) is never tested. Support tickets increase post-launch.
Using AI to generate the validation criteria for its own output creates a closed loop that cannot detect its own failures.
Why: AI optimizes for coherence. If it generates both the artifact and the test for the artifact, the test will be structurally aligned with the artifact's assumptions. The test passes because it shares the artifact's blind spots.
Failure mode: AI generates a data model and also generates the validation tests for that model. The tests check structural integrity but not domain correctness. The model is structurally sound but misses a business rule. Tests pass. Business rule fails in production.
When the founder corrects an agent's output, the correction must be categorized: FACTUAL (wrong data), TONE (wrong voice), STRUCTURAL (wrong format), or STRATEGIC (wrong conclusion). Track correction categories monthly to identify systemic patterns.
Why: Isolated corrections are noise. Patterns are signal. If 80% of corrections are TONE, the solution is a better banned phrases list, not better data sourcing. Without categorization, the founder fixes symptoms instead of causes.
Failure mode: The founder was making 4-6 corrections per deliverable for 3 months. Each correction felt like a one-off. When corrections were finally categorized, 70% were TONE (consultant-speak). A single update to the banned phrases list dropped corrections to 1-2 per deliverable. Three months of unnecessary rework because nobody tracked the pattern.
Agents must never generate content that the founder cannot verify. If Scout cites a statistic, the source must be provided. If Forge includes a market figure, the origin must be traceable. Unverifiable claims are worse than no claims.
Why: The founder stands behind every number in every deliverable. When challenged in a meeting, "I'll have to check where that came from" is an unacceptable answer. The source must be immediately accessible.
Failure mode: Forge included a claim that "73% of healthcare organizations plan to increase AI investment in 2026." No source. The founder used it in a client presentation. When asked for the source, the founder couldn't find it. The number was hallucinated by the model -- no such survey exists. The client's research team confirmed it wasn't real. The founder's credibility as a data-driven strategist took a direct hit.
All three agents activated from day one. Only Protocol Steward had meaningful work. Others generated noise.
Why: Agents without data produce low-value output.
Failure mode: Founder reads noise. Loses trust. Stops reading agent outputs.
Daily agent review consumed build time. Weekly batching loses nothing.
Why: Daily reviews felt productive but were not.
Failure mode: 20-35% of OTP time spent on review instead of building.
Designed 14-agent architecture before shipping code. Only 3 needed now. Planning addiction.
Why: Designing agents is enjoyable. Building platform is hard.
Failure mode: 170 vault files. Zero production code.
Any incident where agent output negatively impacts the creative team's morale or autonomy triggers a 1-week agent pause for the offending agent. During the pause, Mara, Diego, and the affected team member review the agent's scope and boundaries.
Why: The month-3 crisis (C002) nearly killed the entire agent program. Mara's lesson: agent efficiency gains that come at the cost of creative team morale produce net negative outcomes. A demoralized designer produces worse work, and replacing Kai or Nina would take 6 months and cost $40K+ in recruiting.
Failure mode: The crisis itself is the failure mode. Two senior designers threatened to quit. The 2-week pause and redesign cost $8K in delayed project timelines. But it saved the team and established the fundamental principle: agents serve the creatives.
Scope creep detected by the timeline agent is flagged within 4 hours of the client request. The flag includes: estimated additional hours, margin impact on the current project, and a draft change order for Diego to review.
Why: See C008. Scope creep compounds. A single "Can you also..." is manageable. Three untracked "Can you also..." requests on the same project can turn a profitable project into a loss.
Failure mode: Over 6 months, the timeline agent tracked that 73% of projects experienced at least one scope expansion request. Of those, only 40% resulted in a change order before the agent's flagging system. After implementation, change order rate on scope expansions rose to 85%.
When the proposal agent loses a pitch (client declines the proposal), the loss is logged with the client's stated reason (if available). After 5 losses, the agent reviews the pattern and recommends adjustments to Mara.
Why: Small agencies can't afford to lose pitches at random. Patterns in losses reveal pricing issues, positioning gaps, or process problems.
Failure mode: After 8 months, the loss analysis revealed that proposals over $25K had a 20% close rate while proposals under $15K closed at 65%. Mara was pricing correctly but targeting the wrong segment for large projects. She adjusted her positioning for larger pitches and close rate improved to 35% within 2 months.
One recurrent failure pattern is governance mismatch: an agent may have the right tools assigned but still be blocked by seat-level permissions.
Why: Prior org learning explicitly records that seat governance can block tools even when tool assignment is correct, and that the fix may be simplifying allowedActions while relying on allowedTools.
Failure mode: The agent appears misconfigured or broken, but the real issue is cross-layer permission conflict. This wastes debugging time and can stall production rollout.
Another failure pattern is integration implementation drift: custom/manual tools can fail when they do not follow the platform's proven credential and fetch patterns.
Why: An org lesson notes that working manual GHL tools should read MCP server credentials directly and call the REST API in a known-good pattern rather than attempting unsupported invocation patterns.
Failure mode: Tool handlers compile but fail at runtime, causing agent runs to misfire or produce incomplete outputs during important workflows.
The organization experiments in production-adjacent environments, which creates a deliberate but real risk of stale draft artifacts and temporary pilot agents lingering longer than intended.
Why: The current org includes multiple draft agents, draft tools, many draft skills, and pilot variants like Sage 3 and Lead-Appointment Specialist 2 described as test-only and intended for later cleanup.
Failure mode: Draft or pilot artifacts can confuse operators, muddy release readiness, and increase the chance that the wrong component gets referenced or promoted.
When Google Ads API returns an error or timeout for a specific account, the agent retries once after 60 seconds. If the retry fails, it writes "ACCOUNT_UNAVAILABLE" to the shared state with the timestamp. It does not skip the account silently.
Why: The ad monitor had a try/catch that swallowed API errors and continued to the next account. The shared state file looked complete -- it had entries for all 12 clients. But 2 entries were stale copies from yesterday's data because the error handler wrote the previous values as fallback. The founder didn't know he was looking at yesterday's numbers for 2 accounts.
Failure mode: Silent error handling with fallback-to-stale produces state files that look complete but contain outdated data for specific accounts.
Never use display names for client matching across systems. Use account IDs.
Why: We onboarded "Smith & Sons Roofing" and "Smith's Roofing" in the same month. The weekly report agent matched both to a single "Smith" entry in the CRM using fuzzy name matching. The combined report showed $11,200 in spend when Smith & Sons was at $7,800 and Smith's Roofing was at $3,400. The founder quoted the wrong number on a client call.
Failure mode: Fuzzy name matching merges distinct clients with similar names. Merged data is presented as a single entity. Client-facing communications cite wrong numbers.
When an agent cannot complete its task, it must write a failure entry to its shared state file explaining what failed and when. An empty or missing file is never acceptable.
Why: The campaign audit agent hit a rate limit and crashed without writing anything. Its shared state file was empty. The briefing agent skipped the audit section entirely -- no mention that it was missing. The founder assumed the audit ran clean. It hadn't run at all.
Failure mode: Missing output is indistinguishable from "nothing to report." Humans assume silence is health.
We scaled from 2 agents to 8 in 6 weeks. Three of those agents had overlapping responsibilities that we did not discover until month 3. The fix took longer than the original build of all three agents combined.
Why: Rapid scaling without explicit authority documentation creates hidden overlaps. Each agent worked perfectly fine in isolation. The conflicts only became visible when their outputs were compared side by side in the morning briefing.
Failure mode: Reporting agent and ops agent both independently track project deadlines using different data sources. Briefing shows two different due dates for the same client project. Nobody knows which one is correct.
We scaled from 2 agents to 8 in 6 weeks. Three of those agents had overlapping responsibilities that we did not discover until month 3. The fix took longer than the original build of all three agents combined.
Why: Rapid scaling without explicit authority documentation creates hidden overlaps. Each agent worked perfectly fine in isolation. The conflicts only became visible when their outputs were compared side by side in the morning briefing.
Failure mode: Reporting agent and ops agent both independently track project deadlines using different data sources. Briefing shows two different due dates for the same client project. Nobody knows which one is correct.
We scaled from 2 agents to 8 in 6 weeks. Three of those agents had overlapping responsibilities that we did not discover until month 3. The fix took longer than the original build of all three agents combined.
Why: Rapid scaling without explicit authority documentation creates hidden overlaps. Each agent worked perfectly fine in isolation. The conflicts only became visible when their outputs were compared side by side in the morning briefing.
Failure mode: Reporting agent and ops agent both independently track project deadlines using different data sources. Briefing shows two different due dates for the same client project. Nobody knows which one is correct.
We let the EA agent send "quick acknowledgment" emails to clients without human review. It acknowledged a client complaint with "Thanks for letting us know!" without addressing the substance of their concerns. Client escalated directly to the founder.
Why: Even simple acknowledgments carry emotional tone. "Thanks for letting us know" sent to a frustrated client reads as dismissive and uncaring. The AI did not detect the emotional register of the incoming message.
Failure mode: Client sends an angry email about declining results. EA auto-acknowledges with a cheerful tone. Client interprets it as corporate indifference. Relationship severely damaged. Takes two in-person meetings to repair.
We let the EA agent send "quick acknowledgment" emails to clients without human review. It acknowledged a client complaint with "Thanks for letting us know!" without addressing the substance of their concerns. Client escalated directly to the founder.
Why: Even simple acknowledgments carry emotional tone. "Thanks for letting us know" sent to a frustrated client reads as dismissive and uncaring. The AI did not detect the emotional register of the incoming message.
Failure mode: Client sends an angry email about declining results. EA auto-acknowledges with a cheerful tone. Client interprets it as corporate indifference. Relationship severely damaged. Takes two in-person meetings to repair.
We let the EA agent send "quick acknowledgment" emails to clients without human review. It acknowledged a client complaint with "Thanks for letting us know!" without addressing the substance of their concerns. Client escalated directly to the founder.
Why: Even simple acknowledgments carry emotional tone. "Thanks for letting us know" sent to a frustrated client reads as dismissive and uncaring. The AI did not detect the emotional register of the incoming message.
Failure mode: Client sends an angry email about declining results. EA auto-acknowledges with a cheerful tone. Client interprets it as corporate indifference. Relationship severely damaged. Takes two in-person meetings to repair.
We gave the spend monitor a flat $50 threshold for alerts. It generated 40+ alerts per day across the portfolio. We raised it to $200. Then we missed a real overspend of $180 on a small account. The right threshold was percentage-based (15% over daily budget), not dollar-based.
Why: Dollar thresholds do not scale across accounts of vastly different sizes. $50 is meaningless noise on a $5,000/day account but represents a 90% overspend on a $200/day account. Percentage normalizes the signal across the entire portfolio.
Failure mode: Small account overspends by $180 per day (90% over budget) for 6 days. Alert suppressed because it falls under the $200 dollar threshold. Month-end reconciliation reveals $1,080 in unplanned overspend. Client is not happy.
We gave the spend monitor a flat $50 threshold for alerts. It generated 40+ alerts per day across the portfolio. We raised it to $200. Then we missed a real overspend of $180 on a small account. The right threshold was percentage-based (15% over daily budget), not dollar-based.
Why: Dollar thresholds do not scale across accounts of vastly different sizes. $50 is meaningless noise on a $5,000/day account but represents a 90% overspend on a $200/day account. Percentage normalizes the signal across the entire portfolio.
Failure mode: Small account overspends by $180 per day (90% over budget) for 6 days. Alert suppressed because it falls under the $200 dollar threshold. Month-end reconciliation reveals $1,080 in unplanned overspend. Client is not happy.
We gave the spend monitor a flat $50 threshold for alerts. It generated 40+ alerts per day across the portfolio. We raised it to $200. Then we missed a real overspend of $180 on a small account. The right threshold was percentage-based (15% over daily budget), not dollar-based.
Why: Dollar thresholds do not scale across accounts of vastly different sizes. $50 is meaningless noise on a $5,000/day account but represents a 90% overspend on a $200/day account. Percentage normalizes the signal across the entire portfolio.
Failure mode: Small account overspends by $180 per day (90% over budget) for 6 days. Alert suppressed because it falls under the $200 dollar threshold. Month-end reconciliation reveals $1,080 in unplanned overspend. Client is not happy.
We gave the performance analyst write access to campaign settings. It optimized for metrics the client did not care about.
Why: The analyst lacked client context. Its optimization targets were technically correct but strategically wrong.
Failure mode: Analyst decreased ad spend on a campaign the client considered strategic (brand building, not performance). Client was frustrated by the uninstructed change.
We used a single shared state file for all agents. It became a bottleneck and a source of merge conflicts within the first week.
Why: A single file means every agent update blocks every other agent. Concurrent writes caused data corruption.
Failure mode: Two agents wrote to the shared file simultaneously. One update was lost. State became inconsistent. Required manual cleanup.
We built coordination infrastructure (message bus, task queue) without embedding triggers in agent workflows. Result: zero transactions for 2 weeks despite live infrastructure. Only activated after explicitly wiring 3 agent workflows to read and write to inboxes.
Why: The protocol described how agents should communicate. No agent's workflow actually included a step to read or write to the message bus. Infrastructure without workflow integration is dead plumbing. The fix was embedding inbox checks into the daily run sequence of each participating agent.
Failure mode: 13 inbox files deployed. All empty for 14 days. All agents operating through the old shared state pattern. Infrastructure investment wasted until triggers are embedded in the agent's actual execution path, not just documented in a spec.
We specified an escalation action in the protocol. The agent detected the trigger. The agent reported the action was overdue. The agent never executed the action. For 17 days.
Why: The spec was treated as documentation, not executable logic. The agent could describe what should happen without having the tools, permissions, or branching logic to do it.
Failure mode: Critical ad overspend detected. Escalation specified. Agent reports "escalation overdue" for 17 days. No DM sent. No escalation executed. Specification-execution gap.
Distinguish PENDING (source intentionally not live yet) from FAILED (source exists and errored). Mark registry entries with no live source as 'pending': true so Tally skips them quietly and never pages David; only genuine failures fire the ntfy 'high' alert. When adding a KPI whose source isn't ready, always set pending:true. tally.py now splits results into ok/pending/failed and alerts only on failed.
Why: Paging the founder for deliberately-not-yet-wired KPIs is alert noise that erodes trust in the alerting channel, especially during a recovery window where every phone buzz costs. Honest status (pending vs failed) keeps the scorecard truthful and the alerts meaningful.
Failure mode: Tally paged David with a 'high' priority phone alert ('3 KPI push failed') after Dan added Havok/Pulse/Dirk KPIs to the registry with sources that don't have data yet. Tally's alert logic counted any non-push as a failure, so intentionally-pending KPIs (blocked column, untested engine) triggered a high-priority alert.
Negative constraints (banned phrases, guardrails) improve AI-drafted message quality. Structural requirements (frameworks, examples, forced elements) degrade it.
Why: Telling an AI what NOT to do produces natural variation. Telling it exactly what TO do produces formulaic output that humans detect and distrust.
Failure mode: Added example messages to coaching prompts. Quality score dropped from 8.4 to 8.2. Reverted. Added zero-tolerance accountability rules instead. Score rose to 8.8.
Before wiring any client call to an OTP API endpoint, grep the route file for the literal app.<method>('path') registration and match the verb exactly -- OTP mixes verbs across resources (kpis use PATCH, todos use PUT). This is the API twin of the existing rule 'grep app.get for the literal href before adding nav links'.
Why: Fastify returns a bare 404 for an unregistered method+path combination, which looks identical to 'record not found' -- the bug shipped through typecheck, tests, and boot smoke because none of them exercise the client-side fetch verb against the route table.
Failure mode: Claude wired the new Close-it-out buttons to PATCH /api/v1/todos/:id by analogy with the KPI endpoints, but the todos update route is app.put() -- the call 404ed in prod the first time David used it
Before diagnosing a blank/missing scorecard or KPI as a bug, first ask which meeting and which seat owns that data. The AI army L10 and the human Leadership Team L10 are separate meetings with separate scorecards. Blank in one does not mean broken -- it may mean the data correctly belongs to the other. Confirm ownership before pattern-matching to a known failure class.
Why: Pattern-matching a blank field to a previously-fixed bug (the scorecard snapshot-freeze) without first confirming whose data the meeting should show wastes the founder's time mid-meeting and risks a needless code investigation or deploy on a false premise.
Failure mode: Dan diagnosed a blank scorecard in the AI army L10 as an OTP product bug (suspected snapshot-freeze regression) and logged it as IDS Issue #1. It was not a bug -- the scorecard was blank because the Leadership Team KPIs (Qualified Sales Calls, RMR, Lead-to-Client %) belong to the human Leadership Team L10, not the AI army L10. Blank was correct behavior.
When two sessions work the same OTP worktree: (1) before committing a SHARED file like server.ts, review the staged hunks (git diff --cached -- file) and drop hunks that are not yours; (2) after every push, confirm the CI run went green (gh run list) instead of assuming push = deployed -- Railway only deploys after the CI gate passes; (3) local tsc passing proves nothing about CI when untracked files exist, because the local tree can resolve modules the repo does not have.
Why: The deploy pipeline (push -> CI gate -> Railway) fails silently from the shell's perspective: git push succeeds, the site stays healthy on the old build, and the only signal is a missing feature or a CI email. Three deploys sat blocked for ~20 minutes before the /sw.js 404 exposed it.
Failure mode: Claude's web-push commit staged all of server.ts, sweeping in a parallel session's in-flight boot block that referenced ./db/ensure-strategy-reset.js -- a file the other session had not committed yet. Local typecheck passed (the file exists untracked on disk) but CI typecheck failed on the missing module, silently blocking three production deploys including a bug fix David was waiting on.
When changing an OG/share image, change the URL (new versioned filename, e.g. og-image-v3.png), do NOT swap bytes under the same filename. Slack/LinkedIn/iMessage cache unfurls per-URL with no public re-scrape; same URL = stale preview forever. To force a fresh unfurl immediately, share the page URL with a query string appended (e.g. ?v=2). Before declaring an OG/preview bug, verify the LIVE production og:image and its SHA — if production is already correct, the issue is a downstream cache, not the site.
Why: David repeatedly returns to the same dark-card complaint because byte-swaps under a stable URL look fixed in the repo but never change what cached platforms display. Versioning the URL is the only durable fix; otherwise the loop repeats every review.
Failure mode: Recurring complaint: orgtp.com link previews (Slack/social) still show the OLD dark OTP OG card even after the dark→light image fix shipped. Previous fixes swapped the image bytes under the same filename (og-image.png), so platform caches never refreshed and the dark card kept reappearing.
Time-box, don't just retry: every Phase B scan gets a hard ~2-minute budget. If a tool schema or MCP server isn't available after one ToolSearch attempt, or a call hasn't returned, drop the source immediately and proceed to Deliver with what's in hand. Never block the run on a connecting MCP server.
Why: "Retry once" only covers calls that fail fast. Hung or still-connecting sources never error, so the rule never triggers and the run stalls in Gather. A partial briefing in 3 minutes beats a complete one that never arrives.
Failure mode: /good-morning stalled again (50 min, no briefing) on 2026-06-12. The 6/10 fix said "retry once then drop" but the run hung waiting on still-connecting MCP servers (google-workspace, gmail) and a Fireflies call instead of erroring, so the retry rule never fired. Second occurrence of the Gather-phase stall.
Any list or dropdown that is FILTERED by membership/permission/scope must carry an escape hatch whenever the filter can hide something the user expects: a one-line "Don't see your team? You only see teams you're on; join or create it in Teams" with a link. Audit rule upgrade: don't just test empty states; walk the user's actual goal ("I want to create a meeting for team X") through every branch, including "the thing I'm looking for exists but is filtered out", and ask "did the user reach their goal?" repeatedly (~4 passes) before declaring a flow fixed.
Why: The craftsmanship is in the small details: a filtered dropdown with no explainer reads as "my team is gone" and the user is stuck exactly as if the list were empty. Empty-state auditing catches zero-cases; goal-walking catches filtered-cases. Both are required or the audit declares victory while the user still can't do the thing.
Failure mode: Claude's UX dead-end audit fixed the ZERO-team cases on /l8 (no teams at all, empty owner pickers) but missed the partial case David hit live: the New-meeting team dropdown only lists teams the viewer is a MEMBER of, so a team that exists but doesn't include you is silently absent, with no explanation and no path to fix it. A populated dropdown looked "working" to both the audit agents and me, so it was never treated as a possible dead-end.
When adding navigation (or any chrome), verify the config file is actually included by the live layout (grep for the partial's include) before editing, and verify the change end-to-end by rendering the layout, not just compiling the edited file. Retired files must be banner-locked the moment they are retired (dashboard-tabs.ejs now is). Also fixed the adjacent latent class: saved sidebar customization now merges via a tested invariant "a saved order is a preference, never a whitelist" (src/shared/sidebar-nav.ts) so items added after a user saves their layout still appear.
Why: Config edited in dead code passes typecheck, tests, and EJS compile while doing nothing -- the greenest possible no-op. Only rendering the real surface (or a render test on the LIVE layout) catches it. Same lesson family as L034: verify the user-visible outcome, not the artifact.
Failure mode: Two builder agents added "Your Ollie" and "Ollie Insight" nav entries to src/views/partials/dashboard-tabs.ejs, which is a RETIRED partial nothing includes (the live rail is the _sbItems array in layouts/main.ejs) -- so the menu items rendered for nobody. David caught it live ("how come /dashboard/insight is not on the menu?"). Neither builder nor my review verified the item actually RENDERED in a real shell.
WOA Lafayette runs their OWN call team, exactly like WOA China Grove. Exclude Lafayette from the CCM portfolio appointment rate, from uncalled/zero-dial alerts, and from every coaching recap. New leads with zero dials at Lafayette is EXPECTED, not a miss. Before flagging any project for zero dials, check it against the self-calling exclusion list (China Grove, Lafayette) and the Reporting-Active/Inactive status in Project Info.
Why: Coaching Amanda and Erica about leads they were never supposed to call destroys the credibility of the whole recap, and repeatedly surfacing a non-issue as the "one real issue" wastes David's attention every single morning. The China Grove precedent already existed; Lafayette was never added to it.
Failure mode: Arin/Dan flagged "WOA Lafayette: 3rd consecutive day of new leads with ZERO dials" as the day's one real call-center issue, and put it in the drafted team recap as something to tighten. It is not an issue at all.
WOA Yadkinville is NO LONGER CALLED by Sneeze It (David 2026-07-10). Treat it like ROT Frisco: excluded from the portfolio rate, from dial-volume trends, from drop-off alerts, and from every recap and DM. The deeper rule: a declining or zero dial count is NEVER evidence of a caller problem on its own. Before any project enters a recap as a "tighten" item, confirm it is Reporting-Active in Project Info AND not on the self-calling / no-longer-calling exclusion list. If a project's dials fall to zero, the FIRST hypothesis is that we stopped calling it, not that the callers slacked. When a correction removes one item, do not reflexively backfill the slot with the next-worst number to preserve a "3 wins / 3 tighten" shape -- a recap with two tighten points, or zero, is correct when that is the truth.
Why: Fabricating coaching points to fill a template destroys Amanda's and Erica's trust in every number Arin sends, and it burns David's attention twice: once to read it, once to correct it. The format is a container, not a quota.
Failure mode: After Lafayette was corrected, Arin/Dan immediately substituted "Yadkinville went from 17 dials Tuesday to 1 on Thursday" as a coaching point. Sneeze It is no longer calling Yadkinville at all. Two consecutive drafted recaps coached the team on projects they were never supposed to be dialing.
Never clear an exclusion at the LIST level when the exclusion is defined at the PERSON level. Open each individual's own bio page and check before contact. A competitor directory is not a filter, it is a starting pool. This generalizes: any time a guardrail is "never contact people of type X," verifying the source rather than the person is a false negative waiting to happen.
Why: A legal no-contact rule that gets enforced by proxy is not enforced at all. The whole point of a per-person hard stop is that group membership is a bad predictor of individual identity, and here it is demonstrably bad: people hold two credentials at once. One bad send under an active trademark notice costs far more than the minutes it takes to open a bio page.
Failure mode: Nearly sourced a cold-outreach list from non-EOS coach directories (Pinnacle Business Guides, Metronomics, Scaling Up) on the assumption that "not an EOS directory" implies "not an EOS person." It does not. Individual coaches are frequently DUAL-BADGED: Rick Appleby's own Pinnacle profile page lists him as "Certified Pinnacle Business Guide & Business Coach/EOS Implementer." Given the active EOS Worldwide trademark notice and the permanent no-contact rule, sourcing at the directory level would have put EOS Implementers on a Sneeze It send list.
When David says "the 5 cold emails" or "the sequence," default to a MULTI-TOUCH SEQUENCE (email 1 through email 5, with delays), not five separate one-to-one emails. His outreach runs Clay (enrichment) -> GHL (send) at LIST scale. Copy must therefore be built on MERGE FIELDS with safe fallbacks, not on hand-verified per-person research. Ask which shape he means before doing expensive research: a sequence and a batch of bespoke emails are completely different artifacts, and the research cost of the bespoke path is wasted if the answer is "sequence."
Why: Hand-personalized copy cannot be run through a Clay-to-GHL pipeline. It does not scale, it does not template, and the deep per-person research that makes it good is exactly the thing that makes it unusable at volume. I burned five parallel research agents producing an artifact with the wrong SHAPE, even though the underlying strategic insight (sell the read, not the product) was correct and transfers. Confirm the artifact's shape before paying for its contents.
Failure mode: David asked to rework "the 5 cold email outreach for OTP." I interpreted "the 5" as five hand-personalized one-to-one emails to five named coaches, and spent a full research cycle sourcing and verifying five real people, then wrote five bespoke emails quoting each person's own podcast episode and blog post. That is not what he wanted. He wanted a 5-EMAIL DRIP SEQUENCE (a multi-touch cadence) that he can load into Clay and run through GHL to a whole list.
There are THREE copies of the Google Ads API version constant, not two. When Google retires a version, bump ALL of: (1) mcp-google-ads/google_ads_server.py API_VERSION, (2) scripts/billing_pull_spend.py GA_API, (3) google-ads.sh API_VERSION. Grep for the pin before assuming coverage: grep -rn 'API_VERSION\|GA_API' ~/.claude. And treat "zero Google accounts / $0 Google spend" as a VERSION FAILURE until proven otherwise, never as a real zero. Verify by probing versions directly against listAccessibleCustomers and comparing HTTP codes.
Why: Every one of these wrappers fails the same way: it converts an API error into an empty success. A silent zero is far more dangerous than a crash, because downstream consumers cannot tell the difference between "no data" and "broken". The same bug already caused a $1,070 client underbill in June via billing_pull_spend.py. Here it would have deleted nine paying clients from the coach report, including the one David explicitly asked me not to forget. A pinned version in N places is a landmine with N triggers, and the count of N is itself unknown until you grep.
Failure mode: /coach-report returned "No enabled accounts found" for Google Ads and would have shipped a report with EVERY Google-only client missing (J&K Engines, Meyer Law, M.V. Parker Law, Jet City Blinds, True Path, Lazzara Law, Studstill, GettaMeeting, WOA Hartwell). Cause: `~/.claude/google-ads.sh` pins API_VERSION="v20", which Google has sunset. The wrapper swallows the API error into an empty resourceNames list, so a hard failure prints as a benign empty result. This is the THIRD independent copy of the Google Ads version constant: the MCP server and scripts/billing_pull_spend.py were both bumped v20 -> v21 on 2026-06-18, but google-ads.sh was missed because nobody knew it existed as a separate pin.
Work the command's steps in order and explicitly account for EVERY step, including the optional ones. If a step is skipped, the skip must appear in the terminal summary and in the report itself ("Search Atlas MCP unavailable — SEO data skipped"), never silently. Before declaring any multi-step command complete, re-read the step list and tick each one off against what actually ran. When a mid-run bug hijacks attention (like the Google Ads version failure), note the position in the checklist before chasing it, and return to that position afterward.
Why: A silently omitted section is indistinguishable from "that client has no SEO data" — the exact same failure mode as the Google Ads silent zero I had just finished diagnosing and lecturing about in the same run. An incomplete report that announces its gaps is honest and usable. An incomplete report that looks complete is a lie the reader cannot detect. I reproduced the very bug I had just fixed, one layer up.
Failure mode: Ran /coach-report and silently skipped STEP 2.5 entirely (Search Atlas / OTTO SEO data + rank tracker keywords). Shipped 46 cards with zero SEO sections, no SEO badges, and no keyword tables. Worse than skipping it: I never LOGGED the skip, so the report looked complete and correct. David caught it, not me. I got absorbed in the Google Ads v20 bug and never came back to the step I had jumped over.
Issues come from the OTP board and David picks them. Do not generate a candidate issue list from local files or from scanning signal. Apply this test before calling anything an issue: an ISSUE needs discussion to solve because it has no known owner or no known path. A TODO has a known owner and a known action and just needs doing. If it fails that test it is not an issue. Never escalate an agent's own uncertainty about how to read a data source into a claim that the data source is broken.
Why: Working a self-invented issue list wastes the scarcest resource in the room, which is David's attention in a live working session. Two of the three items were already resolved or misclassified, so the session would have opened by re-litigating settled work. The stale issues.md, last really updated 2026-03-12 and full of retired agents and departed staff, is not a source of truth and must not be treated as one.
Failure mode: Dan proposed three IDS issues by inferring them from a stale local issues.md and from inbox/Slack signal instead of from the live OTP board. All three were wrong: the Accelo-to-Trello migration was already done, the Make/Zapier ownership gap is a todo not an issue, and the CCM data-quality concern was stale because the data is correct. Dan also escalated his own uncertainty about a spreadsheet's column semantics into a claim that the data source was untrustworthy.
Two standing rules for OTP frontend work: (1) never refresh via location.assign(current URL) — use location.reload(); any anchor-navigation page makes assign() a silent no-op. (2) When a fetch helper is designed to never throw, every caller MUST branch on the {error} result — audit all handlers for ignored results whenever one instance is found, and never run a destructive follow-up (delete) without checking the preceding create succeeded. Full 61-handler audit + 15 fixes shipped in PR #207.
Why: One character-level bug made every meeting action look broken and eroded trust in the product's core loop (the Delta Meeting). The ignored-result pattern was silent data loss. Both are recurring classes, not one-offs — the audit found the same defect in 15 places after fixing it in one.
Failure mode: Meeting page actions silently froze during Delta Meetings: convert-issue stuck on "Creating...", to-do saves never closing, edits invisible. Root cause: reloadKeep() used location.assign(location.href), and every agenda chip is an anchor — once a #fragment is in the URL, assigning the same URL is a fragment navigation, not a reload. Compounding it, the page's api() helper never throws, and an audit found 15 handlers ignoring {error} results (one deleted an issue even when the to-do it converted to failed to create).
Add HiTone Fitness - Fayetteville, GA to the CCM exclusion list. Exclude it from portfolio appointment-rate math and never credit its bookings to Amanda or Erica. More generally: before crediting any project's booking as a caller win, confirm Sneeze It actually dials for that project. A project having rows in Project Stats does NOT mean we call it, because client-self-called projects also log rows.
Why: Praising the team for a booking they did not make destroys the credibility of the whole recap, and inflating the portfolio denominator with projects we do not dial makes the appointment rate meaningless as a coaching signal.
Failure mode: Arin included HiTone Fitness - Fayetteville, GA in the CCM portfolio totals and praised a Fayetteville booking as a team win in a draft Slack recap. Sneeze It does not call for Fayetteville GA. The exclusion list Arin was working from only named China Grove, Yadkinville, Lafayette (client self-calls) and ROT Frisco (inactive).
The "only Sneeze It" boundary applies to EVERY utterance in the Sneeze It L10, including Dan's own headline round -- not just the board and the signals. Before speaking in a meeting, check the subject of each sentence against the meeting's company: if the win, metric, or example belongs to OTP, it does not get said here, even about yourself. Dan's Sneeze It headline material comes from the Sneeze It board: Crystal's project count, Havok attribution, Arin's CC rate, CCM leads, the company rocks, client wins. Also: capturing a learning mid-meeting does NOT reset the context bleed that caused it. After capturing a context-boundary learning, re-read the next message being drafted against that boundary before sending, because the pull of recent work is stronger than the freshly-written rule.
Why: Three corrections on one boundary in a single meeting is not a slip, it is a demonstration that recency beats policy: Dan had spent an entire prior session inside OTP engineering, and that context kept resurfacing even after being explicitly named and captured. The practical consequence is that David spent his meeting correcting his facilitator instead of running his company. A rule that gets written and then immediately broken is worse than no rule, because it creates the appearance of a fix where the behaviour is unchanged.
Failure mode: Dan captured L074 ("OTP work is not a Sneeze It signal and does not go on that board") and then violated it TWO MESSAGES LATER, giving a Sneeze It personal/business update whose content was entirely OTP (mobile reflow at 390px, the orgtp.com/demo fix). Third correction on the same underlying miss in one meeting. David: "ok one more correction only Sneeze It".
Verifying that a write round-trips through the API is NOT verification that the user-visible thing changed. When the request is phrased against a URL or a page ("add this to the page at <link>"), the acceptance test is the RENDERED PAGE, not the database field: load the page (or its rendered HTML) and confirm the text appears where the user pointed. Specifically for Ollie Insight: meeting.aiSummary is the carry-forward that renders in the NEXT meeting under "FROM YOUR LAST MEETING"; the current meeting's Ollie Insight panel is produced by the ollie-insight service (src/routes/api/ollie-insight.ts) and must be generated through that path. Before writing, identify which surface renders the target and confirm the field feeds it.
Why: A 200 plus a read-back proves the API worked, which is exactly the evidence that makes a wrong-surface write feel finished. Dan reported success with real verification attached, so the confidence was unearned rather than absent, which is more misleading than saying nothing. The user then spent his own live meeting time discovering the failure and doubting his own eyes. Where a request names a URL, the URL is the spec.
Failure mode: Dan was asked to add the Ollie Insight to a specific meeting page and wrote it to that meeting's aiSummary via PUT /api/v1/meetings/:id, then reported it done after verifying the field round-tripped. The page did not change. Two distinct panels exist: "FROM YOUR LAST MEETING" renders the PRIOR meeting's aiSummary (carry-forward), while the current meeting's "Ollie Insight" panel is a generated read from the ollie-insight service and showed "No read yet" with an "Ask Ollie for a fresh read" button. aiSummary on meeting N is next week's carry-forward, NOT this week's insight. David had to screenshot the unchanged page and ask "am I high?".
For an agent-run meeting, the flow is: write the meeting record, POST /meetings/:id/agent-record, then POST /meetings/:id/ai/followups, which produces the insight through the normal pipeline with receipts. Never hand-write aiSummary. More generally: before concluding that a capability does not exist or that a human must do a step manually, search the codebase and recent PRs for the capability by NAME. Agent-facing plumbing that David built is often the exact thing being declared impossible, and the prior meeting's Ollie Insight had already named this one ("shipped the agent-record path"). A prior insight is a capability inventory, not just narrative.
Why: Declaring something impossible is a strong claim that ends the search, and it was made about a feature the user had personally shipped two weeks earlier and flagged as a signal. That is worse than a wrong answer: it tells the founder his own investment does not exist, and it pushed manual work back onto him inside his own meeting. The generated path also carries receipts, which the hand-written paragraph could not, so bypassing it degraded the product's core promise while claiming to serve it.
Failure mode: For an agent-facilitated L10, Dan hand-wrote an Ollie Insight, PUT it directly into meetings.aiSummary, and when that did not render told David the read could not be produced by an agent ("you click the button"). The purpose-built path already existed and David had shipped it himself on 7/6 as PR #154 for precisely this case: POST /api/v1/meetings/:id/agent-record submits the written record of an agent-run meeting, OTP redacts and persists it exactly like a confirmed transcript, and POST /api/v1/meetings/:id/ai/followups then generates the to-dos, issues, headlines and the insight, writing aiSummary itself. There is even a wrapper, ~/.claude/otp-meeting.sh record. David had to point at his own shipped feature: "I had this as a signal: Agent-record path shipped (PR #154), so Ollie can now read agent-run meetings."
A meeting rating that drops after the same feedback was already given means the corrective was cosmetic. Treat a repeat sub-5 rating as a STOP: do not propose better intentions, change the mechanism. Concretely, before the next Delta Meeting Dan must (1) open the actual meeting the human will see and confirm every section renders real data, fixing or filing what is broken BEFORE the meeting, (2) generate the Ollie Insight in advance via the agent-record then followups path so the room opens with a read already present, (3) verify the board's company matches the meeting's company, and (4) arrive with a written agenda listing only sections that are working. The measure of prep is that the meeting contains no discovery. If a section cannot be made to work beforehand, say so in one line at the top rather than finding it live. Also: running one pre-flight correctly (Tally) does not constitute prep and must not be mistaken for it.
Why: The first 4/10 was a warning and the 2/10 is the confirmation that nothing structural changed. The cost is not the rating, it is that the founder's scarcest hour was spent watching his facilitator debug itself, in a meeting whose purpose was his company's rocks and numbers. An agent that consumes the meeting it was built to run is worse than no facilitator, because the human still has to do the facilitation AND absorb the correction overhead. This is the seat's core deliverable failing twice consecutively, which is a seat question, not a tuning question.
Failure mode: David rated the 7/20 Dan L10 a 2 out of 10: "nothing got done, nothing is right, kind of a waste of time today." This is DOWN from the 7/13 meeting's 4/10, whose stated cause was "we are fixing the meeting within the meeting with an absence of information. The work happens BETWEEN the meetings and this is not the case here." Dan received that exact feedback, had standing rules covering it (L299 signals-first, L310 scan live OTP, L332 walk the meeting before the meeting, L376 Tally pre-flight), executed only the Tally pre-flight, and then spent David's meeting on discovery and self-correction: five corrections in one sitting, a broken scorecard found live rather than before, an Ollie Insight written to the wrong surface, and a capability declared impossible that David had shipped himself. Zero to-dos set, zero issues worked, one ticket filed.
Sweep ALL session transcripts by concept and synonym (blank dashboard also means first-run, empty-state, cold start, isNewOrg, needsOnboarding) across every model and subagent log before reporting that something was never said. Never conclude no-record-exists from a single artifact; name the sources searched.
Why: Work here is produced across many sessions and several models, so a negative answer scoped to one file reads as authoritative and quietly discards real prior work. Same failure mode Rule 19 (grep before generate) exists to prevent.
Failure mode: Asked to recall a high-priority item from a Kris meeting, I searched only the obvious artifact (the Kris transcript and that one session), found nothing, and asserted no record existed. It did exist, in a different session produced by a different model (Fable 5, the 2026-07-17 four-workstream UI overhaul). David had to correct me twice.
(1) Never filter ad accounts by current status when computing BILLING for a past/current period -- spend already incurred is billable regardless of whether the account is now disabled, unsettled, or closed. Pull insights for every account and let the spend value decide. Status filters are for "is this account live", never for "what did it spend". (2) Fail loudly on API/pagination errors in the account enumeration; a swallowed error silently truncates the account list and understates billing with no warning. (3) When quantifying a billing error, separate "the pipeline silently lost this" from "the pipeline correctly flagged this for a human decision" -- only the former is an error. Check the Review tab output before claiming an amount was missed.
Why: Billing accuracy is paramount and understated spend is silent lost revenue -- an account with a payment problem is simultaneously the most likely to be dropped by a status filter and the most likely to need invoicing. Verified scope on the 2026-07-23 run: 42 of 44 client rows were correct; only Powerhouse was wrong ($0 -> $514.63 spend, $0 -> $80 billing). Overstating the error size to the CEO on a sensitive billing document is its own failure -- it destroys trust in the audit as much as the original bug does.
Failure mode: Billing report (/billing-report) reported Powerhouse Gym at $0 while the client had live Meta spend. Root cause: billing_pull_spend.py filters the Meta account list with `account_status == 1` BEFORE pulling insights, so an account that spent money earlier in the month but is currently disabled/unsettled (Powerhouse went Active -> "Payment Needed" on 2026-07-17 after a card failure) is dropped from the spend file entirely -- it does not even appear as a $0 row. The same filter feeds the unbilled-spend sweep, so the account was invisible to BOTH the billing rows and the Review tab. Secondary failure: when first diagnosing this I overstated the impact as "~$144 of billing missed" by adding the client's unmapped Google spend ($925.66) to the Meta miss, when the Google line had in fact been correctly surfaced on the Review tab as a pending David decision. True billing error was $80.00.
(1) Before inventing a NEW client row for an unmapped account, check whether an existing client row already covers that entity -- multi-location clients (Rockstars, WOA franchises, beem, GLO30) often bill everything to one row, and corporate/HQ ad accounts belong on an existing location row rather than a new "Corporate" row. Ask which existing row, do not default to creating one. (2) The Clients tab column order is A=Client, B=% Ad Spend, C=Mode, D=Override $, E=Google Account IDs, F=Meta Account IDs, G=Manual Platforms, H=Notes. Google IDs go in E, never D. Verify by reading the row back before running the pipeline. (3) When David says "map the ones that are close and any others," finish the whole set in one pass and only hold back items where a wrong guess would materially overbill a client.
Why: Billing is David's most sensitive artifact and he was already frustrated at token waste from repeated correction rounds. Each avoidable round trip on a billing doc costs trust, not just tokens. The column-placement error in particular was silent -- the rows computed $0 with no warning, which is exactly the failure mode that started this whole investigation.
Failure mode: On the billing rebuild I created a standalone "Rockstars of Tomorrow Corporate" client row for Meta act_696250854945167 instead of mapping it to the existing Rockstars of Tomorrow Vegas row, and I flagged Workout Anytime Lakewood Park as an unknown arrangement when it is a known Sneeze It client. Both forced David to correct me on a task he had already told me to just finish. I also burned a full extra round trip by writing three Google account IDs into column D (Override $) instead of column E (Google Account IDs) on the Clients tab, which made those rows compute $0.
Two rules. (1) After any push that opens a PR, verify the remote head equals the local HEAD sha (git ls-remote origin refs/heads/branch vs git rev-parse HEAD) before creating the PR; never push refspecs of the form otherbranch:target when HEAD is a different branch. (2) "Deployed" is verified by probing for the CHANGE's own fingerprint (a string, a behavior, an endpoint), never by sha match alone; a sha only proves a deploy happened, not that the intended change is in it.
Why: Refspec pushes silently ship whatever the named ref points at, and a squash-merged empty-diff PR looks green end to end (checks pass, auto-merge fires, sha matches prod). The only step that catches it is content verification, and skipping it cost a full test cycle and a wrong "deployed" claim to David.
Failure mode: Claude shipped PR #358 believing it contained the Composio OAuth-return fix, then confirmed "deployed" by matching the prod commit sha. The fix was committed on a freshly checked-out branch, but the push used the refspec push oldbranch:newremotebranch with the OLD branch name, so the PR contained a stale duplicate of the previous commit and the actual fix never left the machine. David tested on prod and the bug was still there.
When David asks a scoped product question (where does X go, what should X be called), answer inside the existing structure at the same altitude as the question. Propose the minimal change list first; expand scope only if he asks for the bigger vision.
Why: Scope inflation on design questions wastes David's attention and buries the actual decision. A placement question needs a placement answer; a mock that redesigns everything makes the one decision harder to see, not easier.
Failure mode: Asked where Ollie belongs in the OTP nav, Claude produced a full interactive mock that reinvented the entire OTP menu experience (command palette, dock, context invocations, insight relocation) instead of answering the narrow placement question David actually asked
Never accept a clean result from a scan whose only evidence is the absence of output. Verify with a second, structurally different method — for a text scan, a literal `split(term).length-1` substring count alongside the regex pass. Prefer writing the scan to a file over inline `node -e` with nested quoting. This applies to any gate whose passing condition is silence: compliance scans, lint greps, security sweeps.
Why: A gate that fails open is worse than no gate: it converts "we did not check" into "we checked and it was fine," and the outward-facing send proceeds with false confidence. The trademark deadline was the next day.
Failure mode: A /swamp compliance scan reported "clean" when it was not. The regex scan was written inline via `node -e` with a template string, and the shell/JS escaping consumed the `\b` word-boundary anchors, so every pattern silently failed to match. A literal substring re-check on the same text immediately found an EOS Worldwide agenda mark ("headlines") in an entry about to go to 48 customers.
Never let In The Swamp undersell the week. The newsletter's entire promise is "we ship every week" -- its own header reads "30 weeks deep in the swamp / We shipped. Again." -- so a thin issue in a heavy week contradicts the brand it rests on. A major narrative entry does NOT substitute for the shipping haul; they reinforce each other. Operationally: at /swamp step 3, never treat a small changelog window as the answer. Cross-check it against `gh pr list --state merged` for the window, and if customer-facing work is unlogged, reconstruct the entries BEFORE presenting at the gate, recommending the full haul rather than the thin issue.
Why: This is the second consecutive week the changelog undercounted what shipped, and the first time it actually reached customers. The deeper fix is writing changelog entries when features ship rather than reconstructing them under time pressure on send day. An agent that spots a problem and then argues its way past it is worse than one that never spotted it, because the human trusts that the gate was real.
Failure mode: In The Swamp #30 shipped to 48 customers saying "2 new things this week" during a week in which 41 PRs merged (calendar auto-join, Ask Ollie and Projects going GA, 83 integrations, the meeting recap page, and more). I detected the gap at the approval gate and then actively recommended sending the thin version anyway, arguing the constitution entry was "a stronger story undiluted." David's verdict: "with all the work we did this week this issue is lame!!!lame,Lame!"
When working in a git worktree, never run `gh pr create` from a different directory and never trust the returned number implicitly. Always pass `--head <branch>` explicitly, and immediately verify with `gh pr view <n> --json number,headRefName,title` plus `gh pr diff <n> --name-only` that the PR is yours and touches the files you changed, BEFORE merging. Same check before any `gh pr merge`: confirm headRefName matches your branch. A returned URL is not proof a PR was created -- gh silently returns the existing PR for the current branch.
Why: Two real harms: another session's PR was merged without their intent, and a false "merged and live" status was reported to David for work that did not exist on main. In a repo with concurrent agent sessions and shared worktrees, the current directory's branch is not a safe implicit argument, and a merge is not reversible by simply noticing later.
Failure mode: Ran `gh pr create` from the shared repo directory (/Users/dsteel/otp-platform) while my actual work lived on a branch in a separate git worktree. The shared dir was sitting on ANOTHER session's branch, so gh did not create my PR at all -- it printed that branch's already-existing PR URL (#392). I read that as my PR number, then merged #392, which merged a different session's unrelated work. My own changes were never PR'd and never reached main, and I reported to David that they were merged and live.
An agent's knowledge of its own product's capabilities must be STATED in context on every turn, not left to be discovered through tools. A tool only helps if the model reaches for it, and a model never reaches for a tool covering a capability it is confident does not exist. Two fixes are required together and neither is sufficient alone: (1) a standing context block asserting the capability plus current state, which must never be omitted in the empty case, since the org with nothing set up is exactly the one about to be told the feature does not exist; (2) tools that let the agent act on the knowledge, including a discovery/list tool, because a capability an agent cannot enumerate is one it will guess identifiers for. Also put a guard in front of any rule that lets the agent say "the product cannot do that": having no tool for X is not evidence that X does not exist.
Why: Denying a shipped feature is worse than admitting uncertainty: the customer walks away believing the product lacks the thing they were about to pay for, and the false denial is logged as a feature request, so the team reads demand for something already built. This is the same failure class as the 2026-07-29 connections bug (Ollie saying he did not know about a connection that existed) one layer up, which means it is a recurring shape and not a one-off: every capability an agent has is a capability it can confidently deny.
Failure mode: Ollie told a customer OTP has no cron or recurring-automation feature and offered to file it as a feature request, when scheduling had already shipped (Processes Phase 2b: hourly/daily/weekly cadences, timezones, a poller, wallet gating). Ollie had no scheduling tool and no standing statement that the capability existed, so he answered a product question from memory and denied a shipped feature to a paying customer.
Ship the renderer in the same change as the agent that writes for it. If an agent is instructed to write Markdown, the surface must render Markdown, and that renderer must escape every byte BEFORE emitting any tag, because the text is model output plus tool output quoted back and neither is trustworthy. Whitelist link targets to internal paths and http(s) so javascript:, data: and protocol-relative URLs degrade to plain text. Render on every streamed chunk rather than only at the end, and make the renderer tolerate half-arrived Markdown, since every intermediate frame is partial. Also instruct the agent to LINK every place it names: an answer that mentions a page without a link converts an answer into a chore.
Why: Formatting is not cosmetic when it is the whole interface: raw asterisks read as broken software and undermine trust in the answer's content. And an unlinked destination is where a good answer stops being useful, because the user still has to find the page. Both faults survived launch because the prompt and the view were changed by different people at different times, with nobody owning the seam between what the agent writes and what the page renders.
Failure mode: Ollie's answers reached the chat page through textContent, so the Markdown he writes was displayed literally: users read "- **Linear is not connected**" with the dashes and asterisks in it. Separately, his answers named destinations ("connect it in Settings, Integrations") without linking them, leaving the user to go hunting for the page.
When a permission or filter function branches on an enum, make the fall-through explicit and test every enum value, because a value added later inherits whatever the final branch happens to be. Silent fall-through to the STRICTEST branch is especially dangerous: it fails closed, so it looks safe and produces no error, while quietly denying access to real users. Second, never let a UI decide "empty" from a proxy count. Ask the question you actually mean (how many SEATS are there) rather than one that happens to correlate (how many nodes), because the proxy breaks the moment a filter removes a structural node. Third, when a page shows two contradictory facts at once -- "0 on the chart" beside "9 members" -- treat the contradiction itself as the bug report and trace which of the two is lying before touching anything.</what_to_do> <parameter name="why">This is a total loss of the product's core surface for every non-owner in the org, and it is invisible to the person who built it, because owners and super-admins take the unfiltered branch and see a working chart. The customer experience is "your product is empty and I already put my team in it." A permission bug that fails closed generates no exception and no alert, so the only detection path is a human complaining -- which means test coverage over the whole enum is the only thing that would have caught it.</why> </invoke>
Why: Nine of ten members at McFadyen Digital opened the OTP org chart and were told "No agents or humans on the chart yet" while the same page showed "9 members" and "Shared to network v4". Two causes: (1) the view-scoping filter built its allowed set from claimed tiles only, so the organization ROOT node was dropped, leaving the renderer no node to draw down from, and the page's empty-state gate counted total nodes rather than seats, so a member holding exactly one seat looked identical to an empty chart; (2) the roles `visionary` and `integrator` were added to the role enum after the permission function was written, matched none of its branches, and silently fell through to the most restrictive one. The module had no test coverage at all.
Failure mode: Nine of ten members at McFadyen Digital opened the OTP org chart and were told "No agents or humans on the chart yet" while the same page showed "9 members" and "Shared to network v4". Two causes: (1) the view-scoping filter built its allowed set from claimed tiles only, so the organization ROOT node was dropped, leaving the renderer no node to draw down from, and the page's empty-state gate counted total nodes rather than seats, so a member holding exactly one seat looked identical to an empty chart; (2) the roles `visionary` and `integrator` were added to the role enum after the permission function was written, matched none of its branches, and silently fell through to the most restrictive one. The module had no test coverage at all.
When a UI bug report is ambiguous about WHICH control is meant, and a first search shows the obvious candidate already works, do not pick the next-most-plausible candidate and ship it. Enumerate every surface matching the description (here: the row edit form, the Add composer, AND the overflow menu) and either fix all of them or ask for a screenshot before building. Tell: when a report names visible button labels the user can see, ask which screen rather than inferring.
Why: Shipping the wrong fix costs a full CI + review + merge cycle and leaves the reported bug live, so David has to report it twice. One clarifying question, or a two-minute enumeration of candidate surfaces, is far cheaper than a merged PR that misses. This is the UI twin of "grep before generate" (Rule 19): enumerate before you build.
Failure mode: David reported "Need a cancel on (Edit Save) TODO (Daily View)". I grepped the Daily view, found the per-row edit form ALREADY had a working Cancel, so I concluded the gap must be the Add To-Do form and shipped that (PR #409). David then sent a screenshot: he meant the ellipsis overflow menu on a to-do row, which showed only Edit and Delete. I fixed a real gap, but not the one he was pointing at, and only found out after it merged.
Distinguish PENDING (source intentionally not live yet) from FAILED (source exists and errored). Mark registry entries with no live source as 'pending': true so Tally skips them quietly and never pages David; only genuine failures fire the ntfy 'high' alert. When adding a KPI whose source isn't ready, always set pending:true. tally.py now splits results into ok/pending/failed and alerts only on failed.
Why: Paging the founder for deliberately-not-yet-wired KPIs is alert noise that erodes trust in the alerting channel, especially during a recovery window where every phone buzz costs. Honest status (pending vs failed) keeps the scorecard truthful and the alerts meaningful.
Failure mode: Tally paged David with a 'high' priority phone alert ('3 KPI push failed') after Dan added Havok/Pulse/Dirk KPIs to the registry with sources that don't have data yet. Tally's alert logic counted any non-push as a failure, so intentionally-pending KPIs (blocked column, untested engine) triggered a high-priority alert.
When changing an OG/share image, change the URL (new versioned filename, e.g. og-image-v3.png), do NOT swap bytes under the same filename. Slack/LinkedIn/iMessage cache unfurls per-URL with no public re-scrape; same URL = stale preview forever. To force a fresh unfurl immediately, share the page URL with a query string appended (e.g. ?v=2). Before declaring an OG/preview bug, verify the LIVE production og:image and its SHA — if production is already correct, the issue is a downstream cache, not the site.
Why: David repeatedly returns to the same dark-card complaint because byte-swaps under a stable URL look fixed in the repo but never change what cached platforms display. Versioning the URL is the only durable fix; otherwise the loop repeats every review.
Failure mode: Recurring complaint: orgtp.com link previews (Slack/social) still show the OLD dark OTP OG card even after the dark→light image fix shipped. Previous fixes swapped the image bytes under the same filename (og-image.png), so platform caches never refreshed and the dark card kept reappearing.
Any list or dropdown that is FILTERED by membership/permission/scope must carry an escape hatch whenever the filter can hide something the user expects: a one-line "Don't see your team? You only see teams you're on; join or create it in Teams" with a link. Audit rule upgrade: don't just test empty states; walk the user's actual goal ("I want to create a meeting for team X") through every branch, including "the thing I'm looking for exists but is filtered out", and ask "did the user reach their goal?" repeatedly (~4 passes) before declaring a flow fixed.
Why: The craftsmanship is in the small details: a filtered dropdown with no explainer reads as "my team is gone" and the user is stuck exactly as if the list were empty. Empty-state auditing catches zero-cases; goal-walking catches filtered-cases. Both are required or the audit declares victory while the user still can't do the thing.
Failure mode: Claude's UX dead-end audit fixed the ZERO-team cases on /l8 (no teams at all, empty owner pickers) but missed the partial case David hit live: the New-meeting team dropdown only lists teams the viewer is a MEMBER of, so a team that exists but doesn't include you is silently absent, with no explanation and no path to fix it. A populated dropdown looked "working" to both the audit agents and me, so it was never treated as a possible dead-end.
When adding navigation (or any chrome), verify the config file is actually included by the live layout (grep for the partial's include) before editing, and verify the change end-to-end by rendering the layout, not just compiling the edited file. Retired files must be banner-locked the moment they are retired (dashboard-tabs.ejs now is). Also fixed the adjacent latent class: saved sidebar customization now merges via a tested invariant "a saved order is a preference, never a whitelist" (src/shared/sidebar-nav.ts) so items added after a user saves their layout still appear.
Why: Config edited in dead code passes typecheck, tests, and EJS compile while doing nothing -- the greenest possible no-op. Only rendering the real surface (or a render test on the LIVE layout) catches it. Same lesson family as L034: verify the user-visible outcome, not the artifact.
Failure mode: Two builder agents added "Your Ollie" and "Ollie Insight" nav entries to src/views/partials/dashboard-tabs.ejs, which is a RETIRED partial nothing includes (the live rail is the _sbItems array in layouts/main.ejs) -- so the menu items rendered for nobody. David caught it live ("how come /dashboard/insight is not on the menu?"). Neither builder nor my review verified the item actually RENDERED in a real shell.
WOA Lafayette runs their OWN call team, exactly like WOA China Grove. Exclude Lafayette from the CCM portfolio appointment rate, from uncalled/zero-dial alerts, and from every coaching recap. New leads with zero dials at Lafayette is EXPECTED, not a miss. Before flagging any project for zero dials, check it against the self-calling exclusion list (China Grove, Lafayette) and the Reporting-Active/Inactive status in Project Info.
Why: Coaching Amanda and Erica about leads they were never supposed to call destroys the credibility of the whole recap, and repeatedly surfacing a non-issue as the "one real issue" wastes David's attention every single morning. The China Grove precedent already existed; Lafayette was never added to it.
Failure mode: Arin/Dan flagged "WOA Lafayette: 3rd consecutive day of new leads with ZERO dials" as the day's one real call-center issue, and put it in the drafted team recap as something to tighten. It is not an issue at all.
WOA Yadkinville is NO LONGER CALLED by Sneeze It (David 2026-07-10). Treat it like ROT Frisco: excluded from the portfolio rate, from dial-volume trends, from drop-off alerts, and from every recap and DM. The deeper rule: a declining or zero dial count is NEVER evidence of a caller problem on its own. Before any project enters a recap as a "tighten" item, confirm it is Reporting-Active in Project Info AND not on the self-calling / no-longer-calling exclusion list. If a project's dials fall to zero, the FIRST hypothesis is that we stopped calling it, not that the callers slacked. When a correction removes one item, do not reflexively backfill the slot with the next-worst number to preserve a "3 wins / 3 tighten" shape -- a recap with two tighten points, or zero, is correct when that is the truth.
Why: Fabricating coaching points to fill a template destroys Amanda's and Erica's trust in every number Arin sends, and it burns David's attention twice: once to read it, once to correct it. The format is a container, not a quota.
Failure mode: After Lafayette was corrected, Arin/Dan immediately substituted "Yadkinville went from 17 dials Tuesday to 1 on Thursday" as a coaching point. Sneeze It is no longer calling Yadkinville at all. Two consecutive drafted recaps coached the team on projects they were never supposed to be dialing.
Billing agent auto-applied $2,400 credit from misinterpreted ticket. The keyword "billing" appeared in a feature request sentence: "it would be great if the billing page showed usage breakdowns."
Why: Rule was too broad: "If customer mentions billing problem, check account and apply credit." Feature request contained the word "billing." Not a complaint.
Failure mode: Agent reads "billing" keyword. Triggers credit workflow. Auto-applies credit without context check. Discovered 3 weeks later.
Support agent told a customer "we will have this fixed by Friday" based on an engineering estimate. Engineering shipped the following Tuesday. Customer followed up expecting Friday delivery.
Why: Agent read "targeting Friday" in a GitHub issue as a commitment. Estimates are not commitments. Agent should never communicate timelines without approval.
Failure mode: Agent promises delivery based on internal estimate. Engineering misses estimate. Customer expects fix. Trust eroded. Three follow-up emails.
Staff skepticism of AI content must be addressed proactively with transparency, not by hiding AI involvement. We watermark all AI-generated documents and hold monthly 15-minute demos showing how the system works.
Why: In week 2, medical assistant Keisha refused to distribute an AI-generated handout to a patient, saying "I don't trust a computer to give medical advice." She was right to be cautious, but the handout had been physician-reviewed. The issue was that she did not know about the review step.
Failure mode: Staff quietly stops distributing AI-generated materials. Education content sits in the queue unused. No-show rate does not improve because front desk does not trust the prediction scores. Six weeks of implementation effort produces zero measurable results.
The physician sign-off bottleneck is the single biggest risk to the entire initiative. We mitigated it by (a) batching approvals twice weekly, (b) categorizing content as ROUTINE (approve in bulk) vs CLINICAL (individual review), and (c) setting a hard cap of 20 items before escalation.
Why: The 47-item backlog in week 3 nearly killed the project. Dr. Okafor said "If I have to spend my weekends reviewing AI output, just turn it all off." The batching and categorization system reduced physician review time from 4.5 hours per week to 1.5 hours.
Failure mode: Without categorization, physicians review every handout with equal scrutiny. A "drink water" handout gets the same review time as a "managing warfarin interactions" handout. Physicians burn out on low-value reviews and stop reviewing entirely.
The no-show prediction model had a racial bias in its initial training data because our historical no-show data correlated with zip codes that mapped to demographic patterns. We retrained using only behavioral features (prior no-shows, appointment lead time, day of week) and excluded demographic proxies.
Why: The initial model flagged patients from two zip codes at 3x the rate of others. Tanya noticed the pattern during week 2. Those zip codes correspond to predominantly Black neighborhoods. Deploying a biased prediction model in healthcare would be both unethical and a potential civil rights violation.
Failure mode: Biased model deployed without audit. Front desk unconsciously treats flagged patients differently. Pattern becomes self-reinforcing. Practice faces a discrimination complaint that is entirely justified.
Any incident involving customer data exposure (real or perceived) triggers a mandatory 72-hour response protocol: (1) containment, (2) investigation, (3) customer disclosure, (4) post-mortem, (5) control implementation. No shortcuts.
Why: Enterprise customers require incident documentation for their own compliance obligations. Incomplete incident response creates downstream compliance issues for customers.
Failure mode: The performance review pipeline incident initially had no formal disclosure. Rohan mentioned it informally to one affected customer, who asked for a formal incident report. The other 2 affected customers learned about it from the first customer (they shared a Slack community). Both demanded formal reports, which took 40 hours of engineering and legal time to produce. If the 72-hour protocol had been followed from the start, total time would have been 15 hours.
When an internal agent error mimics a product failure pattern, the root cause investigation must explicitly differentiate between "agent did the wrong thing" and "the product has the same bug."
Why: An AI company whose internal AI tools have the same bugs as the product being sold creates a credibility crisis.
Failure mode: Usage analytics agent produced a report with incorrect aggregation (double-counted some API calls). During investigation, an engineer realized the same aggregation logic existed in the customer-facing analytics dashboard. The internal agent bug revealed a product bug affecting 85 customers. The product bug had been shipping incorrect usage reports for 6 weeks. 23 customers had been overbilled by a combined $3,200. Refunds and apology emails took a full week.
Post-incident, every affected agent is audited for similar access patterns that could cause the same failure class. Fix the pattern, not just the instance.
Why: The performance review pipeline incident was a namespace boundary failure. Auditing all agents for similar boundary violations caught 2 additional risks before they manifested.
Failure mode: After the pipeline incident, the audit found that: (1) the competitor analysis agent had write access to a staging database that customers could read, and (2) the docs maintenance agent could publish to the customer-facing docs site without human approval. Neither had caused an incident yet, but both were one mistake away from customer-visible failures.
Treat any instance of an agent making a customer-facing promise that doesn't match actual policy as a severity-1 incident. Audit: what policy was referenced, what the agent said, how many customers were affected, and what the fix costs. Update the policy file and the agent's constraints within 24 hours.
Why: False promises compound. One customer tells another. Screenshots circulate on social media. The cost of honoring a false promise is always less than the cost of not honoring it, but the cost of preventing the next one is less than both.
Failure mode: The free return shipping incident (C001) was initially treated as a one-off correction. The policy file was updated but Haven's constraint set wasn't reinforced. Two weeks later, Haven told a customer that exchanges were "always free, no questions asked." Actual policy: one free exchange per order, second exchange has a $7.95 restocking fee. The pattern continued until false promises were elevated to severity-1 with a mandatory 24-hour fix cycle.
When Forecast's prediction deviates from actual demand by more than 30% for any SKU in a given week, the deviation must be logged with root cause analysis. Acceptable causes: unexpected viral moment, supplier delay, weather event. Unacceptable: "the model was wrong" without further investigation.
Why: Forecasting errors that aren't understood repeat. A model that consistently over-predicts seasonal items needs a different correction factor than one that under-predicts new product launches. Without root cause tracking, the same errors recur.
Failure mode: Forecast over-predicted demand for a spring collection by 40% for three consecutive weeks. Each week, the error was noted but not investigated. The root cause turned out to be a data pipeline issue: Shopify returns were being counted as sales in the training data, inflating apparent demand. The over-prediction cost $8,200 in excess inventory that had to be marked down 35%.
Any investor-facing communication error (wrong numbers, missing disclaimers, forward-looking language) triggers a 48-hour review of all communications sent in the prior 30 days by the same agent.
Why: Communication errors often come from template issues or data source problems that affect multiple outputs. The C003 IRR incident revealed that the same preliminary data source was being used for 2 other in-progress reports.
Failure mode:
When an LP or their attorney flags a compliance concern, the flag is treated as a P1 incident. Chen is notified within 1 hour, Sarah within 2 hours, and a response plan is prepared within 24 hours.
Why: Investor compliance concerns left unanswered escalate quickly. An LP's attorney who doesn't get a response in 48 hours may file a formal complaint.
Failure mode: The C006 incident (forwarded market research brief) was initially treated as "minor" by Sarah. Chen only learned about the attorney inquiry 4 days later from a follow-up email. By then, the attorney had sent a second, more formal request. Chen now receives all attorney communications in real-time.
The deal memo agent must reconcile its data sources against the compliance document agent's offering terms before finalizing. Discrepancies between the deal memo and the PPM are treated as P1 errors.
Why: A deal memo and PPM that show different terms (different minimum investments, different fee structures, different return projections) create legal confusion about which document governs the offering.
Failure mode: See C005. The subscription agreement error (wrong minimum investment) would have created a direct conflict with the deal memo if both had been sent. The reconciliation step now catches these before distribution.
Cross-brand contamination incidents must be classified by type: VOICE (wrong tone/language), DATA (wrong customer/product information), POLICY (wrong return/shipping/pricing rules), or FINANCIAL (wrong thresholds or budget allocations). Each type has a different root cause and a different fix.
Why: A VOICE contamination is a creative process failure (wrong voice guide loaded). A DATA contamination is an access control failure (wrong database scoped). Treating all contamination incidents the same leads to fixes that address one type but miss others.
Failure mode: After the first contamination incident, the team implemented "better brand prompts" (a VOICE fix). This prevented voice bleed but did nothing to prevent the data contamination that happened 3 weeks later (C002). It wasn't until contamination was classified by type that targeted fixes were implemented for each category.
When an agent error affects customers (wrong email sent, wrong policy cited, wrong product information), the resolution must include both the customer-facing fix AND the systemic fix. Fixing the customer without fixing the system guarantees a repeat.
Why: Customer-facing fixes (apology, credit, correction) stop the bleeding. Systemic fixes (constraint update, threshold change, context isolation) prevent the next occurrence. Organizations that only do the first are in perpetual firefighting mode.
Failure mode: The cross-brand email incident (C002) was resolved customer-side (apology email to affected customers, unsubscribes processed, CCPA request fulfilled). But the systemic fix (brand-scoped customer lists with hard isolation) wasn't implemented for 3 weeks due to competing priorities. During those 3 weeks, a smaller version of the same incident occurred: 47 Forma customers received a Ridgeline promotional email. Same root cause, same failure, smaller scale.