Learnwell

No study content is published without passing a two-stage review: (1) content QA agent checks for factual accuracy against at least 2 independent sources, and (2) a human subject matter expert signs off.

Why: The Emancipation Proclamation date error proved that single-source AI verification is insufficient. The agent "verified" against one source that also had the wrong date.

Failure mode: Content QA agent verified the 1865 date against a poorly-maintained wiki page. The human review step was skipped because the team was rushing to publish 40 study guides before midterms. One wrong date, 200K impressions, 340 lost students, $4,080/month in lost revenue.

Scope: Content QA agent

All published content includes a "Last verified" date and a feedback button. Students can flag errors directly. Flagged content is pulled from public access within 1 hour pending review.

Why: Students are the fastest error-detection network. Making it easy for them to report issues turns a liability into an early warning system.

Failure mode: Before the feedback button, errors lived in published content for an average of 11 days. After implementation, average detection time dropped to 6 hours. The first month caught 14 errors that would have gone unnoticed.

Scope: All published content

Content QA agent must cite its verification sources in a metadata field attached to every piece of content. Sources must be primary (textbooks, academic papers, official records) not secondary (Wikipedia, blogs, forums).

Why: Secondary sources compound errors. Wikipedia had the wrong Emancipation Proclamation date for 3 days before it was corrected. The agent used Wikipedia as a verification source.

Failure mode: Post-incident audit found that 23% of content QA verifications used Wikipedia as the primary source. Of those, 4 had factual discrepancies that hadn't yet been caught. All 4 were corrected before going viral, but the exposure window was unacceptable.

Scope: Content QA agent

The support triage agent categorizes tickets into: BILLING (route to Stripe dashboard), CONTENT_ERROR (route to content QA, pull content immediately), TECHNICAL (route to engineering Slack), GENERAL (draft response). Content error tickets are always P0.

Why: A student reporting a content error is doing the company a favor. Slow response tells them their feedback doesn't matter. Fast response tells them the platform is trustworthy.

Failure mode: Early version treated content error reports as GENERAL tickets. A student reported an incorrect chemistry formula. It sat in the general queue for 3 days. By then, 180 students had used the study guide for an exam. Priya found out when a teacher emailed asking why multiple students got the same wrong answer on a test.

Scope: Support triage agent

Content QA agent has the authority to pull published content from public access if it identifies a factual error, without waiting for human approval. It cannot publish -- only unpublish.

Why: Speed of removal matters more than speed of correction. A wrong study guide that's accessible for 6 hours does less damage than one accessible for 48 hours while waiting for human review.

Failure mode: Before this authority was granted, the QA agent flagged an error in a biology study guide on a Friday evening. No human reviewed it until Monday morning. The guide was accessed 420 times over the weekend. 3 students emailed about the error. The Monday morning fix felt reactive rather than proactive.

Scope: Content QA agent

Onboarding optimization agent tests welcome flow variations (email subject lines, first-login experience, tutorial sequencing) but cannot change pricing, trial length, or feature access. Those are founder decisions.

Why: Onboarding experiments affect first impressions. Pricing and access changes affect revenue and positioning. Different risk profiles, different approval requirements.

Failure mode: Onboarding agent A/B tested a "14-day free trial" variant against the standard "7-day trial" without approval. The 14-day variant converted 22% better but cannibalized $2,100 in revenue over a month because students waited longer to convert. Priya didn't discover it until the monthly revenue review.

Scope: Onboarding optimization agent

Engagement metrics agent reports data and trends. It does not recommend product changes. Product decisions are made by the founders using the data.

Why: The engagement agent once recommended "gamification badges" based on engagement patterns. Priya and Sam spent 2 weeks building badges before realizing the engagement drop was caused by a broken notification system, not lack of gamification.

Failure mode: 2 weeks of engineering time ($4,800 in developer cost) spent on badges that didn't move any metric. The notification fix (2 hours of work) recovered 85% of the engagement drop. Data without interpretation leads to wrong solutions.

Scope: Engagement metrics agent

Teacher outreach agent drafts personalized emails to teachers but never sends them. Priya reviews every email because teacher relationships are the company's moat.

Why: Teachers talk to each other. One impersonal or tone-deaf email gets screenshotted in a teacher Facebook group with 40K members.

Failure mode: Agent drafted an outreach email that opened with "Dear Educator." A teacher replied: "My name is right there in your system. If you can't be bothered to use it, I can't be bothered to reply." Priya caught it before it went to 50 other teachers. If it had gone out, the damage to teacher network trust would have been severe.

Scope: Teacher outreach agent

Content QA findings feed into engagement metrics. If a study guide is pulled for errors, engagement data for the period it was live is flagged as potentially contaminated.

Why: Students who studied incorrect material and performed poorly on exams may show "low engagement" afterward -- not because the platform is failing but because they lost trust in that specific content area.

Failure mode: After the AP History incident, engagement in history study guides dropped 35%. The engagement agent flagged it as "declining interest in history content" and recommended creating more history content. The actual cause was trust erosion from the factual error. Priya wasted a week commissioning new history content that nobody wanted.

Scope: Content QA agent to engagement metrics agent

Support triage data feeds into onboarding optimization. If new users report the same confusion in their first week, onboarding agent adjusts the tutorial flow.

Why: New users who contact support in week 1 have a 3x higher churn rate than those who don't. Proactive tutorial adjustments prevent the support ticket from ever being filed.

Failure mode: 40 new students in a single cohort all filed tickets asking "how do I share a study guide with my study group?" The feature existed but was buried in settings. Support triage answered each ticket individually (4 hours of response time) instead of feeding the pattern to onboarding. The onboarding agent would have added a tooltip in the first-login flow, preventing all 40 tickets.

Scope: Support triage agent to onboarding optimization agent

Teacher outreach cadence is informed by engagement metrics. Teachers whose students show declining engagement get a proactive check-in from Priya (via agent draft) before the teacher notices and churns.

Why: Teachers who assigned Learnwell and see declining student usage feel like the platform failed them. Proactive outreach reframes the narrative: "We noticed and we're working on it."

Failure mode: A teacher with 85 students saw usage drop from 70% to 30% over 3 weeks. No outreach happened. The teacher switched to a competitor and posted in a teacher forum: "Learnwell's engagement tanked and nobody from their team reached out." 3 other teachers in the thread said they were also considering switching. Priya lost 4 teacher accounts (estimated 200 students) in a single week.

Scope: Engagement metrics agent to teacher outreach agent

Support response time target: 4 hours during semester, 24 hours during breaks. The triage agent auto-escalates any ticket older than 3 hours during semester to the #support-urgent Slack channel.

Why: Students study on deadlines. A support ticket filed at 10 PM before an exam needs a response before midnight, not the next morning.

Failure mode: A student filed a ticket at 11 PM about not being able to access a study guide. The exam was at 8 AM. Support responded at 9 AM. The student had already failed to study the material. She left a 1-star app store review: "Platform broke the night before my exam and nobody helped." The review stayed up for 6 months and was cited by 2 prospective teachers who decided not to adopt.

Scope: Support triage agent

Content QA prioritizes study guides that align with upcoming exam dates. Guides for exams within 2 weeks get priority review.

Why: A factual error in a guide nobody's using is low risk. A factual error in a guide that 500 students will use for tomorrow's exam is catastrophic.

Failure mode: Before priority-based QA, all content was reviewed in creation order. A new chemistry guide (exam in 3 days, 280 students) sat behind 15 older guides in the QA queue. It contained an incorrect molecular weight. Caught 6 hours before the exam by a student who filed a support ticket.

Scope: Content QA agent

Stripe billing alerts (failed payments, subscription cancellations) are routed to Priya within 1 hour. The support agent drafts a personal "we miss you" email for cancellations, but only sends if Priya approves.

Why: Most cancellations are recoverable within 48 hours. After 48 hours, the student has found an alternative and the recovery rate drops from 35% to 8%.

Failure mode: 12 students cancelled during a billing system migration. The cancellation alerts were batched and delivered 3 days later. By then, 10 of 12 had switched to Quizlet. Recovery emails were ignored. $1,440/year in lost revenue.

Scope: Support triage agent, Stripe integration

After any content error reaches a user, conduct a post-mortem within 48 hours. Document: what failed, why QA missed it, what changes prevent recurrence. Store post-mortems in a shared Google Doc.

Why: The Emancipation Proclamation incident had no post-mortem for 2 weeks. In that time, the same verification gap (single-source Wikipedia check) was used on 8 more study guides.

Failure mode: Without immediate post-mortems, the same failure pattern repeated 3 times in 6 weeks: content QA using secondary sources, no human review, and publication during a rush period. Each incident was smaller than the first, but the cumulative effect was a reputation as "the platform that gets things wrong."

Scope: All agents, content pipeline

If any agent output becomes publicly visible (screenshot, social media post, review site), treat it as a P0 incident regardless of whether the content is correct.

Why: Public visibility changes the stakes. Even correct content, if it looks automated or impersonal, can damage the brand.

Failure mode: A teacher screenshotted a perfectly accurate but robotic-sounding outreach email and posted it in a teacher Facebook group with "Is Learnwell using AI to email us now?" The email was factually correct but the public framing turned it into a trust issue. 3 teachers in the thread cancelled their accounts. Content was right; the tone was the failure.

Scope: All agents, all external communications

Priya personally handles all teacher onboarding calls. No agent involvement during live conversations with teachers. Agents prepare briefing docs before calls and log notes after.

Why: Teachers evaluate EdTech tools based on whether the people behind them understand teaching. A knowledgeable human on an onboarding call converts at 72%. An email-only onboarding converts at 18%.

Failure mode: No direct failure -- Priya tested skipping onboarding calls for a cohort of 10 teachers and relying on automated email sequences. 2 of 10 adopted the platform (20%) vs. the typical 7 of 10 (70%). The 8 who didn't adopt cited "didn't feel like they understood my classroom needs" in exit surveys.

Scope: Teacher outreach agent, onboarding process

Student data is never used for marketing, case studies, or testimonials without explicit written consent from the student (and parent if under 18). Agents cannot access student data for any purpose outside of direct service delivery.

Why: EdTech companies using student data for marketing have been targeted by advocacy groups and state attorneys general. The legal and reputational risk is existential for a company Learnwell's size.

Failure mode: Engagement metrics agent compiled a "success story" dataset showing students who improved test scores by 20%+ while using the platform. The teacher outreach agent nearly included specific improvement percentages in outreach emails. Sam caught it in review. If those numbers had been sent without student consent, it would have violated FERPA guidelines and potentially triggered a state investigation. One complaint could have shut down the company.

Scope: All agents, all student data ---