BarPlaybook
AI-graded bar-exam essay platform. Every scoring change is checked against a 32-essay human-graded golden set (MAE 2.58; 84.4% within 5 points).
Overview
BarPlaybook grades practice bar essays and explains where the writer lost points. I built it after taking the California bar exam, so the product starts from a problem I know from the user side. The important part is the evaluation harness: every scoring change runs against 32 essays graded by humans. On 2026-05-05, the grader reached a mean absolute error of 2.58 raw points, with 84.4% of essays within five points of the human score.
Key Features
- Structured essay grading with defensible per-issue reasoning, not a single opaque score
- 32-sample golden set, MAE 2.58, within-5 = 84.4% as of 2026-05-05
- February 2026 essay set imported (20 essays across Q1–Q5) for cohort-fresh practice
- Every scoring change checked against essays graded by humans
Challenges & Solutions
In July 2026, a stricter scoring rule passed all 38 unit tests and still made the product worse. The golden set showed that it reduced agreement with human graders, so I removed it from the holistic grading path that day and kept only the narrower part the evidence supported. Unit tests proved that the code followed my rule. The golden set showed that my rule was wrong.