A tutor who forgets you between sessions is a grader

A tutor who forgets you between sessions is a grader. The grade can be accurate and the feedback can be sharp, but if the next essay starts from zero, the student is paying for judgment and getting no memory, and memory is the part that changes how you study. BarPlaybook was a grader in that sense until April. It analyzed each essay well and each essay alone, so a student who missed the same negligence element three weeks running got three separate notes about it and nothing that said "this is the third time."

The fix came from an idea that wasn't about tutoring at all. In April, Andrej Karpathy described how he uses a model to keep a personal knowledge base, where raw sources go into one directory, the model compiles them into a wiki of linked markdown pages in another, and he asks his questions of the wiki rather than of the pile. The distinction he drew is between compiling knowledge once and re-deriving it on every query, which is what retrieval systems do when they pull chunks of raw text each time you ask. A wiki is a persistent artifact that gets better as you use it, since the answers to your questions get filed back into it. A day or two later he added the part I found most useful, which is that the memory is explicit. You can open the wiki and see exactly what the model does and does not know about you, as opposed to an AI that "allegedly gets better the more you use it" in some way you can't inspect.

I read that and thought about what a bar student actually needs remembered. It isn't the law, since they have outlines and a bar-prep course for that. What nobody keeps for them is the record of their own failures: which issues they don't spot, which element of a rule they keep dropping, whether the problem is getting better or just moving around. A human tutor keeps that record in their head, and it's the most valuable thing they carry, but it lasts one session and it lives with the tutor rather than the student. Karpathy's shape fit that problem closely enough that I had a plan written four days after his post, and the first working version a day after that, once a first attempt that had built a search client for Wikipedia was thrown out.

The adaptation changed three things about his design, and each change was forced by who the reader is.

The first is that the raw sources are the student's own graded essays, and every student gets their own wiki. Karpathy's raw layer is whatever research he's reading; BarPlaybook's is the per-issue verdicts the grader already produces on every submission, the determination that an issue was fully answered, partially answered, or missed, and which rule element went missing when it was. The compiler turns those into two kinds of page. An issue page tracks one legal issue across everything the student has written, and a mistake-pattern page tracks one specific way of getting it wrong, so "negligence" and "negligence, missing the duty element" are different pages with different counts.

The second is that the model does not write the wiki. In Karpathy's version the model owns the wiki layer entirely and the human rarely touches it, which is right for research notes, where the writing is the work. Here the judgment has already happened during grading, so the compiling is plain code. It canonicalizes the mistake, creates or updates the page, increments how many times the pattern has been seen, and attaches a pointer to the essay and the analysis it came from. I did it this way because a record of someone's failures has to be traceable. If a page says you've dropped the duty element four times, a student should be able to click through to the four essays, and a page that can't show its evidence shouldn't exist. The lint pass Karpathy runs for contradictions and orphans became a lint pass for exactly that, flagging pages missing evidence pointers, pages nobody has updated in 45 days, and pages nothing ever retrieves.

The third is that the query isn't the student asking a question but the coach reading before it speaks. Before BarPlaybook writes its overall feedback on a new essay, it pulls the most relevant pages from that student's wiki for that subject and hands them to the coaching step with an instruction to connect the feedback to the long-term pattern. That is the mechanism that finally lets the feedback say "this is the third time," and it adds no new analysis at grading time, since the pages were compiled when the earlier essays were graded rather than assembled from scratch now.

What the student sees is a section called the Playbook. It lists their pages with the weak spots first, and each page shows how many essays it has been seen in, when it was first and last seen, how confident the record is, and a list of the essays it came from. The dashboard carries the top recurring mistakes so the pattern is visible without going looking for it. This is Karpathy's explicitness point turned into a product surface. The student can read what the tutor believes about them, check it against their own essays, and watch a count go up or stop going up, which is a very different experience from being told to "work on issue spotting" at the end of every session.

The catch is the one you'd expect. A wiki compiled by code is only as good as the verdicts it compiles, and a wrong determination doesn't vanish the way a wrong sentence of feedback does. It becomes a durable belief about the student. That's why the evidence links matter more than any other feature on the page, and it's why the grader upstream is checked against essays scored by human graders before any scoring change ships, since memory raises the stakes on being right.

I don't think this is only a bar-exam idea. Any product that gives people judgment on repeated work, code reviews, writing feedback, coaching of any kind, can keep the same explicit record, and the interesting design questions are the same three I hit: what counts as a source, whether the model or the code does the compiling, and who the reader of the wiki really is. A grade tells you where you landed this time, but the wiki is the record of where you keep landing, and that record is the part a student can actually study.