Estimating likelihood

When a lawyer first reads a file, the case feels unique. The documents, the people and the arguments are all vivid. That feeling pulls the estimate toward whatever the strongest piece of evidence suggests, and away from how cases like this usually end.

The better habit is to ask first: what usually happens in cases like this one? That is the outside view. Only then do you look at the inside, the specific facts that make this case better or worse than typical.

A base rate is how often something happens in a group of similar past cases. The group is the reference class. Choosing it is the hardest part:

• Too broad (“all disputes”) and the rate says little about your case.
• Too narrow (“this judge, this month, this product”) and there are too few cases to trust.
• A good class is similar in the things that matter (type of claim, size, legal rule, kind of parties) and large enough to be stable.

Where do base rates come from? Court statistics, published studies, a firm’s own closed files, or an insurer’s records. Where none exist, say so plainly and widen your range. Never invent a base rate and present it as data. The figures in this course are invented for practice and are labelled as such.

From the base rate, move up or down for each factor that makes Alderwick’s case different: the strength of the documents, the credibility of the witnesses, how clearly the law favours the client, and the strength of the other side. Do it one factor at a time, and write one sentence of reasoning for each move.

The lab below moves the estimate in steps on a scale called log-odds. You do not need the mathematics. What matters is what it does: the answer can never leave the 0 to 100 per cent range, and a step of evidence counts for more near the middle than near the ends, which is how real evidence behaves.

• Anchoring. The first number you hear, perhaps the client’s confidence, drags your estimate toward it. Start from the base rate.
• Overconfidence. People who say “90 per cent sure” are usually right far less often than that. The Calibration Game below measures yours.
• Hindsight. Once the result is known it seems obvious. Write your estimate down before the result, so you can learn from it honestly.

You are well calibrated if, of all the things you call 70 per cent likely, about 70 per cent happen. A simple score for yes or no forecasts is the Brier score: the average squared gap between the chance you stated and what happened. Zero is perfect. Always saying 50 per cent scores 0.25. Lower is better.

Lawyers rarely get this feedback, because outcomes arrive months later and nobody compares them with the original estimate. You can practise it now, on questions with known answers.

• Skipping the base rate and starting from the client’s view or the strongest document.
• Using a tiny group of cases as if it were a reliable rate.
• Moving the estimate with no written reason, so nobody can check it.
• Reporting 71 per cent when the honest statement is 60 to 80 per cent.