Module 1
What litigation analytics is, and what it is not: questions, data, limits and honest claims
Clients ask lawyers four questions over and over: Will we win? How long will it take? What is it worth? What will it cost? For most of the history of the profession, the answers came from experience, memory and instinct, sometimes very good and sometimes badly overconfident. Litigation analytics is the practice of answering those questions with data from past cases. It can make a lawyer’s answers more honest, more consistent and easier to explain. It can also give false confidence, because case data is full of traps: the cases that settle disappear, the cases that are still open look as if they never end, small samples look like laws of nature and the facts that mattered most were never recorded. This course teaches the data side of those four questions and the honesty that goes with it. It builds on Legal Risk Quantification, which turns judgement into numbers, and Matter Economics and Pricing, which looks at the money of a matter. Everything is jurisdiction-neutral, and every case in the course is invented.
Hover over an idea to light up what it connects to, click it for its definition, minimum requirements, an example and the usual trap, press Play to watch the map build step by step, or trace a short case through it.
Simplified teaching map, in general terms. Check the law where you work. On a small screen, scroll the map sideways.
By the end of this module you can
- Name the questions analytics can and cannot answer.
- List the sources of case data.
- Explain why case data is biased.
- Make honest claims from numbers.
- State the ethical limits.
Part 1. The lecture
Four questions and a family of answers
| The question | The data that speaks to it | The honest form of the answer |
|---|---|---|
| Will we win? | Outcomes of comparable past cases: how often a claim of this kind succeeded in this kind of court | A base rate with an interval and caveats, not a promise. |
| How long will it take? | Times from filing to closing, including cases still open | A range or a median with the share closed by a date. |
| What is it worth? | Awards and settlements in comparable cases, brought to today’s money | A range with a median, not an average. |
| What will it cost? | Costs and time recorded in past matters, by stage | A range by stage, with the assumptions stated. See Matter Economics and Pricing. |
| Which court, which route, which opponent? | The same data split by forum, route or party | A comparison with its limits, and a note of the ethical rules that apply. |
| What would change the answer? | How outcomes differ with particular facts (documents, witnesses, claim size) | A short list of factors that move the odds, each with a reason. |
Analytics is not a replacement for legal analysis. The law of the case, the quality of the evidence and the particular judge’s approach are still the lawyer’s work. Analytics gives a frame, the outside view of how cases like this one have gone, against which the inside view of this case can be tested.
The inside view and the outside view
- The inside view looks at the details of this case: its documents, its witnesses, the law and the opponent. It is rich and persuasive, and it is where overconfidence breeds, because every case feels special.
- The outside view asks how often cases of this kind have gone a certain way. It is blunt and often humbling, and it is a check on the inside view.
- Good forecasting starts with the outside view (the base rate) and then adjusts for what is particular about the case. Module 6 shows how. Starting from a vivid story and adding up its features, without a base rate, is the commonest way for lawyers to get probabilities wrong.
- Data cannot see everything. The decisive fact in a case is often not in any table: a witness’s credibility, a document that appears late, a judge’s irritation. Analytics gives the odds before those things are known.
Where case data comes from
| Source | What it contains | Typical problems |
|---|---|---|
| Court dockets and registers | Case numbers, parties, dates of filing and closing, types of claim, sometimes outcomes | Outcome fields are often missing or coarse; dates may be of filing in a different court; access and terms vary by country. |
| Published judgments | Reasoned decisions, with facts, law and outcomes | Only some decisions are published, and they are not typical: cases that reach a reasoned judgment are a selected group. |
| Law reports and commercial databases | Searchable judgments and, in some services, extracted analytics | Coding choices are hidden inside the product. Check how outcomes and categories were defined. |
| Your firm’s or your client’s own files | Outcomes, durations, costs and awards in matters you handled | Small samples, a particular kind of client and your own recording habits. |
| Regulators, ombudsmen and tribunals | Statistics on complaints and decisions | Different populations from courts; reporting rules vary. |
| Litigation funders and insurers | Portfolios of cases with outcomes and costs | Confidential and selected by what they chose to fund. |
| Surveys and published studies | Aggregates and research findings | Check the sample, the period and the country. |
- Check the licence and the law before you collect, store or use data. Court data may contain personal information and may be subject to data protection law and to restrictions on reuse. See GDPR and AI and Law and Cybersecurity.
- Record where each row came from and the date you collected it. Data without provenance cannot be checked.
- Keep client confidentiality. Never put client-identifying material into a tool or a shared dataset without permission and proper controls.
Why case data misleads
| Trap | What happens | What to do |
|---|---|---|
| Settlement selection | Most disputes settle and leave no judgment. The cases that reach a decision are those the parties could not agree about, which tends to make them closer calls than the average dispute. Win rates among decided cases therefore say little about a typical claim. | Say what the population is (decided cases, all filed cases, all claims including settled ones) and do not generalise beyond it. |
| Open cases | Cases still running are the slowest ones. Leaving them out makes courts look quick; counting them as finished makes them look quicker still. | Treat them as open. See Module 4. |
| Survivorship | You see the cases that made it into the dataset, not those that were abandoned, struck out early or never filed. | Ask what is missing, and why. |
| Small samples | A handful of cases gives a rate with a very wide margin. Twelve cases and nine wins is not a 75% win rate; it is a guess between about 47% and 91%. | Quote intervals. See Module 3. |
| Changing law | A rule changed, and earlier cases no longer apply. | Split before and after, or restrict to the current law. |
| Different kinds of case | Pooling easy and hard cases, or large and small claims, hides what matters. | Stratify, and check for reversals (Simpson’s paradox). |
| Measurement choices | What counts as a ‘win’ (a partial win? a settlement above the offer?) changes every number. | Define outcomes before you look, and write the rule down. See Module 2. |
| Unrecorded facts | The strength of the evidence is rarely in the table. | Use data as the outside view, and add the inside view. |
| Ecological and base-rate fallacies | Treating a group average as a fact about a particular case, or ignoring the base rate in favour of a vivid story. | Keep the two views apart and combine them carefully. |
What an honest claim sounds like
| Overclaim | Honest version |
|---|---|
| ‘We have a 75% chance of winning.’ | ‘In the 40 comparable decided cases we found, 30 were won by the claimant (between roughly 60% and 86%). Your case has these strengths and these weaknesses, which could move it either way. Cases that settle are not in the figure.’ |
| ‘This will take eighteen months.’ | ‘Half of the comparable cases closed within about fourteen months, and nine in ten within about thirty, including cases still open. Your case has a feature that has been associated with delay.’ |
| ‘The average award is 80,000.’ | ‘The middle award in comparable cases was about 35,000, and the middle half were between 25,000 and 45,000. A few very large awards pull the average up, so I would not plan on it.’ |
| ‘This judge always rules for defendants.’ | ‘In the twelve decisions we found, the judge ruled for the defendant in nine. That is too few to say much, and our data does not show why. Please do not rely on it.’ |
| ‘The model says 82%.’ | ‘A model trained on similar past cases gives about 82%, but we have checked its past forecasts and it was overconfident above 80%. I would use a lower figure.’ |
- Say what the population is, how many cases and what period.
- Give a range, not a point.
- Say what is missing or uncertain.
- Separate the data from your judgement, so that the client knows which is which.
- Do not hide behind the numbers. A number is not a decision. The decision is the client’s, with advice.
Ethics, fairness and rules
- Honesty with clients: do not present analytics as more certain than it is. A client who litigates because of a false percentage, or abandons a good claim because of one, has been let down.
- Fairness to people: analytics about individuals (judges, lawyers, litigants, experts) can be unfair or inaccurate if data is thin. It can also affect the independence of judges and the fairness of proceedings. Some countries restrict or prohibit the use of data to evaluate or predict the behaviour of individual judges, and others regulate the reuse of court data. Check the rules where you work, and see Module 7.
- Access to justice: analytics can help ordinary people to value claims and decide whether to proceed, and it can also be used to screen out weak claims, push undervalued settlements and favour repeat players. Think about who benefits.
- Bias in the data: if past outcomes reflect bias (against a group, a kind of claim or a kind of lawyer), a model built on them can repeat it. See Checking Legal AI Tools.
- Confidentiality and data protection: use client and court data lawfully and securely.
- Professional duties: a lawyer remains responsible for advice that uses analytics, and must understand it well enough to explain it. ‘The tool said so’ is no defence.
Where this course goes
The next modules follow the work: cleaning a table of cases, win rates and what they can and cannot say, durations when cases are still open, award ranges, forecasting a case and scoring your forecasts, and finally the analysis of courts and judges, and how to report all of it. Each has a working tool with invented data.
Part 2. Lab: Can Data Answer It?
Sort each question by how far case data can help.
Situation 1 of 6
How long do claims of this type usually take in this court?
Where does this belong?
Part 3. Lab: Choose the Honest Claim
Choose the best response in each situation.
Situation 1 of 3
Your data shows that 9 of 12 comparable decided claims succeeded. A client asks, ‘So we have a 75% chance?’
What is the best response?
Part 4. Lab: Spot the Overclaim
Click the statements that claim more than the data supports.
Statements about cases
Click every sentence that causes a problem, then press Check. There may be 3 problems, and some sentences are fine.
Part 5. Check your understanding
Ten questions drawn from a larger pool. Sign in to have your score saved towards your certificate.
You have finished the free module
Here is what you can do now, and where the course goes next
Module 1 gave you
- Name the questions analytics can and cannot answer.
- List the sources of case data.
- Explain why case data is biased.
- Make honest claims from numbers.
- State the ethical limits.
Next, 6 more modules and a capstone
- Module 2: The case table: what a row is, how to code outcomes, and how to clean what you collectYou will learn to define the unit of analysis.
- Module 3: Win rates and base rates: what a percentage can and cannot tell youYou will learn to choose the right denominator.
- Module 4: How long it takes: durations, percentiles, open cases and honest timelinesYou will learn to decide what to time.
- Module 5: What it pays: awards, settlements, ranges and what a client can actually expectYou will learn to separate awards from settlements.
- Module 6: Forecasting a case and scoring your forecasts: from base rate to probability, and whether you were any good
- Module 7: Courts, judges, opponents and ethics: analysing people fairly, and reporting results to clients
- Capstone: Capstone: advising Wexcombe Fabrics on a supply dispute
Each module has its own practice labs and a graded quiz, and a Certificate of Completion follows when you pass them all. The whole course is $15 once, or included in an All-Access Membership.
Two Ways In
Introductory PriceLitigation Analytics and Outcome Forecasting: buy it on its own, or get it with All-Access
Unlock all of Litigation Analytics and Outcome Forecasting on its own with a one-time purchase, or get it as part of an All-Access Membership, which also covers every course and the other paid tools. A member never needs to buy it separately.
This is an early, introductory price for a brand-new product — it’s expected to rise as we add more content. Buy now and it’s yours at today’s rate, forever.
No account needed: you pay, and we email you a one-click sign-in.
All-Access is $19.99 a month or $149 a year, and covers every course and most tools, this one included. Create a free account to subscribe.
Look first, then buy. Free previews on every product page. Secure payment by Stripe. Pay once, keep it forever. Details
See what is included in All-Access →Not ready to buy? Create a free account to save your quiz scores and your place in this course, and to get the printable reports from the practice rooms.
General education on analysing litigation data, using invented cases, courts and figures. It is not legal advice and does not predict the result of any real case. Rules on court data, data protection, confidentiality and the analysis of judges differ between countries and change: check the rules that apply to you.
Teaching? Share this with your class
Free to use in class. The free parts open for everyone, no account needed.