Law and Data · Pilot

Checking Legal AI Tools

Tools that review contracts, draft clauses and pull out obligations are arriving in every firm and every legal department. Their makers publish how many suggestions people accept and, in discovery, how well they find what matters. This course teaches the skill nobody sells: how to test a tool, read its numbers honestly, catch its mistakes, and decide whether to rely on it.

Every tool output in the course is written by us with mistakes planted in it, so the lessons are exact and repeatable. No tool is run, and you need only spreadsheet-level maths. It works in any country. Your fictional client is Alderwick Exports.

Pilot. All modules and the capstone are open to try while we finish the course. Quiz scores are not saved yet, and there is no certificate yet.

Module 1
What the tools do and how they fail
The five jobs, the common ways they go wrong, and why a smooth answer proves nothing. Lab: Find the Error.
Module 2
Measuring a tool
The confusion table, recall, precision and elusion, and why accuracy misleads. Labs: Review Scorer and Why Accuracy Misleads.
Module 3
Sampling and validation
How to test on a random sample, how big, and how sure you can be. Lab: Sampling Simulator.
Module 4
Acceptance-rate analytics
Tracking accepted, rejected and edited suggestions to find a tool's weak spots. Lab: Acceptance Dashboard.
Module 5
Checking extraction
Comparing an extracted obligations table with the source. Lab: Extraction Checker.
Module 6
Playbooks and instructions
Preferred, fallback and red-line positions that a tool can follow. Lab: Playbook Tester.
Module 7
Governance, confidentiality and ethics
What never to paste, which questions to ask a vendor, and when not to use a tool. Lab: Vendor Questions and Go or No-Go checklist.
Capstone
Evaluate a tool
Test a simulated review tool on a batch of contracts and decide go or no-go.

A companion to Legal Risk Quantification, and part of the Law and Data series. Not legal advice.