What the tools do and how they fail

Tools sold to legal teams do a handful of jobs, inside the programs lawyers already use:

1. Draft a document from an approved template and a few instructions.
2. Review a contract against a set of preferred positions and suggest changes, often shown as tracked changes.
3. Extract facts, such as parties, dates, amounts and obligations, from one document or thousands.
4. Summarise a document, a set of emails, or a whole matter.
5. Research: find and explain law, precedent or market terms.

Their makers report speed, and often a measure of use, such as the share of suggestions that lawyers keep. That is useful, and it is not the same as being right. This course is about the difference.

Errors fall into a short list. Learn it, and you will know what to look for:

• Wrong figure. 60 days becomes 90. An extra zero turns USD 1,000,000 into USD 10,000,000.
• Wrong unit. 1.5% per month becomes 1.5% per year. One word, opposite meaning.
• Wrong place or law. The courts of England become the courts of New York.
• Wrong party. The Supplier’s duty is given to the Client, or a company is called Ltd in one place and Inc. in another.
• Wrong date. Counting days backwards from a deadline and landing two days off.
• Invented term. A clause that sounds standard appears in the output but is nowhere in the source.
• Omission. The carve-out, exception or condition that mattered most is left out, and the rest reads smoothly.

Two more failures are about behaviour, not text: a tool that agrees with whatever you suggest, and a tool that is used with a client’s confidential material without anyone checking what it stores.

A wrong output rarely looks wrong. It is well written, confident and neatly formatted. A human error often shows in clumsy wording. A tool’s error does not. So you cannot judge a summary by how it reads. You judge it by matching each claim to a line of the source.

1. Go claim by claim. For each sentence in the output, find the line in the source that supports it. No line, no claim.
2. Re-check every number, unit, name and date yourself, and recompute any date.
3. Look for what is not there. Which important terms in the source do not appear in the output?
4. Write down what you found, including what was right. The record is how you learn how reliable the tool is (Modules 2 to 4).

The lawyer. Professional responsibility does not move to the software. If a wrong summary reaches the client, the lawyer who relied on it and sent it is answerable. A tool is an assistant that must be supervised, like a junior who works very fast and never admits to doubt.

• Reading the output for sense instead of checking it against the source.
• Checking the first paragraph carefully and the rest lightly.
• Not looking for omissions.
• Trusting a date or sum because it was formatted neatly.
• Pasting a confidential document into a tool without checking where it goes.