Module 7
Governance, confidentiality and ethics
By the end of this module you can name what must never go into a public tool, put the right questions to a vendor, apply a simple go or no-go rule, and set up monitoring so that a decision to rely on a tool is not treated as final.
Part 1. The lecture
The responsibility stays with you
A tool is a way of doing your work, not a way of passing it on. Whatever the vendor’s terms say, your duty to your client does not move. That is why every other module of this course ends up in a person checking an output before anyone relies on it.
Confidentiality: what goes in
- Never put real client material into a public tool whose terms you have not read. It may be stored, reviewed by staff, or used to improve the vendor’s models.
- Ask whether your inputs and outputs are used for training, and get the answer in the contract, not in a sales call.
- Ask where data is stored and processed, and who the sub-processors are. Location decides which laws apply, and a refusal to say is itself an answer.
- Ask what happens at the end: deletion, return and confirmation.
Duties of confidentiality, privilege and disclosure to clients differ from place to place and change over time. This course teaches the testing skills, not any one jurisdiction’s rules. Check your own regulator’s current guidance and your clients’ terms, and treat that checking as part of the job.
A go or no-go rule
Before approving a tool, write the gates in advance. Split them into critical gates (no training on your data, confidentiality protected, tested on your own sample and meeting your target, a person reviewing every output) and important ones (agreed data region, audit log, disclosed sub-processors, exit terms). Decide the rule too: any critical gap means no-go; an important gap means a conditional go, fixed before use. Silence from the vendor counts as a no.
The point of a written rule is that a polished demonstration cannot outweigh a missing safeguard.
After the go
- Keep an audit log of suggestions and decisions, so you can show later what was relied on.
- Re-test on a fresh sample on a schedule and after any model update. Ask the vendor to give you notice of changes.
- Write the stop rule: the result that would make you stop using the tool.
When not to use a tool
Do not use one when you cannot verify the output, or cannot protect the confidentiality of the input. Convenience, a tight deadline or a colleague’s enthusiasm is not a reason. If the task needs judgement that only you can give, the tool can prepare the ground, but the judgement is yours.
Part 2. Lab: Vendor Questions
Alderwick is considering a contract-review tool and has one hour with the vendor. Twelve questions are on the table. Tick the ones that are worth the vendor’s time and yours. Some are strong because they get a checkable answer. Others invite a sales pitch.
Part 3. Lab: Go or No-Go
Alderwick uses a simple rule. Four gates are critical: if any is not a clear yes, the answer is no-go. Four are important: a gap there means a conditional go, to be fixed before go-live. A silence from the vendor counts as a no. Decide each tool, then check.
| Gate | Type | Tool A | Tool B | Tool C | Tool D |
|---|---|---|---|---|---|
| Our data is not used to train the vendor's models. | Critical | Yes | Not answered | Yes | Yes |
| The contract protects confidentiality and privilege. | Critical | Yes | Yes | Yes | Yes |
| We tested it on our own sample and recall met our target. | Critical | Yes | Yes | No | Yes |
| A person reviews every output before anyone relies on it. | Critical | Yes | Yes | Yes | Yes |
| Data is stored and processed where we have agreed. | Important | Yes | Yes | Yes | Yes |
| The tool keeps an audit log of suggestions and decisions. | Important | No | Yes | Yes | Yes |
| The vendor discloses its sub-processors. | Important | Yes | Yes | Yes | Yes |
| There are clear deletion and exit terms. | Important | Yes | Yes | Yes | Yes |
Tool A
A contract-review assistant, tested on 300 of our contracts. The vendor has not yet set up an audit log.
Tool B
A drafting assistant. The vendor says only that it takes data protection seriously and will not say whether customer data is used for training.
Tool C
An extraction tool with a strong contract and a full audit trail. Our test sample found recall of 62 per cent, below our 85 per cent target.
Tool D
A review assistant with every protection in place and a recall of 91 per cent on our own sample.
Part 4. Check your understanding
Ten questions drawn from a larger pool. In this pilot your score is shown but not saved.
This course teaches how to test and supervise tools, using invented documents, fictional parties and made-up tool outputs. No real tool is run, and nothing here is legal advice. Never paste a real client’s confidential material into a public tool.