The security questions to ask a legal AI vendor

You are about to hand someone your clients’ medical records and your privileged work product. Most vendor security pages are written to reassure rather than to inform — here is what to ask instead, and how to read the answers.

Every legal AI vendor has a security page. Almost all of them say the same four things: encrypted in transit and at rest, SOC 2, role-based access, we never train on your data. Those sentences are cheap to write and most are true in some narrow sense, which is what makes them useless for telling vendors apart.

The questions below are the ones where a vague answer tells you something. None of them require a technical background to ask, and you can send the lot to a vendor in an email.

“SOC 2 compliant” is not a thing. Ask which report exists.

There is no such thing as being SOC 2 certified. SOC 2 is an attestation: an independent CPA firm examines a company’s controls and issues a report carrying an opinion. Nobody issues a certificate, and a badge on a website means nothing on its own.

What matters is which of the two reports exists:

• Type 1 says the controls were designed appropriately, as of a single date. It is a photograph.
• Type 2 says the controls actually operated effectively across a period — usually three to twelve months. It is a video, and it is the one that carries weight.

A vendor with a Type 1 has shown an auditor a set of policies on one day. That is a real step, and it is not evidence that anyone followed them afterwards. Ask which one they hold, what the observation period was, and when it ended — a Type 2 covering a window that closed two years ago is describing a company that no longer exists.

Then ask for the report itself under NDA. Vendors who have one will send it. The report contains the auditor’s opinion, the scope, and — the part worth reading — any exceptions the auditor noted. A summary page on a website never mentions exceptions.

Ask what stage they are actually at

Most early-stage vendors are somewhere inside a readiness programme rather than holding a finished report, and there is nothing wrong with that so long as they say so. Getting to an audit means mapping controls, closing the gaps, and collecting evidence over months — specialist firms run these programmes, often mapping several frameworks onto a single control set so the same work is not done twice. It is a project with a timeline, not a switch someone flips.

So “we are SOC 2 Type II in progress, the observation window opens in March” is a real answer you can put in a diary. “We take security very seriously” is not an answer, and a vendor who blurs the two in a first email will blur other things later.

If you do personal injury, ask about a BAA

Plaintiff firms handling medical records are working with protected health information. Where your firm acts as a business associate of a covered entity, the vendors you pass that information to are your subcontractors, and HIPAA expects a written business associate agreement down the chain.

Ask directly: will you sign a BAA? A vendor built for healthcare-adjacent work has one ready. A vendor who has never been asked will need weeks — and that delay tells you roughly how many firms like yours they already serve.

Worth a call to your own carrier too. Several professional liability policies now ask specifically about AI vendors and about what agreements are in place with them.

Ask who else touches the data

Every AI product is built on other companies’ infrastructure: a cloud host, a model provider, a transcription service, an OCR service. Those are subprocessors, and your client’s file passes through all of them.

Ask for the list, in writing, with what each one does. Then ask whether you are notified before a subprocessor changes. A vendor who cannot produce the list has not thought about it, which means nobody has checked what those providers are contractually permitted to do with what they receive.

“We do not train on your data” — get the scope in writing

This sentence has more room in it than it appears to. Three follow-ups close the gap:

• Does it cover the model providers you send data to, or only your own systems?
• Does it cover everything derived from the documents — extracted facts, embeddings, generated drafts — or only the original files?
• Is it in the contract, or only on the website? A marketing page is not a commitment, and it can be edited on a Tuesday.

Ask what deletion actually deletes

When you delete a document, the file disappears from the interface. The question is what happens to everything the system built from it: the extracted facts, the search index, the embeddings, the drafts that quoted it, the backups.

A system that removes the file but keeps its derived data still holds your client’s medical history in a searchable form. Ask what is removed, how long backups retain it, and what you get on the way out — if “can we export everything if we leave” draws a vague answer, you have a lock-in problem as well as a privacy one.

What a good answer sounds like

Specific, dated, and occasionally unflattering. “Type 2 report issued in April, observation period October to March, two exceptions noted around offboarding timeliness, both remediated — here is the report” beats any badge. So does “we are not there yet, here is where we are and when the window closes.”

The vendors worth trusting with a case file are the ones who answer the awkward version of the question without being asked twice.

Where we are

Applying the test to ourselves: Luma is working toward SOC 2 Type II and does not hold a report yet. We say so on the Security page, and we will publish it there when one exists, with the period it covers. We would rather lose a deal to a vendor who has one than win it by implying we do.