AI vendor due diligence for a UK SME: the twenty questions, and the answers that should end the conversation.

The five that decide whether a product can hold your data at all. Ask them first; the rest of the conversation depends on the answers.

Which provider and which model does the product run on, and does that change without notice? A vendor that will not say is hiding either a cheap model or a dependence it does not control. What happens to accuracy and cost when the provider retires a model, and who tests the replacement? And can we see the system prompts and the rules the product applies to our data, or at least the categories of them? “Proprietary” is an acceptable answer for a large vendor’s internals; it is not an acceptable answer to whether your client’s data is sent to a third model provider you have not recorded.

What is the measured accuracy on inputs like ours, and can we test it on fifty of our own before signing? A vendor confident in its product agrees to the test; one that offers a demo on its own examples instead is telling you something. How is it monitored after go-live, and what does drift look like? And what is the exceptions path: when the product is unsure or wrong, does it stop and ask a person, or does it guess? A product without an exceptions path is a product that guesses.

What access does the product need to our systems, and is it the least that does the job? What are your controls against prompt injection, given the NCSC’s position that it cannot be fully prevented? Which attestations do you hold (SOC 2, ISO 27001, ISO 42001, Cyber Essentials) and will you share the reports? And how quickly will you tell us of an incident affecting our data? A vendor that asks for full admin access to your mailbox or CRM, has not heard of prompt injection, and has no attestations is a vendor to walk away from, however good the demo.

What is the running cost per unit, per conversation, per document, per call, and what is included in the subscription before overage begins? And what does the bill look like at ten times our volume, with what ceilings? Credits-based pricing, which several platforms have moved to, needs a worked example at your volume, because the headline seat price and the effective price can differ by a large multiple. A vendor that cannot give a per-unit figure has not measured its own product.

For a built system: who owns the code, the prompts, the data and the accounts, and are they in our name from day one? For a product: how do we export everything, in what format, and what happens to our data after we leave? And what is the notice period, the price-change clause and the minimum term? The acceptable answers are ownership in your name, a documented export, and a term you could actually leave at. Our guide on your data when a supplier builds your software sets out the clauses for the build case in full.

The twenty answers, the DPA, the sub-processor list, the attestations, the accuracy test results on your fifty examples, and the worked cost example, in one folder with the contract. That folder is most of what a client’s security questionnaire, an ISO 42001 assessor or an ICO enquiry will ask for, and having it is the difference between an afternoon and a fortnight.