A vendor risk questionnaire that actually discriminates

A vendor risk questionnaire has one job: to produce answers that differ between vendors. If every vendor can answer every question with "yes" or "fully compliant," the questionnaire is measuring politeness, not risk. Two changes fix most of it (asking for artifacts instead of assurances, and asking about exceptions instead of the normal case), plus a disciplined way of reading what comes back.

Ask for artifacts, not adjectives

The same topic, asked two ways. The weak version can be answered in a minute by anyone; the strong version cannot be answered without the thing existing:

• Weak: "Do you obtain consent from speakers?" Strong: "Attach the consent form template from your most recent project, and describe where signed forms are stored and how a single form is retrieved."
• Weak: "Is your collection process ethical?" Strong: "Describe a project you declined in the past two years, and why."
• Weak: "Do you comply with data protection law?" Strong: "For your last three projects involving EU speakers, state the lawful basis used and the transfer mechanism, per project."

Ask about exceptions and refusals

Questions about the normal case get rehearsed answers. Questions about exceptions do not, because exceptions force a vendor to describe reality:

• When a speaker withdraws consent after delivery, what happens to their files in datasets that have already shipped?
• Describe a delivery that failed acceptance, and what the root cause turned out to be.
• What share of recorded sessions is discarded in quality control, and where does the discarded audio go?

Reading the answers

The signals that separate a described process from a rehearsed one:

• Numbers that disagree across sections: speaker counts in one answer that cannot produce the session totals in another.
• Requested attachments missing, replaced by an offer to "walk through it on a call."
• Uniform maximum scores on every topic, including the ones where the honest answer is "partially."
• Answers that weaken between the sales call and the written response: always collect the written version after the call and compare.
• A subprocessor list that names only the cloud provider.

Scoring, without pretending it is science

Sort questions into three tiers rather than averaging everything into one number. Disqualifying: no consent records for collected data, refusal to name subprocessors, inability to describe the rights basis of the corpus being sold. Weighted: depth of the described process, retention specifics, exception handling. Informational: tooling, certifications, office security.

Certifications belong in the informational tier and nowhere else. They describe the vendor's own information systems; they say nothing about whether the data being sold has a sound consent chain, which is usually the risk the buyer is actually carrying.

When to send it

Send the written questionnaire after the first call, not before. The call tells you which claims need testing; the questionnaire tests them. Set a deadline for the artifact attachments, keep the responses in the project record, and make the contract reference the version of the questionnaire that was answered.

One more discipline: ask the same core questions of every vendor, and keep the questions stable across projects. A questionnaire that gets rewritten for each evaluation cannot compare anyone to anyone, which is most of its value. And when a vendor's written answers are weaker than their spoken ones, that gap is itself the finding. The written version is the one that will describe what actually happens.

The questions that only exist for training data

Generic vendor questionnaires cover information security, business continuity and privacy policy. Those matter, but they do not touch the risk a data buyer is actually carrying. These do:

• What rights does the agreement grant: training, evaluation, internal research, or redistribution? These are four different products and they are routinely conflated in a single line of a proposal.
• Does the grant extend to derivative works, and does it survive being incorporated into model weights? This is the question that decides whether a model trained on the data can itself be licensed onward.
• For any dataset not collected by the vendor, can they produce the chain from the original rights holder to themselves, in documents rather than in a description?
• What is the position on synthetic or model-generated content inside the corpus, and is it labeled per record?
• If the vendor also trains models, do they train on the same data they sell, and does that create a competitive overlap with the buyer?
• What happens to the buyer's rights if the vendor loses the upstream license?

Keeping the process honest over time

A questionnaire answered once describes a vendor at a moment, and data supply chains change faster than most vendor relationships. The maintenance discipline is what keeps the answers meaningful.

Re-send the core set on a fixed cadence, annually at minimum, and again on any trigger: a change of ownership, a new subprocessor, a new source country, or a change in what the vendor does with the data internally. Version the questionnaire and record which version each vendor answered, so a comparison across years compares like with like.

Keep the artifacts, not just the scores. The completed questionnaire is a record of what the vendor claimed at a point in time, and if the claim later turns out to be wrong, the artifact is what establishes whether it was a mistake or a misrepresentation. That distinction matters more than the score ever will.