The EU AI Act is now a procurement question

For two years the AI Act was something that concerned model developers. That has changed, because the features it describes have arrived in the software ordinary companies already run: CV ranking inside an HR system, lead scoring inside a CRM, a chat assistant answering customers, a tool that drafts performance feedback.

The part buyers miss is that the regulation does not only bind the vendor that built the system. It also binds the deployer — the organisation using it under its own authority. That is you.

The shape of the rules

The Act sorts AI systems by risk rather than by technology. A small number of uses are prohibited outright. A defined set is high risk, which is where most of the obligations sit — and the list includes uses that turn up in perfectly ordinary business tools: recruitment and candidate selection, decisions about promotion or termination, task allocation and monitoring at work, and access to essential services such as credit. A further layer requires transparency: people should be told when they are interacting with an AI system, and certain synthetic content has to be marked as such.

If your use of a product falls in the high-risk set, the vendor carries the bulk of the conformity work — but you carry duties of your own: using the system according to its instructions, assigning competent human oversight, keeping the logs, and informing affected workers before you put it into use.

Three questions for a vendor with AI features

1. Which of these features do you consider to be in scope, and as what? A vendor that has done the work will answer in the Act's own vocabulary. One that has not will answer with reassurance.
2. What is the model behind it, and where does the inference run? This is a data protection question as much as an AI one: a "summarise this thread" button that posts customer data to a third-country API is a transfer you have to be able to describe.
3. Can the feature be turned off per workspace, per team and per user? If a feature is high risk in one of your use cases and harmless in another, the ability to disable it selectively is what keeps the harmless use available.

Where your own obligations start

• Human oversight has to be real: a named person, with the authority and the information to overrule the system, not a rubber stamp in the workflow.
• Workers and their representatives must be told before an AI system is used to make or support decisions about them at work.
• Staff who operate the system need enough AI literacy to understand what it does and where it fails — the Act asks for this explicitly.
• Keep records. If a decision affecting a person was supported by an AI system, you may have to explain how, months later.

What to put in the contract

• A commitment to provide the technical documentation and instructions for use that a deployer needs.
• Notification when the system materially changes — a model swap can change behaviour without changing a single screen.
• Log retention and access, so you can meet your own record-keeping duty.
• The right to disable AI features without losing the rest of the product.

On the dates

The Act applies in stages, and the timetable has been the subject of active political discussion, including proposals to adjust parts of it. Treat any specific date you read — here or elsewhere — as something to confirm against the current text before you rely on it. What is not in doubt is the direction: AI features in business software are now a procurement question with a paper trail, in the same way GDPR became one a decade ago.

The place this bites first for most companies is HR and marketing. If you are evaluating tools in those categories — Personio and GetResponse among them — ask the three questions above during the trial, not at renewal.