Risk classification under the EU AI Act follows intended purpose and system-level impact — not which model card sits underneath. This is regulatory analysis of publicly available text and guidance, not legal advice for a specific deployment.
Picture two teams at the same company, working from the same enterprise contract with the same API-based general-purpose model. One team builds a tool that rewrites product descriptions in a consistent brand voice. The other builds a tool that ranks incoming CVs so recruiters only need to review the top decile. Same model. Same vendor. Same procurement line item. One of these systems, under the EU AI Act, is close to unregulated. The other is a high-risk AI system with a full compliance regime attached — risk management, technical documentation, human oversight design, logging, a fundamental rights impact assessment, and registration in an EU database — regardless of the fact that neither team trained a single parameter.
The reasoning that gets teams into trouble is almost always some version of: we only call an external API, so we’re not “building AI,” we’re just using a product someone else is responsible for. Nothing in the AI Act supports that conclusion, and the reasoning gets more dangerous, not less, as general-purpose models become the default substrate for internal tooling. The Act does not regulate models in the abstract. It regulates AI systems, defined by what they do, to whom, and what role their output plays in a decision — and a generic model can sit inside systems with radically different regulatory profiles depending entirely on that context.
What is actually being classified
Article 3(1) of Regulation (EU) 2024/1689 defines an “AI system” as a machine-based system, operating with some degree of autonomy, that infers from its input how to generate outputs — predictions, content, recommendations, or decisions — that can influence an environment. The European Commission’s 2025 guidelines on this definition break it into seven components, and one distinction matters more than the rest for this discussion: the guidelines separate a system’s internal “objectives” from its “intended purpose,” where the latter is external, contextual, and defined by Article 3(12) as the use specified by the provider — in the instructions for use, sales material, and technical documentation, not in the model weights.
That single move is why “which model are we using” is the wrong first question. A general-purpose model under Article 3(63) is defined by generality — capable of competently performing a wide range of distinct tasks, integrable into a variety of downstream systems. It has no intended purpose of its own beyond being generally capable. Purpose only attaches once someone wraps that model in prompts, retrieval, business logic, and an interface, and points the resulting system at a specific job. That wrapping step is where an unregulated capability becomes a regulated system.
This is not a settled or trivial distinction. A 2026 academic review that examined nearly 900 papers alongside more than 80 regulatory and standards documents found that the foundational terms “AI model” and “AI system” remain inconsistently defined across the field, tracing most current definitions back to OECD frameworks that compounded rather than resolved the ambiguity. If researchers who study this professionally still find the model/system boundary genuinely contested, it is not surprising that engineering teams default to the wrong mental model — the one where compliance is the API vendor’s problem.
The chain regulators actually look at
It helps to keep the layers separate: general-purpose model → AI component → AI system → business process → decision or outcome. The AI Act imposes obligations at two of these layers independently, and satisfying one tells you nothing about the other.
At the model layer, Article 53 obliges providers of general-purpose AI models to maintain technical documentation and to make information available to downstream providers who integrate the model into their own systems — precisely so those downstream providers can do their own classification work. Foundation model vendors have been under these obligations since 2 August 2025. But GPAI compliance is a model-level regime. It says nothing about what any particular downstream system built on that model does, because the model provider does not control — and often cannot see — the intended purpose of every application built on top of it.
The system layer is where Article 6 does its work, and it runs on two independent tests. Under Article 6(1), a system is high-risk if it’s a safety component of, or is itself, a product covered by EU product-safety legislation listed in Annex I — machinery, medical devices, toys, lifts, and similar regimes that already require third-party conformity assessment. Under Article 6(2), a system is high-risk if it falls into one of the functional use-case categories in Annex III, irrespective of what product safety law says. These two gates are genuinely independent, and a system can walk through either one, or both — the medical-triage scenario below shows why that matters in practice.
The exception most teams misapply
Article 6(3) provides a documented, narrow escape from Annex III classification. A listed use case is not high-risk if it doesn’t pose a significant risk of harm, which the article ties to one of four conditions: the system performs a “narrow procedural task,” it “improves the result of a previously completed human activity,” it detects “decision-making patterns or deviations” without being “meant to replace or influence the previously completed human assessment, without proper human review,” or it performs a preparatory task ahead of an Annex III assessment. Recital 53 supplies concrete illustrations — a system that turns unstructured CVs into a structured, searchable database is a narrow procedural task; a system that tightens the professional tone or brand consistency of a document someone already wrote is improving a completed human activity.
Two things make this exception narrower than it looks in a compliance slide deck. First, Article 6(3) is unavailable the instant the system performs profiling of natural persons — no exception, no case-by-case argument, full stop. Second, and more consequential for anyone building on an LLM, Recital 53 defines the standard the exception has to clear: a system “does not materially influence the outcome of decision-making” when it has no impact on the substance of that outcome, whether the decision-maker is human or automated. A tool that merely organizes information doesn’t touch the substance of a decision. A tool that ranks, scores, filters, or recommends does — even when a human formally signs off afterward. That’s the line the rest of this article keeps returning to.
Same model, eight systems
Below is the same underlying general-purpose model, deployed for eight different purposes inside a typical financial-services or enterprise environment. The classification changes case by case, and the reasoning — not the model — is what changes it.
| Use case | Annex III / Annex I trigger | Article 6(3) exception plausible? | Other obligations regardless of tier | Working classification |
|---|---|---|---|---|
| Marketing copy generation | None | N/A — not an Annex III use case | Art. 50(2): synthetic text output must be machine-readably marked as AI-generated | Minimal risk |
| Internal knowledge search (RAG over policies/wikis) | None, if purely retrieval/summarization | N/A, unless feeding a listed use case downstream | Art. 4 AI literacy for staff using it | Minimal risk — but re-check if its output feeds an Annex III decision |
| CV screening / candidate ranking | Annex III pt. 4(a): recruitment, filtering, evaluating candidates | Unlikely — ranking materially shapes who gets an interview | Registration, FRIA (deployer), Art. 10 data governance, Art. 14 human oversight design | High-risk |
| Employee performance assessment | Annex III pt. 4: monitoring/evaluating performance and behaviour | Unlikely — feeds promotion, remuneration, termination | Same as above; Art. 5(1)(f) prohibits workplace emotion inference outright | High-risk |
| Creditworthiness support (input to underwriter) | Annex III pt. 5(b): evaluate creditworthiness / establish credit score | Narrow — a score that materially informs a human underwriter is still in scope | Same high-risk stack; FRIA is mandatory for creditworthiness use cases | High-risk, even as an “assist only” tool |
| Customer-service routing / intake chatbot | None, as pure routing | N/A | Art. 50(1): must disclose it’s an AI system, unless obvious from context | Minimal/limited risk |
| Fraud investigation support | Explicitly excluded from pt. 5(b) if fraud detection is the actual purpose | N/A if genuinely fraud-only | Re-enters pt. 5(b) if functionally linked to creditworthiness/AML decisions; separate law-enforcement gate (Annex III pt. 6) if outputs go to police or prosecutors | Depends entirely on downstream use |
| Medical triage assistant | Annex III pt. 5(d) names “emergency healthcare patient triage systems” explicitly; may also be Art. 6(1) via the Medical Devices Regulation if it’s a SaMD component | Unlikely — triage materially affects care sequencing | Possible double gate: Annex III and Annex I product-safety conformity assessment | High-risk (potentially through two independent routes) |
A few of these deserve unpacking, because the reasoning is where the value is.
CV screening versus internal knowledge search is the cleanest illustration of the whole article. Recital 53’s own example of a low-risk “narrow procedural task” is an AI system that organizes CVs into a searchable database. The moment that same information is used to rank, filter, or shortlist candidates for a real vacancy, it has crossed into Annex III point 4(a) with no plausible Article 6(3) argument, because ranking is precisely the kind of substantive influence on outcome the exception is designed to exclude. Same document type, same retrieval architecture, arguably the same model — the only thing that changed is whether the system’s output determines who gets seen.
Creditworthiness is where “the AI only assists, a human decides” collapses as a defense. Current guidance is explicit that a credit-scoring or creditworthiness-evaluation system remains inside Annex III point 5(b) even when it produces nothing more than an input for a human underwriter’s final call — the classification test asks whether the system materially influences the substance of the outcome, not who has the final signature. An assistive tool that shapes what the underwriter sees, and how, meets that bar in most real deployments.
Fraud investigation shows the same use case can sit on either side of the line depending on entanglement. Annex III point 5(b) explicitly carves fraud detection out of the creditworthiness category — a system whose actual, documented purpose is catching fraudulent transactions is not automatically high-risk on that basis. But draft Commission guidance on Annex III is equally explicit that anti-money-laundering or fraud tooling that is “functionally linked and simultaneously intended” to inform a creditworthiness decision lands back inside point 5(b). The same fraud-scoring model, wired into two different downstream workflows, gets two different answers. And if a “fraud investigation” tool’s outputs are shared with, or used on behalf of, actual law-enforcement authorities for investigation or prosecution — rather than staying inside a bank’s own risk function — an entirely separate and stricter Annex III gate opens under point 6, covering evidence-reliability evaluation and offending-risk assessment.
Medical triage shows the double-gate problem directly: Annex III point 5(d) names emergency healthcare patient triage systems by name, so a triage assistant is high-risk on the use-case test alone. If that same assistant is also embedded as a component of a certified medical device — software as a medical device under the EU’s Medical Devices Regulation — it can additionally trigger Article 6(1) through the Annex I product-safety route, meaning it inherits obligations from two legal regimes simultaneously rather than one.
Where responsibility actually sits
Article 3(3) defines a provider as whoever develops an AI system — or has one developed — and puts it into service under its own name for its own purpose. Article 3(4) defines a deployer as whoever uses a system, under its own authority, that someone else provided. Calling an external LLM API does not, by itself, make an organization a mere deployer of someone else’s product. The prompts, the retrieval pipeline, the business rules, the interface, and — critically — the stated intended purpose are usually built entirely in-house. Under Article 3(3), that combination is development of an AI system, and the organization that assembled it is very often that system’s provider, with the model vendor sitting one layer down as the provider of a general-purpose AI model under a separate, narrower set of obligations.
Article 25 adds a second trap for organizations that think they’re safely in deployer territory. A distributor, importer, or deployer is automatically reclassified as a provider — inheriting the full provider obligation set — if it rebrands a system under its own name, makes a substantial modification to it, or changes its intended purpose in a way that makes a previously non-high-risk system high-risk. Fine-tuning a model, layering a retrieval pipeline over it, or repurposing a general-purpose chatbot into a benefits-eligibility screener can each independently trigger this flip, regardless of whether the organization intended to take on provider status. Practitioners increasingly refer to this as the “accidental provider” problem, and it is the concrete mechanism behind the article’s opening scenario: the CV-ranking team, not the model vendor, is the one who will need to answer for point 4(a).
RAG, rules engines, and human review don’t change the classification — they change the compliance burden
A frequent, reasonable-sounding objection is that retrieval grounding, deterministic rules engines, or a human sign-off step should move a system out of high-risk territory, since these measures make the output more reliable. They don’t move the classification, and conflating the two is a governance error worth naming directly.
Article 6(3)(c)’s pattern-detection exception only survives if the system is “not meant to replace or influence the previously completed human assessment, without proper human review” — meaning a human rubber-stamping an AI-generated ranking in the ordinary course of business does not satisfy “proper human review” in the sense the exception requires; the review has to be a genuine, capable check, not a formality layered on top of a system already carrying the outcome. RAG grounding, similarly, reduces the odds that a high-risk system produces an ungrounded or fabricated output — which is valuable and, for a system that is high-risk, exactly the kind of measure Article 15’s accuracy requirement and Article 10’s data-governance requirement will demand. But grounding a CV-ranking tool in verified data doesn’t un-rank the CVs. The use case, not the architecture quality, is what Article 6(2) tests.
The timeline is moving; the classification exercise is not
As of this writing, the compliance calendar has shifted meaningfully. The Digital Omnibus on AI — adopted as Regulation (EU) 2026/1744, published 24 July 2026 and in force three days later — postpones most Annex III high-risk obligations from the original 2 August 2026 date to 2 December 2027, and postpones Annex I product-embedded high-risk obligations to 2 August 2028. That relief is real but narrow. Article 50’s transparency obligations — the disclosure duties for chatbots and other systems that interact directly with people, and the marking requirements for synthetic content — took effect on schedule on 2 August 2026 and were not deferred. Article 4’s AI literacy duty has applied since February 2025. GPAI model provider obligations under Articles 51 to 56 have applied since August 2025 and are untouched by the Omnibus.
The practical implication for governance teams is that the deadline moved, but the underlying classification exercise didn’t get any less urgent — architecture decisions being locked in now determine which regime a system falls under in December 2027, and several disclosure obligations are already live. Waiting for the extended deadline to do the classification work simply means doing it later, under more pressure, on systems that are already in production and harder to re-architect.
The question to ask first
“Which model are we using” tells a governance team almost nothing useful, because the model has no intended purpose, sits at the wrong layer of the value chain, and can appear inside a minimal-risk tool and a high-risk one within the same afternoon. The question that actually drives the answer is the one this Act is built around: what does this system do, to whom, and what role does its output play in the resulting decision or action. Answer that first, and the classification — provider or deployer, Annex III or not, exception available or foreclosed — follows from the Act’s own text rather than from which vendor issued the API key.
Leave a Reply