1. The problem in practice
Imagine a hospital introduces an AI system to help identify patients who need urgent medical attention. The system is carefully developed and validated against the hospital's clinical protocols. It learns which symptoms, patient histories, and other information indicate that someone needs to be seen quickly. Everything works as intended.
Fast-forward two years.
The hospital updates its triage protocols based on new clinical evidence. Perhaps certain symptoms now require more urgent assessment, or a patient group that previously received routine care is now considered higher risk. The AI software hasn't changed. It's still running the same algorithm, returning recommendations, and processing every patient interaction without an error.
But is it still making the right clinical decisions?
The criteria used to determine what constitutes an urgent case may have changed. A recommendation that was appropriate when the AI was validated might no longer match current clinical practice. This illustrates a form of concept drift, where the relationship between the information an AI receives and the correct clinical outcome or decision changes over time. Unlike data drift, where the characteristics of the input data change, concept drift challenges whether the AI's original understanding of those inputs remains clinically valid.
And that's what makes it particularly interesting from a regulatory perspective.
We often think of an unchanged algorithm as a controlled algorithm. But just because the software hasn't changed doesn't mean the clinical environment has stood still. For manufacturers of AI-enabled medical devices, maintaining clinical performance means monitoring more than the software itself. It also means recognizing when evolving clinical knowledge, treatment practices, or patient outcomes may affect the continued validity of the model.
2. The regulatory thread
Practical application: Manufacturers can integrate AI-specific clinical assumptions and performance monitoring into their existing clinical evaluation, PMS/PMCF, risk management, and change-control processes. An unchanged AI model still requires reassessment when clinical practices or expectations change.
EU MDR 2017/745: Article 61 and Annex XIV require clinical evaluation to remain current, supported by ongoing post-market clinical follow-up (PMCF). Articles 83–84 and Annex III establish PMS requirements for monitoring safety, performance, and benefit-risk as clinical knowledge and practices evolve.
AI Act and EN 18286:2026: EN 18286 provides AI-specific quality management provisions, including verification, validation, and post-market monitoring. Its harmonized status supports conformity with covered AI Act requirements, but does not automatically establish MDR clinical compliance.
FDA: Under 21 CFR § 820.10, FDA's QMSR incorporates ISO 13485:2016, including requirements for post-production feedback (8.2.1), data analysis (8.4), and design changes (7.3.9).
3. A worked example
Back to the hospital example.
The hospital is using an AI system called TRIAGE-02 to identify patients who need urgent medical attention. The manufacturer spent months developing and validating the software against the hospital's approved clinical criteria. The software evaluates symptoms, vital signs, and patient information to help determine who needs to be seen quickly.
Two years later, the hospital updates its triage guidelines. New clinical evidence suggests that certain combinations of symptoms and laboratory results should receive more urgent attention. The hospital adopts new criteria, but TRIAGE-02 continues operating exactly as it did before. There are no software errors, no complaints, and no obvious signs of a problem. Yet some patients who previously qualified for routine assessment may now require urgent care.
This creates an important question for the manufacturer: Is the AI still making clinically appropriate recommendations?
The first step isn't necessarily to retrain the model. The manufacturer needs to understand what changed and whether it affects the device's intended purpose, clinical claims, or validated performance. This could involve comparing patient classifications under both sets of criteria, reviewing a representative sample of AI recommendations against the updated clinical reference, and evaluating available patient outcomes. Any missing follow-up data or limitations in the sample should be documented.
The review might conclude that the AI remains appropriate without modification. It could identify a need to clarify the instructions for use, update the clinical evaluation, or conduct additional validation before implementing a software change.
The important part is having a documented process that recognizes changes in clinical practice, evaluates their impact, and supports a justified decision. An unchanged algorithm doesn't necessarily mean the clinical assumptions behind it remain valid.
4. What should appear in the documentation
The clinical evaluation should identify the specific clinical guidelines, versions, and decision criteria used to develop and validate the AI. For the example above, this means documenting the symptoms, vital signs, laboratory results, and combinations of findings that classify a patient as requiring urgent assessment. The manufacturer should retain the reference definition, the clinical rationale for each triage category, and the acceptance criteria used during validation.
The PMS Plan and, where applicable, PMCF Plan should identify the clinical performance measurements used to confirm that these decisions remain appropriate. These should include sensitivity for detecting urgent cases, under-triage and over-triage rates, and agreement with an independently established clinical reference. For each measurement, the plan should define the patient population, reference standard, calculation method, acceptance criteria, monitoring frequency, and required sample size or statistical justification.
The manufacturer should also maintain a controlled list of applicable clinical guidelines, with the document version, responsible reviewer, and review frequency. When guidance changes, such as the transition from one year to another, a documented impact assessment should identify which triage criteria changed. (e.g., how many patients would be classified differently, whether clinical performance remains acceptable under the new definition, and whether the clinical evaluation, risk file, instructions for use, or software require updating.). This is not necessarily a new requirement, and is typical of a well-established Clinical Evaluation. What is new, is focusing on the metrics of clinical performance that could be early indicators of concept drift.
5. Implement it in your existing MDR QMS
- Add an AI-specific trigger to your clinical evaluation update procedure: material changes in clinical guidelines, clinical performance definitions and metrics, or reference standards.
- Record the device claims and ensure there are PMS metrics supporting each claim.
- Ensure PMS/PMCF is collecting proper clinical data. Ensure the data is collected from actual use of the device, in the market, used as intended. Avoid retrospective data collection and batching data for the sole purpose of confirming clinical endpoints, as this will lose the impact of the clinical environment.
- Make your endpoints and metrics quantitative. Make sure your SOPs and work instructions explain how to collect the data and analyze it against the pre-established baseline performance.
6. Free resource: use this now
Use the linked Clinical Change Review tab to record one clinical assumption that matters to your product. Note the original definition and source, the new definition, the affected claims or outputs, and the person responsible for assessment.
Start with a fictional exercise: pretend your intended-use population has changed in a clinical guideline. Can your current QMS point from this change to a PMCF question, risk review, and documented conclusion? If those records live in different systems, a cross-reference field will help more than another stand-alone checklist.
A useful outcome is a documented decision with a date, owner, and evidence. A blank 'reviewed' checkbox is not enough.
https://bevrik.net/resources --> Concept Drift Monitoring Template
References
- European Commission - harmonized AI standards: https://single-market-economy.ec.europa.eu/single-market/goods/european-standards/harmonised-standards/artificial-intelligence-ai_en
- Official Journal - Implementing Decision (EU) 2026/2274: https://eur-lex.europa.eu/eli/dec_impl/2026/2274/oj
- EU AI Act - consolidated legal text: https://eur-lex.europa.eu/eli/reg/2024/1689/2026-07-27/eng
- EU MDR 2017/745 - current legal text: https://eur-lex.europa.eu/eli/reg/2017/745/oj
- EN 18286:2026 catalogue/contents preview: https://www.iteh.eu/catalog/standards/cen/34ea911c-a980-4433-85ac-1344f93da01b/en-18286-2026
- FDA — 21 CFR § 820.10, Quality Management System requirements: https://www.ecfr.gov/current/title-21/chapter-I/subchapter-H/part-820/section-820.10
- FDA — Quality Management System Regulation (QMSR), including ISO 13485:2016 incorporation: https://www.fda.gov/medical-devices/postmarket-requirements-devices/quality-management-system-regulation-qmsr
- FDA — QMSR guidance on risk management and design and development: https://www.fda.gov/medical-devices/medical-devices-news-and-events/town-hall-quality-management-system-regulation-risk-and-design-and-development-01142026
Originally published on LinkedIn.