The risk-based approach to process validation
The quality of a medical device can be influenced by many different aspects. In order to avoid a diminishing influence during development and production, processes must often be validated. This is done on the basis of selected sampling. But what sample size should be determined and how can it be justified? The risk-based approach can help in the case of process validation. There are also already established statistical methods for determining sample sizes for process validation.
It must be addressed individually, how these methods can be integrated into the QM system as a whole, also with regard to the general risk management system.
Where do the requirements for a risk-based approach to process validation come from?
The risk-based approach to process validation has already been part of ISO 13485 since 2016.
The MDR and FDA require validation of the processes with the help of recognised statistical methods, whereby the sample size must be justified. The following are excerpts from the regulations, standards and also guidelines that require process validation and offer approaches for successful implementation.
- MDR, Annex II, 3. design and manufacturing information
The technical documentation that must be compiled should include amongst others, “(b) complete information and specifications, including the manufacturing processes and their validation, […]”.
- MDR, Article 17 Single-use devices and their reprocessing
It must be ensured that “[…] reprocessing is performed in accordance with the CS, detailing the requirements concerning: […] – the validation of procedures for the entire process, including cleaning steps”.
- EN ISO 13485:2016 + AC:2018 + A11:2021
Any production process whose result cannot be verified 100% must be validated. Section 7.5.6 (and in section 7.5.7 for sterilisation processes).
The organisation shall validate all production and service provision processes whose outcome cannot be verified by subsequent assessment or measurement. The validation shall demonstrate the ability of these processes to consistently achieve the intended results.
- 21 CFR 820.75(a) [9.3]
If the results of a process cannot be fully verified by subsequent inspection and testing, the process shall be validated with a high level of assurance and approved in accordance with the established procedures.
- Guidance for Industry, Process Validation: General Principles and Practices
This FDA guidance provides principles on the correct planning, implementation and documentation of process validation in the pharmaceutical sector.
- EU Guidelines for Good Manufacturing Practice for Medicinal Products for Human and Veterinary Use: Annex 15 Qualification and Validation
This guideline can be used analogously for medical devices. The GHTF guideline and the described FDA guideline from the pharmaceutical sector result in a sensible combination of practice-oriented specifications for validation!
- Guideline of the Global Harmonization Task Force (GHTF), which applies exclusively to the medical device sector. The Quality Management Systems – Process Validation Guidance
This guideline emphasises the special situation of process validation in the medical device sector. It addresses the fact that factors such as production volume, the number of manufacturing steps or, for example, destructive testing have a major influence on the approach to process validation and offer approaches to solutions via statistical methods. Although the GHTF (Global Harmonization Task Force) has been replaced by the IMDRF (International Medical Device Regulators Forum), this guidance offers sensible solutions for process validation that are still state of the art.
The purpose of statistical methods and the most common representatives.
In the production process, the aim is to minimise risk and producing as little defective results as possible. The requirement of the risk-based statistical method is now to define a sample size and, if necessary, an acceptable error quantity therein to permanently achieve a desired conforming production result.
It must be considered that statistical methods are inherently subject to the rules of probability. Every probability calculation and thus also every statistic is based on theoretical considerations that can already be changed by adjusting the smallest screws. Thus, a complete elimination of the risk of producing incorrect results is not possible.
There are various established statistical methods for validation with different acceptable sizes, each of which can be justified on the basis of process-specific conditions. Among them, there are established methods that all determine the sample size at acceptable error levels. These include, for example, the Success-Run Theorem, Acceptable Quality Limit (AQL) or Lot Tolerance Percent Defective (LTPD) sampling.
Two of these methods are explained in more detail below.
- Acceptable Quality Limit (AQL)
The AQL method assigns acceptance quality limits based on the accepted risk. Sampling by AQL is widely used in the industry. Sampling determines the maximum permitted number of defective products in a manufacturing process.
These standard series are suitable for sampling:
- ISO 2859 Sampling procedures for inspection by attributes
and - ISO 3951 Sampling procedures for inspection by variables
Both series of standards describe an acceptance test according to AQL (Acceptable Quality Level). This states what range of defects is acceptable in the batch.
In ISO 2859, there are single samples, double samples and reduced and tightened tests. In ISO 3951, tests based on standard deviations of the whole or the sample are used. The sampling instructions and acceptance criteria are available in tabular form for both standards.
In one of our previously published articles „Mit dem richtigen Rezept richtig mit den Vorgaben umgehen“(only German), we show how to understand AQL in an abstract example.
- Success Run Theorem
The Success-Run Theorem is a very common method based on a binomial distribution that leads to a certain sample size.
As a starting point, the risk level of a process step is first determined by means of a Failure Mode and Effects Analysis (FMEA). The combination of probability of occurrence, probability of detection and severity of a possible error results in the risk (e.g., low, medium or high). A cleaning process, for example, is usually classified as high risk. According to the risk, the confidence level and the reliability can be selected, which form the basis of the Success-Run-Theorem.
Risk | Confidence | Reliability |
High | 95% | 99% |
Medium | 95% | 95% |
Low | 95% | 90% |
Reliability is the probability that a process step will function satisfactorily for a certain period of time. The confidence level indicates the percentage of certainty that the process step is reliable. If the risk is high, the process step should be 99% reliable with 95% certainty (see table).
The determination of the confidence and reliability level classification is based on an organisation’s risk acceptance criteria, industry practice, guidance documents and legal requirements.
The following formula can be used to calculate the sample size:
An example calculation:
A cleaning process step is classified as high risk based on the FMEA. Accordingly, the manufacturer specifies that this process step should be 99% reliable with 95% certainty.
The above formula can then be applied as follows:
Rounded up, this results in 299 random samples that must be taken and tested. These must not show any defects/deficiencies in defined cleanliness for the cleaning process step to have been successfully validated.
Integration of the statistical methods into the QM system and in particular – the general risk management.
Systematic, risk-based process validation should be an integral part of any quality management system. The minimisation of errors in the production of medical products is already required by the fact that, according to ISO 13485, the risk for users and patients must be kept as low as possible.
Where applicable, each manufacturer shall establish and maintain procedures to identify valid statistical procedures necessary to establish, control and verify the acceptability of process suitability and product characteristics.
Regardless of the method used to determine the statistically valid sample size and the corresponding justification (confidence, reliability or acceptable quality limit), the method should be based on
- determining the risk associated with the process,
- the effort associated with the manufacture of the product, and
- the activities related to inspection, measurement and testing
and applied consistently.
When is a statistical method appropriate as described in ISO13485?
We would like to conclude this article by drawing your attention to an addition in the standard that restricts the use of statistical methods in validation.
Process validation is limited in ISO 13485, chapter 7.5.6 by the addition of “as appropriate”. This restriction is made because there may be processes that are not statistically describable. A statistically describable process has influencing factors that have an equal effect on all elements of a entirety. Among others, this is not always the case
with cleaning processes,. Due to biodynamics, different pre-cleaning influences and other time-variable factors, a justifiable sample size is often not possible.
It is therefore advisable to evaluate every production process from this point of view in order to avoid data volumes. This way, a significant improvement in the quality of the products can be achieved.
Were you hoping to find an always valid approach to sample size calculation in this article? Don’t you yet know how and when to justify your sample? Well, as diverse as the range of medical devices is, so are the risk-based approaches. Feel free to contact us, we will be happy to show you further possibilities.
Please note that all details and listings do not claim to be complete, are without guarantee and are for information purposes only.
