Regulation (EU) 2017/746 on in vitro diagnostic medical devices (IVDR) imposes enhanced requirements on products compared with the IVDD (Directive 98/79/EC). In the context of performance evaluation, clinical performance is a key element and performance studies are becoming increasingly important. Annex XIII sets out how manufacturers must plan, conduct and document the performance evaluation, with Section 2 addressing performance studies in detail.
What is a clinical performance study?
The performance evaluation of an in vitro diagnostic medical device (IVD) is the appraisal and analysis of data to determine or verify the scientific validity, analytical performance and, where applicable, the clinical performance of a product. Where it is necessary to demonstrate clinical performance, i.e. to substantiate the diagnostic accuracy of the IVD, this may be evidenced using data such as literature data, routine diagnostic data or data from previous performance studies. Where the available data are insufficient or not available—particularly in the case of novel or complex IVDs—performance studies must be conducted. A distinction is made between internal performance studies (e.g. in the manufacturer’s laboratories) and external performance studies (in hospitals, large laboratories, etc.). Both types of performance study have advantages and disadvantages. Whilst external performance studies require considerable time and incur high costs, internal performance studies can be conducted quickly and at comparatively low cost. However, the feasibility of an internal performance study depends critically on the availability of specimens. Biobanks and targeted specimen collections can often provide the required material. This article focuses on internal performance studies.
Regulatory basis for a performance study involving an IVD
Regulation (EU) 2017/746 on in vitro diagnostic medical devices (IVDR) sets out, in Annex XIII, Section 2, essential elements of a clinical performance study. The IVDR also refers to ISO 20916:2019 “In vitro diagnostic medical devices – Clinical performance studies using specimens from human subjects – Good study practice”, which serves as guidance for conducting IVD performance studies. In addition, the recommendations set out in MDCG 2022-2 “Guidance on general principles of clinical evidence for in vitro medical devices (IVDs)” are helpful. The German Medical Devices Implementation Act (MPDG) further supplements the IVDR requirements in Chapter 4 by adding national requirements.
Planning an internal performance study
When planning an internal performance study, numerous aspects must be considered. Above all, the question is whether the IVD is suitable for an internal performance study at all.
- Self-test IVDs intended for use by lay persons are, in almost all cases, not suitable for internal performance studies.
- IVDs suitable for internal performance studies are typically IVDs intended for use by professional users.
If the IVD is suitable for an internal performance study, the required number of specimens must be determined and the endpoints defined. Timelines and cost plans, as well as further documentation, must be developed. The manufacturer’s laboratories in which the internal performance study is to be conducted must be appropriately equipped for a performance study. Depending on the type of specimen (e.g. infectious material), the relevant laboratory authorisation must also be in place. In addition, sufficiently trained personnel must be available to perform the tests and to lead and supervise the study.
Some performance studies require an authorisation from the competent authority (IVDR, Article 66), while others are subject only to notification. Depending on the IVD, the competent authority in Germany is either the Paul Ehrlich Institute or the Federal Institute for Drugs and Medical Devices (BfArM). Applications must currently be submitted via the German Medical Device Information and Database System (DMIDS), and later via EUDAMED. The involvement of the ethics committee must be taken into account.
The success of any internal performance study depends on the availability of appropriate specimens. The availability of specimens via biobanks and/or commissioned specimen collections must therefore be researched carefully.
Specimens for an internal performance study
Most specimens that can be procured are stored frozen. Prior to initiating the performance study, it must therefore be clarified whether such specimens can be used at all. A freeze–thaw study is recommended, during which fresh specimens are analysed, frozen and re-analysed repeatedly. If frozen specimens can be used, biobanks must be identified that can provide suitable specimens in sufficient quantity and quality. Care must be taken to ensure that the biobanks have robust quality management systems for storage and that ethics documentation and traceable records for the stored specimens are available. If specimens are sourced from a biobank, it must be considered that transport costs—particularly for biobanks located abroad—can be very high. In addition, depending on the type of specimen, it must be ensured that the required import documentation is available and that the import has been declared. Especially for specimens containing infectious material, this can help prevent the valuable shipment from being detained by customs.
In addition to sourcing specimens from biobanks, specimens may be collected on commission by clinics or laboratories. Where possible, the collection of leftover specimens is recommended, as this reduces the ethics workload. Although an ethics committee opinion must still be obtained in all cases—and in some instances the collecting institution may already hold an ethics opinion covering all of its leftover specimens—patient information documents often do not need to be prepared. Finally, adequate storage capacity must be available in the manufacturer’s laboratories in a temperature-controlled and monitored environment. Once a sufficient number of specimens of the required quality has been obtained, the internal performance study can commence, provided that all documentation has been prepared.
Documents for an internal performance study
A very important element of an internal performance study is the study plan (Clinical Performance Study Plan (CPSP)), for which the IVDR specifies detailed requirements. ISO 20916:2019 “In-vitro diagnostic medical devices – Clinical performance studies using specimens from human subjects” is an excellent guide in this regard. Even if certain points listed in the IVDR are not applicable to internal performance studies, careful and comprehensive wording is essential. For the plan describing how the measurements are to be performed, a standalone document that is unambiguous and clearly drafted is recommended, which can also be used to train the laboratory staff conducting the work. Preparing study labels for the IVDs to be tested is very important to ensure that the test IVDs can be uniquely identified. In addition, depending on the type of specimens used, an ethics application may be required. Furthermore, well-prepared tools for data collection, data processing and data analysis are essential.
Activities during the study
Prior to commencing the measurements, all laboratory staff performing the work must be trained by the investigator and this training must be documented. Thereafter, the study can begin. During the study, the investigator must regularly verify that all measurements are performed as described in the plan, record any deviations where applicable, and initiate corrective actions where necessary. Data analysis can often also begin during the course of the study.
What are the advantages of an internal performance study?
Compared with external performance studies, internal performance studies offer the advantage that—with good preparation—they can be conducted significantly faster. In addition, the investigator can intervene promptly if something does not run quite as expected, as both the investigator and the study are located at the same site. Furthermore, they are usually less cost-intensive and require fewer documents to be prepared.
Do you require your own data for your IVD and are you looking for competent study support? Please contact us. We support you with internal and external studies.
