What is hardware-in-the-loop testing?
Hardware-in-the-loop (HIL) testing is used, for example in medical technology, to check the interactions between a real physical (sub)system and a simulated environment. This test method is particularly common in the development and verification of embedded software, as well as other assemblies, such as purely electronic components.
The test specimen is embedded in an artificial environment. This simulates, as accurately as necessary, the behaviour of the interfaces to the unit under test, with the aim of simulating either the “real world” or borderline areas. Depending on requirements, this virtual environment can contain real hardware (actuators and sensors) or just a software representation of it.
With the help of automatically running test scripts, the inputs and parameters of the simulated test environment can be varied as required and the states and responses of the system can be checked under test. Corresponding test runs can be scheduled or triggered manually, e.g. by providing a new software version. Test results and logs are also generated, collected and distributed to decision-makers accordingly.
Why is HIL testing relevant?
All automatic test procedures (including classic unit tests) are characterised by a high level of reproducibility. In the area of test execution, the “human factor” can be largely eliminated – meaning that different testers, different daily forms and similar influences are no longer relevant.
In terms of costs, the methodology scores twice over. Local HIL test setups replace expensive and potentially dangerous physical prototypes. Human testers are not required for execution. Although there is an initial outlay for setting up the test project and creating the test cases, this point can be quickly relativised by a few repetitions of the test runs within a project. If one considers the exponential increase in costs in relation to the time at which errors are found (NIST, “The Economic Impacts of Inadequate Infrastructure for Software Testing”, 05/2002), it quickly becomes clear that the ability to carry out tests easily and frequently is likely to have the most serious impact on costs and quality. In practice, regular HIL tests during development, in parallel with any existing unit tests, for example at night, are therefore a good way of detecting and correcting problems at an early stage.
What regulatory aspects need to be considered when using HIL?
The use of HIL testing therefore appears to be technically very advantageous. In order to be able to use the procedure for the development of medical devices, the qualification of the entire test environment is necessary.
Chapter 7.6 of the German version of EN ISO 13485:2016 + AC:2018 + A11:2021 states: “The organisation shall document procedures for the validation of the use of computer software used for the monitoring and measurement of requirements. Such software applications shall be validated prior to first use and, where appropriate, following changes to or use of that software. The specific approach and activities related to software validation and revalidation shall be appropriate to the risk associated with the use of the software, including the effect on the ability of the product to meet specifications. Records of the results and conclusions of the validation and necessary actions from the validation shall be maintained […].”
Similar requirements can also be found at the FDA, e.g. in 21 CFR part 820.70 (i): “When computers or automated data processing systems are used as part of production or the quality system, the manufacturer shall validate computer software for its intended use according to an established protocol. All software changes shall be validated before approval and issuance. These validation activities and results shall be documented.”
In practice, this means that the corresponding HIL structure must be recorded, evaluated, specified and tested as part of a computer system validation (CSV) before implementation in the process and therefore before use in projects. This affects both the HIL software and hardware, and possibly also parts of the connected infrastructure. Particular focus should be placed on the risks to be expected when using the tool in relation to the affected sub-processes as well as the precise definition and verification of the requirements for the test equipment.
The typical HIL setup, what should it be able to do?
Common HIL products offer several standard interfaces across the board, such as digital inputs and outputs (GPIO, PWM), analogue inputs and outputs (ADC and DAC) and buses (I2C, SPI, CAN). The setup is usually connected to a PC via USB. The number or distribution of the respective interfaces varies greatly and should be selected according to the expected test tasks. This also applies to the qualitative properties of the interfaces (speeds, resolutions, tolerances, etc.). Specialised purchase solutions exist for specific application areas or business fields, but this does not rule out adaptations or extensions for use.
Usability should not be neglected. This point is largely determined by the software. Should tests be created graphically or using classic programming, i.e. which programming language should be used? How easy is it to set up, commission and integrate the system?
So how do you get started?
The first question is certainly: “Make or buy?” Even if the market offers many ready-made products, it can make sense to develop your own solution that is already designed for the intended use or environment. A usable setup that includes a USB-IO adapter board and peripherals is available for just a few hundred euros each (material costs). In most cases, coupling with a CI/CD environment (Continuous Integration/Continuous Delivery) makes sense – HIL tests are then part of every software development process. In any case, the structure and configuration should be thoroughly documented.
If the HIL system is placed under version control and the corresponding status is “frozen”, the CSV process is continued. Assuming a positive result, the tool is then basically ready for productive use. The CSV process should also safeguard changes and special adaptations for a project from this point onwards.
Final thoughts…
HIL testing is a powerful tool in the development and verification of complex systems. It enables engineers to simulate real-life scenarios, as well as limit ranges, in a controlled environment and to check the reaction of a system long before the medical device is launched on the market. Once the entry threshold has been overcome, the efficiency and safety of the test processes, and therefore also the product quality, can be significantly increased.
Please note that all details and listings do not claim to be complete, are without guarantee and are for information purposes only.
