The most common question I get when delivering courses on risk management and ISO 14971:2019 is how you should be estimating risk. It is something that many of us have to do when we take part in risk management.
I will share with you how it’s supposed to be done, and how most risks are estimated. Learning about this might come as a big relief.
This video is an extract from the online course Risk Management for Medical Devices and ISO 14971:2019.
Estimating risk is when you determine the probability of occurrence of harm and severity of harm. The risk should be recorded in your hazard traceability matrix or risk analysis. You do this both before risk control measures have been taken, as well as after risk control measures have been implemented. The latter estimation being the most important one.
Risk is defined as the combination of the probability of occurrence of harm and the severity of that harm. We will focus on the probability of occurrence of harm, or just Po.
Risk estimation as part of risk analysis
You will do the first risk estimation as part of the risk analysis. Meaning, it is the risk before any risk control measure has been implemented. And this is where the big challenge lies.
You should assume that nothing has been done to reduce risks. It may sound odd, but you should, for example, start by thinking that the device has no packaging, no casing, no insulation on cables, nor any protection or warnings anywhere. It may not be sterilised or even cleaned, and you haven’t thought about selecting biocompatible materials.
Why estimate risk before risk controls?
You will make many decisions in the design and manufacture of the medical device to reduce risks:
- You manufactured the medical device in a clean room,
- you train your personnel,
- you put clear labels on the device, and
- you might have in-process controls.
Many or all of these things will be risk controls. You should record that they are in fact just that, to prevent someone in the future wanting to remove or change the implemented risk controls, without realizing that it is a risk control measure.
If an accident occurs and someone is injured and it leads to a trial, then it is much better to be able to show that you decided to implement insulation on cables, an insulating cover on the device and earth connection instead of just earthed ground wire to reduce the risk as opposed to just taking these risk controls for granted.
Estimate the harm, not the hazard
When estimating risk, it is the probability of occurrence of harm that you should estimate.
It is not the probability of occurrence of the hazardous situation, but the actual harm. Because not all hazardous situations will actually lead to harm every time, and it is the harm that matters. You may still have to estimate the probability of all sequences leading up to the hazardous situation, and the probability that the hazardous situation leads to harm.
The most common way to estimate risk
The most common way to estimate the probability of occurrence of harm or, Po, is by measuring it semi-quantitatively, so that a certain probability of the harm arising is related to a number. For example, if the probability is greater than 0,01% that may be represented by a 5 on a scale from 1 to 5.
When I refer to Po, I am sometimes a bit sloppy, because I refer both to the probability, which is a number between zero and one, or zero and 100%, and sometimes I refer to the numbered scale from 1 to 5. But strictly speaking, the 1 to 5 value is by definition not a probability. I’m quite convinced you will understand what is meant based on the context.
What is the best risk estimation scale?
You don’t necessarily have to have five steps on your scale, but normally, you should have no less than three steps or more than ten.
Fewer steps will give too few possibilities to differentiate one probability from another, whereas too many will be difficult to work with.
If it is not already determined and defined from before in your procedures or elsewhere, a five-point scale is in most cases appropriate.
How to estimate the probability?
When you estimate the risks, you should ideally find information from any of the following sources, and in this order of priority:
- Published standards or articles about similar devices
- Statistical references to products that are already out on the market
- Tests that you do to explore risk
- Results of investigations and analyses. This could be calculations, Monte Carlo simulations or for example fault tree analysis.
- Expert assessment, for example, when consulting internal or external experts on how often things would go wrong or happen.
Of course, it would be great if we could use published standards, articles or statistical references for all the risks that we come up with in our hazard traceability matrix. That is however not the case in reality. What I will be saying next depends on what kind of product you work with, and for how long you have been in the market.
If you have a brand-new product and no one else is manufacturing a similar product, you will find that most of your risk estimates are based on the last option, expert assessment, and this is just a different way of saying that you guessed. Knowing that most estimates are done this way, usually comes as a great relief to many people who are worried about these things. If you are making guesses, trust me, you are not alone.
At the same time, if you have a critical risk, and the consequences of not having a very reliable estimate of the probability of occurrence of harm are severe, then you may simply have to spend the time and money on tests or results of investigations to actually determine the probability. This could for example be simulations, process validations or usability studies.
Would you like to learn more about Risk Management?
Get instant access to our online Risk Management for Medical Devices and ISO 14971:2019 course right here. In 6 hours, you can learn more about how to develop new medical devices and maintain them in organisations where design control requirements apply. This course is taken by quality assurance, project management, design engineering or those involved in R&D and product development teams.
Peter Sebelius (PMP) is a highly esteemed trainer, consultant and entrepreneur in the medical device industry. He is a member of the Joint Working Group that is revising the ISO 13485 and ISO 14971 standards.
He has vast ‘hands on’ experience too, having developed, amongst other things, a mechanical chest compression and an ex vivo perfusion machine for lungs. He has received numerous awards including the Great Design Award and the title “This year’s specialist” by Veckans affärer.