Arbitrary selections in Risk Management are a liability.
Mar 1st, 2012
We can see a day when a case will be challenged in court against a company that used Probability Impact Graphs (PIGs risk matrix) for their risk assessment. The questions that could be asked will be horribly embarrassing and very damaging to the PIGs risk matrix’s user. Likely, they will tend to prove that the approach constituted a professional negligence. Perhaps due to the breach of Duty of Care. Indeed Arbitrary selections in Risk Management are a liability.
A preliminary list of questions
Here is a preliminary list of questions that could be asked:
- So, on which basis did you decide that the probability of the event was “medium” (or “pink”) or whatever your PIG risk matrix shows, and more importantly, why did you neglect to use any of the methods, published from the ’80s on about (subjective, expert driven) approximations of probabilities?
- Which is the basis for defining the consequence (loss) classes in your PIG risk matrix? How did you ended up considering that 20M$ loss was worse then 5 casualties and had to be used as the driving parameter for the selection of the consequence class? Methodologies to define multi-parameter functions have been published at least since the ’80s, why didn’t you use them?
- Which studies did you develop to define the various classes limits of likelihood, losses? On which basis did you select those limits?
- Why did you limit the highest class to -x- casualties and -y- millions? What about any scenario that would overcome that value? Did you imply it does not exist?
State of the Art?
- …in your statements you mentioned that PIGs risk matrix correspond to State of the Art, yet we do not know any Risk Management Standard (ISO, COSO, ONR) that would formally advise to use PIGs risk matrix, neither we know of any standard formal definition of PIGs risk matrix, class limits, methods to define class limits.
- So, did you use PIGs risk matrix just because every one uses them? Are you saying that PIGs risk matrix are State of the Art? (NB: SoA is the highest level of development at a particular time (especially the present time); NOT what everyone does!). PIGs risk matrix are not SoA, they might be assimilated to “common practice”, or “standard practice”, BUT there is ample evidence that appeals to Common Practice constitute a fallacy: using PIGs risk matrix because every body seems to do so is not a justification!
- Commercial PIGs risk matrix software generally bear a disclaimer saying: “beware users”…this software is just a way to display an information treatment that the user produces…the software house does not bear any liability…
- Which criteria did you use to select the colours of your cells, which correspond to various levels of criticality? If we understand well, your criticality criteria is as a pseudo tolerance criteria. Red color means highest risk, risks that one should deal with, mitigate immediately. Yellow means attention and green means “they are ok”, right? What criteria did you use to define those levels of criticality?
- There are publicly available tolerance criteria since the mid ’60s. How come your color threshold does not match any known tolerance criteria, and how come that cells straddle those tolerance criteria?
- Using “credible scenario” is a censoring decision. How come you felt entitled to censor your analysis?
- Using “average p, C (loss)” is a biasing decision. How come you felt entitled to bias your analysis towards the center for each single scenario?
- In your opening statement you say that scenarios entering in your PIG risk matrix have to be credible scenarios. What threshold to credibility did you use? How does that threshold match with your PIGs risk matrix cells limits?
At the end of this drill, we doubt the user/you will be feeling in a strong position to further argue the case. We believe the user would be facing unpleasant consequences because his behavior has been negligent.
Remember, State of the Art is not what everybody does…and common practice is not an excuse, constitutes a fallacy.
Do not set yourself to be the looser by confusing “what every body does” as State of the Art.
We will soon publish a post explaining how you can avoid these pitfalls.
Tagged with: Acceptability, biases, Probability Impact Graphs, risk, State of the Art, tolerance
Category: Optimum Risk Estimates, Probability Impact Graphs, Risk analysis, Risk management, Tolerance/Acceptability