Oil and gas are also victims of risk matrix

A recent paper entitled The Risk of Using Risk Matrices shows that Oil and gas are also victims of risk matrix. The paper analyses the widespread use of Probability Impact Graphs (PIGs, risk matrix) in Oil and Gas (O&G). It comes to very similar conclusions to those reached by other academics and practitioners and us!

“Classic” Risk Matrices deficiencies

Risk-Acceptance Inconsistency.

The regions depicted in common practice PIGs (Risk matrix, Example 1 later on ) feature arbitrary stepped borders which generally bear no relation with actual corporate or societal risk tolerance. Thus those risk matrices lead to misleading decision-making support.

Range Compression.

Risk Matrix and their probabilities, Consequences (Losses), risk indexes using scores to mimick expected-loss calculations do not reflect actual “distances between risks”, i.e. specifically, the difference in their expected loss.

Below you can see Consequences (Losses) split in 5 classes. M dollars are used as a metric for Losses. In the usual log scale (top graph) the intervals seem equally spaced. However, when displaying the same values in decimal scale (bottom graph) we can see a significant “range bias”.

Centering Bias.

People generally avoid extreme values or statements if they can. For example, if a score range goes from 1 to 5, many will select values in the 2 to 4 range as a result of the “centering bias”.

Category-Definition Bias.

Users oftentimes confuse frequency and probability. The confusion is not solved by the risk matrix compilation guidelines generally offered to users.

The results are confused communications, as pointed out by Canadian CEOs. Researchers even talk about the “illusion of communication” generated by the “category-definition bias”.

“Newly recognized” Risk Matrices deficiencies

Ranking is Arbitrary

If the risk matrix uses an index approach (ranks from 1 to n), then indexes can be in ascending or descending order. Either approaches are used in various industries. This adds to risk matrix generated confusion. Ascending or, even worse, descending indexes further the distance between a matrix based risk assessment and reality. Rankings are arbitrary and misleading.

Instability Because of Categorization .

Categories are the cardinal (index) given to probabilities and Consequences ranges. Categories generate “instability” as they influence the “binning exercise” necessary to compile a risk matrix. Arbitrary range limits and categories are not necessary as we have shown in Ten Rules for Preparing Sensible Risk Assessments.

Relative Distance is Distorted.

The fact that risk matrix axes display, explicitly or not, values in logarithmic scale distorts distance between points. This is a phenomenon that occurs in many instances in science and economy.

Unless properly explained and conveyed to users this distance distortion leads to significant misunderstandings.

ISO 31000 identifies additional deficiencies of Risk Matrix (PIGs)

Beyond the publications cited above we discussed this theme in detail in a 2013 blogpost.

Oil and gas are also victims of risk matrix. Can we solve the problem of Risk Matrices deficiencies?

We show three ways:

Correcting a series of common mistakes.

Our 2016 paper explores the idea that correcting a series of common mistakes, which can be done quite easily and inexpensively, would lead to honest and more representative way to assess risks. Here is the reference. Oboni, F., Caldwell, J., Oboni, C., Ten Rules for Preparing Sensible Risk Assessments, Risk and Resilience 2016, Vancouver Canada, November 13-16, 2016.

House keeping of risk register.

In many instances clients call us for help with meaningless third parties risk matrices. We call the cause of that call the “two hundred yellow syndrome”. Indeed, the risk matrix numerous deficiencies lead to bin many risks in the central, generally “yellow”, area of the matrix.

Furthermore confusion in the hazards, consequences and risk scenario descriptions leads to added confusion. The “house-keeping of the risk register consist in eliminating confusion by first strictly abiding to a well defined glossary. Once the register is consistent, we vet one by one the risks, look at possible common cause failures, interdependencies and rationalize probabilities and consequences.

The final result is a p-C graph unencumbered by arbitrary cell limits. We can then help the client to define the risk tolerance threshold which applies to the considered case.

Example 1: Risk Matrix with ascending indexes and four regions: green, blue, yellow and red.

ORE deployment constitutes the logical next step

If the client requires it we deploy ORE.

ORE builds a convergent hazard and risk register that will accompany the project/operation thought its existence. The register is:

• Scalable, i.e. built in such a way that different levels of knowledge can coexhist in different areas.
• Drillable, i.e. allows to perform queries on the threats-from, threats-to, per process thread, etc.
• Concise, meaning it avoids double counting and fuzzy statements.
• Economical, meaning information is never wasted, there is never the need to „start over“, even if considerable amount of new information becomes available. Finally it is
• Updatable, if necessary and if data available, up to real-time.

We can transparently lay out all the argumentation behind the analysis itself, including the assumptions, hypotheses, parameters and their uncertainties for disclosure purposes.

The final result of a ORE deployment is a intolerable risk dashboard as depicted below (Example 2).

Example 2 An example of ORE dashboard.

Additionally we have published a series of papers years ago showing problems and solutions with risk matrix and risk communications:

• Our white paper 
• Oboni, F., Oboni, C., Zabolotniuk, S., Can We Stop Misrepresenting Reality to the Public?, CIM 2013, Toronto
• Oboni, F., Oboni, C., Is it true that PIGs fly when evaluating risks of tailings management systems? Short Course and paper, Tailings and Mine Waste ’12, Keystone Colorado

Tagged with: bias, risk matrix, risk matrix deficiencies

Category: Consequences, Optimum Risk Estimates, Probability Impact Graphs, Risk analysis, Risk management