Stakeholders not satisfied with probability assessments

It is obvious that risk and uncertainty analyses can and will always be challenged by opposing stakeholders not satisfied with probability assessments. They will base the challenge based on their subjective, modeled or even „pseudo statistical“ approaches. Under the excuse of limited or poor available knowledge of the problem at stake some stakeholder may invoke the „unrepresentative“ character of expert analyses. In addition they will try to avoid making sensible decisions based on “mistrust„ in the results. Indeed they will feel their gut feeling is necessarily better than a science based approach.

The advantage of using ORE is that it allows transparent discussions including ranges estimates, variability of parameters and uncertainties. That addressed stakeholders not satisfied with probability assessments, thus leading to healthier debate. In addition,  in our experience, it helps finding consensus.

During the implementation of decisions it is common for decision-makers to seek for further protection. They generally add conservatism and use traditional engineering frameworks of “defense-in-depth.” This is typical of deterministic approaches to hazard and risk management. In those approaches engineers add layers of protection, without explicitly evaluating their effectiveness, to bound known uncertainties to, for example, “credible thresholds”. However, these approaches have limited effect in reducing the “unknown unknowns”, i.e. reducing the completeness uncertainty.

In particular, these approaches may lead to censored results such as:

• identify a group of failure event sequences leading to credible worst-case accident scenarios called design-basis accidents;
• predict their consequences;
• design appropriate safety barriers which prevent such scenarios and protect from, and mitigate, their associated consequences.

As a matter of fact, accidents in all sorts of industries have shown that the „credible“ scenario established in this way oftentimes represent a strong censure of the possible and actually credible ones. Codes are generally not covering the full breath of situations that should be covered. Thus mitigations result severely under estimated.

The underlying principle has been that if a system is designed to withstand all the individual worst-case credible accidents, then it is “by definition” protected against any credible accident. That does not cover for uncertainties, inter-dependencies and common cause failures which unfortunately do characterize accidents in our world.

ORE Risk-Informed Decision-Making (RIDM)

ORE Risk-Informed Decision-Making (RIDM) is a deliberative process that uses a set of performance measures (we call them success criteria), and other considerations, to “inform” decision-making. The ORE RIDM process acknowledges that human judgment has a relevant role in decisions, and that technical information cannot be the unique basis for decision-making.

ORE RIDM is applicable for decisions that typically have one or more of the following characteristics, independently from the financial stakes significance:

• ramifications of alternatives which are difficult to understand without detailed analysis;
• uncertainties in key inputs generates substantial uncertainty in the outcome of the decision alternatives;
• large numbers of objectives requiring detailed formal analyses;
• need to define objectives and derive the corresponding performance measures when facing a variety of stakeholders.

We will teach a hands-on ORE Risk-Informed Decision-Making (RIDM) course in London (July) and Banff  (November) using well documented case histories.

ORE has been used together with its companion methodology CDA/ESM (a Risk Adjusted cradle to grave project evaluation tool which replaces the NPV to support risk based/risk informed decision making in a number of industries).

Here are some examples:

• Mining (operations and environmental rehabilitation)
• Transportation (RR and highways)
• Commercial wharves (including multi-modal logistics)
• Automotive (specialty vehicles)
  M&A, startups

How to fix trust with stakeholders not satisfied with probability assessments

We highlight below some aspects of Risk Informed Decision Making (RIDM), as recently discussed by NASA and USNRC, and compare them to ORE features to show how ORE deployment is feasible and beneficial.

Modern RIDM requires to acknowledge uncertainties, multiple hazards, multiple stakeholders and compliance needs, integrating stakeholders’ concerns and perceptions.

ORE:

• includes the uncertainties by working with ranges. As a matter of fact the key point is to guarantee that uncertainties are taken into account in each step of the risk assessment. Uncertainties have to be
  • systematically identified and classified;
  • represented and described by rigorous mathematical approaches;
  • propagated through the steps of the risk assessment procedure onto the risk measures until the decisions.
• allows to integrate
  • multiple concerns (multiple stakeholder expectations) and
  • information sources as well as
  • multiple hazard types (natural, man-made, technological, etc.).

This leads to greater stakeholder participation in decision-making. In fact, technocratic decision processes, driven purely by rational technical considerations, are modified to integrate the concerns and the perceptions of stakeholders.

The match between ORE and modern RIDM requirements is perfect.

Modern RIDM requires the classic engineering/technical approach to risk assessment which includes:

• safety margins (Factor of Safety),
• engineered redundancy and diversity to prevent and reduce the impact of failures

and stochastic analyses such as:

• Inter-dependencies
• Common Cause Failure
• Scenario building and
• Extreme event analysis

are integrated in the decision making process. That is true even while recognizing that the knowledge of all the aspects of a given system under consideration is imperfect or not ready for analysis simultaneously at different stages of a project/operation development/life..

ORE builds a hazard and risk register that will accompany the project/operation thought its existence. The register is:

• scalable. For its structure allows different levels of knowledge to co-exist in different areas.
• Drillable, meaning that it is always possible to perform queries on the threats-from, threats-to, per process thread, etc.
• Concise, meaning it avoids double counting and fuzzy statements.
• Economical, meaning ORE never wastes information, there is never the need to „start over“, even if considerable amount of new information becomes available.
• Updatable, if necessary and if data available, up to real-time.

It is possible to transparently lay out all the argumentation behind the analysis itself, including the assumptions, hypotheses, parameters and their uncertainties for disclosure points. However ORE register structure and numerical techniques remain proprietary.

Thus ORE is again fulfilling RIDM requirements and provides decision-makers with a clearly informed picture of the problem upon which they can confidently reason and deliberate.

We could make many more considerations, but this is a blogpost, neither a book nor a course. Therefore we will be delighted to answer your questions. Just send them through the contact form or as a comment.

Closing remarks

Building trust with Stakeholders not satisfied with probability assessments is possible if one deploys ORE within a Risk Informed Decision Making (RIDM) process. The approach helps achieving success by:

• supporting the selection of decision alternatives,
• ensuring that decisions between competing alternatives emerge with proper awareness of associated risks all along the life cycle of the operation/project.

When applied to projects ORE RIDM helps avoiding:

• late design changes, which can be relevant sources of risk,
• cost overshoot,
• schedule delays, and
• cancellation.

Contact Riskope to learn more.
Register to Riskope courses: London (July) and Banff  (November)

Tagged with: ORE, probability assessments, RIDM, Stakeholders

Category: Optimum Risk Estimates, Risk analysis, Risk management