Comments on a risk assessment tool for tailings storage facilities
Mar 24th, 2021
A colleague of ours asked for comments on a risk assessment tool for tailings storage facilities paper by Chovan et al (A risk assessment tool for tailings storage facilities (cdnsciencepub.com)).
We accepted and decided to prepare this piece. The reason is that we have seen various other attempts by reputable engineering companies to use similar methodologies leading to similar comments from our end. The application of simplified methodologies, or unproven approximations in the field of tailings dams could harm people and their user and ultimately hurt the industry as a whole. Therefore it cannot be taken lightly.
As a result, this post merges comments we have made not only on the specific paper cited above, but also on risk assessments practices we have reviewed. For the sake of transparency and ethics, we have each time made our remarks known to the authors via private messages or conversations before publishing this post, as far back as June 2020.
The context of the discussion
The paper by Chovan et al uses the Silva Lamb Marr (SLM, 2008) semi-empirical method. SLM links the factor of safety (FoS) from stability analyses to the estimated probability of failure (pf). Indeed the SLM approach is a tool to quantify expert judgement to streamline risk analysis. SLM it is not a geomechanical method.
We started studying the relationship between factor of safety and probability of failure in the mid-eighties. Furthermore we applied the original version of the same SLM approach in our seminal paper Factual and Foreseeable Reliability of Tailings Dams and Nuclear Reactors -a Societal Acceptability Perspective. We published that paper at Tailings and Mine Waste 2013 and we are referring to Figure 2.
We have proven the validity of semi-empirical approaches to estimate the probability of failure in our Tailings Dams Management for the Twenty-First Century book. However, we indicated very clearly, and not only in the book, that SLM does not include tailings dams specific aspects. Indeed SLM is a slope evaluation approach. As a matter of fact, its very title is “Probability and risk of slope failure”. In addition, and likely because of the prior statement, the examples included in the SLM original paper do not include:
- diversion systems,
- penstocks, and finally
- tailings lines and other tailings dams specific features.
Finally, and for the sake of clarity, Chovan et al use the following acronyms:
- LOE for Level of Engineering, a proxy to SLM’s category of the structure, and
- APF for Annual Probability of Failure.
- LOE is similar to LOP, or Level of Practice, as defined by Julien et al (2019).
Our comments on a risk assessment tool for tailings storage facilities
Altering the SLM Fos-pf relationship
The alterations of SLM FoS-pf relationships proposed by Chovan et al significantly underestimate the pf with respect to SLM. That is due to the linearization of the SLM relationships they apply. The underestimation reaches a maximum value well over 250%, with an average of over 200% for FoS=between 1 and 1.3 in all the structures categories (LOPs). We think the pertinence of altering the well though out and empirically derived SLM curves should be justified and demonstrated. The impact of such a underestimation could be significant when prioritizing mitigation.
Altering the LOP/Category criteria
Let’s not forget that the LOP families criteria have different values in Chovan paper than they had in the original LMS. These changes may have an influence on the FoS-pf relationship, adding uncertainties to the construct. Table 3 of Chovan et al shows these values.
Defining minimum APF based on non mining experience
Furthermore, using civil aviation and other non-mining reputable sources to develop minimum APF for each category (LOE/LOP) seems excessively optimistic. That is especially considering the actual rate of catastrophic failures in other industries, including the highly controlled nuclear one. The message delivered by Chovan et al for a high to very quality dam, with FoS>1.9, i.e. a pf <10-7 to 10-8 seems overly audacious considering the assumptions made.
Using ESA and USA and lumping up all the other cases
Chovan et al apparently use drained or undrained conditions as pertinent. They argue that their modified way of evaluating the category of a structure “englobes” all other cases. They state: “more complex failure mechanisms (e.g., strain softening) are not explicitly included. We assume that the best LOE considers all of the potential failure mechanisms for a particular structure. They then add: “lower LOE may not have considered some of these mechanisms – increasing the degree of uncertainty and the APF”. We think this “lumping up” may hinder the efforts of engineers to gain a better understanding of the structures they are evaluating.
Other comments and closing remarks
There are many other points that could be made. However they are less important than the ones we have summarized above. They bear on the:
- evaluations of consequences of potential failures,
- “sad-return” to the usual matrix, and finally on the
- development of corporate and societal risk tolerance thresholds.
Before closing this discussion, we would like to:
- reiterate again how important it is to study a dam system, including ancillary water management, etc., and not only a dam cross section.
- State that if anyone really want to evaluate the performances of a portfolio of structures it is necessary to spend a lot of time and effort in building the knowledge base for each system. Only then the LOE/LOP or category of the structure can be evaluated in a repeatable, consistent way. That leads to causality analyses and possible benchmark with world-wide performances.
- Remind that the effect of monitoring and increasing the knowledge base depth should be investigated for each system, and finally that
- even if using semi-empirical approaches like SLM one should not lose sight of the geology and geotechnical realities of the system at design stage. For instance, given their intrinsic variability, cohesive materials slopes should receive a higher FoS than granular ones. That is if the designer’s aim is obtain the same pf. SLM or similar approaches do not deliver that result because it is not a purely geomechanical method.
Tagged with: estimated probability of failure, factor of safety, quantify expert judgement, risk assessment tool, Silva Lamb Marr, Tailings Dams Management
Category: Consequences, Hazard, Mitigations, Probabilities, Risk analysis, Risk management, Tolerance/Acceptability