Tailings risk assessment methods comparison
Nov 2nd, 2022
In order to develop a Tailings risk assessment methods comparison we will start by:
- reviewing a number of existing alternatives to ORE2_Tailings™. This leads to justify the need for the ORE2_Tailings™ approach,
- discussing what we call the Blackbox objection, which actually applies to all the alternatives and finally
- the ORE2_Tailings™ procedure, algorithm and results.
Alternatives to ORE2_Tailings™
We have grouped possible alternatives into four families discussed below.
FMEA is not a quantitative risk assessment even if it uses indices like probability = 1,2 or… n. Neither it is quantitative if the cell limits have arbitrary “probability look-alike” numbers such as 0.01, etc. In FMEA it is impossible to consider the joint effect (causality) of various failure modes without incurring in “fuzzy acrobatics”. One can implicitly include management, care and other contributing aspects in the “binning” exercise. Researchers demonstrated that the coloring of the cell is “disjointed” from reality and public expectations.
Probabilistic slope analyses
There is an ample body of literature (for instance Christian, Baecher, (2011) (see Oboni F., Oboni C., The Factor of Safety and Probability of Failure relationship, TMW 2020) that shows that this approach delivers excessively high probabilities of failure. In addition it ignores management, care and many other contributing aspects.
Silva Lamb Marr semiempirical approach
We group under this heading two papers. They are Silva Lamb Marr (SLM, 2008 with the updated graphs in its comments) and Altarejos (Altarejos-García et al. 2015). Engineers mostly use SLM (oftentimes with the “outdated” graphs) and the difference between SLM and Altarejos is in any case not really insignificant.
Both papers display very clever set of curves based on a limited set (75) of examples. Among these some were indeed dam slopes and some were retaining structures or other slopes. The empirical curves link the quality of the slope to the factor of safety for stability and deliver the annual probability of failure. The paper does not indicate which deterministic factor of safety to use among the multiple generally available.
We insist on the term SLOPE because SLM does not include any provision for pipelines at crest or ancillary water management typical of dams. Indeed, SLM is a slope analysis despite the keywords indicating “dam” in the list. SLM cites however the possibility of using effective stress or undrained strength.
The paper mentions the need to expand the analysis to other failure modes, but provides no solution.
Probabilities below credibility level?
Furthermore the curves allow the user to pick probabilities of failure that are beyond credibility (less than 10-6 or one in a million). These may apply to “top notch” modern hydraulic structures in “mint conditions”, but certainly not to tailings dams, Rana et al. (2022). Indeed, we demonstrated in 2013 (paper cited earlier) that the factual probability of catastrophic failure of nuclear reactors was hovering around 10-4 (or one in ten thousand). And that despite theoretical values at less than 10-7 (or one in ten millions) at design stage.
In summary, we have used SLM in some papers where we needed a simple approach for an example, or only in very specific (and rare) cases where the dam can be considered a slope.
In our Tailings Management book we clearly show the limits of SLM for dams. Furthermore we illustrated the need to anchoring the results of this type of approach to wide arrays of cases. We did that with ORE2_Tailings™ using hundred years of history (see also Rana et al. 2022) as discussed later in this text.
Chowan et al.
Chowan et al. (2021) arbitrarily modify some aspects of SLM. However they do not offer any justified anchoring to reality beyond perception of the parties involved. Also, the approach encounters the same difficulties and misleading elements of SLM. For instance, it uses “one magic” factor of safety for a dam (see Table 7 of the Chowan et al. paper). It is unclear if they apply that factor to a single dam or an entire TSF.
They offer examples of application but the only “validation” is the comparison to Mount Polley failure after the fact, a case we also dealt with in our book.
Furthermore their approach only peripherally and implicitly alludes at water management, not allowing to include its actual engineering design criteria but only verbiage.
Blended FTA approaches
These use failure trees (FTA) to analyze each failure mode and either probabilistic slope analyses or SLM (exerting LOTS of care) to obtain a blended probability of failure. This is a way to bypass the limitations discussed above.
The probabilities in the FTA are also subject to many black box biases. We explained this approach in our Tailings book (cited earlier), in section 1.2 Linking FoS to Pf: Three Simplified Methods. This was the method a major consulting company used for their QRA. They compared their results with the ORE2_Tailings™ deployment. We were very pleased to see that the results were astonishingly similar, but ORE2_Taiings™ was way faster.
NB: None of the above allows to include in the analyses any interdependency with other HDSs, TSFs, external hazards.
The Blackbox objections to the five alternatives
In this section we go back to each alternative and discuss what we call the “Blackbox Objection” that we oftentimes encounter.
Despite its apparent transparency and simplicity there are numerous hidden “tricks” that may bias the results (Thomas et al. 2014, just to quote one). We explained this in our book Oboni, F. and Oboni, C., 2020. Tailings Dam Management for the Twenty-First Century. Springer International Publishing.
Probabilistic slope analyses
One can think probabilistic slope analysis are transparent but it is not the case! No software vendor explains what happens “inside the algorithm”. Furthermore, if an end user performs a Monte Carlo simulation, the intrinsic assumptions lead to high variability of the results.
Silva Lamb Marr
Silva Lamb Marr (2008) base their set of curves on a limited set (75) of examples. SLM state they used expert judgement to obtain “relative estimates” (among that sample) of the slope stability probability of failure. They did not specify the nature and type of the dams material and cross sections. In addition, not all the 75 structures were dams. One does neither know if the dams of the set were hydraulic dams nor the cross-section types.
Chowan et al.
Engineers around the world (including Canada and Latin America) have come to us with objections, asking questions. We report a sample of these below in italics. It is important to note that the objections denote in some case hazardous misunderstanding of the development. The cause seems due in part to the imprecise glossary and blurry “mathematics” used in the paper.
Notes by users
- It is suggested in the introduction to use a Factor of Safety (FoS) relative to the level of uncertainty. However no guidance is offered and the example they give shows “one number” FoS with no justification.
- Silva et al. (2008) presents a figure with curves that stops at FoS of +/- 2.2. This makes more sense than their figure which has curves with a FoS of 3. In addition, an inadequately constructed dam remains a high-risk structure, regardless of the FoS. (NB: this last remark is due to our correspondent misunderstanding of the aim of the paper).
- The use of words such as: “considered tolerable” tickles me. Because per se it does not mean anything “measurable” in the paper.
- They talk about “excessive seepage”. But that doesn’t mean anything as some dikes are designed to have seepage and others don’t!
- Table 2 is actually based on what the Quebec government is doing to assess the class of hydraulic dams. The application of such a recipe is debatable. And, I believe, should not be presented as “the solution” but rather as a list of questions to assist in risk assessment.
- Table 3 is not very convincing.
- It is written that ”… dam classification can be used to complete this task” (P9 of 17). How can the classification of a dam help to establish Annual Probability of Failure?
- One should not allow wording such as: ”would eliminate the possibility…” in this type of paper.
- Mixing Lifespan Probability of Failure and Annual Probability of Failure seems to me a risky approach. Because in the end TSF must be efficient for an indefinite period. However, here I can be wrong, because my knowledge of statistics is limited.
The blended approach inherits all the “blackbox” aspects of the single approaches it uses, described above.
The ORE2_Tailings™ algorithm converts various sets of data (verbiage and factual data) into a set of proxy variables (numbers). The algorithm combines them mathematically to deliver the annualized probability of failure of the dam “body” in various conditions. For instance drained, undrained, seismic, liquefaction, etc., pertinent with the site, studied by the engineers.
In addition, ORE2_Tailings™ evaluates the causalities of the failure. As a result it allows to propose a series of risk informed controls enhancements.
Simultaneously to the dam “body” analysis, the algorithm deals with the ancillary water management and pipelines potential failures (if applicable), external erosion potential (river, lake or sea/ocean). This analysis is rather complex as:
- malfunctioning of the diversions and water balance mismanagement may lead to overtopping;
- malfunctioning of weirs and spillway may lead to scouring and ultimately failure;
- external water courses, water bodies and sea/ocean may generate toe ablation which can lead to liquefaction, etc. and finally,
- penstock and other elements as applicable, may lead to hazardous hydraulic conditions.
The presence of active pipes (and possibly traffic) at the crown further complicates this side of the analysis.
Indeed, the two sides (dam “body” and ancillary water management) of the system interact in various ways due to potential for:
- downstream erosion,
- toe ablation and finally,
- other factors.
The maths involved in the set of operations happening in the algorithm are linear interpolations, series and parallel system reliability equations, statistical functions.
ORE2_Tailings(tm) blackbox objections
As per the Blackbox objection we hear formulate against ORE2_Tailings™, we can give the following two replies:
- The methodology is no more a black box than the other commonly accepted misleading methods discussed earlier. However its results reflect the world-wide portfolio behavior (see below).
- The information we deliver is enough to understand how the algorithm works without delving into the subtleties of its structure .The development of ORE2_Tailigs(tm) required decades to formulate and calibrate. Note that civil and geotechnical engineering is full of examples of methodologies that are known only in principle. Indeed no one can reproduce their results “by hand”, same as ORE2_Tailings™. Among these:
- finite element analyses,
- sophisticated 3-D stability analyses, and finally
- break analyses.
For instance, when asking consultants to perform dam break analysis engineers understand the general idea behind the code, But the computer code (often proprietary) is not available to “review”. This is also the case for ORE2_Tailings(tm).
ORE2_Tailings™ experience, results and anchoring to reality
Finally, Riskope’s experience bears to date on hundreds of HDSs. Thus we can show the result of ORE2_Tailings™ on hundred dams all over the world, anchoring to reality by our benchmarks, as visible in the graph below.
The HDSs we used to prepare the TSF summary in the figure belong to an array of mining companies. They are active/inactive/closed, with upstream, centerline, downstream/rockfill methods. The owners/operators asked Riskope to deploy ORE2_Tailings™ due to their concerns. As one can see from the graph, in some cases ORE2_Tailings(tm) confirmed the concerns. However, there were also cases that we could “dismiss” even among this “worrisome” sample of dams.
Note that a great majority of the TSFs in this world-wide portfolio hovered around or within the benchmark lines. Thus they reflected the results brought in by Rana et al. (2022).
As a matter of fact, the portfolio depicted in the graph predicts little less than three catastrophic failures per year. That is, if our sample was representative of the world-wide portfolio.
Now, the number of yearly catastrophic failures around the world is slightly above three on long term average. Hence we can infer that our sample portfolio is marginally better than the world-wide one. In other words, none of these owners asked us to look at “real bad apples” or have “real bad apples” in their inventory.
This rests the discussion of the representativeness of the ORE2_Tailings™ results. Indeed it shows they reflect historic reality thanks to years of calibration and observation.
Closing remark on Tailings risk assessment methods comparison
We consider the very act of undertaking a rational and serious risk assessment as a very important step towards risk reduction.
Clients asking for a study of their dam foster enhanced risk awareness, care and understanding.
However, one should remember the limitations and approximations of each method and ensure that the results are anchored to reality.
Tagged with: decision, risk, Risk Assessment, Tailings Dam, tailings dam failures
Category: Mitigations, ORE2_Tailings, Risk management, Uncategorized
Leave a Reply