Worldwide tailings benchmarks ten years later
Aug 31st, 2022
A colleague sent us a paper by Rana, Ghahramani, Evans, Small, Skermer, McDougall and Take. We reference it herein as Rana’s 2022 paper. The title is “Global magnitude-frequency statistics of the failures and impacts of large water-retention dams and mine tailings impoundments”. It is available online since Aug. 3rd 2022 at Earth-Science Reviews (2022), .
I must confess at first I was quite intimidated. Almost a decade earlier we had written Factual and Foreseeable Reliability of Tailings Dams and Nuclear Reactors -a Societal Acceptability Perspective, TMW 2013. We had produced a first estimate of the rate of occurrence of major catastrophic tailings dams failures in the world. Thus, it was with a mix of anxiety and trepidation that I undertook the reading of the paper by such a group of experts and academics.
Rana’s 2022 paper scope
The paper analyzes worldwide datasets of large reservoir facilities (LRFs) and tailings storage facilities (TSFs). It covers the period 1965-2020. Rana’s 2022 deals with recording unavoidable data gaps by adopting multiple estimation/extrapolation approaches. It discusses the assumptions and illustrates the resulting uncertainties.
In our discussion we will focus on TSFs only, due to our specific experience in this matter. We will reference herein our paper as Oboni’s 2013. Let’s note right away that the uncertainties are very significant even with basic data such as the number of tailings dams. Indeed Rana’s 2022 paper indicates a low estimate of constructed TSFs at 6,810, respectively a upper estimate of 20,230.
Let’s start with a couple definitions
Rana’s 2022 clearly defines the glossary it uses. We note the following definitions:
- Tailings storage facility (TSF): an impoundment of tailings, supernatant including a dam, tailings and water. We note that in our experience many TSFs include more than one dam to impound tailings in a specific pond. This generates uncertainties on the census. We also note that the failure definition in Rana’s 2022 does not necessarily include a breach/catastrophic failure of impoundments.
- Annual failure rate: the number of failures per the number of constructed facilities worldwide in a given year. Rana’s 2022 expresses this as a percentage and finally,
- Cumulative failure rate: the cumulative number of failures per the cumulative number of constructed facilities worldwide at the end of a given period> Rana’s 2022 expresses this also as a percentage.
In Riskope practice we prefer the scientific notation to the percentage as it better covers the hopefully small numbers depicting the probability of failure.
Oboni’s 2013 estimated the following
Assumptions
Following Davies & Martin (M. P. Davies, T.E. Martin, 2000; N. Lemphers, 2010) there were more than 3,500 TDs (of various kinds and construction type, with at least 50% of the upstream design) around the world. In absence of more accurate data, and more particularly, on the evolution of this number, Oboni’s 2013 considered a portfolio of 3,500 dams globally and 1,000 dams for the US alone. Portfolios were assumed constant over the years. However, we noted that even if that assumption was wrong by one order of magnitude, there would be no significant alterations in the conclusions of the paper.
From UNEP (1998) world data we read that there were 44 major reported TSFs failures in the decade around 1979 and 7 in the decade around 1999.
Results
Using the numbers above in 2013 we evaluated for the worldwide portfolio in the decade “around 1979” (that is 1974-1984) a rate of 44/(3,500*10)= 10-3 per annum, respectively 7/35,000= 2*10-4 per annum for the 1994-2004. Oboni’s 2013 also gave values for the US portfolio alone.
Oboni’s 2013 vs. Rana’s 2022 worldwide tailings benchmarks ten years later
First of all let’s note that Rana’s 2022 state that the global rate of reported TSF failures has remained fairly constant since the mid sixties. Indeed, the annual number of TSF failures worldwide mostly ranges between 2 and 6, closely matching the Oboni’s 2013 estimates (appx. 1 and 7).
Rana’s 2022 low TSF estimate
So, Rana’s 2022 and Oboni’s 2013 agree that the annual number of TSF failures stayed relatively constant, despite a reported increase in the annual construction rate of TSFs. Rana’s 2022 states that the cumulative failure rate of TSFs declined over time. When assuming Rana’s lower-estimate of the number of constructed TSFs (6,810), they estimate a cumulative failure rate is ~4.4% by the end of 2020.
Rana’s 2022 upper-estimate generates some questions
Counter-intuitive result?
However, when adopting their upper-estimate (20,230 TSFs), Rana’s obtain a rate of ~1.5%. This value is in the same order as the corresponding rate of Large Reservoir Facilities (LRFs). We perceive this result as “strange”. Indeed the general perception is that hydro reservoirs are “safer” than tailings dams. At Riskope we believe this “strange” value is the result of an exaggeration of the TSF count and “mixing oranges and apples” in the TSF world-wide basket.
“Advocates” bias?
Let’s also note that some recent “advocate” papers seem to have inflated the number of dams in order to demonstrate the hazard. Actually, this move does not help their discourse because inflating the number decreases the rates. Hence an inflated number shows exactly the opposite than the desirable “advocate” result. One more thing: should the rate of tailings and hydro dams be “similar”, public opinion should be as worried by hydro than tailings. This is blatantly not the case nowadays. However, things may change as LFR age and slowly start crumbling under our eyes, world-wide.
Using the same Oboni’s 2013 data and converting them into the Rana et al. glossary definition we would have:
Annual failure rate:
The estimation follows a few simple steps: 1974-1984 44/10= 4.4 major failures per annum, per 3,500 assumed TSF= 10-3 or 0.1%, respectively 7/10=0.7 per 3,500 leading to 2*10-4 or 0.02%. As we will see in the next section these estimates are in very good agreement with those of other researchers.
For benchmarking of dams, a standard procedure in the ORE2_Tailings™ quantitative risk assessment, compliant with GISTM, the annual failure rate is paramount. Indeed it allows to anchor results to factual reality. Furthermore we can tell clients if their dams are better or worse with respect to the world-wide portfolio performance.
Cumulative failure rate:
By using the 1965-2020 timeframe (56 years) and a long term average yearly number of failures of 3.5, again with 3,500 TSFs and apply the Rana’s 2020 definition we get 3.5*56/3,500=0.06 or 6%. If we use the average values of the two decades (see earlier), we find (4.4+0.7)/2=2.55. Thus, by applying Rana’s 2020 definition we get a cumulative failure rate of 2.55*56/3,500=0.04 or 4%. Again, the match between Oboni’s 2013 and Rana’s 2022 is excellent.
The cumulative failure rate is not useful for benchmarking single dams as it is victim of bluntness due to “excessive averaging”, a point we discuss below.
Other “statistical studies” and estimates
Azam and Li
In our Tailings Management book (https://link.springer.com/book/10.1007/978-3-030-19447-5 ) we quoted, like Rana’s paper does, Azam and Li (2010). These researchers suggested a historical, global failure rate of 1.2% for TSFs. One gets to that value by dividing 220 recorded failures by 18,400 mines, assuming 1 TSF per mine. It is important to note that this value covers over hundred years of records. Thus the actual annualized rate of failure is 1.2%/100= 1.2*10-4 in full agreement with the value by other researchers, such as Halabi et al. (2022). Indeed, they assumed the global estimate of 18,400 existing TSFs and calculated the annual probability of TSF failures to have been in the order of 10-4 on average in the period 1950-2019.
Other statistical approaches
Other statistical approaches bear on the rate of failure, potential consequences and potential failure modes, Their discussions in 2018-2019 are present this blog as follows:
Thus, the benchmark values we use in ORE2_Tailings™ since 2014 are confirmed by the Rana’s 2022 paper as well as the other researchers referenced above. Worldwide tailings benchmarks ten years later is a success!
Using rates and evaluating “returns”
At Riskope we are reticent to use returns. That is because layman tend to understand return as “the next failure will occur in “so many” years”. This is not the only reason as we think the return is a general disservice to the public.
However, for the sake of comparison, we have picked a selected number of dams portfolios, with varying volumes, in various parts of the world. We analyzed these 57 dams for various clients using ORE2_Tailings™. They are a subset of the large number of structures we have evaluated to date using the same methodology.
Three sample portfolios
- A: in Central Asia a portfolio of 4 upstream dams with an average yearly probability of failure of 7.80E-03. These are poorly documented, monitored, maintained and give a return of ~32 years.
- B: a selection of 14 dams in the US with excellent documentation, analyses, mitigations and good monitoring. Their yearly average probability of failure of 7.69E-04 gives a return of ~93 years and finally,
- C: 39 Canadian mix-quality dams with a yearly average probability of failure of 6.79E-04 give a return of ~38 years.
Three samples conclusions
- For the entire portfolio A+B+C of 57 dams the evaluated return is one failure on average every 21 years because a few bad apples spoil the lot in terms of expected returns.
- If we assumed that the entire portfolio of 57 dams is representative of the worldwide portfolio, and use the values above, we would get 2.9 failures per year. This value falls well within the observational range of the last hundred years. However,
- the fact that the long-term average world-wide is around 3.5 dams failures per year shows that the world-wide portfolio is “worse” than our sample of 57 dams.
Important notes
Thus, not only the benchmarks are correct, but ORE2_Tailings™ results are proven to be realistic and well calibrated at world-wide scale. Furthermore, the returns yielded by the ORE2_Tailings™ analyses in our day-to-day practice also match the returns evaluated by Rana’s 2022. This constitutes another proof of the correct calibration of the methodology based, for example, on on more than hundred dams evaluated in our forthcoming paper at TMW 2022 . By the way, the title of our forthcoming TMW 2022 paper is Optimizing mitigation of tailings dams portfolios.
Averages needs to be taken with a grain of sand (pun intended).
As we have hinted earlier, one should consider averages with care because they tend to smooth out data. Thus, they depict an optimistic image of the whole portfolio. This hinders good mitigative decisions and overall management.
To illustrate this, we have prepared a graph of the annual probability of failure (vertical axis) for the three portfolios A, B, C and the global one (A+B+C selected portfolios) results. The graph depicts the Oboni’s 2013 min-max benchmarks and pre-catastrophic threshold as horizontal lines. For each portfolio we show: the min-max annualized estimated probabilities of catastrophic failure, the average, as well as the standard deviation range (blue bar).
We can observe the following:
- A: C Asia portfolio has a low variability because of the small sample size (dam number).
- B: US sample size is bigger. Despite having an average within the worldwide benchmark, it tends to overlap the worldwide benchmark with some dams.
- C: The selected Canadian dams portfolio has some dams with their max estimates being pre catastrophic, despite having an average within the worldwide benchmark. Finally,
- the global portfolio (A+B+C) carries the min-max estimates of the best and worse estimates of the overall portfolio. It has an average and standard deviation influenced by each single dam. However, note that the nefarious influence of the C Asia four dams gets “smoothened” down by averaging.
- This leads to a statement we have oftentimes made. When looking at tailings dams the devil is in the details and oversimplification or “bulk” judgements can be extremely misleading.
One last source of satisfaction in Worldwide tailings benchmarks ten years later
We were very pleased to read in Rana’s 2022 their “Note that many TSF failures have been caused by multiple variables (our note: ORE2_Tailings Potential Failure Causality Analysis – Riskope). The proportional contributions of structural drainage deficiency and weather hazards as causative variables has increased, whereas that of seismic activity, embankment deficiency and unstable foundation has decreased.”
A caveat
This note validates Riskope’s view that causalities rather than failure modes are important for risk assessment ( Why everything we know about Tailings Dams failure is wrong – Riskope). Furthermore, “slope methodologies” such as Silva Lamb Marr and its clones should be avoided. That is because they fail to explicitly capture essential aspects of a dam’s life such as:
- ancillary water management,
- main pipes,
- traffic, and finally
- ponding,
- etc…
Aknowledgement
We want to close this Worldwide tailings benchmarks ten years later blogpost by thanking Rana et al. for their outstanding academic paper (Rana’s 2022). This has indeed allowed us to prove the validity of the benchmarks we use in ORE2_Tailingstm. The confidence that the methodology is indeed well calibrated, world-wide has significantly increased thanks to them.
Tagged with: benchmarks, dams failures, decision, risk, tailings, Tailings Dam
Category: ORE2_Tailings, Probabilities, Risk analysis, Risk management
Leave a Reply