Risk assessment and automation bias

Risk assessment and automation bias

Jan 27th, 2021

In these days of AI, IoT and other technological developments Risk assessment and automation bias have to be discussed.

Risk assessment and automation bias

The definition of Automation bias covers the propensity users have to favor suggestions from automated decision-making systems. Victims of automation bias will ignore contradictory information made without automation, even if it is correct.

Automation bias is the last of a long dynasty

Automation bias is heir to a dynasty of biases based on the same principle, but seemingly very different one from another, for instance, and in what we believe may be a sort chronological order:

  • trusting a vendor because the price is written,
  • blindly following instructions or orders, even if they appear grotesque,
  • trusting the results of a software, without checking if they make sense,
  • following GPS instructions even if the area they make us enter looks shady, and finally,
  • accepting the conclusions of a AI, IoT or Space Observation program as good to go without any critical cross-check analyses.

We will discuss how Risk assessment and automation bias may lead their users to inappropriate decisions.

Risk assessment and automation bias

ISO31000 and other norms are extremely clear, Risk assessments must first state the context and the conditions of validity of the evaluations. Those two statements clarify the reasons behind the assessment and the contour of its valid. Neither should ever be forgotten and any user of the assessment results should always refer to them.

Many biases afflict Risk assessments especially if the two statements are not precise enough and transparent.

In our more than thirty years of quantitative risk assessments practice we have seen a number of flawed risk assessments due to that lack of clarity and biases. That is why we offer risk assessments audits as a service and why we have written so much on the subject. We have created a list of what leads to wrong assessment of risks, we summarize below.

  • Uncritical acceptance of step by step modus operandi rather than attention to the factual environment/situation development. This fallacy leads to prioritizing risks with what is urgent rather than what is important.
  • Accepting the extant reports as face values, unawareness of conditions noted in the model that they use for the decision making.
  • Unfamiliarity and/or ignorance of the context.
  • Outdated reports that do not reflect on the changes of the context or situation.

Tagged with: , , ,

Category: Consequences, Probabilities, Risk analysis, Risk management, Uncategorized

Leave a Reply

Your email address will not be published. Required fields are marked *

Riskope Blog latests posts

  • India dam break Facebook and fake news
  • 24-02-2021
  • India dam break Facebook and fake news finds its inspiration in a series of very recent events. Very unfortunately one…
  • Read More
  • Factor of Safety and probability of failure of geostructures
  • 10-02-2021
  • Today we discuss the Factor of Safety and probability of failure of geostructures. We presented this discussion  at TMW 2020…
  • Read More
  • The eternal confusion between tailings hazard and risks
  • 3-02-2021
  • A comment to a blogpost on tailings failures inspired this comment on the eternal confusion between tailings hazard and risks. Some…
  • Read More
  • Get in Touch
  • Learn more about our services by contacting us today
  • t +1 604-341-4485
  • +39 347-700-7420

Hosted and powered by WR London.