Automation bias

Automation bias is when humans prefer outcomes that are generated by automated systems over the outcomes generated by non-automated ones, even if the contradictory information indicates otherwise.

Example: During detecting defective products in mass production, automation bias occurs when the quality inspectors prefer to use an image classification model with 20% lower accuracy than a human inspector.

Human inspector with higher accuracy compared to automated systems

How to prevent automation bias

To avoid automation bias you should:

  • Reason why you need AI and reason if you need AI to avoid overuse of AI when it is not needed.

  • Compare the performance of automated systems with non-automated systems.

Was this page helpful?