Automation bias is when humans prefer outcomes that are generated by automated systems over the outcomes generated by non-automated ones, even if the contradictory information indicates otherwise.
Example: During detecting defective products in mass production, automation bias occurs when the quality inspectors prefer to use an image classification model with 20% lower accuracy than a human inspector.
How to prevent automation bias
To avoid automation bias you should:
Reason why you need AI and reason if you need AI to avoid overuse of AI when it is not needed.
Compare the performance of automated systems with non-automated systems.