Failures in the Loop: Human Leadership in AI-Based Decision-Making

The dark side of AI has been a persistent focus in discussions of popular science and academia (Appendix A), with some claiming that AI is “evil”
[1]
. Many commentators make compelling arguments for their concerns. Techno-elites have also contributed to the polarization of these discussions, with ultimatums that in this new era of industrialized AI, citizens will need to “[join] with the AI or risk being left behind”
[2]
. With such polarizing language, debates about AI adoption run the risk of being oversimplified. Discussion of technological trust frequently takes an
all-or-nothing
approach. All technologies – cognitive, social, material, or digital – introduce tradeoffs when they are adopted, and contain both ‘light and dark’ features
[3]
. But descriptions of these features can take on deceptively (or unintentionally) anthropomorphic tones, especially when stakeholders refer to the features as ‘agents’
[4]
,
[5]
. When used as an analogical heuristic, this can inform the design of AI, provide knowledge for AI operations, and potentially even predict its outcomes
[6]
. However, if AI agency is accepted at face value, we run the risk of having unrealistic expectations for the capabilities of these systems.