Designing for Fallible Humans

Security and privacy solutions today are designed with an assumption of a rational user. System designers assume that the user is able to review all information shown to them, consider it along with other information they have, and user priorities, and make a conscious, rational decision in their best interest. We all know that these assumptions are wrong. Even worse, they are simply excuses for technology-centric, best-effort design. This paper argues for designing for fallible humans, taking into account human cognitive limitations, human bias and human preferences. Such design means anticipating human error and compensating for it with built-in safeguards, it means presenting information in a way palatable to humans, it means soliciting user input and working collaboratively with the user's cognitive biases and preferences. It means helping users weave security and privacy into their daily routine, and not view them as obstacles or overhead to other, more desirable tasks.