Choice, Uncertainty, and Decision Superiority: Is Less AI-Enabled Decision Support More?

Providing decision makers with more information is often expected to result in more informed and superior decisions. This is especially true when leveraging artificial intelligence (AI) to explore and find complex patterns in vast amounts of data. Although AI can enable an “information advantage,” truly intelligent systems should buffer scarce human cognitive resources from information overload and be well adapted to the environment in which they are deployed. Paradoxically, some practitioners have conflated AI's information processing superiority with a contradictory decision-support goal: to provide human decision makers with more, higher quality, or more novel courses of action, regardless of context, than they could generate without AI. In this article, I review the evidence examining the costs and benefits of providing decision makers with more or less choice and identify the factors that moderate the relationship between the amount of choice and decision effectiveness. Although providing more information and choice increases confidence and certainty in one's decision, it can make decision making more difficult, decrease satisfaction, and result in poorer decision outcomes. The research indicates that such negative effects are influenced by the level of entropy and variety provided and can be reduced with increased familiarity but are further compounded when decisions are increasingly effortful, difficult, or complex. The review concludes with guidance on how designers might leverage knowledge of choice overload and associated moderator effects to create more adaptive and effective decision support systems.


I. INTRODUCTION
A DEEPLY rooted view in modern society is that more information equates to more informed decision making, which, in turn, leads to better decisions [1], [2]. Some have argued that decision makers should consider all available information unless they can demonstrate that information is undoubtedly irrelevant and does not impact the decision outcome (i.e., principle of total evidence) [3]. These views are consistent with the rational perspective that more information is better than less when operating under conditions of certainty, unless the cost of obtaining information outweighs its value [4]. From a technological standpoint, this rational perspective has manifested itself in the deployment of more sensors, collection of more data, and the development of more technologies that promise to translate data into more useful information and better decisions. Today, artificial intelligence (AI) methods, such as machine learning (ML), are highly effective and disproportionately more efficient than humans at exploring vast amounts of data and finding complex, multifactorial, and/or nonlinear statistical relationships within that data [5]. The logical conclusion, one could argue, is that the cost of obtaining more information in many instances may no longer outweigh its value.
Recent U.S. policy on decision advantage exemplifies this technologically-oriented rational perspective [6], [7], [8]. This concept is built on the premise of technologically-enabled information superiority, that is, coordinated access to and better use of an uninterrupted and more voluminous data flow, to which an adversary's access is limited [9], [10]. Although most would agree that high technology can enable an information advantage (compared to those who do not have access to similar capabilities), one of the founders of AI and Nobel Laureate, H. A. Simon [11], described the purpose of any intelligent system as one that also "buffers … the scarcity of human attention … from the over rich information in which it swims" (p. 44) and is well adapted to the environment in which it is deployed. Paradoxically, some AI engineers have conflated AI's superiority in processing voluminous data with a pragmatic yet contradictory goal: To generate more, higher quality, or more novel courses of action (COAs), irrespective of context in which they are generated, than humans could generate without AI [12] (also see [13]). Although some designers of decision support systems are guided by a domain culture concerned with information overload, 1 many express a preference for providing more AI-enabled choice with the belief that this will improve decision outcomes. Designers often justify these preferences based on the assumptions that: 1) AI generated options are of higher quality than those generated without AI; 2) providing more options reduces the uncertainty inherent in complex decisions; or 3) providing more choice promotes the exploration of otherwise overlooked alternatives [13]. These preferences and assumptions are echoed by practitioners, for instance, military leaders discussing human-machine teaming requirements for multidomain operations [14].
Consistent with the belief that more choice is better, research indicates that people prefer more choice (and information) for many reasons: it increases confidence in one's decision and certainty about decision accuracy [15], [16], [17], increases perceived freedom of choice [18] and choice flexibility [19], and increases enjoyment in the decision making process [20]. Moreover, having more information supports post hoc justification of a decision better than having less, which facilitates defensive decision making [21]. Yet a preference for more choice does not necessarily result in better decisions. An important question to ask, therefore, is: what is the value of providing human decision makers with more, technologically-supported choice? This is especially pertinent if one assumes that AI is better equipped than humans to identify important relationships and/or present novel options without incurring additional cost to the decision maker.
In contrast to rational arguments and human preferences for more information, numerous researchers have demonstrated that humans can make effective decisions without access to all of the available information or by ignoring some of it when available [22], [23]. While some have shown that more information or choice can have negative consequences for decision quality, e.g., [24], others have shown that less complex models can perform at least as well as information-heavy complex models under conditions of uncertainty [2], [25]. The common message is that, in some situations, less is more. Although many of the studies reviewed below examined relatively simple decisions compared to those made in high-consequence domains (e.g., military decision making), the research indicates that the increased complexity, difficulty, or uncertainty associated with high-consequence domains only compound the observed effects.
In this article, I review the scientific evidence across a range of disciplines, domains, and contexts on the effects of providing decision makers with more or less information or choice. In Section II, I review research examining the information and choice overload 2 effects and their impacts on decision making. This section is divided into six sections: First, as a prelude to the choice overload effect, I examine the effect of providing more or less information to decision makers in Section II-A. Then I highlight how subjectivity in the model development process impacts the amount of choice presented to decision makers in Section II-B. Then, in Section II-C, I review the potential costs and benefits of providing decision makers with more or less choice, and highlight the factors that moderate choice overload in Section II-D. In Section II-E, I review the effect of providing more information or choice on uncertainty, and in Section II-F, I examine how perceived variety and entropy influence the relationship between the amount of choice and decision effectiveness. In Section III, I provide guidance on how designers and developers might leverage this research and knowledge of these empirical effects to create more adaptive and effective decision support systems. Section IV concludes this article.

A. Information Overload Effect
In many situations, access to minimal essential information is better than having no information, which can lead to negative consequences or poor outcomes [15]. However, providing more information may be useful only up until a point. Thereafter, it may have diminishing returns [15], [26], [27]. Too much information-referred to as the information overload effectcan result in negative outcomes, including increased information search costs and reduced decision accuracy [15], [16], [21], [28].
Numerous studies have documented these effects. In a classic study, Jacoby et al. [15] presented university students with 2, 4, or 6 pieces of information about multiple retail brands, and asked them to select the one that most closely approximated their ideal brand. Their data showed an inverted-U relationship between the ability to select the best brand and the amount of information considered: Worse decisions were made with the least (8 pieces) or most (74 pieces) information, whereas the best decisions were made with a moderate amount (e.g., 24-48 pieces) (for similar effects in accounting, see [29]). These authors also observed an information by choice interaction effect (for a discussion of Choice Overload, see Section II-C). When participants were presented with four brands to choose from, they made better decisions when given more compared to less information about each brand. But when presented with 12 brands, decision making was generally poor irrespective of the amount of information provided. In general, when minimum information was coupled with minimal choice, or maximum information was coupled with maximum choice, decision making suffered. In contrast, decision making was best when the effects of providing more information about each choice were considerably dampened by fewer choices.
Despite demonstrating a decrease in decision quality with more information (or choice), participants in the Jacoby et al. [15] study were more certain that they had made the best choice when given more information. Slovic [17] observed a similar increase in certainty with more compared to less information, without any improvement in actual decision quality. Expert horse-race handicappers were asked to identify 5, 10, 20, or 40 variables (from a list of 88 variables describing a horse's past performance) they would use to handicap a race if access to all information was limited. Handicappers subsequently judged 40 horse races. As the number of variables they considered increased from 5 to 40, decision confidence increased. However, judgment accuracy remained unchanged and judgments became more inconsistent. The smaller variable sets (e.g., 5, 10) meant handicappers made their decisions with fewer cues that, on average, were more predictive. However, their increased confidence with more variables suggests that they expected the additional information to have some, if not more, predictive value. This expectation is consistent with the finding that more confident decision makers prefer more information [30].
A study on command-and-control decision making in a simulated fire chief decision making task further examined the effect of using additional information that was relevant to the task at hand [31]. Experienced fire fighter commanders were tasked with deploying resources to control the spread of fire. Varying degrees of information (i.e., incomplete versus detailed) was provided under a range of constraints (i.e., more/less reliability, more/less control over resource deployment, and in/ability to communicate with subordinates). When additional relevant information was made available (i.e., landscape features, wind details, fire warnings), commanders were compelled to use it and seek it when not available. While consistent with prior work showing that information seeking increases decision confidence [32], [33], the desire to use or seek additional relevant information proved costly. Participants saved significantly more assets when less information was available or sought, even when it was highly relevant to the task (also see [24]). Similar information seeking costs have been shown in other domains. For instance, in a spatial prediction task, Wickens et al. [34] demonstrated that although participants could identify better deals (i.e., when information was more useful), they were biased toward obtaining more information even when its cost exceeded its expected value. In a related study, Shields [35] observed an inverted-U relationship between judgment accuracy and the amount of information extracted from a report used to judge organizational performance. Beyond a moderate amount of information, extracting additional relevant information impeded accuracy.
Collectively, this research indicates that decision making effectiveness increases in a curvilinear manner with more information. After a moderate amount of information, decision makers may confound the feelings (e.g., confidence, certainty) associated with receiving (or seeking out) more information with the ability to make a good decision and overlook information search costs, negative return, and related overload effects. Several authors [2], [21], [36] have conjectured that more information is likely to be beneficial in situations where an optimal COA can be calculated: when uncertainty is low, the environment is stable, and when judgment requires only few risk factors to be estimated. Under these conditions, they argued that large amounts of data or information are likely to be enormously helpful for estimating risk and consequences and determining optimal choice. However, these authors argued that optimization may not be possible when uncertainty is high. Section II-B highlights how subjective processes in decision model development can impact the amount of choice provided to decision makers.

B. Subjectivity in Decision Modeling and Its Effect on Choice
One of the primary purposes of decision support systems is to increase the accuracy of human decisions, in part, by easing the cognitive burden on decision makers when decisions are difficult, complex, or unfamiliar. Decision support systems are often perceived as objective, especially when enabled by AI or ML. This may be due to their highly quantitative and computational nature, which can increase confidence in the accuracy of their decisions. However, many ostensibly objective processes are based on subjective assumptions or decisions that can influence the amount of choice provided to decision makers.
In AI recommender systems, for instance, the approach developed to score, rank order, and select potential options is typically based on the developer's specific goals or objectives. Different objectives or associated scoring functions could result in different recommendations, reprioritization of options, or an increase or decrease in the final number of options provided [37]. Likewise, in reinforcement learning, decision options are generated and rank ordered based on the probability of maximizing reward. However, rather than rewarded behaviors being determined empirically, they are usually based on subjective preferences for specific behaviors and subjective assumptions about how they should be measured [38]. If different rewards are maximized or measured differently, different options may be generated [39]. In unsupervised learning methods, such as K-means clustering, different similarity metrics (e.g., Euclidean distance, cosine, correlation)-used to determine cluster quality-can result in different values of K being optimal [40], [41]. Although the value of K can be optimized statistically, subsequent validation of generated clusters with subject-matter experts has given little consideration to the (mis)match between the model and expert's bases for categorization, how experience or context might influence that judgment, or the extent to which experts may disagree [42], [43], [44].
Arguably, the greatest impact of human subjectivity on the number of decision options generated is on cut-off placement. That is, once a rank-ordered list of options has been generated, where do developers draw the line (i.e., under the top-3, -7, or -n)? At present, the basis for specifying this threshold or modifying it under different conditions is largely based on subjective choices made in model development. In some instances, such as principal components analysis-an unsupervised ML technique-a statistical norm is used to determine the threshold for selecting components (i.e., Kaiser-Guttman rule: Retain components with eigenvalues greater than one) [45], [46], [47]. This common practice is preferred despite the acceptance of multiple other objective cut-off methods that could change the number of components (e.g., scree test; minimum average partial method; parallel analysis). A primary determinant of which method is implemented is not the objectivity or suitability of each one but rather the default setting within the modeling software being used [48], [49].
These are just a handful of examples of the subjective factors that can impact the amount of choice provided to decision makers. Counter to Peirce's [50] recommendation to base such decisions on science, many decisions are based on subjective developer preferences for presenting more or less options or domain cultures with different values (e.g., fewer COAs [51] versus more alternatives/exploration [52]). In Section II-C, I examine the assumption that more choice is better.

C. Effects of Too Much Choice on Decision Making
Much like too much information, which often results in negative outcomes (e.g., increased information search costs and reduced decision accuracy), too much choice often results in decision-making difficulties and reduced satisfaction with a choice. Paradoxically, however, people enjoy or are attracted to more extensive choice [53]. The negative process-related attitudes and outcome-based behaviors associated with too much choice have been termed the choice overload effect [54], [55].
A meta-analysis of 50 experiments found that choice overload effects were sometimes but not always observed, and concluded that there was no reliable choice overload effect (d = 0.02) [54]. However, considerable heterogeneity was reported in this data (I 2 = 68%), indicating that the absence of a choice overload effect could be explained by unidentified moderators. In a subsequent meta-analysis examining 99 choice overload experiments (including 78% of the 55 observations analyzed in [54]), Chernev et al. [55] found a significant and negative choice overload effect in studies without a moderator and a significant moderating effect of four variables on choice overload. The influence of these moderators is discussed below. First, I describe the typical effects of providing more or less choice.
In a seminal series of studies on consumer and student decision making, Iyengar and Lepper [20] provided grocery store customers with either a limited (n = 6) or extensive (n = 24) choice of flavors of jam. Although visitors were more attracted to the more extensive display, significantly more people made a purchase when given limited (30%) compared to extensive choice (3%). In a follow-on experiment, participants given a more extensive choice of chocolates (i.e., 30) enjoyed the selection process more. But, compared to those given less choice (i.e., 6), they also felt they were given too many options, found the choice more difficult, and were more frustrated and less satisfied with their choice.
Several researchers have examined whether similar effects are observed in more consequential tasks. For instance, Iyengar et al. [56] analyzed thousands of employee records across multiple industries to examine the effect of choice on 401(k) retirement plan participation. They demonstrated that employee participation in any retirement plan reduced when given too much choice. For instance, plan participation reduced from 75% when two funds were offered to around 60% with 59 plans (also see [57]). Likewise, Johnson et al. [58] investigated whether presenting decision makers with more or less health plans impacted the quality of plan choice for themselves, a partner, and a child, after considering copayments, yearly premiums, deductibles, and out-of-pocket expenses. When provided with four (compared to eight) plans, participants chose the best plan more often (42% compared to 21%) and, on average, incurred lower choice costs (i.e., ∼$200 compared to ∼$250 overpayment).
In a related study, Schram and Sonnemans [59] simulated the choice between more or less health care plans and examined the plan choice and the cost of switching given changes in health risk profiles over time. When provided with four (compared to ten) health plans, participants were more likely to choose the optimum plan. Although those given more choice digested more information in total than those given less, they considered a smaller proportion of the available information. More choice led to higher search costs, longer decision times, an increased likelihood of switching between plans (when there was no switching cost) but a decreased likelihood of switching to a better plan (for similar effects, see [60]). Similar effects have been observed in other medical contexts. For instance, when physicians were presented with a more extensive choice of drugs that could be prescribed to their patients, they were less likely to prescribe any of them compared to when given less choice [61], [62].
Two studies of abstract decision-making suggest that the choice overload effect may also depend on how stringently decision effectiveness is defined (e.g., optimal versus near-optimal versus good enough) [63], [64]. As in previous studies, providing more choice reduced the likelihood of selecting the optimal or a near-optimal option (i.e., within 10% of optimal). However, participants' ability to select a good-enough option (i.e., in the top 25%) increased from 46% with 4 options to 66% with 8 options and remained relatively stable with 12 options [64]. This apparent contradiction-worse optimal but better good-enough option selection-was explained by participants' use of more appropriate heuristic methods that reduce the cognitive effort associated with a decision. With 4 options, participants relied on a tallying heuristic (i.e., count the number of cues favoring each option), whereas with 8-12 options, they used lexicographic (i.e., prioritize options based on the most important attribute) and undominated heuristics (i.e., eliminate least desirable options) that work better under uncertainty.
Although decision support systems are not designed to leave human decision makers to their own devices, the observed negative impact of too much choice on decision outcomes mirrors the findings on option generation, where participants have to generate their own options without assistance. For instance, Johnson and Raab [65] asked intermediate-level handball players to generate conceivable COAs and then choose the best one after viewing excerpts from handball games. Decision makers generated relatively few options and generated better options first. Consistent with the choice overload effect, as the number of generated options increased, the likelihood of selecting the best option decreased (for similar results, see [66]). In a similar study, Ward, et al. [67] asked skilled and novice soccer players to make decisions about their opponents'-rather than their own-COAs. As in the previous study, relatively few options were generated with better options first. This time, decision effectiveness declined as the number of bad options generated increased. These results have been replicated under various constraints (e.g., with/out time-pressure) and in other complex domains (e.g., [68], [69], [70], [71]).
Much like information overload, the research on choice overload demonstrates that people are attracted to more choice. But more choice increases decision making difficulty and reduces satisfaction. More importantly, it leads to poorer decision outcomes on average in both abstract and real-world decision making tasks, including greater search costs, longer decision times, worse and more costly decisions, and a decreased likelihood of switching to better options. However, some evidence suggests that while decision effectiveness generally declines with more information, people can continue to satisfice (i.e., make a good-enough choice) when they use ecologically fit and effort-reducing cognitive heuristics. In Section II-D, I highlight the constraints that moderate the relationship between amount of choice and the associated overload effects.

D. Moderators of the Choice Overload Effect
Meta-analytic research has identified several moderators of the relationship between the number of options and choice overload effects. For instance, Scheibehenne et al. [54] identified two significant moderators. First, they showed that, for those studies using consumption quantity as the dependent variable, more choice resulted in greater consumption. Consuming more, however, is not a useful proxy for effective decision making and is more analogous to searching for more information than choosing a good option [31], [35], [59], [72].
Scheibehenne et al. [54] also showed that when more options were presented to participants who were proficient or familiar with the decision, or who had a strong preference for more options, they were more satisfied with their choice or more likely to make a choice. Chernev et al.'s [55] first moderator, preference uncertainty, exemplified this effect. Negative effects were greater when preference uncertainty was high, such as when decision makers did not have a meaningful basis for making a decision (i.e., limited expertise, knowledge, or criteria for making choice, or did not have a strong preference). This resulted in greater switching, more choice deferral, poorer option selection, and decreased satisfaction (Mean Cohen's d = 0.4 approx. 3 ) (see also [73]; for similar effects, see [25]). However, the choice overload effect largely disappeared or was reversed (Mean Cohen's d = −0.2 approx.) when preference uncertainty was low [55]. That is, more rather than less choice was facilitative when expertise, task familiarity, or preference for a large choice set size was high.
Chernev et al.'s [55] analysis also demonstrated that more choice coupled with more complex choice sets resulted in more negative overload effects (Mean Cohen's d = 0.2 approx.). The complexity moderator captured the contextual factors associated with the value-based relationships between options. When provided with more choice but without a dominant or attractive option, when options were not complementary, or their attributes easily aligned, decisions were more likely to be deferred or were less effective. However, when complexity was low, the effect was reversed (Mean Cohen's d = −0.9 approx.). These data are consistent with the claim that less is more as the level of complexity increases [2], [36].
Decision difficulty also moderated the choice overload effect [55]. When more choice was provided under more severe time constraints, with more attributes per option, or with greater levels of accountability, the result was a greater likelihood of deferring the choice, poorer option selection, decreased satisfaction, or increased regret (Mean Cohen's d = 0.5 approx.) [55]. However, when decision difficulty was low, the trend was reversed (Mean Cohen's d = −0.2 approx.). These findings are consistent with those who have shown how the number of options interacts with or is mediated by task-related difficulty factors, such as time pressure (e.g., [74], [75], [76]). 3 Mean Cohen's d was estimated from [55, Fig. 3a-3d].
Finally, more cognitively effortful goals, such as buying rather than browsing or choosing a specific option rather than a particular choice set, moderated the choice overload effect [55]. The choice overload effect was greatest with more effortful decision goals. However, more rather than less choice was preferred when decisions were less effortful (Mean Cohen's d = −1.6 approx.). Although this effect was considerably tempered with more effortful decisions, more choice was still favored (Mean Cohen's d = −0.6 approx.) (see also [26], [73], [77]). The negative impacts of increased information processing demands are consistent with the claim that a reduction in choice may be facilitative when decisions impose considerable burden on limited cognitive capacity (e.g., [2]). Since the amount of effort imposed is highly contingent on one's level of proficiency, these effects may be offset by preference uncertainty or option familiarity when individuals have the necessary expertise to make a meaningful choice [78].
While this section provides empirical and meta-analytic insights into some of the factors known to moderate the relationship between the amount of choice provided and choice overload effects, this list is unlikely to be exhaustive. For instance, a range of factors that have been shown to impact decision making more generally may also affect this relationship, including the level of risk posed, the criticality of the impending decision, the need to be adaptive and/or resilient, or the trust one may have in the decision support system. To the best of our knowledge, however, research has not explicitly or directly examined the moderating influence of such variables on the relationship between the number of options and choice overload. In the next two sections, I examine the effect of increased uncertainty and variety on choice overload.

E. Effect of More Information or Choice on Uncertainty
Many modern-day work contexts and decisions are characterized by varying degrees of uncertainty. A popular misconception is that more information or choice may reduce uncertainty [13]. On face value, this is consistent with a preference for more information and the illusion of certainty that conveys [15], [16], [17]. It is also consistent with reasons why people delay their decision making until receiving more information or choice. That is, more information or choice increases a decision maker's perception they are choosing from the full spectrum of possible options, including the best option [79]. However, providing more or all possible options is not equivalent to reducing uncertainty.
Van Herpen and Pieters [80] argued that decision makers make worse choices when those choices are high in informationtheoretic entropy, which is a function of the information conveyed by an option (i.e., the number of bits 4 ) and the level of uncertainty associated with each option (measured by its known probability [p]) [81]. In uncertain situations, for instance, when decision options are equally likely, entropy or the amount of information conveyed (i.e., the degree of uncertainty) increases logarithmically with the number of equally probable options presented. In contrast, when armed with the knowledge that some options are more likely than others (e.g., four options with p = 0.5, 0.25, 0.15, 0.1, respectively) compared to the same number of equally likely options (e.g., four options each with p = 0.25), inherently, there is less uncertainty and less information conveyed in the unequal probability choice set (i.e., 1.74 bits compared to 2 bits in the case of four options, equal probability set). In sum, when options are equally likely, adding more options increases uncertainty. The extent to which uncertainty increases with each option can be dampened by making additional options more or less likely or, for instance, by increasing or decreasing the number or probability of attributes for any given option.
Numerous studies have demonstrated information and choice overload effects on decision outcomes in entropy terms. For example, Lurie [76] asked undergraduate students to choose the best calculator from 18 or 27 different options within a 2-min time period. Entropy was manipulated by providing ratings of each calculator on seven different attributes (e.g., versatility, ease of use, battery life) whose probability was either evenly or unevenly distributed. The probability of choosing the best option reduced from 0.96 when presented with less choice with an uneven attribute distribution (i.e., where entropy was lowest; 36.28 bits) to 0.65 when given more choice with an even attribute distribution (i.e., where entropy was highest; 52.75 bits). Moreover, their analyses showed that entropy mediated the relationship between the number of alternatives and choice quality. That is, when more choice contained less information (measured in terms of entropy), the quality of choice was better than when presented with fewer choices containing more information.
In a subsequent simulation study, Lurie [76] examined the interactive effects of choice set size and entropy on choice quality under two time-pressure conditions. Their results showed that as entropy increased (either by providing more attributes or a more even probability distribution), decision quality decreased. However, they also showed that this relationship was mediated by the decision strategy used and the amount of information processing or mental effort involved. The level of effort required (which increased as entropy increased) to make a choice under normal conditions (i.e., without time pressure) mediated the relationship between the number of options and decision quality under time pressure.
Using a variation of Lurie's [76] simulation, Fasolo et al. [82] examined whether similar effects would be observed in a more naturalistic context. They simulated decision making in large and small grocery stores and found that decision making strategies were substantially more effortful in the large store-where choice was more extensive, and entropy was high-compared to the small store-where choice was limited and entropy was low. In addition, they found that when customers chose a product with few competing brands, lower effort strategies used in the small store typically led to more accurate decisions compared to when making a choice in a large store. Even when choosing from a product with more competing brands, low effort strategies used in the small store resulted in comparable levels of accuracy to those used in the larger store. Only when choosing from very large product categories with an extensive range of competing brands, the strategies employed in the small stores typically result in lower levels of accuracy than more effortful strategies used in large stores.
Collectively, this research suggests that more choice or information increases rather than decreases uncertainty. Additionally, entropy mediates the relationship between the number of options and decision effectiveness. That is, the negative effects resulting from too much choice can be explained, at least in part, by the higher level of entropy in the choice provided. Since cognitive effort has typically been shown to increase when entropy increases, these effects may be explained by the moderating effect of task effortfulness [55]. That is, effort may moderate the relationship between choice and decision because of the mediating effect of entropy.
It is worth noting that in each of the aforementioned studies, entropy was increased or decreased by changing the probabilities of events, options, or attributes. However, in many real-world, high-consequence domains, such probabilities may be unknown or even unknowable, which may increase uncertainty, and further compound the negative effects of more information or choice. In Section II-F, I examine the related issue of how more choice, with more or less entropy, impacts variety.

F. Effect of Variety on Choice Overload
One argument for providing decision makers with more AI-enabled options is that it may add more novel options to the option set; options the decision maker would not or could not generate themselves. More novel options could provide an opportunity to explore new parts of the option space and/or facilitate useful contrast between alternative options [19], [83]. Rather than being based on the number of novel options presented, the ability to compare options or choose the best one is partially determined by the variety of choice provided (i.e., the dis/similarity between options) and the degree to which a choice can be justified [84], [85], [86]. When options differ markedly on some utilitarian attribute, decision makers find it easier to decide [82], [87]. In contrast, when attractive options become increasingly similar, the ability to differentiate between them and to justify one's choice becomes more difficult and can result in deferred choice [87].
Structural aspects of the choice set, including the number of options, level of entropy, and choice set organization, have been shown to influence perceived variety [80], [88], [89]. However, differing perspectives exist on what constitutes variety. For instance, Hoch et al. [88] proposed a product-or option-based approach in which variety is viewed as a function of the dissimilarity between pairs of options, often measured as the number of attributes on which they differ (i.e., Hamming measure). Option pairs with more attributes that differ from each other are considered more varied. Using a related option-based measure, Hwang and Lin [28] conducted a meta-analysis of 31 experiments to examine the effect of presenting varied information (i.e., the number of different measures of financial performance provided for each option) on predictions about whether a company would go bankrupt. When variety was defined in option-based terms, prediction accuracy was significantly lower when participants received more (Mean = 0.672) compared to less varied choice (Mean = 0.767).
In contrast, Van Herpen and Pieters [80] proposed an attributebased approach in which variety is a function of the entropy conveyed by options and attributes (i.e., described as dispersion) and the degree to which attributes are dissimilar (i.e., described as dissociation, measured as 1-Lambda). From this perspective, an option set is considered more varied when dispersion and dissociation are greater. The authors presented participants with either 8 or 16 products, each with either less (i.e., evenly distributed; high entropy) or more dispersed (i.e., unevenly distributed, low entropy) attributes, and with either a high, partial, or low level of attribute dissociation. Their results indicated that option-based measures (e.g., Hamming measure) were poor predictors of perceived variety (i.e., 3.4% variance explained), albeit were highly correlated with the number of options (r = 0.99). Attribute-based measures (i.e., dispersion, dissociation), on the other hand, were much stronger predictors of variety (i.e., 62.5% variance explained). Dispersion and dissociation were only moderately correlated with the number of options (r = 0.55, 0.48, respectively) and were not correlated with each other (r = 0.06), indicating that each one captured a unique aspect of variety.
In a study examining the structural aspects of variety, Kahn and Wansink [89] demonstrated that the level of entropy and organization of options (i.e., in a way that makes their variety more perceptible) moderated the relationship between actual variety and perceived variety. They demonstrated that perceptions of variety increased more so when attributes were unevenly (i.e., dispersed; low entropy) rather than evenly distributed, and when options were presented in an organized compared to disorganized manner. In other words, when cues were available that made variety more apparent (i.e., more organized, unevenly distributed), desired behaviors increased. Unfortunately, these studies did not examine the effect of perceived variety on decision effectiveness. However, the research examining entropy effects suggests that decisions are less effective when more choice is accompanied with higher entropy.
The decision difficulty and choice complexity moderators identified by Chernev et al. [55] capture several elements of the structural aspects of choice associated with variety. For instance, components of decision difficulty (e.g., number of attributes per option, presentation format) have been used as measures of variety, entropy, or organization (e.g., [88], [89]). Similarly, components of choice complexity (e.g., attribute alignment, option complementarity) have been used as measures of entropy or dissociation [80]. Hence, the moderating effects of task difficulty and choice complexity may further be explained, at least in part, by the aforementioned influence of structural aspects of choice [72], [76], [82], [89]. That is, the negative effects of more choice are compounded by increased difficulty or complexity when there is no attempt to dampen these effects by decreasing entropy, increasing organization, or dissociating attributes.
In sum, the research on variety of choice indicates that structural information can be used to increase perceptible variety and moderate choice overload effects. These findings are consistent with a range of cognitive science research, which suggests that structural, functional, and conceptual dissimilarities between options (not just their similarities) provide useful opportunities for sensemaking, comparison, and evaluation, and can encourage decision makers to restructure their own thinking or change their perspective, which can lead to a better solution [90], [91], [92], [93], [94]. Without structure, increasing the amount of choice or information as a means to add variety will likely have unintended and undesirable effects (e.g., [28]).

III. GUIDANCE FOR AI-ENABLED DECISION SUPPORT
What does this research mean for developers of decision support systems? While AI may be an integral component of an advanced information superiority strategy, intelligent technologies should also buffer humans from information or choice overload and adapt the amount of choice based on the contextual constraints [11]. Although some have argued that the need for information superiority may increase as the complexity of the environment and scope of the mission challenge increases [95], [96], it is precisely these conditions where less is more. A design principle based on "the more information the better" ( [11], p. 44)-which is often reflected in a decision maker's preference for receiving, and an engineer's preference for providing more choice-is likely to negate the potentially beneficial effects of AI-enabled decision support. Counterintuitively, the empirical evidence reviewed in this article indicates that developers should heed Simon's [11] warning-that human "attention is scarce and must be preserved" (p. 44)-by erring toward providing less rather than more choice to human decision makers, except when the context suggests otherwise. Table I summarizes advice, derived from the current review of the state-of-the-science, about when and how much choice should be provided to decision makers during decision support under a range of constraints.
To instantiate this advice, at least one question remains: How much, exactly, is less, more, or the right amount of choice? Anecdotally, some designers have used Miller's [97] limits of short-term memory (i.e., 7+/−2) as a guide to the amount of choice to provide [13]. However, this estimate is based on the cognitive limits associated with passive storage in human memory rather than active processing of that information. Others have proposed more conservative estimates of human working memory limits that take both information storage and processing into account (e.g., 4+/−1; [98]). Whether intentional or otherwise, Chernev et al.'s [55] meta-analysis indicates that researchers who investigated the effect of providing less versus more choice on decision making used similar estimates in the "less choice" conditions (∼6+/2 options), especially in those studies that explicitly examined difficult or complex decisions (∼4+/−2) or examined the effect of choice on decision outcomes (∼3+/−1).
Although these numbers are consistent with estimates of working memory limits, Miller and Cowan's estimates are only directly applicable to certain types of situations: tasks that are entirely mental (e.g., where there is no opportunity to use external memory aids to offload cognitive work; cf., [99]) or tasks that are novel or unfamiliar (cf., [100]). Effective decision support systems, by definition, provide external support and so, in most situations, decision makers can circumvent these cognitive limits through intuitive or ecological design [1], [101], [102].
Further, in situations that are not novel or unfamiliar (e.g., where the information or choices are meaningful), the size of a unit of information is largely determined by one's level of proficiency and decision-related knowledge. Hence, decision makers routinely circumvent the working memory limits via alternative cognitive mechanisms developed with expertise (e.g., long-term working memory, retrieval structures) [78], [103], [104]. In sum, these limits provide a quantitative guide but are highly context-dependent and are relevant to only a subset of situations (e.g., low familiarity or proficiency, high effort) [54], [55].
The fact that these limits can be circumvented with effective mechanisms should not be considered a license to provide more skilled decision makers with ever increasing amounts of choice. Despite the general advice provided in Table I that more choice may be beneficial when familiarity is high but uncertainty, complexity, and difficulty are low, generating more options has been shown to negatively affect expert performance in complex and dynamic tasks, and when skilled decisions are based on intuition (e.g., perceptually automated motor actions or recognition-primed decisions) [65], [67], [69], [105]. The decision to exceed these limits should be context dependent and deliberate, such as to make situational complexity visible, or to encourage exploration, divergent thinking, or a shift in perspective [106]. For instance, more exploration may provide a better picture of the risk distribution across the option space or help decision makers anticipate future surprises or flexibly shift between options as conditions change [19], [107]. However, research suggests that a better mechanism for increasing complexity visibility or promoting exploration and comparison between options is to increase variety (and reduce entropy) by manipulating the informational structure of choices rather than to provide more choice or information (see Table I) [11], [21], [80], [106], [107].
Using parsimony as a guide (i.e., as simple as possible to explain the phenomena in question rather than the simplest possible explanation), one way to address the "how much?" question is to state the answer in general, albeit context-dependent terms: just enough choice to allow the current goal to be achieved under the current constraints without oversimplification. A growing body of research on ecological rationality suggests that under uncertainty where anomaly, surprise, and complexity are the norm, simpler decision strategies that rely on relatively few information cues and minimal choice may be as effective as more complex strategies that rely on large amounts of data. For instance, in an often cited example, DeMiguel et al. [108] demonstrated that, in the uncertain world of financial investing, 14 different complex optimal asset allocation models (e.g., sample-based mean-variance, minimum-variance, and Bayesian models) performed no better than a statistical model based on a simple investing heuristic that needs no data (i.e., 1/N: divide assets equally across investments) (for other examples, see [109]). In contrast, in well-defined, stable and more certain environments where the future rarely deviates from the past, more historical information is likely to be of much greater utility in predicting future behavior (also see [2], [25]).
The advice presented in Table I is based on the current state of the science summarized in this review of empirical and meta-analytic research. As a result, advice is absent on how the relationship between the number of options and decision outcomes is moderated by other equally important factors. For instance, research has not explicitly or directly examined the moderating influence of the level of risk posed, the criticality of the impending decision, the need to be agile, innovative, and/or resilient, or the trust one may have in the decision support system. Future research should prioritize examination of these and related effects. For instance, although trust may increase with greater freedom of choice and increased confidence gained from more choice or information, especially when it is more relevant or reliable, the concomitant degradation in decision making from overwhelming amounts of information may erode that trust (e.g., [110], [111]).

IV. CONCLUSION
In summary, research suggests that, on average, decision makers find more choice (and information) more attractive, are often more confident in their decision making, and more certain in the accuracy of their decisions compared to when receiving less choice. Given human preferences, such as the desire for more information or choice, or the subjective decisions made throughout the model development process, it is easy to see why some decision support systems engineers may err toward providing more choice. However, research also indicates that, on average, more choice results in more difficult decisions, less choice satisfaction in hindsight, and poorer decision outcomes. These effects are mediated by the level of entropy contained within a choice and can be mitigated by the amount of perceptible variety provided.
Previous meta-analytic research questioned the reliability of choice overload effects [54]. However, subsequent meta-analytic research demonstrated that this observed heterogeneity can be explained by moderators of these effects [55]. This research showed that the negative effects of more choice can be reduced when participants are in a position to make a meaningful decision. In contrast, negative effects are typically compounded when decisions are more effortful, and when both decision complexity and difficulty are increased. These findings are consistent with the observation that "less is more" when uncertainty is high and suggests that more choice should be limited to conditions where complexity, difficulty, entropy, and/or cognitive burden are low, or when familiarity is high. Always favoring less choice or information irrespective of the circumstances may bias decision makers toward a narrow, oversimplified, and potentially inaccurate view of the decision space. Likewise, always favoring more choice or information irrespective of the circumstances may result in more variable, inferior, or more effortful decision making, especially in information-rich, uncertain environments (e.g., [109], [112]). Hence, to support the goal of decision superiority, developers of decision support systems should consider the impacts of more or less choice on decision effectiveness and the contextual factors that influence that relationship.
More research is needed on the moderating and mediating effects of other critical features of adaptive decision making, such as innovation, resilience, perceived risk, and trust. Likewise, more research is needed on the effect of more or less choice on the kinds of decisions typical of decision making in very complex and uncertain environments (cf., compound, concurrent, and interactive option generation versus discrete option selection/forced choice). Such research would provide valuable insight into the question of whether more or less choice during AI-enabled and other technologically aided decision support is better.