Experimental designs for testing metal detectors at a large sports stadium

When utilizing metal detectors at a large venue such as a sports stadium, there are the competing objectives of accuracy of the patron screening and the speed of throughput. This research, carried out in collaboration with the security staff at MetLife Stadium in New Jersey as well as other stadiums, analyzed two patron screening methods: handheld metal detectors (“wands”) and walk-through metal detectors (“walk-throughs”). An initial experimental design was created to understand the effectiveness of wanding. This design was used with MetLife Stadium security during three training sessions. The data collected was used to understand (a) if the prohibited item was found and (b) in how many attempts. Various prohibited as well as allowable metal items were hidden at random in various locations on the body of individuals, who were then scored based on the importance weight of the item (guns were given more weight than keys for example). Trainees were then assigned a performance score based on speed and accuracy and were tested until they reached a minimum required score. Building on this initial experiment, a second more formal experiment was created to help MetLife Stadium staff understand how walk-throughs would perform outdoors at different security settings. This experiment focused less on training security staff and more on understanding the performance of walk-throughs in real situations (as opposed to idealized lab situations). This experiment was created to understand the walk-through performance at each setting in the outdoor environment; e.g., does a walk-through catch each of the pre-specified prohibited items, and is this consistent across machines on the same setting? Because of the number of factors to be considered (type of item, location, orientation, walk-through setting, etc.), designing the experiment required a sophisticated approach called Combinatorial Experimental Design. The experiment was part of two DHS-supported projects on best practices for stadium security.