A Survey of the State of the Art in Sensor-Based Sorting Technology and Research

Sensor-based sorting describes a family of systems that enable the removal of individual objects from a material stream. The technology is widely used in various industries such as agriculture, food, mining, and recycling. Examples of sorting tasks include the removal of fungus-infested grains, the enrichment of copper content in copper mining or the sorting of plastic waste according to the type of plastic. Sorting decisions are made based on information acquired by one or more sensors. A particular strength of the technology is the flexibility in sorting decisions, which is achieved by using various sensors and programming the data analysis. However, a comprehensive understanding of the process is necessary for the development of new sorting systems that can address previously unresolved tasks. This survey is aimed at innovative researchers and practitioners who are unfamiliar with sensor-based sorting or have only encountered certain aspects of the overall process. The references provided serve as starting points for further exploration of specific topics.


I. INTRODUCTION
Sensor-based sorting is an increasingly important technology in various industries where material streams must be processed with high throughput.The umbrella term refers to a family of systems that enable the separation of individual particles from a bulk material stream, where the sorting decision is based on sensor data.The term bulk material describes a powdery, granular, or lumpy mixture in pourable form, i.e., unpacked.They are used, for example, to remove fungus-infested seeds before sowing and foreign or inferior substances from food, to enrich ores in mining, e.g. for copper or gold extraction, or to sort waste, e.g.lightweight packaging by material type or waste glass cullet by color.Since sorting decisions are typically calculated based on information from one or more imaging sensors, it falls within the scope of machine vision.The manual counterpart to sensor-based sorting is hand picking, in which impurities are removed by hand from a product to be recovered.The development of The associate editor coordinating the review of this manuscript and approving it for publication was Mauro Fadda .
sensor-based sorting systems in the broadest sense has been around for almost 100 years.For example, there exists a relevant patent from 1926 [1].It has found wide application, mainly in the fields of mining, recycling, and the processing of agricultural products and foodstuffs.
A schematic representation of the processes is shown in Fig. 1.The material is fed into the system by means of a conveyor mechanism, for instance, a vibrating feeder.The material is then transported further, for example, by a conveyor belt.Sensor-based data acquisition also takes place during this transport phase.The sensor data are processed with the goal of detecting and classifying individual particles in the material stream.The classification result serves as the basis for the sorting decision, which is executed by means of actuators.Since classification and separation are performed in separate steps, sensor-based sorting is sometimes also referred to as indirect sorting in distinction from mechanical sorting processes such as screening, wind sifting, or float/sink processes [2].In theory, an arbitrary number of classes can be distinguished, and separation into multiple fractions is also possible in principle.In industrial applications, however, the task is usually implemented as a binary sorting task, that is, sorting into "product" and "residue", since this is both simpler and more efficient to implement than multi-way sorting.
A particular strength of the sorting technology lies in the flexibility with regard to the detectable material properties, and thus the sorting criteria to be applied.This is mainly due to the variety of sensors that are suitable for use in sensor-based sorting systems.Due to their suitability for systems with high material throughput, imaging sensors dominate at this point.An example for a sensor-based sorting system is shown in Fig. 2.
From this introductory overview, it becomes obvious that research in sensor-based sorting is a highly interdisciplinary field, including aspects from the fields of process engineering, mechatronics, sensor and actuator technology, image processing, expert knowledge about the product to be sorted, and probably more.In addition, only economically attractive systems find their way into industrial applications, which means that economic considerations are also of great importance.Related work therefore typically sets a corresponding focus, depending on the authors' main discipline.However, a multidisciplinary approach is always required to implement high-performance sorting systems or to solve new sorting problems.
At the time of writing, 131 articles and conference papers are indexed in the Scopus database for the search query "sensor-based sorting" for the period between 2006 and 2023.For the preparation of this paper, those were clustered according to the main application field addressed and the sensor technology focused on, where applicable.From Fig. 3 it can be seen that, according to the indexed work, there exist three main applications, namely the processing of agricultural products and foodstuffs, mining, and recycling, the latter two representing the clear majority of contributions.Although most publications in the past have dealt with the mining sector, it can be noted that, especially in recent years, this FIGURE 2. Example of a sensor-based sorting system with its components numbered as chronologically passed by the material stream: (1) feeding, here: vibrating conveyor (2) transport, here: conveyor belt (3) sensor box (4) schematic illustration of scan-line (5) illumination, here: halogen floodlight (6) separation, here: pneumatic valves.
trend has shifted towards the recycling sector.Fig. 4 further provides a qualitative overview of the relationship between the sensor technologies considered most frequently and the corresponding field of application according to the indexed publications.As can be seen, there appears to be a dominant application of sensors in the infrared (IR) spectrum for recycling applications and of X-ray-based systems in the mining field.Sensors operating in the visible (VIS) spectrum are represented in all fields of application with a focus on the processing of agricultural products and foodstuffs.Laserinduced breakdown spectroscopy (LIBS) and laser-induced fluorescence spectroscopy (LIFS) appear to be of current interest both in mining and recycling, but are not considered to the same extent as the aforementioned technologies.
This survey aims to cover a particularly broad spectrum of the state of the art and research in sensor-based sorting.For this purpose, we selected references from the year 2000 onward, focusing on recent publications.The references were further selected based on their relevance in order to cover the broad spectrum of research on sensor-based sorting as well as their credibility, allowing a comprehensive view of the topic.Existing surveys on the technology are typically  limited to a certain field of application, such as mining [3], [4] or the processing of municipal solid waste [2], or certain sensors, such as infrared and laser-induced breakdown spectroscopy [5].The survey therefore aims at researchers as well as practitioners who are either new to the application of sensor-based sorting or have restricted themselves to certain involved aspects.
This survey is organized as follows: Following this introduction, we provide a detailed overview of the application fields and exemplary sorting tasks that have been addressed so far in Sec.II.These are clustered according to the main fields of application in sensor-based sorting, i.e., processing of agricultural products and foodstuffs, mining, and recycling.In Sec.III, we take a closer look at the different system components involved, i.e., those illustrated in Fig. 1, the common instantiations and their parameters.The main process parameters that define the sorting task are discussed in Sec.IV.Furthermore, an overview of common means to quantitatively describe the efficiency of sensor-based sorting systems is provided in Sec.V. Finally, Sec.VI concludes the information provided and discusses the potential for future research directions.

II. FIELDS OF APPLICATION OF SENSOR-BASED SORTING
Sensor-based sorting is used mainly in the fields of agricultural products and foodstuffs processing, mining, and recycling.All fields of application have in common that large material streams need to be processed at high throughput.Exemplary materials to be sorted from these different areas can be seen in Fig. 5.In the following, exemplary tasks in the respective areas are presented.

A. AGRICULTURAL AND FOOD INDUSTRY
In order to meet the great challenge of supplying food to a constantly growing population in the face of declining arable land, the agricultural industry is under constant pressure to increase efficiency.There exist numerous examples of the use of machine vision systems to support solving this challenge [6].Sensor-based sorting also plays an important role in meeting this challenge.
Numerous articles deal with the grading of seeds for crop production, that is, the detection of defects, damaged, shrunken, and broken kernels, as well as foreign materials.
Corn with approximately 5% and rice and wheat with approximately 19% each are among the most substantial sources of calories in the human diet [7].Consequently, sensor-based sorting is widely used in these applications.For example, it is a suitable sorting technology for the processing of wheat grains [8].High material throughput and low system costs represent an important factor for the profitability of the systems [9].A prominent application example in this context is the detection and removal of fungus-infected wheat grains before sowing [10], [11].Long-term conditioning of wheat seeds is also expected to increase resistance to disease over several generations [12].Other research is even concerned with estimating seed vigor during grading [13].Comparable studies have also been published in the context of corn seed [14], for rice grading [15], [16], [17], and sunflower seed [18].
In addition to tasks related to seeds and hence mainly agricultural products, sensor-based sorting is also used in the industrial processing of a wide variety of foods, for example bulgur [19].Safety-relevant application examples can be found in the reduction of aflatoxin contamination in apricot kernels [20] or the removal of coherbs containing liver-toxic pyrrolizidine alkaloids from the spice and medicinal plant crop [21].Numerous applications also deal with the quality enhancement of nuts, for example pistachios [22] or almonds [23].Sensor-based sorting was even successfully used to sort grapes according to their sweetness [24].Other applications include beans, corn for consumption, and small fruits such as tomatoes and cherries [25].

B. MINING
In the field of mining, sorting tasks mainly consist of separating ores to be extracted from non-metallic accompanying minerals, called gangue.A particular challenge in this area of application is that the materials to be detected are always present as mixtures and virtually never in pure form.In addition, very heterogeneous and complex geometries have to be handled.A comprehensive overview of the application field is presented in [4].
The processing of mineral resources typically involves energetically very intensive and therefore expensive process steps.This applies in particular to crushing and wet processes, such as float/sink processes.In addition to energy costs, the need for electricity and water, which can be contaminated during the process, plays an important role in the operating permit of a mine [26].Due to increasing demand for certain mineral resources and advanced mining, the industry also faces decreasing ore grades and more complex ore types [3], [27], [28].This is also why the use of sensor-based sorting is becoming increasingly economically attractive for this area of application.By implementing pre-concentration before the aforementioned process steps, unwanted by-products can be removed from the material flow, thus reducing energy requirements.Additionally, the freed-up capacity of the process steps can increase the throughput of the target product.In some cases, it is also possible to extract other valuable materials from the stock stream under the by-product through sorting [29], e.g., quartz [30].Furthermore, economic analyses on the use of sensor-based sorting in this context have been published [31], [32].
Many published works deal with the question at which point in the preparation process, which is usually visualized in the form of a flow chart, sensor-based sorting can be used most efficiently.In general, it is agreed that pre-concentration has the greatest effect if it occurs at the earliest possible point in the processing chain, e.g., directly on the run-ofmine material [4], [33].As part of pre-conditioning, the rock is often screened to a defined particle size distribution [34].

C. WASTE PROCESSING AND RECYCLING
In waste processing, manual sorting by humans is still widespread today.However, the purity and thus the value of recovered material can be significantly increased by using sensor-based sorting systems [52].This is urgently needed because, for example, a directive of the European Union mandates an increase in municipal waste recycling rates of one percentage point per year from 55 to 65% by weight for the years 2025 to 2035 [53].
Similarly to mining applications, sensor-based sorting in the context of recycling is typically only a small component of a complex processing chain, which also includes many mechanical sorting processes [54].Typical processing steps include screening, magnetic separation, air classification, ballistic separation, sensor-based sorting, and manual sorting [55].It typically serves as a pre-treatment step with the goal of generating fractions as suitable as possible for a recycling process.Two general tasks can be distinguished: the sorting of highly heterogeneous mixed waste, such as unseparated municipal waste, and the sorting of already separately collected waste fractions.
A comprehensive overview of the use of both mechanical and sensor-based sorting methods for highly heterogeneous municipal waste is presented in [2].Sensor-based sorting is typically used for such heterogeneous material streams only late in the treatment chain, i.e. subsequently to various other processes.One reason for this is that sorting into a large number of fractions is technically very complex.Frequently, however, pre-sorting by mechanical processes is simply more practicable.However, sensor-based process monitoring can also provide valuable insights into material flows early in the preparation [56], [57].Occasionally, sensor-based sorting is also applied early, i.e., on heterogeneous stock streams, for example, to selectively extract glass [58] or to remove impurities for further processing as refuse-derived fuel, e.g., polyvinyl chloride (PVC) [59].
In the work considered below, an already pre-sorted material stream, for instance due to the way of collection, is assumed.Glass, metal, and plastic recycling are presented as the most prominent representatives.The processing of construction and demolition waste is also considered.

1) GLASS
In glass recycling, sensor-based sorting has been used since the 1990s.For reuse, the color purity of the waste glass cullet is of great importance.For example, for the production of white container glass with an addition of 50% waste glass cullet, there must be a purity of color, that is, the addition of actual white glass, of 99.7% [60].Green glass behaves most tolerantly.Here, an off-color content of up to 15% is tolerable.Furthermore, foreign matter, especially ceramics, stones and porcelain [58], as well as heat resistant [61] and lead-containing glass [2] must be removed from the material stream by sorting.All these tasks are realized using sensorbased sorting.

2) METALS
Metal recycling plays an increasingly important role both in the recycling of shredded components, such as those from buildings or vehicles, and in the recycling of waste electrical and electronic equipment (WEEE).For example, steel demand and production have doubled in the last 30 years [62].The use of recycling processes results in tremendous energy and CO 2 -emission savings [63].An overview of existing sorting and sensing technologies and requirements for WEEE recycling can be found in [64].
For metal recycling, magnetic separators are used first in the treatment pipeline [62].This results in an already iron-free fraction, which can be further processed by sensorbased sorting.Thereby, for example, separation of aluminum, magnesium, copper, and brass can be achieved [65], [66].A distinction between iron, chromium, and nickel is also possible [62].In addition, the differentiation of various aluminum alloys, especially cast and wrought alloys, represents a special task [67].Further investigations are concerned with the preparation of materials of specific end products, for example, screens [68].

3) PLASTICS
Sensor-based sorting has been established as a pre-treatment step in plastic waste processing for several years.To prepare the material for subsequent recycling processes, it is typically necessary to create fractions as pure as possible with regard to the polymer type.Obviously, those can occur in arbitrary colors, hence the sorting needs to be material selective.However, especially with the so-called waste electrical and electronic equipment plastics (WEEP), complicated mixtures or compounds of polymers that make the task even more difficult occur quite often.For such, even float/sink methods, which are often used for the separation of different polymers [69], reach their limits in this task, since the densities in mixtures of different polymers often overlap and no longer allow differentiation [70].
A general task for sensor-based sorting in the pre-treatment of plastic waste is the general differentiation of various polymers such as polyethylene terephthalate (PET), low and high density polyethylene (LDPE, HDPE), PVC, polypropylene (PP), polystyrene (PS) and acrylonitrile-butadiene-styrene (ABS) [71], [72].In many cases, sorting of polymer groups, such as polyolefins, including PE and PP, is of particular interest [73].The task typically depends on the characteristics of the waste stream.For example, distinguishing PET and polylactides (PLA) is particularly important in the sorting of food packaging, as these polymers are mainly used in these products [74].With the development of compostable and biodegradable plastics, sorting of these has also drawn scientific attention [75].
Furthermore, during their lifetime, plastics undergo aging and possibly contamination, leading to changes in material properties that have an impact on recycling [70].The probably most extreme case of this effect is in the field of landfill mining [76], where raw materials must be recovered from already landfilled waste [77].Plastics that experienced degradation may differ in the sensor signal from virgin ones [73], which leads to the need for adapting data processing [78].The effects may also depend on the type of degradation, for example, due to multiple re-extrusion processes in the case that the material was already recycled, or thermal aging [79], [80].Beyond that, it is even desirable to consider the state of aging as a potential sorting criterion by predicting the state of aging based on sensor data [81].
Further studies focus on the detection of additives and polymer mixtures [82].Since additives and fillers also play a crucial role in plastic recycling, special attention is paid to the detection of heavy metals in polymers [83], [84], brominated plastics [82], [83], other flame retardants [85], and chlorinecontaining plastics [86].
An alternative approach to reduce the complexity of the methods presented and to enable high-purity sorting is the use of markers [87].The idea is to provide polymers with fluorescent markers during production that can then be detected with the aid of sensors during pre-treatment.This is also called tracer-based sorting [88].
For technical reasons, which are explained in Sec.III-B, and because they occur predominantly in WEEP, the sorting of black plastics is currently receiving high attention [89].Several studies focus on general polymer discrimination [82], [90], [91] or on specific sub-tasks, such as styrenic plastics and polyolefins [70], [92], particularly for black plastics.
Various research priorities and developments are possible for the further spread of sensor-based plastic sorting [5].One possible improvement lies in focusing more on the heterogeneity of plastic waste, which is often not adequately addressed in studies.Also, the potential of multi-sensor solutions, which yield promising performance, has not yet been fully exploited.Last but not least, there is a great need for extensive datasets in the application domain to fully exploit modern machine learning techniques.

4) CONSTRUCTION AND DEMOLITION WASTE
In the wake of, among other things, the request of the European Union towards its member states to significantly increase the use of recyclate in the building materials sector [93], the use of sensor-based sorting for construction and demolition waste (C&DW) has been intensively investigated in recent years.Among others, C&DW may contain concrete, bricks, tiles and ceramic, asphalt, wood, and gypsum [94].Around 600 million tons of mineral raw materials are used annually in the construction sector in Germany alone [95].
Currently, primary raw materials are used predominantly in the production of new building materials, since only 81 million tons of C&DW are recycled for construction applications each year.Much of the demolition material is landfilled or used as fill material in road construction [96].
Sensor-based sorting can be essential to increase this rate and create high-quality building materials from recyclate [97].For reasons comparable to those of plastic recycling, sorting tasks can be classified as relatively difficult, partly because multiple sensors are supposedly required [98].By using appropriate fragmentation technologies, for example, electrodynamic fragmentation [99], composites can be separated along grain boundaries for subsequent sorting.Besides the characteristics of the material stream, e.g., depending on the source, the sorting of the coarse [96] and the fine fraction [97] is typically distinguished.In addition to the generation of pure fractions in terms of material, the removal of particularly critical materials, namely organic, gypsum, and aerated concrete, from the material stream for the recycling process plays an important role [100].Two special recycling lines, which are said to have a special potential for industrial establishment, are ''gray-to-gray'', referring to the purification of concrete and plaster, and ''redto-red'', referring to bricks and ceramics [98].In addition to the treatment of material originating from the demolition of buildings, another exemplary task is the sorting of road debris, for example, into the categories tar, bitumen, and minerals [101].

III. SYSTEM COMPONENTS AND PARAMETERS
The overall sorting process performed in sensor-based sorting systems is typically divided into 4 phases [4], [43], [102], [103]: Presentation refers to feeding and transport of the material within in the system.Examination refers to the observation of objects contained in the stream by one or more sensors.Data analysis refers to the processing of sensor data.Separation describes the physical separation of individual objects from the material stream.
In order to implement efficient sorting systems, the various components of the system must be optimally matched to each other.Several works further consider pre-conditioning of the material, e.g., in the form of crushing or screening, as a separate first phase [3], [26], [104].All process phases are based on various parameters and can also be divided into geometric and process parameters [34].They are discussed in more detail below.

A. FEEDING AND TRANSPORT
In addition to the obvious task of feeding material into the system, the technical realization of feeding can already have a significant impact on the quality of sorting.Objects should be fed to the transport phase as individually as possible, that is, with the greatest particle spacing possible, while providing Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
the desired mass throughput.Furthermore, it is desirable to ensure that the material is evenly distributed throughout the entire width of the sorting, since this allows the utilization of the best possible capacity and generally has a positive impact on the quality of the sorting.A common technical realization is the use of vibratory conveyors.Optionally, those can be equipped with an additional feed hopper [105].However, the feeding process typically depends on the overall process in which the sensor-based sorting system is embedded, for instance, prior pre-treatment steps.
Similarly, the transport phase also has additional objectives in addition to the simple transportation task.First, it needs to be ensured that no objects lie on top of each other, since this would lead to occlusions.Second, the formation of clusters of objects should be avoided.This goes hand in hand with the above-mentioned requirement for the feeding mechanism to feed objects as individually as possible.On the one hand, it is important to ensure enough spacing between objects such that single object separation is still possible.Whether this is the case also depends highly on the temporal and spatial resolution of the separation, as will be discussed in Sec.III-D.Whenever objects are too close to each other, a single activation of an actuator can erroneously separate several objects from the material stream.Also, it must be possible for data analysis to recognize single particles as such.Third, the transport phase must ensure optimal flow control.This means that all objects are transported at uniform velocity and no relative motion of individual objects exists.In case of non-uniform velocity, objects may present a trajectory not predicted by the system, which may cause separation failure [58], [106].The fulfillment of all three mentioned tasks has a strong impact on the quality of the sorting [105].
There are three transport mechanisms predominantly used in sensor-based sorting: conveyor belts, chutes, and freefall transport.A schematic representation of these variants is provided in Fig. 6.In the following, these three variants are discussed with respect to their advantages, disadvantages, and parameters.Also, there exists related work on methodology to evaluate the transport behavior of individual objects, for example, such as those presented in [107].
Flat conveyor belts, as depicted in Figures 2 and 6a, supposedly meet the requirements mentioned above on the transport phase best.They support the elimination of relative motion of the particles by accelerating or decelerating them to the set belt speed, hence ensuring a uniform transport velocity.This particularly holds true for rough belts.However, in the application area of agricultural products and food industry, this may not be an option due to hygiene and cleanliness.In this case, the belt should be as smooth as possible.A common setup to increase throughput while ensuring a high separation quality is to transport the material sequentially over several conveyor belts with increasing belt speed.In this way, the material is pulled further apart throughout the transportation process [105].However, the mentioned advantages are mirrored in the costs.This not only concerns comparatively high investment costs, but also an increase in the space requirement to set up the belt, as well as operating costs, e.g., electricity and wear and tear on the belt.As conveyor belts include many moving parts, they can also be considered maintenance-intensive.As a consequence, condition monitoring systems and the like are sometimes employed [108].The parameters of this type of transport are the length of the belt, the speed of the belt, and the material of the belt.In industrial settings, belt speeds typically range from 1 m s −1 to 4 m s −1 [105].
The realization of material transport through chutes, as shown in Fig. 6b, is a cost-effective alternative compared to conveyor belts.Acceleration of the material occurs via gravity.Cost advantages exist not only due to lower investment costs, but also due to lower maintenance requirements.Only the wear of the chute surface may need to be accounted for [16].However, these cost advantages are countered by the fact that chutes typically achieve poorer flow control.This is partly due to friction of the surface with the accelerating objects.One measure of compensation, which is used especially in the food industry, is the use of rills on the chute by means of which the material is guided.This, in turn, has a negative effect on the scalability of the system in terms of possible mass throughput, as additional rills have to be provided.For chutes, possible design parameters include the length of the chute, the angle of inclination, and the surface material [109].
The most cost-effective solution is the transport of the material in free fall, see Fig. 6c.As is the case with chutes, the material is accelerated by means of gravity.However, whenever the air resistance of the particles in the material stream varies too much, there will also be variations in the transport speed.This, in turn, may lead to issues with material separation and data acquisition, such as color fringes when using color line-scan cameras.For foreign, previously unknown objects, which are typically to be removed in the course of the sorting process, no information regarding air resistance can be provided.Therefore, the field of applications for such sorters is severely constrained.

B. SENSOR TECHNOLOGY
The wide range of applicable, commercially available sensors leads to flexible application areas of sensor-based sorting.Imaging, line-scanning sensors are primarily used to enable high material throughput.From X-rays to the terahertz range, a large part of the electromagnetic spectrum is covered here.The properties of the material to be "made visible" determine the best sensor type.Multi-sensor systems are used when a single sensor cannot perform the task [110].Information from the various sources must then be processed accordingly during data analysis, and information fusion approaches have to be applied.
The most widespread sensors for sensor-based sorting are described in detail below, along with how they work.Examples of applications are also mentioned.Examples of non-imaging systems are included in addition to the prevalent The green spheres represent objects to be accepted ("product") and the red squares objects to be rejected ("residues").The yellow beam indicates the scan-line of the sensor, and the blue beam indicates the actuator's action zone.In the schematics of (a) and (b), sensory examinations can also be performed on the transport medium (i.e., on a conveyor belt or chute) instead of after discharge.imaging sensors.Table 1 provides an overview of the sensors discussed.

1) X-RAY SENSORS: XRF AND XRT
X-rays are in the wavelength range below 10 nm.The two mainly used measuring principles in sensor-based sorting are dual-energy X-ray transmission (DE-XRT) and X-ray fluorescence (XRF).
For XRT measurement, which is well-known from airport security checks, a source is placed on one side of the transport medium, e.g., above the conveyor belt, and a sensor on the opposite side, e.g., under the conveyor belt [51].The source emits x-rays of a certain energy that pass through the material to be inspected.In this process, the beams are absorbed proportionally.The absorption is material-specific, with elements with a low atomic number having lower absorption than elements with a high atomic number.The sensor therefore measures the intensity of the attenuated X-rays that have penetrated the material and have not been fully absorbed.Combining these intensities pixel by pixel, an x-ray image is obtained.A major advantage is that the penetrating measurement reveals information about the internal structure of particles [111].In multi-energy X-ray transmission (ME-XRT) imaging, several of such images at different energy levels are acquired using this principle.DE-XRT results in a high-and low-energy level image.An advantage of this measurement principle is that the thickness of the scanned particles, which, in principle, has a direct influence on the absorption properties, does not affect the measurement [51].
In contrast, XRF is used to measure the fluorescence caused by X-rays.Source and sensor are mounted on the same side of the material, that is, above a conveyor belt [2].
X-ray sorters are mainly found in the application fields of mining and recycling and are only occasionally used for sorting agricultural products and food.An example of the usage of XRF is the determination of the copper content in ores [33], [40].Other examples using XRF include the determination of heavy metal concentration in wood [112], the discrimination of PVC and PET [2] or the sorting of non-ferrous metals in waste deposits [113].With respect to (DE-)XRT, examples of application include the detection of rare earths in ores [26], [51], coal preparation [48], sorting of non-ferrous metals [65] and the detection of enclosed diamonds [114].

2) COLOR LINE-SCAN CAMERAS
Color line-scan cameras are sensitive to light, that is, the range of the electromagnetic spectrum visible to humans, which ranges from 380 nm to 780 nm.With reference to "visible", such cameras are also called VIS cameras.They record three color channels within the aforementioned wavelength range, which correspond to the colors blue, green, and red.
There are different sensor patterns for capturing the three wavelength ranges.In the tri-linear layout, there are three physically separated sensor lines, which are provided with corresponding optical filters.The sensor lines are read out with a certain time offset, which must be configured orthogonally to the sensor rows according to the transport speed of the scene, so that the same scene is recorded in all colors.Deviation from this configuration can lead to unwanted color artifacts since the registration of the color channels is no longer performed appropriately.The effect can be prevented by using a prism in which the incident light is distributed among the three separate line sensors, which are then read out simultaneously.In bi-linear sensor patterns, two of the three color channels are combined in one sensor line by alternately making one pixel sensitive to the corresponding wavelength.In this way, the problem of registration is alleviated, but not circumvented.In the monoline pattern, only one sensor line is used, in which macropixels consisting of three adjacent pixels, each sensitive to one color range, are located.Thus, all colors are captured at the same time without the costly use of a prism, but with minimal local offset.Color linescan cameras are used for both reflectance and transmittance measurements.The latter are used, for instance, in color glass sorting.
Color line-scan cameras are used in all fields of application of sensor-based sorting.Tasks in the agricultural products and food industries include the sorting of various wheat grains [8], [9], [12], cleaning bulgur [19], rice sorting [17], and the sorting of sunflower seeds [18].One of the most prominent examples of application in the recycling field is the sorting of colored glass cullet [58].The use of ultraviolet (UV) illumination further creates fluorescence effects that can be observed in the visible spectrum and which allow lead glass to be detected and sorted out [2].Applications in the mining sector include the sorting of already processed minerals [115] and the processing of lignite [115].Color line-scan cameras are also frequently used in multi-sensor systems, especially in conjunction with infrared cameras [73], [98] or alternatively in the form of four-channel VISNIR cameras.If the information to be extracted cannot be taken from visible light, VIS cameras are nevertheless frequently used as complementary sensor technology for the precise localization of individual particles due to the high spatial resolution available.

3) COLOR AREA-SCAN CAMERAS
Other than the sensor patterns discussed in the context of line-scanning VIS cameras, Bayer sensors are usually used to obtain color information for area-scan cameras [116].Here, the sensor is covered by a color filter array, whereby half of the sensor is provided with green filters and the rest equally with red and blue filters.This creates a checkerboardlike pattern.The raw camera image thus initially contains information for only one color per pixel.The missing color values for all pixels are calculated by interpolation during the so-called demosaicing process.
In contrast to commonly used line-scanning sensors, the use of area-scan color cameras for sensor-based sorting has been proposed [117].A supposed advantage in using area-scan cameras is the ability to observe the movement of individual particles.This is achieved by using sufficiently high frame rates, which in turn enable multiple observations of the particles.The authors propose a multiobject tracking algorithm, in which the parameterization of a motion model can be updated incrementally with each observation and eventually used to predict a particle's future trajectory [118].This in turn can be used for the calculation of the actuator control signals for separation, see Sec.III-C.The potential of this approach has been demonstrated both by numerical simulation [119] and experimentally on a lab-scale sorting system [120].The presented results suggest that the approach is particularly advantageous for sorting scenarios in which non-uniform transport velocities (see Sec. III-A) exist.Furthermore, it enables material characterization based on motion-related features [121], [122].However, although the use of color area-scan cameras has been addressed in various scientific publications, no industrial application is known.

4) INFRARED AND HYPERSPECTRAL CAMERAS
Hyperspectral cameras (or Hyperspectral Imaging, HSI) combine optical spectroscopy with spatially resolved imaging, bringing together the advantages of both methods.Depending on the sensor technology and the imaging spectrometer used, hyperspectral cameras capture specific regions of the electromagnetic spectrum.Technically, these are divided into the ultraviolet range (UV, 220 to 380 nm), visual to near-infrared range (VISNIR, 400 to 1000 nm), shortwave infrared or extended shortwave infrared range (SWIR, 1000 to 1700 nm and ESWIR, 1000 to 2500nm ) and mid-infrared rang (MIR, 3 to 5 µm).
A hyperspectral image provides a large number of spectral channels of closely neighboring wavelengths (sometimes several hundred), so that each pixel is assigned a continuous spectrum in the corresponding spectral range.Examples of spectra as acquired per pixel with a SWIR-HSI camera are provided in Fig. 7.This allows for a spectroscopic analysis of objects or scenes, which can be used to determine chemical material properties.This is especially true in the SWIR region where molecules absorb light because of the vibrational motions of their bonds, especially CH, OH, NH, and SH, which are common to all organic molecules.Plastics, for example, exhibit characteristic absorption patterns in the SWIR region as a result of their carbon-based structure and, therefore, can be differentiated by polymer.SWIR-HSI cameras are predestined for material-specific optical sorting and have in particular become one of the most widely used sensor technologies in this field in recent years.Ever since this camera technology has also become economically Examples of spectra obtained using a SWIR-HSI camera for the two plastic types PE and PP.Here, the camera used captures 256 bands in the wavelength range between approx.1000 nm and 2500 nm.The occurrence of absorption bands is material-specific and can hence be used to distinguish materials and calculate sorting decisions.
attractive due to decreasing prices, HSI has found its way into all areas of application.
In the agricultural and food industries, SWIR sensors are used for rice sorting [15] and wheat grain evaluation [13].SWIR-HSI is also considered a standard method for the detection of fungal infection in cereals [123].In the mining application area, HSI is cited as one of two technologies that have contributed to the expansion of the application area in the last decade [4].SWIR-HSI is used to determine the copper content in ores [35], [36] or to perform a general mineral characterization [46], and to determine the silver, gold, carbon, and sulfur content [47].In the recycling sector, SWIR or SWIR-HSI cameras are mainly used for sorting C&DW [98], [100] and plastic sorting.With regard to the latter, the technology has the potential to replace manual sorting for lightweight packaging and is considered an integral part of recycling processes [105].A typical task in plastics sorting that can be solved using SWIR-HSI is the general discrimination of different polymers [72].However, many works focus on specific plastic types or groups, such as the differentiation of polyethylenes and polypropylenes [73], of PET and PLA [74] or the detection of PVC [59], a typical contaminant in plastic recycling.
One weakness of SWIR-HSI, especially noticeable when used in the recycling sector, is the limited detection of black materials, such as black plastics.Black colored plastics use mainly carbon in the production process.This causes very high radiation absorption in the SWIR range, so that only weak or non-usable signals can be detected.This limitation does not apply to the MIR range.Until recently, MIR-HSI cameras were considered too slow, too low in spectral resolution, and as having poor signal-to-noise ratios [90].However, recent technological developments have resulted in the successful use of appropriate sensors in sensor-based classification.The use of MIR-HSI is particularly promising for WEEP, since the corresponding plastics are often black [70], [92].It is capable of general discrimination of different polymers, including black particles [90], and therefore it is a promising technology for the sorting of packaging waste [124], for example.It also has the potential to detect different additives and mixtures [82].Commercially available MIR-HSI technology has also already been experimentally verified for application in sensor-based sorting [89].

5) TERAHERTZ SENSORS
Radiation in the wavelength range between 30 µm and 3 mm at a frequency between 0.1 THz and 1 THz is termed Terahertz radiation.At the time of writing, the first corresponding line-scanning sensors have been developed, enabling the technology for in-line use.However, it is still of scientific interest and no industrial application in the field of sensor-based sorting is known.
The potential of the technology for sensor-based sorting, however, has for instance been studied by means of a line-scanner in the low terahertz range between 84 GHz and 96 GHz [91].The ability to discriminate carbon-containing black plastics and polymers in general was confirmed.
A challenge when using THz-scanners in sensor-based sorting is that scanners in the millimeter range suffer from a limited resolution due to the comparatively long wavelength.A possible solution to overcome this is the use of multiaperture approaches.

6) LASER INDUCED BREAKDOWN SPECTROSCOPY
The elemental composition of a sample can be ascertained using the low-destructive technique known as laser-induced breakdown spectroscopy (LIBS), also known synonymously as laser-induced plasma spectroscopy (LIPS).In this measurement technique, a pulsed laser beam is focused on the sample.A small portion of the sample -on the order of a few micrograms -is vaporized by the brief, intense laser pulse, which also creates plasma by ejecting electrons from the atoms' outer shells.A spectrometer detects these characteristic emissions of the material a few nanoseconds later, allowing identification and quantification of the elements present in the sample [125].
The method has entered the industrial realm as a result of recent advances that enable its use in real-time settings, such as sensor-based sorting.One benefit of the method is that the samples -in this case, bulk material -need little to no pre-conditioning.A serious disadvantage of the method is its design as a point measurement system and the need for complex calibration routines.Due to the single line in which the material is fed, only low throughput is possible.
LIBS-based sorting systems are typically not found in the agricultural products and food industries but are used in mining and recycling applications.In addition to their dominant use for metal sorting [62], there exist studies on the differentiation of different thermoplastics [126] or polymers in general [71].There are no limitations in the characterization of black plastics.Further work evaluates the 6482 VOLUME 12, 2024 Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.

7) LASER INDUCED FLUORESCENCE SPECTROSCOPY
Laser-induced fluorescence spectroscopy (LIFS) is a non-destructive technique that has been increasingly used in the field of materials research and characterization in recent years.Just like LIBS, the sample is excited by a laser.UV lasers are used for this purpose in LIFS.The energy input briefly lifts atoms from a stable state to a higher, unstable level.When they return to the stable ground state, photons with a higher wavelength are then emitted.This leads to short-lived fluorescence in certain materials.This fluorescence is in a wavelength range which is slightly higher than that of the excitation, and is observed by hyperspectral sensing in the UV-VIS range.The intensity of the fluorescence is recorded over a time period, which also allows the decay behavior to be observed.The resulting spectral information over time then forms a kind of fingerprint that can be used to characterize materials.However, elemental analysis is not possible [127].
LIFS is used in mining applications because the described fluorescence can be excited in almost all minerals.For example, the evaluation of the time-resolved fluorescence allows the differentiation of different ores [128] and is applied in the enrichment of lignite [129].Likewise, LIFS is used in the recycling application area for plastic sorting [130].In particular, there is no limitation regarding carbon black plastics [127], [131].

8) NON-IMAGING SENSORS IN SENSOR-BASED SORTING
In addition to the dominant imaging sensors, non-imaging sensors are also used for sensor-based sorting in rare cases.These include acoustic sensors in particular.The material characterization is carried out by evaluating sounds after a collision of bulk material particles with a surface.This methodology is called impact acoustic (IA).A schematic representation of a possible realization of such systems is shown in Fig. 8.
A main application of impact acoustic is the sorting of nut-like products, e.g., chestnuts [132], hazelnuts [133] or almonds [134].Furthermore, the method is suitable for detecting damaged wheat grains [135].In this system, the grains are fed individually to the sensing system which consists of microphones.A laser barrier serves as a trigger to record collision sounds.Similar setups have been proposed for the sorting of plastic flakes [136].Here, laser triangulation is also used to determine the size of the flakes and is included in the classification based on the acoustic data.
One limitation of such sorting systems is the scaling with respect to the increase of the material throughput.The particles must always be fed to the detection in lines.A higher sorting width is accompanied by an increase in the number of lines and requires a separate sensor system for each line.

C. DATA ANALYSIS
As can be seen in Sec.III-B, imaging sensors dominate in sensor-based sorting.Data evaluation takes place accordingly in the form of image processing.The goal of image processing in sensor-based sorting is to compute control signals for the actuators to eject the particles to be removed.For this purpose, individual particles must be detected and classified.As data analysis platforms, Field Programmable Gate Arrays (FPGA) [9], PC systems [23], [26] or a combination of both [120] are primarily used.
If the sorting decision based on the classification result is that a particle should be ejected, the location of the detection is used to determine which actuators have to be activated at which time to remove the particle.In the case of binary sorting systems, the distinction of two classes, that is, "accept" and "reject" or "product" and "residue", respectively, would suffice.However, to obtain further information about the input stream, it may be of interest to distinguish between several classes.
Currently, conventional rule-based image processing pipelines are still dominantly used to solve this task.However, recent research focuses on utilizing data-driven approaches, that is, machine learning.Therefore, both approaches are considered in what follows.
As is typically the case in machine vision applications, image acquisition takes place under very defined conditions.This means that the illumination of the scene is a design parameter and is typically constant to optimize image quality.The background of the scene can also be freely selected during system design.A conventional processing chain then corresponds to the typical machine vision pipeline [137] and consists of the steps image pre-processing, information compression and extraction, decision and action.
Optical effects, such as distortion or inhomogeneities of the illumination, are corrected during image pre-processing, the first step of the processing chain.In the course of object localization, image segmentation takes place.Due to the constant image acquisition conditions mentioned above, this can often be done by comparatively simple methods, for example, by pixel-wise application of a threshold followed by connected component analysis [138].To detect particles completely, individual image lines of the line-scanning sensor must be combined over time, which in turn produces two-dimensional images of the material stream.For further information compression and extraction, features are calculated to describe the individual particles.Considering a color line-scan camera, such features can, for example, be based on color, texture, or geometry [115], [139], [140].The resulting feature vector is then the input for a classifier that has the task of assigning a material class to the detected particle.Several classification algorithms are being used, ranging from comparatively simple rulebased approaches, e.g., by defining thresholds [38], [50], to machine learning methods, e.g., Support Vector Machines (SVM) or Random Forests [13].If spectral data need to be classified, matching of reference spectra [77] or regression models to map, for example, a NIR [35] or X-ray [33] signal to material properties, is also used.
In addition to conventional image processing pipelines, recent studies investigate the use of modern machine learning approaches, especially from the field of Deep Learning [141], as a replacement for several or even all processing steps.In the most extreme case, a single model is trained end-toend to derive sorting decisions directly from sensor data [18].In addition, Deep Learning is being used to re-evaluate material characterization tasks that were previously considered intractable, such as the characterization of black plastic based on SWIR data [142] or the selection of wavelengths for plastics characterization in the IR spectrum [143].As discussed in Sec.III-B, only very low reflectance is obtained in this case.Using N-BEATS [144], a Deep Learning-based solution originally proposed for interpretable time series forecasting, promising detection rates have been achieved for this task.Similarly, various VGGNet structures were evaluated to discriminate and eventually sort typical components of construction waste [145] and autoencoder structures for the detection of minerals embedded in asphalt composites [101].
For particles to be ejected, a control signal for separation is calculated and transmitted in a final step.As discussed in Sec.III-D, all common separation mechanisms are designed as an array of individual actuators aligned perpendicular to the transport direction.Accordingly, both a spatial component, which determines which actuators in the array are actuated, and a temporal component, which determines in which time period the actuators are to be actuated, must be calculated.This is also called the deflection window.For the determination of the spatial component, the location of the particle is used as determined by the sensor data, and the sensor coordinates are converted into actuator coordinates.For the temporal component, a fixed temporal offset is added to the time of detection in the sensor data.This offset can be calculated based on the velocity of the particle and the spatial distance between the sensor and the separation.However, since particle-individual velocities are not known -except for the approach with an area-scan camera mentioned in Sec.III-B -an average particle velocity must be assumed, since velocity cannot be assessed by line-scanning cameras.This also results in the need for a uniform particle velocity, as discussed in Sec.III-A.Furthermore, for the final definition of these deflection windows, different strategies can be followed.One common approach is the use of a rectangle that surrounds the particle (bounding box) or a targeted pulse on the geometric center of gravity [146].
In addition to achieving the highest possible result quality, processing time plays an important role in data analysis for sensor-based sorting.The latency between sensory detection and separation results in a firm real-time criterion with binary utility function, which must be met.To maximize precision in the separation, this delay should be kept as small as possible.It typically amounts to a few milliseconds [61].Within this time, both the sorting decision must have been calculated and the control signal for the separation must have been transmitted.If the real-time barrier is broken and the particle has already passed the separation stage, it can no longer be removed from the material flow.Algorithmic work is therefore concerned with reducing computational time by selecting suitable features [147] or using approaches from the field of approximate computing [148].

D. SEPARATION
In sensor-based sorting, the physical separation of the material occurs, with few exceptions, via an array of actuators positioned perpendicular to the direction of transit.After being discharged from a conveyor belt or chute, for example, the actuators act on the particles that need to be separated from the material stream during the flight phase.The action alters the path of the particles that need to be separated in such a way that they become physically isolated from the rest of the material and, for instance, land in different containers or on different conveyor belts for subsequent conveyance.Pneumatic fast-switching valves are the de facto standard for material separation [4].A short impulse generated by dry air blasts separates individual particles from the material stream, see Fig. 9.

1) PNEUMATIC SEPARATION
When using pneumatic fast-switching valves (and also alternative separation technologies), the effective range is divided into a discrete grid by means of the individual elements of the array, e.g.individual compressed air nozzles.The cell sizes of this grid perpendicular to the transport direction define what we refer to as the spatial resolution.Similarly, there exists a temporal resolution, which figuratively defines the cell sizes in the transport direction.The highest achievable temporal resolution is usually dependent on the electronics used to control the individual actuators, such as the bus system used  for communication or the minimum switching times of the individual actuators.The impact of spatial and temporal resolution is illustrated in Fig. 10.As can be seen, fine resolution is needed to ensure that single particle separation is possible without erroneously co-deflecting other particles nearby.Clearly, with increasing proximity of particles (see Sec. IV), an even higher resolution is required.
The design parameters for realizing a pneumatic separation include the size of the nozzles and the overpressure.They must be chosen depending on the particle size distribution and density [3].Their suitability for use in sensor-based sorting also depends on switching times and achievable volume flow [149].Using a multi-hole probe, the volumetric flow rate and the spatial distribution of the air flow velocity of the valves can be measured for a steady [150] or transient state [151].It allows for measuring the total pressure, static pressure, temperature, flow velocity, and orientation.
Various works also exist that study the pneumatic deflection process in sensor-based sorting by simulation.Computational fluid dynamics (CFD) is a popular tool for modeling air flow, as is the discrete element method (DEM) for modeling particles [152].By using a DEM-CFD coupling, the complete sorting process can be simulated and the simulation results can be compared with those obtained from experiments [119].Such a simulation is a valuable tool to identify optimal parameters for pneumatic separation, for example, the optimal position and orientation of the valve array for varying transport parameters, such as transport velocity [153].
The consumption of compressed air is a significant factor in the operating costs of sensor-based sorting systems.In the mining field, the generation of compressed air may account for approximately 70% of the total operating costs [50].However, this depends heavily on the field of application and the resulting circumstances.For example, significantly different ratios are reported for grape sorting, an example from the food industry [154].In this case, the costs of the staff, which are incurred for cleaning and the like, account for the largest share.

2) ALTERNATIVE SEPARATION TECHNOLOGIES
The two most common non-pneumatic separation technologies used in sensor-based sorting are mechanical flaps [26] and water jets, while the latter is rarely used due to maintenance and cleaning problems [103].Both are found mainly in the field of mining.The reason for this is that they are particularly suitable for deflecting comparatively heavy objects.One disadvantage of mechanical flaps is that they are comparatively maintenance-intensive due to contact with the material.Water jets conflict with a major advantage of sensor-based sorting, namely that it is a dry process [65].This is also a reason why they are typically not found in the sorting of agricultural products and foodstuffs.In general, in many cases, the need for the use of energy-intensive drying processes arises.
In addition, the idea of using robotics, for example, grippers or suction valves [155], to separate the material stream has been around for quite some time [156].In recent years, this idea has been revived, especially for sorting of C&DW [157], [158].This field of application appears particularly interesting for robotic sorting due to potentially very large objects [159].Furthermore, it should be noted that human sorting is also not an alternative due to limitations in weight and dust generation during the process, which may contain hazardous materials such as asbestos.Also, in the processing of municipal waste, especially packaging waste [160], the combined use of robotic technologies and optical sorting systems with pneumatic separation is considered a promising solution.

IV. PROCESS PARAMETERS
In addition to the geometrical parameters, a group of so-called process parameters must be considered [34].Examples of process parameters are the particle size distribution, material throughput, or the characteristics of the material itself.Another process parameter, which will be discussed in the following, is particle proximity.
In contrast to the system parameters discussed in Sec.III, the process parameters can only be freely selected to a limited extent, since they are partially defined by the sorting task itself.However, they have a decisive influence on the efficiency of sensor-based sorting systems.

A. PARTICLE SIZE DISTRIBUTION
The particle size distribution describes an interval of the geometric size of the particles.Sensor-based sorting systems are fed with a particle size distribution as closely defined as possible by means of suitable pre-conditioning, since the particle size distribution has a decisive influence on the selection of system and process parameters, e.g., the achievable material throughput or the parameterization of the separation unit.This can be achieved by, for instance, screening or crushing.As a general rule of thumb, a size ratio of 1:3 between the smallest and the largest particles within the distribution has been established as a desirable limitation [32].
In general, sensor-based sorting is used for particle sizes as small as 1 mm, e.g., for diamonds.However, in mining, in particular, it is customary to use sensor-based sorting systems only for significantly larger particle sizes, since the material throughput, and thus the economic efficiency of the sorting systems, increases with increasing particle size.The sortable particle sizes range up to approximately 300 mm [32].There is a quasi-linear relationship between particle size and material throughput, the latter increasing with increasing particle size [161].The results of certain experimental investigations suggest a higher quality of sorting with larger particles [115].In general, the sorting of small particles can be seen as a more difficult problem, since additional boundary conditions have to be considered.For example, it must be ensured that the spatial resolution of the sensor is sufficiently fine to allow the data analysis to correctly characterize individual particles.Equivalently, a fine resolution is also required for material separation.

B. MASS FLOW, OCCUPANCY DENSITY AND PROXIMITY
To describe the amount of material processed by a sensor-based sorting system, the terms mass flow and occupancy density need to be distinguished.Both parameters depend on the particle size distribution (see Sec. IV-A) as well as the sorting width and the transport speed of the transport medium (see Sec. III-A).Moreover, the parameters usually exhibit a strong correlation.
The mass flow rate is defined as the weight of material fed to the sorting system over a period of time, and thus also depends on the density of the material to be sorted.It is often expressed in tons per hour and is particularly relevant for economic considerations.Mass flows of up to 250 t h −1 have been reported for sensor-based sorting [32].If the transport parameters are fixed, different mass flows for the same sorting task have strong effects on the sorting quality.Increasing the mass flow rate has a principally negative effect on the quality of the sorting for a given sorting task [115], [161].Based on Monte Carlo simulation, it has been shown that the sorting quality decreases exponentially as the mass flow rate increases [162].However, such results must also be considered together with economic aspects [76].For this purpose, the resulting costs of sorting per ton are included in the consideration, in addition to the sorting quality and the mass flow.This consideration is particularly helpful if, due to the field of application, certain limit values have to be observed instead of maximum purity of the sorting result.
According to the above-mentioned definition, however, it is not so much the mass flow that is responsible for the influences on sorting quality shown above, but rather the occupancy density [163].The density of occupancy describes the proportion of the sensory image that is actually occupied by material, compare Fig. 11.It is thus, so to speak, a unit of measure for the mass flow from the sensory point of view, since the scene is projected onto a 2D representation.From a data analysis point of view, the occupancy density is also of great interest, since in many cases it is directly reflected in the required runtimes of the algorithms used.However, for the reasons mentioned above, the occupancy density also strongly correlates with the mass flow.If feeding succeeds in distributing the material evenly across the entire sorting width, the occupancy density also strongly correlates with the proximity of the particles.

C. MATERIAL COMPOSITION
The material composition describes the ratio of "product" to "residue" of the input material stream to be sorted.It also determines which of the two groups is to be deflected by activation in the course of physical separation.Generally, this 6486 VOLUME 12, 2024 Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
is the fraction that is expected to be in the minority.This is partly because energy is required for the deflection process and partly because the deflection process itself potentially leads to sorting errors and should therefore be performed as rarely as possible [161] (see Sec.Sections III-D and V).Based on the Monte Carlo simulation [162] and experimental studies [163], an exponential decrease in the achievable mass flow is obtained with an increasing fraction of particles to be deflected while fixing the sorting quality.

D. PARTICLE PROXIMITY
Particle proximity results from the number of particles fed into the system within a period of time, their size distribution, and the quality of the feeding and transport unit (see Sec. III-A).High proximity has two main negative effects on sorting quality.First, touching objects create clusters [162], which must be resolved by data analysis to evaluate individual touching particles.However, such algorithms often also rely on assumptions about particle shapes and generate additional computational load, which must be taken into account to meet the required firm real-time requirements (see Sec. III-C).Second, object clusters cause errors during separation, even if the particles were evaluated individually and correctly during data analysis.In all separation methods presented in Sec.III-D, the deflection of one particle can falsely co-deflect other particles located nearby.This effect has also been studied on the basis of numerical simulation [152].One statistical parameter to describe particle proximity is, for example, the smallest particle distance.

V. EVALUATION OF THE SORTING QUALITY
The efficiency of sensor-based sorting systems is typically evaluated using the two competing parameters material throughput and sorting quality.As discussed in Sec.IV, the quality of the sorting typically decreases as the throughput increases [163].
There are two potential error types that can occur during a sorting process [152], [163]: 1) Data analysis can fail to characterize an individual particle correctly, which can lead to confusing a particle to be accepted with one to be rejected and vice versa.Such errors in material recognition are influenced by the presentation, examination, and data analysis phases.2) A particle may not be physically removed from the material stream, e.g.due to poor actuator control, although it was correctly characterized.Likewise, it can occur that a particle that should not have been deflected is mistakenly co-deflected.Such errors in material separation are influenced by the phases presentation, data analysis, and separation.
Analyzing the final sorting result does not allow one to draw conclusions about which of the error types led to the sorting errors.The types of error can be visualized in the form of a decision tree [161].Considering a binary sorting task, many established parameters for describing the sorting quality can be traced back to a confusion matrix as shown in Fig. 12 [34], [115], [164].The terms true positive, false positive, false negative, and true negative given in Fig. 12 are abbreviated by TP, FP, FN and TN below.
There are three widely used parameters to describe sorting quality [55].They are typically based on mass ratios and can also be expressed on the basis of a confusion matrix.
The first parameter is Recovery and calculated according to the notation from Fig. 12 by It hence describes the amount of material removed, i.e., rejected in the sorting process, and does not provide information about the correctness of the sorting result by itself.
The second parameter is Yield.It is calculated as hence describing the ratio of the proportion of "product" prior to sorting to the proportion of "product" after sorting.The third parameter is referred to as Purity and is given by It describes the the actual proportion of material to be recovered in the corresponding sorting fraction.Beyond these three established parameters, further work is concerned with quantifying the sorting performance.An example is the parameter referred to as Separation Efficiency which is given by Separation efficiency (SE) % := R d − R c (4) in its original definition [161], [162].In this notation, R d refers to the ratio of material to be rejected that was successfully rejected in percent, and R c to the ratio of material to be accepted that was successfully accepted.Hence, it describes the difference between the purity of the accepted material and the product loss.With reference to the notation used in this paper, the definition can be formulated as Separation efficiency (SE) % := A key difference is the inclusion of product loss, i.e., material to be accepted that was falsely rejected, in the parameter.Furthermore, based on the concept of a Receiver-Operating Characteristic (ROC) curve, which has also been used to determine the classification and economic performance in sensor-based sorting [28], so-called Sorting Optimization Curves (SOC) have been proposed [164].The approach is intended to support the evaluation and optimization of system configurations.For this purpose, the quality and yield are calculated as described above.As an extension, the influential costs are also considered, such as the composition of the input material or the yield as given in Eq. ( 2).In this way, operation characteristics can be derived for different configurations and adjusted according to the desired sorting result.

VI. CONCLUSION
In the following, we briefly summarize this survey and provide an assessment of potential future research directions.

A. SUMMARY AND FINDINGS
Within the identified main application fields of mining, recycling and the processing of agricultural products and foodstuffs, a wide variety of tasks exist, ranging from the sorting of highly heterogeneous material streams to the removal of rather rare occurring impurities.In addition, there are often boundary conditions for specific areas of application which must be taken into account in the system design, such as easily cleanable components in the field of agricultural products and foodstuffs.Therefore, a deep understanding of the application area and the processing procedures is essential.
In order to achieve advances in sensor-based sorting, the complete sorting process must be understood and considered holistically.This concerns both the system components used in material feeding and transport, sensor technology, data analysis, and material separation, as well as the consideration of any process parameters specified by the context of use.Conveyor belts, chutes, and free-fall transport were presented as the most common transport mechanisms, and their advantages and disadvantages were discussed.
The wide range of possible applications for sensor-based sorting is largely due to the large number of applicable and industrially available sensors.The most common sensor technologies used in sensor-based sorting have been named and briefly explained.Here, for the recent past, the role of HSI is to be explicitly emphasized.The research considered suggests a dominant application of HSI in the IR spectrum for recycling applications and of X-ray-based systems in the mining field.VIS technology is represented in all fields of application.LIBS and LIFS appear to be of current interest in both mining and recycling, but not in the field of processing agricultural products and foodstuffs.
However, the potential of any sensor technology can only be exploited by means of adequate data analysis.A particular challenge lies in the development of real-time capable systems.The difficulty of this task increases with the demand of plant operators for ever higher material throughputs to increase the economic efficiency of sorting plants.With regard to the actuators used for the physical separation of individual particles, it can be stated that pneumatic separation by means of fast-switching valves can be regarded as the standard.However, by achieving finer spatial and temporal resolutions for valve control through technological advances, sorting quality can be further improved and higher occupancy densities can be handled.
For system design, it is crucial that the process parameters of particle size distribution, mass flow and occupancy density, composition, and material proximity are well known and, as far as possible, controlled.These parameters must also always be considered in combination due to their dependencies.A particular challenge may be that these parameters are rarely static for a sorting task, but are subject to fluctuations during the sorting process, so that a certain bandwidth must be assumed in the system design.
With regard to the evaluation of sorting quality, various parameters have been established which, however, are ultimately always based on a confusion matrix.Thus, depending on the goal of an investigation, it is necessary to evaluate which parameter offers the maximum information content to answer a particular question.

B. POSSIBLE FUTURE RESEARCH DIRECTIONS
Although sensor-based sorting technology can be considered at a mature stage for many applications, we believe that there exist many research directions for further advances.This is not limited to quantitative advances, i.e., developments leading to higher sorting efficiencies, but also concerns qualitative ones, i.e., enabling the technology for new applications.Due to the interdisciplinary character of the technology, it appears that there are many directions to pursue in future research.However, in the following, we highlight three possible future research directions, which we consider to have a particularly high potential for future process advances.
We expect that the development of new fields of application for sensor-based sorting will be made possible, in particular, by advances in sensor technology.A recent example of this is the upgrade of MIR-HSI cameras for industrial use.Developments in the field of LIBS and THz sensor technology also appear promising.In addition to specific sensor technologies, this also applies to multisensor systems.The integration of more and more sensors coupled with intelligent data fusion algorithms also creates opportunities to tackle previously unsolved sorting tasks.
Furthermore, it can be stated that machine learning has become more and more important for sensor-based sorting in recent years, just as in many other areas of application.Although there are already some papers that address the topic in the context of sensor-based sorting, the potential seems to be far from exhausted.In some cases, the trend towards data-driven evaluation processes goes hand in hand with the development of new sensor technologies.This is particularly true if the acquired sensor information can only be interpreted by humans with difficulty or not at all.The same applies to the use of multi-sensor systems as described above.In particular, the fairly new research field of explainable artificial intelligence (XAI) is expected to increase industrial acceptance for the use of such methods in the future.
Eventually, in the wake of the fourth industrial revolution, sensor-based sorting is going to be no longer regarded as a closed system with the purpose of material separation but rather as a rich source for process data.More precisely, information about the material stream obtained via sensor data and processed by the algorithms may not only be used to calculate sorting decisions but also to gain valuable insights about the stream characteristics.This allows, for example, conclusions to be drawn about upstream conditioning steps, or their parameterization to be optimized.However, this requires future systems to provide data interfaces in a way that is standardized across the industry in order to make the information available to other systems.

FIGURE 1 .
FIGURE 1. Schematic illustration of the sensor-based sorting process.

FIGURE 3 .
FIGURE 3. Journal and conference papers indexed by Scopus per year and field of application matching the search query ''sensor-based sorting''.

FIGURE 4 .
FIGURE 4. Qualitative comparison of the most widely used sensor technologies and the respective application fields based on Journal and conference papers indexed by Scopus matching the search query ''sensor-based sorting''.

FIGURE 5 .
FIGURE 5. Examples of materials to be sorted from mining, recycling and agricultural products and foodstuffs.

FIGURE 6 .
FIGURE 6. Schemes of the most common transport variants: (a) conveyor belt (b) chute and (c) free fall.The green spheres represent objects to be accepted ("product") and the red squares objects to be rejected ("residues").The yellow beam indicates the scan-line of the sensor, and the blue beam indicates the actuator's action zone.In the schematics of (a) and (b), sensory examinations can also be performed on the transport medium (i.e., on a conveyor belt or chute) instead of after discharge.

FIGURE 7 .
FIGURE 7.Examples of spectra obtained using a SWIR-HSI camera for the two plastic types PE and PP.Here, the camera used captures 256 bands in the wavelength range between approx.1000 nm and 2500 nm.The occurrence of absorption bands is material-specific and can hence be used to distinguish materials and calculate sorting decisions.

FIGURE 8 .
FIGURE 8. Schematic illustration of an impact acoustic sorting system.

FIGURE 9 .
FIGURE 9. Lateral view of the separation process by means of pneumatic fast-switching valves.

FIGURE 10 .
FIGURE 10.Schematic visualization of the impact of separation resolution.Here, the particle indicated by the red diamond is meant to be deflected, while the one represented by the turquoise circle should remain unaffected.Grid areas with a yellow background align with activated individual actuators, such as the fast switching valves being opened.

FIGURE 12 .
FIGURE 12.Visualization of a confusion matrix to describe the sorting quality.

TABLE 1 .
Overview of the sensors discussed in Sec.III-B.The sorting tasks, advantages and disadvantages cited are illustrative; each technology possesses numerous advantages and disadvantages.