Shadow Segmentation With Image Thresholding for Describing the Harshness of Light Sources

Light sources are usually described by luminous intensity, color temperature and color rendering index, while the harshness of the light source is often overlooked. It is also referred to as the softness or light quality and can be determined by analyzing the shadows it produces. Shadow detection is often addressed in image analysis research, but the methods usually provide solutions for removing shadows. In this research, a method is proposed for isolating shadows from test images, applying thresholding for segmentation of the shadow into umbral and penumbral regions, calculating the shadow metric, and finally calculating the final numerical evaluation of the quality of the light source, i.e., the degree of its softness or harshness. The method proved to be efficient in analyzing halogen, LED and xenon light sources in conjunction with light-shaping attachments that soften the light. The method is not influenced by the intensity of the light source and focuses exclusively on the quality of the light. The method provides a single value to describe this light property and has the potential to become a standardized method for describing the quality of light sources and light-shaping attachments.


Shadow Segmentation With Image Thresholding for
Describing the Harshness of Light Sources Veronika Štampfl and Jure Ahtik Abstract-Light sources are usually described by luminous intensity, color temperature and color rendering index, while the harshness of the light source is often overlooked.It is also referred to as the softness or light quality and can be determined by analyzing the shadows it produces.Shadow detection is often addressed in image analysis research, but the methods usually provide solutions for removing shadows.In this research, a method is proposed for isolating shadows from test images, applying thresholding for segmentation of the shadow into umbral and penumbral regions, calculating the shadow metric, and finally calculating the final numerical evaluation of the quality of the light source, i.e., the degree of its softness or harshness.The method proved to be efficient in analyzing halogen, LED and xenon light sources in conjunction with light-shaping attachments that soften the light.The method is not influenced by the intensity of the light source and focuses exclusively on the quality of the light.The method provides a single value to describe this light property and has the potential to become a standardized method for describing the quality of light sources and light-shaping attachments.
Index Terms-Image analysis, light harshness, light quality, shadow segmentation, thresholding.

I. INTRODUCTION
A PRECISE description of lighting conditions is crucial for the repeatability of scientific research projects whose results could be influenced by changes in lighting conditions.Their current descriptions usually include the type of light source, its intensity, correlated color temperature (CCT) and distance from the observed scene.However, this information does not necessarily fully describe the lighting conditions at the scene, as light sources emit light with different levels of harshness and are often coupled with light-shaping attachments to alter the original light.
Our ongoing research aims to characterize artificial light sources and the influence of light-shaping attachments on the original light.While in our previous research [1], [2], [3] we have investigated the influence of light-shaping attachments on color properties of a light source using widely used metrics such as color differences in the CIE Lab color space and the color rendering index (CRI), we have not encountered an available metric that would describe the harshness of a light source, let alone characterize the influence of a light-shaping attachment on this metric.
This study describes our approach to developing a method that describes the harshness of a light source.Although appearance of light can be influenced by the surrounding environment, the acquired metric describes the harshness value native to the light source, since it is acquired in a lab environment.Therefore, the method has the potential to become a new standard for quantitative assessment of the harshness of artificial light sources, while the final value could be presented along the already standardized metrics that are native to the light sources, such as CCT, CRI and power.

II. LIGHT AND SHADOW
When light rays hit an object, an occluder, it blocks them on their way to the projection surface and casts a shadow.Depending on the properties of the light, the shadow can be soft or harsh, and it can also contain color information [4].
To accurately determine the harshness of a light source with or without a light-shaping attachment, several factors must be taken into account.The spatial orientation of the light source in relation to the observed scene must be considered as well as several properties of the light source, e.g. the size, shape and material properties of the attached light modifiers.The material composition, surface properties and porosity of the material play a crucial role in the effect on the original light [1], [3], leading to a variety of possible application combinations.While geometric projection helps to predict shadows to a certain extent, it has the limitation that it ignores the influence of the materials used in the potential light modification.However, a closer look at the resulting shadow can provide information about the properties of the light source as a whole [5].
While a single-point light source produces a uniform shadow with a clear boundary, a non-point light source results in a more uneven shadow with a gradual gradient, which can be divided into a fully shaded umbra and a partially shaded penumbra region [4], as shown in Fig. 1.The ratio between these two areas and rate of change from one region to another is visually perceived as light harshness, also known as light quality [4].

III. SHADOW ANALYSIS APPLICATIONS AND METHODS
In computer graphics the existence of shadows is essential to achieve photorealism [6], [7], [8].To accurately represent  shadows in virtual environments, researchers analyze the lighting conditions in real scenes and transfer these findings to the development of software for designing three-dimensional environments [9], [10], [11].
The influence of the shape of an object and the reflectivity of its surface on the image properties has already been well researched [12], [13], [14], while the third parameter, the arrangement of the light sources and their properties, is still under development.This points to the complexity of recognizing and analyzing shadow shapes to obtain illumination data.MacDonald et al. [15] investigated the importance of the position of a light source in relation to an illuminated object, while Okabe et al. [16] showed that analyzing cast shadows is a reliable method in retrieving lighting properties.
The presence of shadows can lead to a loss of information in the shadowed areas of the image.Most researchers, especially in the field of computer vision and visual recognition, are therefore concerned with researching the methods to remove them from visual images [17], [18], [19], [20].
Shadows are usually perceived by analyzing brightness, colors, textures and geometric shapes within an image and then removed [21], [22], [23], [24].Sanin et al. [25] show that methods based on color values are the fastest but can be unreliable.Analyzing textures is most effective in detection of shadow areas but is the most demanding.Gong and Cosker [26] evaluated several methods for shadow extraction and analysis and showed that the success rate of shadow detection and removal algorithms depends primarily on the colorfulness or brokenness of the shadow in the analyzed image, followed by its softness and texture.
Another common method for defining shaded areas is to remove the background of the scene, isolating the object and its shadow [22], [27].Such methods provide us with a binary image of the shape of the object and its shadow, from which we cannot subsequently define the individual segments, along with umbra and penumbra.Khan et al. [28] propose a method for shadow perception that uses multiple deep convolutional neural networks (CNNs) that indicate shadow shapes by detection of the object edges in the image and learning the network.This method enables the definition of a shadow from only one image and does not require additional input information about the background, while the areas of umbra and penumbra can be detected by analyzing the image gradient.Since this study, the performance of shadow detection with CNNs has been further improved.Now, data for shadow edge, shadow count and shadow region are combined to achieve better shadow detection [29].
Dong et al. [30] follow a similar approach with gradient analysis.The results show that the normalized values of pixel brightness in the transition region from umbra to penumbra and then to the fully illuminated region are consistent with the logistic sigmoid function.Astronomers and astrophysicists analyze the histogram intensities of sunspot images and obtain data on the signal strength at each pixel, while sudden changes in the gradient are used to define the boundaries between the umbra and penumbra [31].

IV. METHODOLOGY
While existing shadow analysis methods are efficient in shadow removal from images from an uncontrolled environment, they are computationally demanding and do not meet our needs to the desired extent.Since our aim is to analyze light sources and describe them without the influencing factors from the surrounding environment, we propose a novel method that analyzes shadows created under controlled conditions, while quantitatively determines the harshness of a light source.Main steps are shown in Fig. 2.
First, two photographs of a predefined projection plane are taken, one with and one without the shadow area caused by a predefined occluder.The photographs are registered according to a reference image and the background is removed from the photograph with the shadow by subtracting the image with the shadow from the image created without the occluder.
Second, we perform an image thresholding for 256 thresholds to obtain the shadow shape at each brightness level.The centroids of these shapes are calculated and formed into a function representing the shadow gradient.
Finally, partial derivatives are used to calculate the point of change within the shadow gradient to define the size of umbra (U h ) and penumbra (P w ).The ratio of P w /U h represents the harshness of the tested light source, which we propose as a new metric to describe the properties of light sources.

A. Test Environment
To obtain a shadow produced by a light source, we set up a test scene including a projection plane, an occluder, a light source and a camera.The test scene was set in a darkroom to fully exclude ambient light and minimize environmental effects [32].The scheme of the test scene is shown in Fig. 3.
For the projection plane, we used a 50 × 50 cm gray PVC plate with a surface reflectance of 27.70% (SD = 3.86%) in the visible spectrum and an average surface roughness of 1.9 µm to 2.6 µm, depending on the direction of measurement.These surface properties proved the material as uniform enough not to affect the quality of shadow reproduction.
Using a UV inkjet printer, we printed a series of visual markings on the surface of the projection plane.The first was a set of 16 ArUco markers, binary unique patterns that, in combination with the associated algorithm from an opensource library [33], enable fast and reliable identification.These markers, each measuring 2 × 2 cm, were used as reference points for the alignment of images in later image analysis steps.
In addition to the ArUco markers, a square outline was printed on one edge of the surface to define a specific position for the occluder.A 1 mm thick L-shaped aluminum profile was used for the shadow cast, with the lower, horizontal part measuring 6 × 6 cm and the upright part of the occluder measuring 6 × 8 cm.It was painted matt black to minimize reflection.The average measured reflectance of the surface in the visible spectrum was 2.36% (SD = 0.07%).
To illuminate the scene, a light source was placed to the side of the projection plane at a distance of 1.2 m from the edge of the plane and the positioned occluder.The center of the light source was positioned 10 cm higher than the projection plane and set at an angle of 0 • relative to the surface of the plane.
To capture the surface of the projection plane, a camera was mounted above the scene at an angle of 90 • relative to the surface of the plane.The height of the camera was adjusted so that the entire plane could be captured and was set to approximately 200 cm above the surface.

B. Input Data
The method required three input images.The first was the reference image with ArUco markers, which was used for image registration and enabled a pixel-by-pixel comparison of the photographs.We used the file for printing the markers on the projection plane (Fig. 2) and exported it to a square PNG file with a length of 5906 pixels per side to use as a reference image.
The second input image was a photograph of the lit projection plane without the occluder in place, while the third image was captured under the same lighting conditions but with a placed occluder.This gave us an image of the background and an image of the shadowed background, respectively.All photographs were taken in RAW format to obtain the highest possible quality and not to influence the results by image compression that other file formats are subject to.

C. Image Registration
The photographs had to be registered to the reference image to extract the shadow.The flowchart of the image registration algorithm for one photograph is shown in Fig. 4.
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
The first input for the image registration algorithm is the PNG reference image.After importing it, we converted it to grayscale, as we were not interested in the color values.The conversion was performed using the equation where Y stands for the luminance, R for the red value of the pixel, G for green and B for blue, while the coefficients are predefined so that the result closely represents the relative perception of the brightness of the color components by the average person [34].The next step was the detection of the ArUco markers on the grayscale reference image using an algorithm as in [35], which returned the 2D positions p The second input was a captured image, which we first converted from RAW format to JPG format, ignoring the camera's calibration profiles and applying no gamma correction to keep the recognized data as close as possible to the captured data.In the next step, we converted the image to grayscale according to (1), saved it as a test image and again detected the ArUco markers p test ArU co = (x test id , y test id ) (Fig. 2b).If not all markers were found, a loop was initiated in which the image brightness was gradually increased for the value of 0.5 until the position coordinates of all markers were determined.
Using the IDs of the markers, we were able to assign the coordinates to the individual markers and calculate the homography matrix with equation    x r e f id y r e f id w r e f id where A is the 3 × 3 homography matrix and w r e f id is a scaling factor.A homography matrix was used to align the test image with the reference image and ensure an appropriate level of image registration and image comparability.
The aligned image (Fig. 2c) corresponds to the size of the reference image since the image registration method crops the input image during the process.However, we have additionally cropped the aligned image to the area of interest (shadow area) to minimise the amount of data to be processed in the next steps.Since the shadow is symmetrical about the vertical axis, we first split the image in half and only use the left part of the image.The right edge of the cropped image was therefore set to the centre of the input image, the bottom edge of the cropped image was set to the shadow origin, i.e. the edge of the occluder, while the left and top edges were set to remove the ArUco markers and area that could not contain the shadow.The cropped image was 1453 px wide (I w ) and 3370 px high (I h ).The final aligned and cropped grayscale image was saved in 8-bit JPG format.

D. Shadow Detection
To extract the shadow shape from the registered images and obtain data for shadow description, we set up another algorithm.The flowchart is shown in Fig. 5.A pair of images consisted of two pre-processed images, one without the occluder (I B ), i.e. without the shadow, and the second with the occluder and the shadow area (I B S ).In order to extract the shape of the shadow from the shaded image and eliminate the influence of the background, the images were subtracted No scaling or offset was performed.The resulting image I S shows the change in pixel intensity, i.e. the shadow, caused by the occluder (Fig. 2d).
Image I S was then inverted to obtain the positive shape of the shadow, as governed by where I (x, y) is the final image representation of the shadow (Fig. 2e) and 255 is the maximum possible value of a pixel.Two empty data arrays were created to store the data we generated in the next step.The image I was thresholded with inverse binary thresholding according to where I T (x, y) is the binary image, Y thr is the applied threshold and 255 is the maximum possible value of a pixel.We created a loop in which the Y thr was increased from 0 to 255 in steps of 1.In this way, we obtained a series of 256 binary images of the shadow image, each showing a larger portion of the shadow.
In each binary image, we used the Suzuki algorithm [36] to search for clusters of black pixels and their contours.To determine the area and the centroid of the contour outlining the shadow, we calculated the raw image moments for every contour according to where M i j is the moment.The contour with the largest value for the image moment M 00 was defined as the most prominent to represent the shadow, since M 00 represents the cluster area.This eliminates any additional noise that could be caused by the non-uniformity of the surface.The centroid of the most prominent contour (Fig. 2f) was calculated using the formula where ( x, ȳ) are the centroid coordinates and M i j are image moments calculated according to (6).If no cluster of black pixels was found within the binary image, the centroid coordinates were set to (0, 0).We normalized the coordinates based on the width and height of the images, i.e. ( x, ȳ) = ( x/I w , ȳ/I h ), and saved them to data arrays.The loop was repeated for all 256 binary images within the loop.

E. Shadow Segmentation
A third algorithm was set up to analyze the data and obtain shadow metric H .The algorithm is shown in Fig. 6.
First, we imported data arrays with normalized centroid coordinates ( x, ȳ).We plotted the coordinates and visually defined outliers as noise.Any ( x, ȳ) coordinate that appeared to be an outlier was replaced with (0, 0) and was not deleted at this point to maintain equal data sets.
To find the thresholds where the shadow starts and stops appearing, i.e. shadow start and end, we searched for the positional arguments of the first and last non-zero value (Y last thr and Y f ir st thr , respectively) in the data arrays for x and ȳ.The positional arguments can be translated directly into the threshold value at which the contour for the centroid was calculated.The difference between these two positional arguments is the range of threshold values in which the shadow appears.We can also refer to this range as the brightness range of the shadow S r and can be defined as We combined the data arrays for the same combination of light source type and quality, combining 7 data sets that differed only in light intensity.Once we had combined the data arrays, the elements were sorted according to polar angle.When sorted, we discarded the coordinates with the value (0, 0) to keep only the data describing the shadow.
To smooth the data sets, we applied the Savitzky-Golay filter [37] to each data array by convolving a linear function to the data, using an adaptive length of the filter window for each array.The filter window span D had to be adapted depending on the amount of data and was defined as follows where L is the length of the data array, f is a constant factor and m is the polynomial order of the convolving function.Our analysis showed that the constant factor f = 17 proved to be the most efficient to smooth the data as little as possible, but still achieve a sufficient level of smoothness for further processing.
The consecutive pairs of centroid coordinates x and ȳ in the data arrays formed a curve to which we fitted third-order B-splines ỹ = g( x) passing through all input points ( xi , ȳi ).We then evaluated the spline fits for 100 uniformly distributed values, representing the final shadow data for the light source.
We calculated the derivative of spline fit g ′ ( x).The maximum value of the derivative g ′ ( x) defines the endpoint of the umbra and thus the transition from umbra to penumbra S u/ p .While x additionally needs to fit the conditions to avoid false indications where S u/ p is the positional argument of the transition point from umbra to penumbra and i max is the positional argument of the maximum value of the derivative g ′ ( x).
Once the sections of the umbra and penumbra were defined, we could determine the width and height of these areas.The height of the umbra was defined as where g( x) S u/ p is the value at the transition point from umbra to penumbra.The width of the penumbra was defined as where xS u/ p is the value of x at the transition point from umbra to penumbra and xS e is the value of x at the end of the shadow.Dividing the penumbra width by the umbra height resulted in the final numerical evaluation of light quality, i.e. harshness Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.

V. EXPERIMENT
To test the method, we created an image database with a total of 84 photographs; or 42 photo pairs, since the pair consisted of a photo of the projection surface with and without occluder, i.e. with and without shadow.Each pair differed in the illumination conditions, where type of light source, quality of light, and light intensity varied.We used the proposed method to analyze all 42 image pairs and obtain the harshness of the light sources.Then we compared the results of our method with the results of a commonly used linear shadow projection.
A. Light Sources Three types of light sources were tested: LED (labeled L), halogen (labeled H ) and xenon (labeled X ).Each light source was transformed from its original harsher (labeled H ) light into a softer light (labeled S) using a dedicated light-shaping attachment.Each of these six combinations was tested at 7 intensity levels, from the lowest to the highest available level in equal steps.
The Lupo Superpanel Dual-Color light panel with a color temperature of 5040 K was used as the LED light source.It is a square LED panel with side length of 30 cm.The original light was softened with a square light-shaping attachment Lupo LED Panels Softbox Set with side length of 40 cm and 30 cm depth.
A Kaiser Studiolight H was used as the halogen light source.The halogen light bulb is 2 cm wide and can be considered as a point source.To soften the light, a square softbox with a side length of 70 cm was attached, with depth of 45 cm.
The xenon light source used was an Elinchrom ELC PRO HD 500studio flash, with a round xenon tube with the diameter of 7 cm.A square 66 cm Elinchrom Portalite Softbox was attached to soften the light beam, with depth of 35 cm.
To characterize the light emitted by the tested light sources, we used the X-Rite i1 Pro spectrophotometer and Argyll CMS software.The emission spectra of all tested combinations were measured by placing the device on the projection plane in place of the occluder and pointing it directly at the light source.Fig. 7 shows the emission spectra of tested light sources.

B. Image Capture
To capture the photographs, we used the Nikon D850 camera with a Nikkor 50 mm 1.4G lens.The camera's aperture was set to 5.6 for maximum sharpness [38], while ISO sensitivity, white balance, and shutter speed were set according to the type of light source but did not vary among intensity levels.
The white balance was set for each light source based on the color temperature measured with the spectrophotometer, while ISO sensitivity and the shutter speed were set with the aim of ensuring the lowest possible image noise and maintaining the sharpness of the images.Each photo of the series was taken in RAW format.

C. Geometric Shadow Projections
To obtain the theoretical shadow projections, we used an Adobe Illustrator vector application in which we constructed the entire scene to scale.We defined the areas of umbra, penumbra and illuminated surfaces with vectors for all six light source combinations, ignoring the intensity of the light source.
We then cropped the images of shadow projections on the virtual projection plane to the dimensions I w and I h , thresholded the images, and determined the centroid of umbra U pr oj ( x, ȳ) and penumbra P pr oj ( x, ȳ) according to the methodology from Section IV.We have defined the umbra height U pr oj h as where ȳU pr oj is the ȳ value of the centroid coordinate for umbra.
To define the penumbra width P pr oj w we followed the equation where xU pr oj and xP pr oj are the x values of the centroid coordinates for umbra and penumbra, respectively.To determine the harshness value, we followed (13).

VI. RESULTS
This section comments on the steps of the proposed method and the results obtained.The results are evaluated, while guidelines for the application of the method and its possible improvements are given.

A. Image Registration
The first important step for the proposed method to be efficient is image registration.The detection of ArUco markers on the reference image and the photographs was entirely successful.However, when detecting markers on grayscale images of photographs, which were illuminated with a lower intensity value, the image brightness had to be increased in order to detect all markers.We hypothesize that the reason for the lower success rate in markers is the pronounced uneven illumination of the projection plane when illuminated with a weaker light beam.Consequently, not all markers had sufficient contrast to the background to be recognized.The brightened image was only used for marker detection, while the original, non-brightened image was aligned to the reference image.
The homography matrix A takes into account translation, rotation, scaling, shear and deformation due to perspective.Since we did not use a lens distortion correction profile, image distortions may still occur -lenses with a focal length of less than 50 mm usually result in pincushion positive distortions, with the central part of the image perceived as slightly convex.However, the camera lens used in this study provides low image distortion due to its focal length and highquality construction.In addition, only the central portion of the captured photograph was used for analysis, minimizing the possibility of this type of distortion occurring and affecting our results.
During image registration to the reference image, areas of the overlaid image that extended beyond the boundaries of the reference image were removed by the method.In this way, the width and height of the processed images were aligned with the reference image and the operational data was made more manageable in terms of data volume.This cropping allowed us to work with less data without affecting the shadow.

B. Shadow Detection
We used image subtraction of the shaded image I B S (x, y) from the unshaded image I B (x, y) to isolate the shadow.An image difference calculation would not be sufficient, as it would give positive pixel values (returns absolute difference of pixels) even in the areas without shadow, which would not describe the shadowed area.The subtracted image I S (x, y) was then inverted, which is not mandatory for the success of the method.Still, we considered this step useful for ongoing visual evaluation of the steps taken, since it provided us with a more intuitive visual representation of the shadow.Inverting the image led to the need to use inverse binary thresholding in the following step, while binary thresholding would produce identical results if the input images were only subtracted and not inverted.
When searching for clusters of black pixels in the binary images, we could not always find a distinctive cluster at each Y thr .In these cases, there were no black pixels in the image, which means that the shadow was not reproduced at these specific pixel values.In these cases, we set the coordinate values ( x, ȳ) to (0, 0).In some cases, however, several clusters were found, but these occurred randomly and were of small size.We were able to eliminate these by defining the most prominent contour in terms of the area size of the cluster representing the shadow shape at the specific Y thr .

C. Shadow Segmentation
The method provided a series of 256 coordinates for each tested combination of light variables, where each coordinate represented the weighted center of a shadow shape at a specific brightness level.To better understand the data obtained, we have plotted them in a 3D diagram as shown in Fig. 8h.We can see that some coordinates are at (0, 0), Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.mostly at the lowest Y thr values.After the lower range of the threshold (Y thr ) with coordinates at (0, 0), a certain dispersion of the data can be observed, followed by a section of the data sets that show some continuity, and then again a dispersion at the highest thresholds, i.e. the brightest parts of the image.
The observed continuity of the results showed a similar trend in the data sets corresponding to the tested lights that only differ in the intensity of the light source.This prompted us to overlay the results and disregard the Y thr , as can be seen in Fig. 9, where we can observe similar or completely consistent trends in the data sets per light source combination.The data for lower light source intensities appear sparser than for higher light intensities, which is due to a larger proportion of the dataset with ( x, ȳ) at (0,0).Nevertheless, the coordinates overlap regardless of the light source intensity.This indicates that the shape of the shadow is independent of the light source intensity and that the lower harshness of the light at lower intensities, which is usually perceived visually as in Fig. 8a-g, is only a consequence of the lower contrasts and not a physical change in the light.
In Fig. 9, the previously described scattering of the data is still evident.However, two types of scatter can be observed: random and repetitive.The repetitive scatter can be seen in every data set and only occurs at the lowest and highest Y thr values where x and ȳ are not in (0, 0).The scatter at the maximum values of x and ȳ shows no trend but is limited to the darkest parts of the shadow, where we would expect no change in shadow shape as the area should be completely shadowed by the occluder.This indicates image noise in the darkest parts of the image, where the pixel values are close together and appear visually identical, but still slightly different in terms of signal.Similar observations were made for the largest Y thr used to analyze the brightest parts of the image.For all samples except H_S, we can see a trend similar to the quadratic equation.We believe that this noise is due to the removal of the background, either due to the surface roughness of the projection plane or due to camera distortion.
Since this is a pixel-based method, we expected a certain amount of random noise.We believe that the previously described noise is acceptable if it is removed efficiently and does not affect the quality of the data.Nevertheless, there are data points that do not fit the repetitive noise.One random coordinate pair can be seen for H_H, one for L_H, one for L_S and three for X_H.However, these coordinate pairs are isolated cases and are clearly visible.In addition, they occur in samples with lower light source intensity, where the data set is sparse due to lower contrast values.
All ( x, ȳ) coordinates that were recognized as noise were replaced by (0, 0).We then combined the data sets of the different light intensities for each light source combination and disregarded the (0, 0) coordinates.In this way, we obtained data sets that described the shadow shape for each light source combination, as we had previously found that it was not affected by the light source intensity.Furthermore, by combining the data sets, we increased their density and gained more accuracy in their descriptive properties, as the combined coordinates rarely overlapped but mostly filled the gaps between data points from other datasets.
We smoothed the combined data sets to allow B-spline fitting to ensure final comparability of the results, as the curves were fitted to the same data set sizes.The fitted curves (g( x)) can be seen in Fig. 10, where it can be observed how they describe the shadow shape shown in the background of the plots.The smaller the difference in the x coordinates of the curve, the sharper the shadow appears, and vice versa.Visually, we would rate H_H as the harshest and most direct light source, followed by X_H.L_H appears slightly harsher than L_S, but with minimal difference.X_S and H_S appear the softest, with the latter having the greatest dispersion along the x-axis.
To describe the harshness of the shadow and thus the harshness of the light source H more precisely and numerically, we have defined the point of change S u/ p that enables the shadow to be segmented into umbra and penumbra.The value of the ỹ coordinate at the point of change already partially describes the shadow harshness.The lower the value, the harsher the shadow appears.This again corresponds to the shadow harshness described earlier, with H_H being the sharpest, followed by X_H, L_H, L_S, X_S, and H_S, with similar relationships observed between the samples.Visually, we would refer to this shadow property as the umbra height.The umbra height values U h are shown in Fig. 11a as a function of penumbra width P w , being the change in the value of the x coordinate from the point of change to the end of the shadow (lowest x and ỹ coordinate values).The results for P w follow the reverse order of the results for U h , but on a different scale.Since this indicates a non-linear dependence, both metrics were included in the final light harshness assessment.
Results describing light harshness H are shown in Fig. 11b.The six tested light source combinations show that light sources H_H and X_H produce the harshest shadow, with values of 0.01 and 0.05, respectively.The slight difference in values is a result of the size of the light source, as the halogen lamp is a point-source light source, in respect to the occluder, while the xenon flash tube appears slightly larger and produces a double shadow.Our method does perceive this secondary shadow as a penumbra; however, the penumbra width value is counterbalanced by the umbra height, resulting in an accurate description of the quality of the light source.
The results for L_H and L_S appear close (0.29 and 0.42, respectively), while the tested light source combinations differ only in the use of a light-shaping attachment.Since the difference is much larger for other pairs of light sources with and without light modifiers (3.78 for halogen and 1.54 for xenon), we can estimate the effect of this light-shaping attachment in combination with this light source to be small, since the difference is 0.13.However, the tested LED light source is a panel with multiple LEDs.This way the L_H already has a high dispersion of the light beam itself.Therefore, we cannot predict how the light-shaping attachment would interact with other light sources, but it is originally intended for use with this particular light source.
X_S and H_S have the largest light harshness values of 1.59 and 3.79, respectively.This describes these two light source combinations as the softest among those tested.This is consistent with the visual evaluation of the images where we can clearly see a triangular shape of the umbra.This is a consequence of the light source becoming larger when the light-shaping attachment is used, illuminating the occluder from a much wider angle.Since this is one of the original mechanisms of how a softbox works as a lightshaping accessory, we can equate the change in the size of the light source with its harshness, which is the goal of this method.

D. Shadow Brightness Range
The observation that the shape of the shadow is not affected by the change in light intensity prompted us to investigate this finding further.Since the shadow boundaries, i.e. the start and end of the shadow, were determined, we were able to calculate the brightness range of the shadow.We normalized these values and presented them in Fig. 12 together with the normalized brightness values of the light sources.
Overlayed normalized values for the lights that only vary in light intensity show identical trends.This shows that the shadow brightness range is directly affected by the light source intensity, or in other words, that the light source intensity affects the contrast between the shadow and the background, but not the shadow shape, since the data from Fig. 9 overlaps despite different light intensities.
The analysis of shadow region shown in Fig. 12 proved that overlaying the brightness values can also serve as a tool to determine the success rate of shadow detection for this method.The matching results show a high success rate, however there are differences in the results.The cases where the shadow value is greater than the light value indicate that a larger portion of the surface has been identified as shadow and that some noise may still be present.When the light value is higher than the shadow value, too many data points in the data set may be classified as noise.
H_S in Fig. 12 shows the largest differences in normalized values, ranging from 0.3% to 18.1%.Upon additional inspection of the original and denoised data, we were unable to find any additional data points that would exhibit noise; therefore, we believe these differences are due to the light property measurements.It is well known that halogen light sources vary in brightness with excessive use as the filament inside burns.The light source measurements for H_H were taken immediately before the images were taken, while the images for H_S were taken with an additional pause between the measurements due to adjusting the light-shaping attachment.This may cause the brightness of the light source to vary between brightness measurement and image capture.In addition, Fig. 9 shows complete alignment of all data points within the light sources, except for H_S.This indicates that an element in this light source combination produces differences in shadow shapeeither the light source or the light-shaping attachment.Based on the comparability and matching of other results we assume the difference is due to changes in light source instability.
Nevertheless, a trend can be observed for light sources with different harshness.For the samples for H_H, L_H and X_H, i.e. the harsher light sources, the normalized values for light source brightness are higher than the corresponding normalized values for shadow brightness.For samples for H_S, L_S and X_S, i.e. softer light sources, the results are reversed, i.e. the light values are lower than the shadow values.This could indicate that the method is less efficient when analyzing softer light sources or that the noise in images for softer light sources has not been efficiently removed.Higher light source intensities ( 5)-( 7) lead to better agreement between the two overlaid values, despite the harshness of the tested light source, suggesting that the method is applicable to softer light sources as well.Combined with the fact that higher light intensity values generate larger data sets due to the stronger contrast of the shadow, we again conclude that the method is most efficient when analyzing shadows generated with higher light intensity values.

E. Cross-Reference With a Geometrical Shadow Projection
We have compared the results of our laboratory experiment with the results obtained by analyzing images from a theoretical shadow projection.The images for the latter are shown in the first row of Fig. 13, while the second row shows the corresponding image of a shadow from our experiment at the highest intensity level of the light source and with an equalized histogram to match the visually perceived contrasts  between the samples.The shadow metrics and the harshness of the projected shadows are shown in Fig. 14 along with the previously presented results from our experiment.
We can see that the results from Fig. 14a for geometrically projected shadows follow a similar trend to the experimental values, but with some anomalies between the corresponding samples.All three combinations of light sources without a light-shaping attachment (H_H, L_H and X_H) lead to higher values for the umbra height U h and the width of the penumbra P w .This would indicate that the umbra appears longer and the penumbra wider than perceived by our method, while all three samples have similar variations in U h and P w .However, when comparing the results for combinations with attached light modifiers (H_S, L_S and X_S), this trend is no longer apparent.For samples L_S and X_S, the theoretically determined values are lower for U h and higher for P w , while the opposite is true for H_S.These differences in the values are then reflected in the final evaluation of harshness in Fig. 14b, where the values for H pr oj largely correspond to the values H for samples without light-shaping attachments, while they show large differences for samples with light modifiers in place.
In the visual analysis of the geometrically projected shadows, H_H appears to be the harshest, closely followed by X_H.This ratio is also observed in experimental images; in both cases, however, the penumbra appears smaller in practice than in the theoretical projection.Nevertheless, the harshness values H and H pr oj seem to be close to each other for these two examples, while values close to 0 indicate the highest level of the light source harshness.
The geometric projection for L_H shows a high umbra and a wide penumbra, which can also be seen in the corresponding equalized image.The similarity is also reflected in the harshness, as H and H pr oj have similar values.L_S shows a wider penumbra and a much lower umbra height than the harsher version of the light source (L_H) when looking at the theoretical projections.However, this is not so obvious in the experimental image.The height of the umbra appears lower, but the width of the penumbra is similar to L_H.This is reflected in the comparison of H and H pr oj for the example L_S, where H represents a harshness value that is more in line with the visually perceived value than H pr oj .
While the light sources H_S and X_S should produce a similar height of the umbra and a similar width of the penumbra based on the geometric projection, the experimentally determined values for these two samples differ considerably, which is reflected in the values for H and H pr oj .Visually, we can confirm that H_S shows a more gradual spread of light than X_S.Compared to sample L_S, X_S should be closer to L_S than to H_S on a harshness scale, which is reflected in the results for H , while H pr oj predicts a different ratio that does not match the visually perceived shadows.
All of these variations among harshness values for experimentally and geometrically obtained images show the complexity of shadow rendering in a real environment, especially in situations where light-shaping attachments are used to alter the original light beam.With relying solely on the geometry of a light source and a light-shaping attachment, it is not possible to accurately predict the transformation of the original light beam.

F. Method Evaluation and Further Research
Based on the visual comparison of the harshness values with the experimental and theoretical shadow images, we judge that the proposed method adequately describes the observed metric and provides the first quantitative insights into the harshness of the observed light.Nevertheless, there are parts of the method that could be improved and further explored to achieve full automation of the method.
Although the initial registration method has been improved by adjusting the brightness levels to achieve full applicability, further research is needed to assess the degree of distortion within this setup.Optical camera calibration could be added to the method if it would improve the quality of shadow detection.This could already reduce the noise in the largest thresholds.
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
Another possible improvement to the method would be image filtering to smooth the shadow gradients and reduce noise in the lower thresholds.However, a systematic study of possible smoothing filters should be conducted and their influence on the shadows analyzed.
While image smoothing would reduce the noise to a certain extent, the residual noise should be described mathematically and removed algorithmically to fully automate the method.Although this would have been possible with the current state of the method, we found that our sample size is too small to define conditions that would efficiently limit noise and allow its removal, as we would only define it to 6 specific lighting conditions, since the intensity of the light source plays no role in noise generation.
Although we have merged the data sets for light sources with different intensities, this step is not mandatory for the method to work.However, the method is more efficient when the data sets are larger, as they are also denser.We suggest using the method for light sources set to maximum brightness to enhance the contrast between the shaded area and the illuminated area or taking several images with different brightness levels and merging the data sets according to the method.
The method appears to be efficient regardless of the type of light source while being specific to the observed scene.The results of the method are only applicable and comparable if the method and the experimental part are performed as proposed, suggesting that it is only applicable as a standardized method for the analysis of light sources.
The proposed metric of harshness does pose a challenge when it comes to evaluating its validity, as it cannot be directly compared to shadow information obtained with theoretical geometric projections of light and an occluder, due to possible interactions of materials in the path of light rays (such as light-shaping attachments).The potentially most problematic part of the method is the isolation of the shadow, but we have not been able to evaluate its success rate using a predefined method.In the development of CNNs for automatic shadow detection, the Balanced Error Rate (BER) is often used to evaluate the success rate of shadow detection.However, to calculate the BER, ground truth data about the position of the shadow is required.This data is not available in our study since we wanted to avoid a visual definition of shadow boundaries, as our visual perception of shadow boundaries can be inaccurate due to varying contrasts, as shown in the study.However, we were able to evaluate the success rate of shadow detection by comparing brightness values, which showed promising results.

VII. CONCLUSION
The aim of this research was to develop a method to numerically describe the quality of the light source, i.e. its harshness.The method developed proved to be efficient in its task, as it provides a single value to describe this light property.
The obtained value presents itself without a unit and is somewhat arbitrary at this point since its limits are not yet defined.However, all steps within the method ensure the comparability of the results when the first test images are taken as predicted by the method.This shows that the method could be standardized and the result of the method became one of the standardized metrics in describing light source properties such as the power of a light source, the color rendering index and the correlated color temperature.

Fig. 1 .
Fig. 1.Shadow cast by a point light source that produces hard shadows and a non-point light source that produces soft shadows.

Fig. 2 .
Fig. 2. Main steps of the method (for sample L_H_7; follow Section V-A): (a) reference image, (b) input photograph with detected ArUco marker, (c) aligned image with indicated cropped area, (d) subtracted image, (e) inverted image representing the shadow, (f) shadow with drawn contour and its centroid at threshold 128, (g) centroid at different thresholds forming the shadow gradient.

Fig. 3 .
Fig. 3. Scheme of the test scene (lateral and top view).

Fig. 4 .
Fig. 4. Sequence of actions in image registration algorithm.
id ) of the corners of the detected markers and the identifiers (IDs) of the markers.

Fig. 7 .
Fig. 7. Emission spectra for seven light intensity values for each light source and light quality combination: (a) hard halogen, (b) soft halogen, (c) hard LED, (d) soft LED, (e) hard xenon, and (f) soft xenon light.

Fig. 11 .
Fig. 11.Shadow metrics and the final results for light source harshness.

Fig. 14 .
Fig. 14.Shadow metrics for captured and geometrically projected shadows in (a) and the final results for light source harshness obtained through measurement (H ) and geometrical projection of shadows (H pr oj ) in (b).