Corrections to ‘‘The Structural Information Potential and Its Application to Document Triage’’

In the above article <xref ref-type="bibr" rid="ref1">[1]</xref>, a few of the sections require corrections and some additional experiments to strengthen the work.

. Its publication revealed how multiple U.S. administrations misled the public on the reasons for and actions taken in the Vietnam War, and had consequential repercussions on the legal aspects of freedom of speech and journalism in the United States. The ''papers'' consisted of more than 7,000 pages of documents that were covertly photocopied by Daniel Ellsberg of the RAND Corporation and made available to the press in 1971. Among the major practical problems with which the journalists had to contend was that of rapidly identifying the most informative content and collating related pages so that articles could be finished before editorial deadlines [3, p. 392], [4]. Layout-based classification is a sensible strategy for both tasks under such circumstances. The difficulties posed by document triage for the evaluation of the Pentagon Papers were fictionalized in a memorable scene of the film The Post (2017)

3) PSYCHOPHYSICAL EXPERIMENTS
Our understanding of SIP and other pattern-ordering methods may be deepened beyond what is possible through the use of visual evidence and quantitative evaluations, on the basis of empirical insights obtained from psychophysical experiments. The primary goal of experiments such as those described below is to evaluate the agreement between computational and perceptual pattern orderings, which is especially important for SIP as a model for human behavior. The discovery of further insights into the perceptual, cognitive, and other practical aspects of SIP-for example, in the context of the triage task-represents a secondary goal of the experiments. Experimental design-The participants in the experiments were 10 students in digital humanities from the University of Bern, Switzerland (4 females, MD = 32 years, RG = 22-43, SD = 5.98). The stimuli consisted of the 104 pages of the New Yorker magazine described above [1, Fig. 31], each reproduced on a 3-by-4.5 cm (1.18-by-1.77 in) piece of paper. This size ensured, on one hand, that all pages could be laid out within the confines of a small table and viewed at a glance; on the other hand, the small size helped to focus on the overall layout pattern, reducing the salience of semantic information such as text and picture content. To further emphasize the pattern and inhibit reading, the reproductions were placed upside-down.
The concept of the uniform-clustered-regular pattern spectrum was explained to the participants with reference to representative page samples ([1, Fig. 10] a, b, d) and through discussion. To ensure that the concept had been effectively assimilated, a practical training exercise was conducted in which the participants were required to order 16 sample documents (uniformly selected from the library cards shown in Fig. 21), with corrective inputs from the experimenter.
Ordering experiment-This experiment aimed to evaluate the suitability of SIP as a model of human pattern ordering along the uniform-clustered-regular spectrum. To this end, the participants received a stack of randomly ordered stimuli and were required to order the page reproductions along the uniform-clustered-regular spectrum.
The differences between the perceptual and SIP orderings were quantified using the topological entropy method, and the statistical summary is presented in Fig. 3b We were able to enhance our comprehension of the results by adopting a qualitative approach, based on spontaneous remarks made by the participants when commenting on the experiment, observations of their behavior, and postexperiment debriefings. First of all, participants commented on the tediousness of the task: in terms of the need to physically arrange so many stimuli, in cognitive terms (the difficulty associated with deciding on their proper order), and in motivational terms (due to the duration of the operation, ca. 20 to 30 minutes). Second, it was difficult for the participants to order the stimuli purely in terms of their layout without interpreting ink structures in terms of their semantic classes, such as pictures and titles, despite that some of these were identified as such precisely because of the layout that made them stand out. Third, the ordering of clustered patterns was more challenging than that of uniform and regular patterns; the latter could be intuitively comprehended as ''mostly empty surfaces'' and ''homogeneous ink distributions,'' while the concept of scale-space clustering was less familiar. Perhaps the results would have been different with participants drawn from other demographics, such as document designers, artists, and computer scientists.
In conclusion, the foremost finding from this experiment is the difficulty of perceptual ordering for various practical and cognitive reasons. It also provides evidence for the poor performance of human pattern triage. Since the experiment was more relevant for revealing human factors in pattern ordering than for allowing an evaluation of SIP as a perceptual model, we designed a second experimental setup to address these issues.
Improvement experiment-To diminish the influence of factors affecting free perceptual ordering, in this experiment, the participants were presented with a fixed order of stimuli (the one produced by SIP) and asked to improve this ordering by shifting the page reproductions so as to create a transition from uniform to clustered to regular layouts that they deemed to be perceptually the most ''smooth.'' The numerical results, with median topological entropy of 0.38, now indicate a smaller, albeit non-trivial, divergence between computational and perceptual orderings for the improvement task relative to the case of the free ordering task (M = 0.39, 95% CI = [0.33, 0.45]; Fig. 3c).
The effect size of the participants may be better appreciated through a visual examination of the resulting patterns. Unlike the numerical assessment, this approach reveals that the improvements made were comparatively limited in terms of their number and the distance of pattern shifts along the spectrum. Fig. 3g shows a representative example of the improvement produced by participant MF, which is the closest to the median divergence from SIP; clearly, this ordering is quite similar to SIP (Fig. 3h) and far from the median ordering of other methods (Fig. 3e). In terms of qualitative results, the participants stated that this task was easier than the former, and, indeed, they completed the experiment faster (in ca. 15 to 20 minutes). It appears from this experiment that SIP-induced ordering is very much accepted by humans as producing a smooth transition between uniform, clustered, and regular patterns. It also greatly facilitates triage.
Rating experiment-In this last experiment, we intended to compare on a perceptual basis the smoothness of the pattern spectra induced by various computational methods, which we assessed from a formal quantitative point of view in the preceding section, ''Quantitative Evaluation'' [1, Fig. 19]. To do this, the participants were requested to rate on a five-level scale the smoothness of the five analyzed computational orderings, along with the physical ordering of the pages, based on printed copies of Figs. 31-36 from [1].
The results are presented in Fig. 4. The rating aggregation method employed was majority judgement, chosen because it is a non-parametric ranking method that favors consensual candidates and is based on median scores rather than means [5]. Clearly, not only does SIP produce a highly consistent and smooth uniform-clustered-regular pattern spectrum, but all other methods are also given lower consistency ratings.
General assessment-In conclusion, SIP is a good model of perceptual pattern ordering along the uniform-clusteredregular spectrum, as well as aiding greatly in pattern-based triage. The psychophysical experiments described herein confirm and expand, from an empirical point of view, the conclusions derived from visual evidence and formal quantitative evaluation. In the next section, we evaluate SIP on a much larger dataset containing 1.2 million stimuli.  Fig. 19], as well as the physical order of pages in the document. The ratings are aggregated over 10 participants using the majority judgement method and a five-level rating scale.

5)
In the above article [1, p. 13130, Section V-B],'' add after §1 of col. 2, the following paragraphs and figure. The purpose of this addition is to provide an analytical explanation VOLUME 11, 2023 85713 Authorized licensed use limited to the terms of the applicable license agreement with IEEE. Restrictions apply. The power spectrum of the signal is shown in (a). Note that the amplitude distribution is not flat, as is expected from white noise. (c) Welch's power spectral density estimate of the signal is shown in (a), in which the signal is divided into 200 overlapping windows of equal length, filtered with a Hamming window [6], [7, pp. 415-417]. Note the noisy Gaussian distribution of the amplitudes and the flattening of their sorted plot, which leads to an increase of the Shannon relative entropy, H r , with respect to that of the power spectrum of (b). (d) Averaged power spectrum of 100 white noise signals.
of the randomness phenomena affecting the measurement of SIP.
In fact, the power spectrum of white noise with uniform random amplitude is theoretically flat, i.e. equal at all frequencies, and thus has maximal entropy [7, p. 385]. However, for a finite signal, the power spectrum distribution has a power law shape (Fig. 5 a, b). As in the case of run lengths, this is due to the unequal number of cycles per frequency available to compute the power spectrum. Various methods have been developed to overcome this limitation, such as averaging the power spectra of multiple random signals or averaging overlapping windows of the same signal (Fig. 5 c, d) [7, pp. 371-644], [8]. The distribution of the averaged amplitudes of the power spectra will now converge to a Gaussian, as per the central limit theorem. Increasing signal length also increases the distribution variance, with the power spectrum becoming uniform in the limit and its entropy maximal, as expected. The flattening of the power spectrum also characterizes the SIP method, since it comprises an averaging of the power spectra of all orientations of an image.
In conclusion, uniform random-amplitude white noise patterns of finite extent naturally exhibit substantial homogeneity, while their measured entropy is further increased through spectral averaging as part of the SIP method. 6) In the above article [1, p. 13132, col. 1, Section V-G],'' replace §1 with the following paragraph. The purpose of this correction is to extend the discussion of SIP in relation to psychology.
The field of perceptual organization [9] studies from a psychological perspective many aspects related to SIP, among them structure [10], informativeness [11], entropy [12], uncertainty [13], redundancy [14], complexity [15], randomness [16], Gestalt Prägnanz [17], scene statistics [18], illusions [19], attractivity [20], and aesthetics [21]. Configurations are, however, difficult psychophysical research stimuli; this is more so the case for real data, such as documents, where semantics, aesthetics, and user experience (among many other factors) play an important role in their evaluation. As the pattern ordering resulting from the SIP is primarily intended to have humans as end users, it is nevertheless desirable to investigate in future work the relevance of modifying the SIP method to account for human pattern perception.