Sketching and Stacking With Random Fork Based Exact Signal Recovery Under Sample Corruption

In this paper, we propose a new technique for exact recovery of missing data due to impulsive noise in time-domain sampled acoustic waves, named as sketching and stacking with random fork (SSRF). Existing methods recover the original signal from the corrupted sequence based on the correlation between each element of the signal or between the bases constituting the signal such as in the field of interpolation or compressive sensing. In addition, a partially corrupted signal is retrieved through statistical approaches using a large amount of pre-measured data in the machine learning domain. As a new approach, we hypothesize that if there is a novel method of re-sampling and processing, which can extract the intrinsic information of a corrupted signal, then the original signal can be recovered without information loss. The mathematical backgrounds in our study are twofold; first, a signal made up of overlapped <inline-formula> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> damped sinusoidal waves can be represented as the superposition of <inline-formula> <tex-math notation="LaTeX">$2k$ </tex-math></inline-formula> number of geometric sequences according to Euler’s formula; second, this superposed signal can be decomposed into individual geometric sequences by well-transformed <inline-formula> <tex-math notation="LaTeX">$2k+1$ </tex-math></inline-formula> informative matrices. The proposed method is to extract the parameters of <inline-formula> <tex-math notation="LaTeX">$2k$ </tex-math></inline-formula> geometric sequences by transforming the non-corrupted samples into <inline-formula> <tex-math notation="LaTeX">$2k+1$ </tex-math></inline-formula> informative matrices, and retrieve the corrupted samples by extrapolating the obtained parameters. We reveal the condition where SSRF perfectly fulfills the exact reconstruction, and verify the quality of reconstruction by comparing the results with conventional schemes.


I. INTRODUCTION
Recently, the scope of acoustic signal usage continuously broadens with advances in signal processing techniques. While acoustic signals were mainly used for the sound source itself in the past [1]- [3], the state-of-the-art processing of acoustic signals has shown some promise in terms of information gathering for localization [4]- [7], room shape detection [8], indoor mapping [9] and sound based tracking [10], [11].
To guarantee accurate results of acoustic signal applications, the acquired acoustic signal must be lossless and noise-free as much as possible. However, in numerous applications, one has to deal with recovering the sampled signals corrupted by various types of noises, e.g. thermal, The associate editor coordinating the review of this manuscript and approving it for publication was Ananya Sen Gupta .
impulsive and white/pink random noises [12]. Among these noise types, the presence of impulsive noise severely impairs the quality of the original signal, since it distorts particular samples to an unrecognizable level.
To resolve this problem to recover the original signal, various methods have been studied as follows:

A. INTERPOLATION
The most traditional signal recovery method is the family of interpolation methods which are based on the basic functions, e.g. linear, cubic, logarithmic, and exponential functions [13]. The basic concept of interpolation is to recover the lost parts of the sampled signals using their adjacent samples. For example, cubic interpolation connects the adjacent samples with 3-rd order cubic polynomial, which is smoother than linear interpolation [14]. However, the interpolation technique itself cannot be a universal method because it is difficult to VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ accurately model the correlation between adjacent samples and extract the intrinsic parameters of signals.

B. SINE WAVE FITTING
Relying on the fact that most acoustic signals can be represented as a sum of multiple sinusoidal waves, sine wave fitting is another method to recover acoustic signals. Assuming that the signal does not change its frequency components in the sampling window, sine wave fitting can be effectively used to recover signals under the influence of noise [15]. However, the algorithm must go through extensive iterations [16] and is highly dependant on acquiring a long enough sampling window for robust performance [17].

C. COMPRESSIVE SENSING (CS)
The development of CS has paved the way to a new era of signal recovery [18]- [21]. Unlike precedent signal recovery methods based on the Shannon-Nyquist theorem, CS enables the recovery of sparse signals by changing the dimension of the sampled signal, which makes this method compressive. However, in the case of acoustic signals, their frequencies are randomly generated in the continuous domain. Therefore, even with a small number of acoustic waves, it is quite difficult to form a complete sparse vector and is possible to deal with an approximate sparse vector with energy dispersion. In addition, algorithms based on linear correlation for implementing CS, e.g. orthogonal matching pursuit (OMP), have a philosophy of selecting the basis with the highest correlation sequentially. Therefore, it is obvious that it is vulnerable to interference due to the non-orthogonality among acoustic waves. As a result, the required number of bases must be numerous, which in turn requires high computational complexity, in spite of substantial reconstruction error.

D. DEEP NEURAL NETWORK (DNN)
With the soaring popularity of machine learning, there has been many approaches of combining machine learning methods with signal recovery [22]- [25]. It contains the mechanism of training a neural network to make noisy samples as similar to the original signal as possible. It is based on statistical inference to learn how data is missing. It is very effective in recovering the original signal, but it obviously requires pretraining with a huge number of data sets, which is highly challenging in practice.
To overcome the limitations of the conventional algorithms mentioned above, we propose a novel SSRF method of sparse acoustic signal reconstruction. We deal with the partially corrupted acoustic signal, which was originally combination of several sinusoidal signals. The corruption of the samples by the impulsive noise, is modeled as an independent Bernoulli distribution for each sampling index. We mainly focus on how to re-sample the observed signal and extract the intrinsic parameters for recovering the original signal without information loss. For this, we utilize two mathematical facts: 1) a damped sinusoidal wave can be decomposed into two different complex-valued exponential functions whose gains are the same and the exponents are complex conjugates of each other, 2) an equidistant sampled exponential function can be represented as a single geometric sequence. The main advantage of our proposed scheme is the ability to restore a corrupted signal with the novel processing of sketching and stacking, i.e. it does not require a large dictionary of bases or large amounts of training. The reason for the naming of this scheme as SSRF is based on the re-sampling and augmentation with arbitrary indices of a re-sampler, i.e. a fork. Furthermore, we find the condition in which the corrupted samples can be perfectly retrieved with the proposed scheme.
The rest of this paper is composed as follows. In Section II, the signal model is introduced with mathematical descriptions. In Section III, the devised SSRF methodology is explained and the simulation results and performance analysis follows in Section IV. Finally, Section V presents the concluding remarks.
The following symbols will be used throughout the paper: • ∅: the empty set.
• v l (t): the l-th wave at time t.
• k: the number of superposed waves.
• t: the time interval for sampling.
• P: the number of samples. • s(v; t) ∈ R P : the sampled series of v(t) whose length is P with the time interval t.
• ψ m ∈ N m 0 (random fork): an arbitrary collection of lexicographically ordered m indices where m ∈ N.

A. SIGNAL MODEL
The acoustic signal from the source is the summation of k damped sinusoidal waves, v(t), which can be represented as follows: where V l , γ l , and f l are the peak amplitude, the damping factor, and the frequency of the v l (t), respectively. Then, we can define the original sequence of samples, s(v) ∈ R P , as below: where the time interval for sampling, t, is set to meet the Nyquist theorem, i.e. 1 2 t > max{f 1 , · · · , f k }.

B. CORRUPTION BASED NOISE MODEL
To consider the corruption of s, let us assume that the probability of corruption for each sample follows the Bernoulli distribution B(p) which takes the value ∅, i.e. corruption, with probability p and the value 1, i.e. non-corruption, with probability 1 − p. In addition, let us denote b ∈ {∅, 1} P to the realization of B(p) over P samples of s, i.e. if p = 1, then all elements of b become ∅. Naturally, the corrupted sequence, c ∈ R P , can be represented as follows: where is the symbol of Hadamard multiplication.

C. PROBLEM STATEMENT
The objective is to recover the original signal s with the corrupted signal c. For this, we estimate the wave parameters, V l , γ , and f l for all l, with c, and extrapolate them to reconstruct s. For simplicity, we assume that k is known a priori.

III. METHODOLOGY OF SSRF: EXACT RECONSTRUCTION OF CORRUPTED SAMPLES
This section provides the general methodology of SSRF. The concept of SSRF is depicted in Fig. 1.

A. GEOMETRIC SEQUENTIAL REPRESENTATION
Before describing SSRF, let us look at the interesting mathematical properties of s toward the geometric sequential representation. Recalling Euler's formula as follows for arbitrary A, θ ∈ R: where i is √ −1. It implies that an equidistant sampled wave can be represented by two geometric sequences, whose initial terms are identical and common ratios are of complex conjugate relationship. From this, we substitute V l 2 and e −γ l t+i2πf l t to α l and β l , for all l, respectively. Then, s can be rewritten as follows: where (·) * is the operator of complex conjugate. Thus, the problem is reformed to obtaining the parameters of geometric sequences, α l ∈ R and β l ∈ C for all l, with c, i.e. reconstruction of 2k geometric sequence using randomly corrupted observed sequence.

B. DESIGN OF RANDOM FORK
To obtain the wave parameters, V l , γ l , and f l , i.e. the geometric sequential parameters, α l and β l , for all l, we first design a random fork, ψ m , as follows for an arbitrary m ∈ N: Definition 1: ψ m ∈ N m 0 is an arbitrary collection of lexicographically ordered m indices. This is the set of m-indices for re-sampling. As implied from the name of random fork, this is a technique of randomly extracting and arranging m element values of an arbitrary sequence.
Given an arbitrarily corrupted sequence c and the random fork ψ m , let us define the forked samples, ψ m,c ∈ R m , as follows: For convenience, let c(ψ m ) be equal to ψ m,c , thus we will use both terms interchangeably throughout this paper.

C. DATA AUGMENTATION FOR CONSTRUCTING SEARCH SPACE
Next, we make the informative matrix, m,c ∈ R m×(m+1) , by stacking the nonidentical m+1 number of c(ψ m ) as follows for an arbitrary m ∈ N: (7) where 1 m is the one-vector whose length is m and [A|B] is the stacking operator of vectors (or matrices) of A and B whose dimension of column are equal each other.
We further define the combinatorial informative matrices, [j] m,c ∈ R m×m , for j ∈ {0, · · · , m} as follows: where [j] m ∈ R (m+1)×m is a column-collecting matrix capturing all columns excluding the j-th columns. For instance, [1] 3 is equal to m,c are non-corrupted samples. It enables us to decompose [j] m,c in a convenient way. This property of linear algebra supports the approaches in the following subsections.
1 A homogeneous polynomial is a polynomial whose nonzero terms all have the same degree. For example, x 6 + x 5 y + xy 6 + y 6 is a homogeneous polynomial of degree 6, in two variables; the sum of the exponents in each term is always 6.
Since the rank of all sub-matrices is 2k, c j can be represented as follows: . (14) For brevity, let us substitute V [j] 2k for V j . To obtain det(V j ), we utilize the permutation property of determinants. In V j , for any j, let us switch two arbitrary common ratios, β n and β m . This switch between β n and β m is equivalent to the swap of the n-th and m-th rows regarding the form of V j . Then, the sign of the determinant will be changed from the original one. It implies that det(V j ) includes the factor of (β m − β n ). Furthermore, all possible (β m − β n ) with the condition of 1 ≤ n < m ≤ 2k are included, i.e. 1≤n<m≤2k (β m − β n ).
From this permutation property of determinant, we can further obtain As a result, the final form of {c j } 2k j=0 in (14) can be represented as follows: Here, (16) is the coefficients of polynomial whose roots are {β 1 , · · · , β 2k }, which concludes the proof.
In addition, it is trivial to extract {α 1 , · · · , α k } by a simple matrix inversion after replacing {β 1 , · · · , β 2k } in (10). Each pair of the initial term and the common ratio, i.e. {α j , β j } for all j ∈ {1, · · · , k}, is matched through this matrix operation, and thus there is no paring problem between the initial terms and the common ratios.
Condition 1: Given c, if there exist at least one ψ 2k satisfying the following relationship, then s can be perfectly retrieved by only using c.
where {·} is the operator making a set consisting of all elements of an input. Finally, Algorithm 1 summarizes the SSRF procedure.

IV. PERFORMANCE ANALYSIS
In this section, we evaluate the performance of the proposed SSRF solution in terms of mean squared errors (MSE) of reconstruction. ii) Re-sample c using ψ 2k and construct 2k,c as (7).

A. SIMULATION SET-UP
For a performance comparison, we choose the algorithms of CS and DNN, which are briefly explained in Section 1. The reconstruction method based on CS is implemented by the OMP algorithm with the partial discrete cosine transform (DCT) matrix. The number of bases in the CS based method is 5000. Next, the DNN based reconstruction method is utilized, where the number of training data set is 30000. In addition, the number of hidden layer and the number of perceptrons in each layer are 2 and 40 respectively. The sigmoid activation function is considered in neural network model. In addition, MSE is selected as the loss function in the DNN algorithm. For optimization, scaled conjugate gradient method is applied.
The simulation results are based on Monte Carlo simulation experiments with 10000 cases. The parameter settings for performance comparison is as follows. The peak amplitude V l follows the Normal distribution with zero mean and 1/ √ k as variance. The damping factor γ l follows uniform distribution U(0, 10 3 ). In addition, the frequency follows uniform distribution with U(0, 10 kHz). The sampling time interval t is set to 0.5 × 10 −4 second, and the number of samples P is 30. Fig. 2 shows the cumulative distribution function (CDF) of reconstruction error of each algorithm according to different k and p. The representative challenge of that CS method is that it requires large number of bases due to the generation of f l and γ l in the continuous domain, i.e. there is no guarantee of the orthogonality among superposed signals. In addition, the shortcomings of DNN based signal reconstruction is that it requires an enormous training data set for statistical inference because of the binary corruption affecting the samples. Here, we remark that the SSRF method is superior to CS and DNN even though there is no dictionary of bases and training data sets. For all cases of k and p value, the SSRF method shows less reconstruction error than CS and DNN algorithms. In addition, if condition 1 is satisfied, the SSRF method result in the exact signal recovery. As shown in Fig. 2 (a) and (d), it is interesting that the SSRF technique shows the capability of signal reconstruction close to perfection in an environment with high probability of satisfying condition 1.

C. DETAILED COMPARISON BETWEEN SSRF AND DNN
More intensive comparison between SSRF reconstruction and DNN methods according to the number of training data VOLUME 8, 2020   (M ) is shown in Fig. 3 and 4, where the number of perceptrons in each layer (H ) is fixed to 40. To describe the statistical aspects, the box plot is utilized. In each figure, the top and bottom line of the blue box represents the first and third quar-tiles. In addition, the red horizontal line in the box represents the median value. The dotted vertical line represents the range of data excluding outliers. For all sub-figures in Fig. 3 and 4, as the number of training data gets bigger, the median value of the reconstruction error gets smaller. With more training data, the accuracy of DNN increases. However, given the extremely low reconstruction error of SSRF as seen in Fig. 3 and 4, increasing M cannot excel the SSRF performance. With same k value in Fig. 3 (a) and (b), reconstruction error increases as p gets larger both for SSRF and DNN based reconstruction. Then for a fixed p as in Fig. 4 (a) and (b), reconstruction error gets slightly higher as k gets bigger.
Next, comparison between SSRF and DNN methods for reconstruction according to number of perceptrons in each layer (H ) is shown in Fig. 5 and 6, where the number of training data set is fixed to 3 × 10 4 . As shown in Fig. 5 and 6, reconstruction error decreases as the number of perceptrons increases. However, the increase in the number of perceptrons does not dramatically decrease the reconstruction error. Even in the case of Fig. 5 (b), DNN with H = 40 and H = 60 show almost similar performance in terms of each median, the first and third quartiles. As a result, SSRF results in higher performance in terms of the distribution of reconstruction error regardless of k, p and H .

V. CONCLUSION
In this paper, we introduced a novel method of recovering a partially corrupted signal, namely SSRF. For the proposed method, we hypothesized that if there is a novel method of re-sampling and processing, which can extract the intrinsic information of a corrupted signal, then the original signal can be recovered without information loss. Here, we utilized two mathematical backgrounds to support our hypothesis as follows: first, a signal made up of overlapped k damped sinusoidal waves can be represented as the superposition of 2k number of geometric sequences according to Euler's formula; second, this superposed signal can be decomposed into individual geometric sequences by well-transformed 2k + 1 informative matrices. As a result, we converted the problem of decomposing 2k geometric sequences into a solving of a 2k-th order polynomial equation to extract the intrinsic information of the original signal from the corrupted signal. In addition, we employed the required condition that the corrupted signal can be perfectly retrieved in terms of the indices of non-corrupted samples and the number of superposed signals. Furthermore, we validated that SSRF outperformed in terms of the reconstruction error compared to CS and DNN based algorithms, even if SSRF does not require a dictionary of bases or a data set for training. For future works, we believe that the SSRF technique can be applied to more diverse research fields of signal processing such as wireless communications, automotive radar systems, and image inpainting.
JOO HYUN PARK received the B.S. degree in electrical and computer engineering from Seoul National University, Seoul, South Korea, in 2016, and the combined master's and doctorate degree in electrical and computer engineering from the Graduate School, Seoul National University. Her research interests include related to acoustic wave and its applications, e.g., acoustic signal processing, sound source localization, and so on.
WANJEI CHO received the B.S. degree in electrical and computer engineering from Seoul National University, Seoul, South Korea, in 2019, where he is currently pursuing the Ph.D. degree. His current research interests include radar signal processing, wireless channel modeling, acoustic channel modeling, and DNN applications.