Extracting Human Context Through Receiver-End Beamforming

Device-free passive sensing of the human targets using wireless signals have acquired much attention in the recent past because of its importance in many applications including security, heating, ventilation and air conditioning, activity recognition, and elderly care. In this paper, we use receiver-side beamforming to isolate the array response of a human target when the line of sight array response is several magnitudes stronger than the human response. The solution is implemented in a 5G testbed using a software-defined radio (SDR) platform. As beamforming with SDRs faces the challenge to train the beamformer to different azimuth angles, we present an algorithm to generate the steering vectors for all azimuth angles from a few training directions amidst imprecise prior information on the training steering vectors. We extract the direction of arrival (DoA) from the array response of the human target, and conducting experiments in a semi-anechoic chamber, we detect the DoAs of up to four stationary human targets and track the DoA of up to two walking persons simultaneously.


I. INTRODUCTION
Device free human sensing has gained interest because of its importance in many applications that includes human safety in industry settings [1], security and surveillance [2], heating, ventilation and air conditioning [3], and assisted living [4].Diverse technologies have been considered for human sensing: cameras [5], infra-red [6], visible light [7] acoustics [8] or wearables [9].Compared to these solutions, device-free wireless sensing has the advantages of reduced intrusion of privacy, performance in darkness and occlusion and not having the inconvenience of wearing a device.
Device-free passive human sensing is a challenging task because the signals scattered off the humans are several magnitudes weaker than that of the line of sight (LoS) signal.Previous attempts have looked at signal strength [10], direction of arrival (DoA) [11], time of flight [12] and Doppler shift [13], [14] or a combination of those [15] to extract the parameters of the channel for activity recognition, presence detection or localization and tracking.In this work, we use DoA as the channel parameter but extract the DoA from a novel approach.More specifically, we use beamforming at the receiver end and steer the beam to different directions to construct the array response to detect the human presence and to estimate the direction of that person.Unfortunately, current super-resolution algorithms that detect the DoA, e.g.multiple signal classification (MUSIC) [16], require the number of sources as prior information for an accurate estimation of the DoAs [17].Using beamforming, such problems can be mitigated as the DoAs of the interested targets can be simultaneously estimated from the amplitude of the array response using our approach.Additionally, by focusing the beam towards the direction of the person, activity patterns can be simultaneously recognized for multiple persons [18].However, beamforming has not been utilized for human sensing in the past mainly because of the requirement of transmitter (TX) and receiver (RX) phase synchronization and the requirement of large antenna arrays for high resolution.
Recent developments in 5G with massive MIMO have enabled beamforming with high resolutions.In our work, we utilize a 5G testbed [19] comprised of software-defined radios (SDRs).Beamforming with SDRs is still challenging because the beamformer has to be calibrated for different azimuth angles due to practical limitations like unknown wave propagation conditions and random phase shifts in RF chains.Moreover, training the beamformer for a larger number of azimuth angles is labor intensive and is not practical.We, therefore, use reference directions to train the beamformer by transmitting from a limited number of azimuth angles and measure the steering vectors at the receiver for those directions using the relative phases of the antenna array.The measured steering vectors can still be far from ideal due to, e.g.unknown wavefield propagation conditions and antenna array orientation and geometry errors.Therefore, we sanitize these steering vectors and using them as reference points, we generate the steering vectors for all azimuth angles using a mathematical model.
As contributions, i) we propose an algorithm to estimate steering vectors for all azimuth angles using the reference points even when steering vector mismatches occur.ii) We also provide an approach to detect the human presence from the array response, and iii) a method to estimate the direction of the human when the direct path from the transmitter is several magnitudes stronger than the scattered paths from the human.iv) By conducting experiments in a semi-anechoic chamber of size 22.4m2 , we achieve a 100% detection accuracy and a maximum error for the median direction of arrival of 7 • for a person performing in-place activities in five different locations of the area of interest.We perform experiments placing up to four persons in the given area and show that DoAs can be extracted from stationary persons when sufficient spatial separation among them is present.Finally, conducting experiments up to two walking persons we show that their DoAs can be simultaneously extracted.
The rest of the paper is organized as follows.Section II provides a summary of the most relevant device-free human sensing approaches.Section III introduces our signal model used for beamforming and explains how we construct the array response through beamforming using SDRs.In Section IV, we provide our solutions to problems that occurred due to practical limitations associated with SDRbased beamforming i) fixed phase offsets in RF chains, and ii) steering vector mismatch.In Section V we detect human presence from array response, extract the response of the human and the corresponding DoA.In Section VI we explain the configuration of our testbed, Section VII explains the experiment environment, data collection and evaluation of our algorithms.We discuss the scope and limitations of this work and provide our conclusions in Section VIII.

II. RELATED WORK
Our work is related to Device-free localization, tracking and activity recognition of humans using radio signals.Other competing technologies include vision-based solutions [5], thermal images [6], visible light [7] acoustics [8] or wearables [9].Vision or optoelectronic sensors used for people monitoring exploit the properties of reflectivity with timeof-flight (ToF) cameras, emissivity with thermal images or acoustic sensors.They are, however, constrained by range, occlusions, environmental conditions (smoke/fire), and privacy concerns [20], [21].Other solutions, which demand that people carry a device (Device-based), limit human subjects' mobility, visibility, and communication.Device-free RF-sensing solutions, which do not require people to wear any devices, and which rely on electromagnetic propagation, mitigate these challenges.The proliferation of radio devices inside the area of interest is therefore expected to boost the development of non-intrusive RF based localization, tracking and activity recognition [22], [23].
Among the most relevant approaches that perform devicefree localization and tracking, WiTrack [12] uses a custom made transceiver exploiting a wideband FMCW radar to estimate time delay of signals reflected from a human and a 5×5 antenna array with more than half wavelength separation between two elements to achieve spatial diversity.WiDeo [36] uses WiFi backscatter communication to jointly estimate ToFs and AoAs to localize humans.DynamicMusic [11] jointly estimates DoA and observed time of flight (reflected off human subjects) from channel state information (CSI) by a two dimensional MUSIC algorithm using commodity WiFi devices.However, the approach successfully detects only a walking person, whereas IndoTrack [31] extends this by incorporating AoA with Doppler information to localize a static and dynamic person.xd-Track [15] applies SAGE algorithm for multiple channel parameter estimation (signal strength, DoA, ToF and Doppler) using an SDR platform, thereby improving the resolution of multiple human target tracking despite heavy computation efforts and custom made hardware.Widar2.0 [37] extends this to commercial WiFi devices that uses CSI and further leverages multipath effects to enable absolute ToF estimation.
These approaches are relevant to our work in devicefree estimation of DoA for human tracking, however, our approach is different to all these in the way DoA is estimated.While current state of the art approaches use either an extended version of MUSIC algorithm or a variation maximum likelihood estimation using the relative phases of antenna arrays, we use beamforming techniques and construct the array response to isolate the path corresponding to humans and estimate their DoAs.We show that the approach can estimate DoAs of multiple targets without the knowledge of the target count in advance and the target resolution is only limited by the number of antennas.

A. SIGNAL MODEL
Here we use a point source narrowaband signal model and a uniform linear array (ULA) of M omnidirectional antenna elements.The narrowband signal received at time instant t by the ULA is modeled as where s(t), i(t) and n(t) are C M ×1 vectors representing the line of sight (LoS) signal, signal scattered off a person and noise respectively.Under these conditions i(t) is the desired signal and it can be amplified by finding the correct beamformer weight w.However, as the person is not always stationary, we cannot use a fixed weight as in beamforming used for fixed transmitter and receiver positions.Therefore, we construct the array response using the beamformer weights for all azimuth directions.

B. ARRAY RESPONSE OF A DELAY AND SUM BEAMFORMER
In this work, we use a 5G testbed (Sec.VI) consisting of software defined radios (SDRs).In our testbed, a 12.48 MHz channel is divided into 52 OFDM subcarriers each having a bandwidth of 240 kHz.In OFDM, a channel is divided into multiple subcarriers and data is modulated in each subcarrier.The in-phase and quadrature (IQ) samples of the 52 subcarriers measured at runtime are fed to the beamformer as a post-processing step to obtain N spatial streams of the array response such that signals arriving from direction i ∈ N are amplified at the i th spatial stream.
Here, we implement a delay and sum beamformer using OFDM beacons.Even though, other commonly used beamformers like minimum variance distortionless response (MVDR) or linearly constrained minimum variance (LCMV) beamformers improve the signal to interference plus noise performance, they achieve this by having the nulls of the beam at interference directions [38].This can be detrimental for human sensing, as we rely on the signals scattered off the humans behaving as the interference paths.
We implement the beamformer as follows.The transmitter sends a beacon with a known sequence a(t) each subframe.IQ samples x m (t) ∈ C are received by antenna element m ∈ M where M is the number of receiver antenna elements.
x m (t) can be represented as where due to the time delay τ m j to propagate to antenna element m where c is the speed of light and λ is the wavelength.We estimate φm j ≈ φ m j by computing the crosscorrelation r(φ m ), and φm j maximizes the correlation between transmitted and received beacons This approach is possible because a truncated version of frequency domain Zadoff-Chu sequence used in LTE is adopted here as A(f ) [39].The sequence was truncated because it was longer than the number of available subcarriers.The beamformer weight w i is computed using w i = p i /M where p i is the steering vector for azimuth direction i ∈ N .We apply the weight vector on r j (t) at time instant t where r j (t) = [r( φ1 j , t), r( φ2 j , t), . . ., r( φM j , t)] T to obtain array response a(t) where a(t) = [a 1 (t), a 2 (2), . . ., a N (t)].

IV. PRACTICAL IMPAIRMENTS TO BEAMFORMING AND OUR SOLUTIONS
Even though the array response can be computed as above in an ideal setting, practical limitations in real world testbeds introduce errors on the relative phase of signals among the received streams and the steering vectors.To tackle these problems, we implemented calibration techniques.

A. PHASE OFFSETS OF THE RF CHAINS
The problem of time-varying phase-offset due to clock drift is solved by using a separate transmitter that sends reference signals before receivers compute the phase difference between RF chains for run time compensation by [39].Phase offsets caused by non-identical antenna feed cables, connectors and other RF components are mitigated by first, performing phase measurements of each component via an external vector network analyzer and measuring a crude phase offset for each RF chain.Then, finding the phase offset α m in each antenna element by using the line of sight (LoS) direction as the reference.If a plane wave impinges on the antenna array when a signal is emitted from the LoS, the relative phases among the antenna elements are almost zero.We use this assumption to calibrate the phase offsets.We first model the measured signal xm (t) as xm (t) = x m (t) • e jαm where α m is the unknown phase offset at antenna element m.Then, we calibrate α m such that the steering vector q LoS of the LoS direction is q LoS = [0, 0, 0 , . . .0] 1×M .This calibration helps us in two ways: i) to mitigate fixed phase offsets in RF chains, and ii) to use the LoS direction as the reference direction for a signal coming from 90 • azimuth angle to the ULA.

B. STEERING VECTOR PROCESSING
In this work, we estimate the steering vectors and learn the beamformer weights from a particular azimuth direction by transmitting a signal from known anchor points.In practice, the steering vector associated with a plane wave impinging from a particular direction may not be known precisely.This results in a mismatch between the presumed and the measured steering vectors.This can occur due to imprecisely known wavefield propagation conditions, antenna calibration errors, antenna array geometry and orientation errors and/or signal pointing errors [40], [41].Therefore, we choose to first measure the phase rotations from signals transmitted from k reference directions, convert them to steering vectors and assign them as anchor points.Thereby, we derive the directions of the signals arriving directly or from positions near to those anchor points.Next, we use a model of the delay and sum beamformer to sanitize the steering vectors and to assign azimuth angles to those anchor points.

1) Steering Vector Estimation using Anchor Points (calibration phase)
We estimate the steering vector q i of receiver antenna elements by placing a transmitter at k anchor points, measure the azimuth angles Θ = [θ 1 , θ 2 . . .θ k ] and use them as reference angles for the k steering vectors.The phase shifts corresponding to the steering vectors for the k reference directions are estimated using the method explained Sec.III.
The IQ samples are correlated with the TX beacon as in Eq. 3 and the phase rotation φm i that maximizes the correlation r(φ m ) is estimated so that q i normalized to the 1 st antenna element is As mentioned above, the steering vector derived from the anchor point is not ideal because of residual phase offsets in RF chains, measurement offsets and orientation errors in the antenna geometry.

2) Steering Vector Sanitation and Array Response Computation
As a solution for steering vector mismatch, we estimate the presumed steering vector using a method explained in [41].Note that this method is executed for a minimum variance distortionless response (MVDR) beamformer under heavy interference conditions.Even though we use a delay and sum beamformer, this method can be applied to our case because the steering vector is determined by the physical properties of the channel independent of the type of beamformer being used.We adapt their approach to estimate the optimum steering vector as follows.The measured steering vector is modeled as q i = p i + e i where p i is the presumed steering vector and e i is the error for the anchor point at  [41] as a convex optimization problem Once the presumed steering vector is known, it is possible to estimate the corresponding azimuth angle of the anchor point by comparing with a modeled steering vector.We model the steering vector pi for direction i as a function of azimuth angle where • sin θ i and −90 • < θ i < 90 • is the azimuth angle in the xy plane measured from x axis for a signal arriving from the i th direction.By minimizing the minimum mean square error (MMSE) between pi and p i using azimuth angle as a variable, we are able to obtain the optimum azimuth angle θ opt i of the anchor point and the corresponding steering vector Finally, we generate steering vectors for all azimuth angles using Eq. 8 and construct the array response using Eq. 5. Algorithm 1 summarizes this process.Fig. 2a and Fig. 2b illustrates the array response computed from a signal emitted for a duration of 15 s from a TX at A and and a TX at C. Fig. 2c illustrate the array response for a person at A and and Fig. 2d for a person at A and at B when the transmitter is at C. From the first two figures it is visible that the array response of a particular azimuth angle is constant.Compared to the first two figures, it can be noticed that there are more perturbations in the array response of the final two figures.Additionally, compared to Fig. 2c, Fig. 2d has more perturbations closer to the LoS response.However, the LoS response is quite strong in both situations which tends to shadow the response of the human.Result: Array response a(t) for i ← 1 to k do Estimate e opt i by solving (7) Update vector p i using q i = p i + e i Estimate popt i and θ opt i by solving (9) end Generate steering vectors using (8) for −90 ≤ θ ≤ 90 Construct the array response using (5) Algorithm 1: Array response computation.

V. DETECTION OF THE HUMAN IMPACT
In this section we analyze the array response in the presence of a human from the spatial streams computed according to Eq. 5 from the steering vectors processed according to Sec.IV-B corresponding to azimuth angles −90 • ≤ θ i < 90 • .

A. HUMAN PRESENCE DETECTION
We infer human presence from the array response for a given time interval T as follows.We construct the array response a hf (t) for a human-free setting using Eq.5.r j (t) in Eq.5 is computed from the IQ samples collected from the testbed when no human exists.Then we find the difference, ∆a T , between a hf (t 0 ) and the array response measured with a human subject present a hp (t) for all θs during the interval t 0 ≤ t < t 0 + T (since the array response for the human free environment is constant over time, we use only the response at time instant t 0 ).
Human presence is detected when ∆a T > T h, where T h is the maximum ∆a for array responses measured in a human free environment T h = max(∆a T1 , ∆a T2 . . .∆a T N ) during N time intervals.Note that ∆a Tn where n ∈ N is not zero due to noise and multipath effects.

B. DETECTION OF THE ARRAY RESPONSE OF HUMANS
When human presence is detected by the above approach, we remove the effect of the line of sight path from the array response a hp (t) through subtraction and construct the array response of a human a h (t).The intuition here is that the array response of the LoS has a constant gain for a particular angle over time.However, from the collected data we notice that when the human is closer to the LoS beam, the subtraction of a constant value from the array response of human and the LoS does not completely remove the LoS effect over a long duration.Therefore, we estimate the mean array response for a short time interval T and subtract that value from the array response such that where t 0 ≤ t ≤ t 0 + T .Fig. 3a illustrates the behaviour of |a hp (t)| and Fig. 3b shows the behaviour of |a h (t)| when a human is present at anchor point A for T = 15s.According to Fig. 3a, the LoS path overshadows the paths from the human, however, between azimuth angles {−50 • , 0 • } perturbations can be observed in the array response than between the azimuth angles {0 • , 90 • }.When the effect of the LoS path is removed, human's impact can be observed as in Fig. 3b.Interestingly, the two figures show that the highest amplitude of the human response is as much as 4 times smaller than the LoS response.

C. ESTIMATION OF A HUMAN'S DIRECTION
We use the array response of the human a h (t) when the LoS response is subtracted to estimate the direction of arrival corresponding to the human.As the beamwidth of the unit response of a signal emitted from a particular direction is fairly wide (half power beam width of ≈ 25 • for a ULA of 4 antenna elements in our case) and the residual response of LoS signal may still exist, the estimation of the human's direction is not trivial.As a naïve approach, we can detect the angle corresponding to the maximum gain at a particular time instant, however, this introduces noise due to large beamwidth, residuals of the LoS beam and sidelobes.Again, MVDR or MUSIC algorithms cannot be used because the number of sources are not known in advance.Therefore, we use the following procedure to estimate the DoA corresponding to a human.We first model the array response of the ULA for a signal emitted from each azimuth angle.As an example, the array response âj of a signal emitted from direction j can be modeled by multiplying the beamformer weight w j with all steering vectors of the interested K azimuth angles s.t.
where P = [p 1 , p2 , . . ., pK ]. Figure 4b illustrates the amplitude of the array response |â j | for a unit source impinging on the ULA from directions θ = 0 • and −27  The idea here is that when the peaks of |a h (t)| and |â j | are aligned for a signal incoming from a certain azimuth angle, this produces a high correlation.Here, we compute the matrix Â manually instead of shifting |â j | over different angles because the array responses for different azimuth angles are not equal to their shifted versions.As an example, according to Fig. 4b, |â j | for a signal arriving from −27 • is not the same as when |â j | for a signal arriving from 0 • is shifted by 27 degrees to the left.

VI. TESTBED DESCRIPTION
We used a software-defined radio platform consisting of a single-antenna transmitter and a receiver with a four element uniform linear array (ULA) with λ 2 inter-element spacing.Dipole antennas were used as array elements.The transmitter is comprised of a Universal Software Radio Peripheral (USRP) X300 series with UBX-160 and SBX RFdaughterboards as the radio front-ends connected to a host computer.The receiver was comprised of three USRPs, of which two served to collect the signal from the other highlevel node (= transmitter).The third device is the reference USRP.It operates by synchronizing over the air with the primary two USRPs and then transmitting a reference signal to calibrate the starting offset between them.Phase coherence between the two receiver USRPs was accomplished using a clock distribution system.It provides both a pulse-persecond signal and a 10 MHz reference signal to discipline the local oscillators of the USRPs.The reference USRP was necessary due to inter-device random phase offset at start-up and component variability in each RF chain.Additionally, the reference USRP helps to ensure that phase coherence, and therefore beamforming performance is maintained throughout the measurement collection.
The air-interface configuration of the testbed is an OFDM frame structure at a carrier frequency of 3.42 GHz.The total bandwidth of the system is 15.36 MHz with 52 subcarriers each with a bandwidth of 240 kHz yielding a total bandwidth of 12.48 MHz.Each USRP stream IQ samples at a rate of 16.66 megasamples per second.This is subdivided into subframes of 3082 samples in length in time domain, yielding approximately a rate of 5408 subframes per second.This rate of the subframes is essentially the sampling rate of the IQ samples used as input for beamforming.

VII. EXPERIMENTS AND EVALUATION
We conducted experiments in an environment as shown in Fig. 5a which is a semi-anechoic chamber where ground reflections are still possible as the ground was not insulated with radio wave absorbents.Each experiment in below sections lasted for 15s.As the testbed provides samples at a rate in excess of 5400 samples/s we downsampled it by factor 10 and obtained 8112 samples for each experiment.

A. EVALUATION OF STEERING VECTOR SANITATION
As mentioned in Sec.IV-B we generate the beamformer weights using a semi-supervised learning approach where measurements are collected from anchor points and then assigning the closest steering vector generated from a model using the MMSE criterion.In this section we analyse the performance of this approach using the MMSE of 17 steering vectors collected during two measurement campaigns as shown in Fig.For all the directions in both campaigns, except the direction having the index 7 in campaign 2, MMSE<0.2.Generally, the directions having a smaller |azimuth angle| tend to have low errors and vice versa, e.g., directions with indices 5, 6, 7 and 8 have the highest errors while C, D, 3, and 10 have low errors.For the direction indexed 7, the error is 2.65 and the corresponding azimuth angle is −20 • .Given that direction 7 should be further away from directions with indices 6 and 8 (having angles of −55 • and −52 • respectively), it can be concluded that the measured steering vector for direction 7 has a very high error which cannot be sanitized by the proposed algorithm.Therefore estimation of the azimuth angle for that direction is also erroneous.We conjecture that the error is caused by the increase in azimuth angle where the direction with index 7 has the highest azimuth angle.It is well known that higher azimuth angles can introduce errors in DoA resolution.Comparing the five single person experiments, it can be observed that ∆a is above T h during the entire period.The same can be observed with experiments involving 2-5 persons.The main difference is that the worst case performance of ∆a for two persons (DE in Fig. 7b) is higher than that of 1 person experiments and it is even higher for experiments involving 3 − 5 persons.This is expected behavior, more perturbations in ∆a can occur, since more persons are present.

C. DIRECTION OF ARRIVAL ESTMATION OF A HUMAN
In this section we compare the impact of a person placed at directions A, B, C, D or E i) on the DoAs of the LoS signal and ii) on DoAs corresponding to the single person .
Fig. 8a shows a boxplot representation of the DoAs of the LoS signal when a human subject is at A, B, C, D or E. The median DoA of the LoS from A to E are 0 • , 1 • , 2 • , −2 • and 0 • .It is clear that LoS DoAs are not affected when the person is furthest from the LoS, e.g.A and E. Additionally, we observe that the outliers of experiments A and E are also low.When the person gets closer to the LoS, the low resolution of the beamformer causes the LoS angles to deviate slightly from the ideal, e.g., B (−1 • ), C (2 • ) and D (2 • ).Fig. 8b shows a boxplot representation of the raw DoAs corresponding to the human derived from the approach described in Sec.V.The median DoAs of the human are −32 • , −16 • , −4 • , 2 • , and 16 • .Compared to the DoAs of anchor points of A to E shown in Fig. 6, the absolute errors are 0 • , 7 • , 2 • , 3 • and 6 • respectively which illustrates the accuracy of our DoA estimation approach.As positions furthest from the LoS have most outliers, we remove these outliers by applying a moving median filter.Finally, Fig. 8c  Note that for these three figures, the experiments were performed separately and the results were merged to the three figures for ease of comparison.Figures 9d and 9e show the distributions for 3 and 4 human subjects respectively.
Clearly, the angles for the LoS signals have the most consistency.TX at A, B, C and E have azimuth angles −32 • , −23 • , −6 • and 20 • 100% of the time and D at 5 • around 65% of the time.However, when a human is placed in the same five positions, the distributions show a Gaussian behaviour.The mean angles can be computed as −32 • , −16 • , −4 • , 5 • , and 14 • .This can be attributed to the paths scattering off of different points from the subject's body as opposed to the LoS path not going through such disturbance.
When two persons coexist in the environment as in Fig 9c 0.5 m away from each other, the distributions spread even further resembling a mixture of two Gaussians.However, the DoA corresponding to the two persons cannot be directly inferred from the mean or the peaks of the distributions.The distributions only provide a rough estimate of the area where the two persons are located, e.g. the distribution for AB is shifted to the left of BC and BC's distribution is shifted to the left of DE.When three (Fig. 9d) and four (Fig. 9e) persons coexist 0.5 m away from each other, the distributions are spread even further.Based on this information, it can be concluded that when people are stationary in a given area, the number of persons can be estimated by evaluating the spread of the distribution even though there DoAs cannot be directly estimated.11c.Again, it can be observed that it is possible to capture both the persons at the same time.Note that the maximum DoA this time is ≈ 30 • while the minimum is around ≈ −50 • .This can be attributed to the path that the person is taking where the anchor points of the two extremes have DoAs of 32 • and −55 • .Fig. 11f shows the DoAs when two persons walk simultaneously this time between ≈ +50 • and ≈ −50 • .Interestingly, both paths were captured while discontinuities exist when a person moves closer to the LoS e.g.1.9-3.8s.Another important observation is that the noise levels when two persons exist are higher compared to when a single person walks.We note that this is caused by low TX power levels used in the two experiments involving the two persons.

VIII. DISCUSSION AND CONCLUSIONS
This work presents an approach to capture a human and the direction of arrival through beamforming utilizing SDRs as the hardware.The main contributions are an algorithm to generate the steering vectors from few reference points and imprecise prior information, and detection of the human and the direction of that person from human induced paths amidst strong LoS signal.From experiments we detect human presence 100% of the times, estimate DoA up to a maximum error of 7 • of a single person, show that detection of multiple stationary persons is possible when sufficient separation among them is present, and finally demonstrate the capability of tracking up to two walking persons.Currently, one of the main limitations of our approach is the requirement of a calibration step to calculate the steering vector for a few reference directions.The proposed solution relies on the beam sweeping capabilities of the receiver as a post processing step which relies on accurate steering vector estimation.Even though, new wireless standards such as IEEE 802.11ac, ax and 5G support such capabilities, currently there is no commodity hardware that can be utilized for our purpose.Therefore, we use USRP hardware for this work.However, with the current expansion of research on device-free human sensing, we expect chipset manufacturers to expose such capabilities in a wide range of products.
The experiments are performed in a semi-anechoic chamber which has only the ground reflection, this setup is more similar to an open area that has no reflections.In future work, we intend to perform experiments in a cluttered environment with more reflections.However, as the reflections from stationary objects remain static, we expect to subtract those effects using the same approach used to remove the LoS effect.For the experiments, currently a 4 antenna ULA is used at the receiver and the minimum distance between two nearby human subjects is 0.5m.In the current setup, the 0.5m distance between humans is not enough to separate them with the current ULA.It is well known that, as the number of antenna elements increase in the linear array, the beamwidth can further be reduced so that the interference on adjacent beams can be further attenuated.We will, in further investigations, study settings with 8 and 16 antennas in order to increase the directional perception accuracy and the count of people that can be recognized simultaneously.In future work we expect to extend this approach to detect and localize multiple persons and simultaneously recognize their activities with a larger ULA that supports a higher resolution.

FIGURE 1 :
FIGURE 1: A plane wave impinging on a antenna array at an azimuth angle of θi.

FIGURE 2 :
FIGURE 2: (a) Array response for a signal emitted from anchor point A at −28 • during 15 s and array response when a signal is emitted from C at 0 • (b) no human exists in the environment (c) a human exists at A and (d) a human exists at A and at B.
Author et al.: Extracting Human Context through Receiver-End Beamforming

FIGURE 3 :
FIGURE 3: (a) Array response of the LoS when a person exists at A. (b) Array response of the person when LoS response is removed.

FIGURE 4 :
FIGURE 4: (a) Comparison of the amplitude of the array response |a hf (t)| when environment is human-free vs |a hp (t)| where a human exists at −27 • at a particular time instant.(b) Amplitude of the modeled array response |âj| for a unit source impinging on the ULA from directions θ = 0 • and −27 • .

5 .
In the first campaign we used 5 directions (indexed A, B, C, D & E) as shown in Fig. 5a and in the second campaign 12 reference directions (indexed 1-12) were used as shown in Fig 5b.Fig 6 illustrates the MMSE for each direction between the presumed steering vector p i and modeled steering vector pi together with the azimuth angle (θ i ) computed according to Sec.IV-B2.

FIGURE 6 :
FIGURE 6: MMSE between modeled and presumed steering vectors of 17 directions from two measurement campaigns along with the computed azimuth angles.
illustrates the result when those DoAs are filtered.The filtered DoAs have medians • , −16 • , −4 • , 2 • , and 16 • .D. EVALUATION OF THE RESOLUTION OF DOA OF MULTIPLE HUMAN TARGETS 1) Stationary Human subjects with 0.5 m Distance Here we compare the distributions of the LoS signal with TX at A, B, C, D and E, single human subject at A, B, C, D, and E, two subjects at AB, BC and DE, three subjects at ABC and four subjects at ABCD.Fig 9a shows the distribution of angles corresponding to the LoS paths, Fig 9b for the single human subject positions and Fig 9c for the two human subjects case.

2 )
Human Subjects with increased Distance Next, we conducted two experiments placing two persons at positions 2 and 4 marked in Fig. 5b, 1.6 m away from each other and 1 and 6, 4 m away from each other.Figures 10a and 10b show the distributions of the DoAs for the two experiments.Unlike in the previous case, the distributions for the two humans have a clear separation.For the positions 2 and 4, the mean DoAs can be estimated as −26 • and 25 • .The DoAs of the anchor points for the two positions were estimated This work is licensed under a Creative Commons Attribution 4.0 License.For more information, see https://creativecommons.org/licenses/by/4.0/.This article has been accepted for publication in a future issue of this journal, but has not been fully edited.Content may change prior to final publication.Citation information: DOI 10.1109/ACCESS.2019.2945476,IEEE Access Author et al.: Extracting Human Context through Receiver-End Beamforming

FIGURE 7 :FIGURE 8 :
FIGURE 7: Human presence detection from the array response (a) single person at A, B, C, D and E, compared with human-free case, (b) two persons at AB, BC, CD, DE compard with human-free case, (c) three persons at ABC, BCD, CDE, four persons at ABCD and BCDE and five persons at ABCDE compared with human-free case.

FIGURE 9 :FIGURE 10 :
FIGURE 9: Direction of arrival distribution in different scenarios.(a) No human exists, a TX is placed at A, B, C, D & E in separate measurement campaigns, (b) one human exists at A, B, C, D & E in separate measurement campaigns, (c) two humans exist at AB, BC or DE within 0.5 m in three separate measurement campaigns, (d) three humans exist at ABC 0.5 m from each other and (e) four humans exist at ABCD 0.5 m from each other.

FIGURE 11 :
FIGURE 11: (a) A person starts moving from position 6 marked in Fig 5b, walks two rounds along the path marked in red and stops at 6, (b) DoA of the person from (a), (c) A person starts moving from position 1, walks between positions 1 and 6 along the path marked in red for two rounds and stops at 3. Another person is fixed at 3, (d) DoA of the two persons from c, (e) person 1 starts from position 6 and person 2 starts from position 12 and walk along the paths marked in red for two rounds, (f) DoAs of the two persons from (e).
This work is licensed under a Creative Commons Attribution 4.0 License.For more information, see https://creativecommons.org/licenses/by/4.0/.This article has been accepted for publication in a future issue of this journal, but has not been fully edited.Content may change prior to final publication.Citation information: DOI 10.1109/ACCESS.2019.2945476,IEEE Access Author et al.: Extracting Human Context through Receiver-End Beamforming