Channel Measurements at 300 GHz for Low Terahertz Links in a Data Center

This article presents comprehensive double-directional channel measurements with time-domain channel sounding at 300 GHz that characterize the channel of wireless links in a data center. The channels are classified into three scenario-dependent use cases and are individually analyzed providing channel parameters as a function of the required signal-to-noise ratio of the prospective communication system. The spatial and temporal analysis of the channel reveals relevant propagation effects such as the influence of scattering and derives relations between the channel parameters of the propagation and the radio channel. The analysis shows that multipath propagation becomes relevant for systems with high signal-to-noise ratio requirements despite high-gain directional antennas. A first order approximation of relevant propagation effects complements the analysis. The measurement data of the whole measurement campaign with 18250 calibrated impulse responses including all meta data is published so that the research community has a collective benefit.

ABSTRACT This article presents comprehensive double-directional channel measurements with timedomain channel sounding at 300 GHz that characterize the channel of wireless links in a data center.The channels are classified into three scenario-dependent use cases and are individually analyzed providing channel parameters as a function of the required signal-to-noise ratio of the prospective communication system.The spatial and temporal analysis of the channel reveals relevant propagation effects such as the influence of scattering and derives relations between the channel parameters of the propagation and the radio channel.The analysis shows that multipath propagation becomes relevant for systems with high signal-to-noise ratio requirements despite high-gain directional antennas.A first order approximation of relevant propagation effects complements the analysis.The measurement data of the whole measurement campaign with 18250 calibrated impulse responses including all meta data is published so that the research community has a collective benefit.
INDEX TERMS THz, channel, channel sounding, data center, top-of-rack, multipath propagation, delay spread.

I. INTRODUCTION
A PROMINENT application of terahertz (THz) commu- nications represent the integration of additional wireless links in data centers [1].Data centers are undergoing a transformation, shifting from primarily storage-driven applications, like managing large databases, to more complex and computationally intensive tasks that focus on processing and analyzing requests [2].Consequently, these new demands call for ultra-low latency, high capacity, reliability, scalability, and flexibility in system design and implementation [3].The efficiency of current data center networks is constrained by short bursts on links that are sparsely used on average; nonetheless, these bursts result in a decrease in the quality of service [4].
Incorporating wireless links into a data center network offers numerous advantages.When paired with beam steering [5], wireless links enable a full dynamic reconfiguration of the data center network.Traditionally, modifying a deployed network is complex, lengthy, and costly.However, with the introduction of wireless connections, the links are no longer limited by physical cabling.The software-defined network controller will be able to dynamically adjust the network topology in response to current demands and traffic patterns [6].In addition, a direct communication between nodes that bypasses the aggregate and core switches is favorable for latency requirements.Fig. 1 illustrates the conceivable types of wireless connections in the data center.
The potential for data center applications was recognized early on [7] and has since been actively addressed by the research community.For instance, the European Project "Terahertz Based Ultra High Bandwidth Wireless Access Networks" (TERAPOD) has investigated the feasibility of the data center use case by pushing the development of THz sources and detectors, providing device and channel measurements and conducting physical layer (PHY) and higher layer simulations [8].Nonetheless, it is only in recent years that advances in device development have made THz communications feasible, with considerable progress still to be achieved.
The number of publications with respect to THz communications has increased dramatically in the recent years proving the great interest of the research community in this area, whereas the number of channel measurements with explicit relation to data centers are limited.The first measurement campaign in a real data center was reported by the authors in [9] in 2019 providing first, limited information on path loss, power delay profiles (PDPs) and power angular profiles (PAPs) in inter-rack scenarios.
After that, a measurement series in an office room with rack mock-up at 300 GHz was presented by Cheng and Zajic in [10] and [11] that deal with the measurement of inter-rack links in line-of-sight (LoS) and obstructed-line-ofsight (OLoS) scenarios reporting frequency dependent path loss, PDPs and an estimated coherence bandwidth.However, being located in an office room without realistic server racks, the measurements cannot capture the impact of a realistic environment and materials.
Inter-rack measurements of medium height (MH) links in a data center were presented by Cheng et al. in [12] in 2020 analyzing the PDP and transfer function in LoS, OLoS, and non-line-of-sight (NLoS) scenarios.Here, the distance between transmitter (TX) and receiver (RX) is not varied and kept constant with 1.75 m for all inter-rack setups.
In [13], Cheng et al. examine channel characteristics for a virtual multiple-input multiple-output (MIMO) setup in a data center with a short transmission distance of 15 cm between TX and RX, and the influence of the vibration of cables on the fading that is measured in a similar setup in an indoor lab environment.Finally, Song et al. conducted unidirectional measurements for an access point (AP) to rack scenario at 140 GHz in [14] presenting channel parameters such as path loss exponent, root mean square delay spread (RMSDS) and angular spread (AS).To sum up, there are no measurements reported and no research data available that examine the THz channel for different link types under different propagation conditions in a real data center.

The contribution of the paper is threefold:
• The definition of channel parameters based on discrete multipath components (MPCs) allows for a clear and comparable evaluation of the channel data.In contrast to the widely use approximation based on the RMSDS, the definition of the coherence bandwidth for timeinvariant double-directional channel measurements is based on the correlation function of the time-variant transfer function that is given for discrete MPCs.• A detailed measurement campaign in a real data center analyzes the wireless channel for all different types of links in a THz enhanced data center, and evaluates the channel parameters in a system-oriented way as a function of the required signal-to-noise ratio (SNR), • The first open-access research data of measured doubledirectional complex channel impulse responses (CIRs) at 300 GHz is provided [15].The paper details measurements and the associated analyses for double-directional propagation and radio channels of top-of-rack (ToR) links and MH links.point-to-point (P2P) radio channels and intra-rack links further round out the measurement campaign.The data presented is the basis for a detailed analysis of the propagation characteristics with a focus on multipath propagation and constitute the most comprehensive measurement series in a specific THz communications use case to date.In addition, more than 18250 calibrated CIR and the corresponding extracted MPCs are published as research data, made available for the research community [15].
The rest of the paper is organized as follows: Section II introduces the environment, measurement equipment, and methodology, which are relevant to all subsequent sections.Then, inter-rack links are analyzed in Section III discussing the double-directional propagation channel and its channel parameters for ToR links and links at MH.The discussion on inter-rack links is extended to the radio channel of the same scenarios applying a realistic antenna diagram of a steerable patch antenna array in Section IV.The P2P radio channel with regard to the different types of links is presented afterwards in Section V where different aspects from an application point of view are discussed.Lastly, intra-rack links for three different distances are evaluated in Sections VI and VII concludes the paper.

II. MEASUREMENT METHODOLOGY A. MEASUREMENT ENVIRONMENT
A data center is generally located in a big room or a hall.The walls of the indoor environment are characterized by typical building materials such as concrete, glass, metal, plastic, and wood.A distinguishing feature of data centers is represented by server racks that are usually organized in rows in a regular structure so that each server rack is accessible.Electric and fiber cables, that provide the power supply and connect the servers with each other, are either organized in cable ducts or laid in the floor.In addition, some ventilation system or air conditioning system is in charge of handling the heat dissipation.Measurements that characterize the channel of additional wireless THz links are conducted in the Dell EMC Research Data Center, Cork, Ireland.Also in this research data center, the server racks are organized in rows and the cables are laid in the floor.Cooling is realized by cold air that streams out of the floor and is suck into the front of the racks.The heated air is emitted by the back side of the racks and collected by ventilation openings located at the ceiling.Plastic curtains in the form of polyvinyl chloride (PVC) strip curtains mounted at the ceiling above each row of racks separate the air in hot and cold aisles in order to ensure an effective cooling flow.Fig. 2 shows a floor plan of the data center and illustrates the arrangement of the server racks, the PVC strip curtains and the resulting cold and hot aisles.
The research data center is equipped with many different types or racks.The most common rack is the VMAX Dell EMC server rack that is in addition identical in construction with Unity All Flash rack and similar to the EMC Symmetrix racks.It is a powder-coated steel cabinet having a width of 0.61 m, a depth of 1.18 m, and a height of 1.90 m.It holds 40 rack units (Us) of 19-inch rack-mounted servers.The front is a perforated metal plate that has a 26 cm high illuminated plastic plate at a height of 1.25 m.The sides are metal sheets and the rear is also a perforated metal plate to enable an air flow through the servers.The top is divided into three parts that can be either a closed or a perforated metal cover.
In the given structure of the data center, different regions are identified that will by nature provide different propagation characteristics due to different application scenarios, varying geometrical characteristics, and different surrounding materials.The first distinction can be made between links within the same rack -the intra-rack links -and links between different racks -the inter-rack links.The intra-rack links propagate within the metal cabinet whereas the interrack links propagate in the indoor environment.
The inter-rack links can again be split into links at MH that propagate below the top of the rack and ToR links that propagate above the top of the rack.MH links face a corridor-like environment with racks along both sides and can barely propagate to an adjacent aisle.In contrast, ToR links are situated in an open-space region with only individual obstacles like columns.As a special feature, the plastic curtains mounted above each row of racks impair a link over long distances and favor multipath propagation.Fig. 3 illustrates the classification of the wireless links in the data center that are also schematically visualized in Fig. 1.

B. MEASUREMENT EQUIPMENT AND POST-PROCESSING
The measurements are performed with the Ilmsens m:mimo sub-mmWave correlation-based time-domain channel sounding system [16].Being able to handle up to four TXs and four RXs, the base unit creates a common clock of 9.22 GHz that is distributed via coaxial cables to all TXs and RXs in order to guarantee the synchronization.Also, the power supply for all components is provided by the base unit [17].In the TX ultra-wide band (UWB) modules, an M-sequence with 4095 chips and a chip duration of 108.5 ps is created with a linearfeedback shift register and converted to the intermediate frequency (IF) band from 5.22 GHz to 13.22 GHz resulting in a measurement bandwidth of 8 GHz.Additional frequency extenders convert the test signal via a subharmonic mixer into the low THz band from 300.26 GHz to 308.26 GHz.The local oscillator (LO) is created by a chain of frequency doublers and amplifiers making use of the common clock signal in order to assure a coherent phase information.More details are provided in [18], [19].
The measured CIRs are post-processed offline with MATLAB R .After a calibration process in order to obtain comparable and exchangeable measurement data, the averaged CIRs can be combined for a specific measurement position to multidimensional sets of CIRs.For time-invariant measurements without rotational scanning, the combination yields in a one-dimensional space that contains the single CIR as a function of the delay.For double-directional measurements containing information on angle of departure (AoD) and angle of arrival (AoA) the three-dimensional space AoD×AoA×Delay is obtained [18].On the basis of these sets of CIRs, the MPCs are extracted, by a local maxima search of the amplitude.In order to correctly detect the maxima in the spatial domain at the edges of the given matrix, the periodicity of the angular domain is taken into account by replicating the lowest and highest columns and rows to the opposite edge.The added elements are removed again after the maxima detection.The MPC extraction considers an adjustable threshold that is oriented towards the cut-off level of the noise floor.Finally, a list with all detected discrete MPCs characterized by their respective complex amplitude A l , delay τ l , AoD ϕ AoD,l , and AoA ϕ AoA,l is obtained.
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.

C. CHANNEL PARAMETERS
Channel parameters are suitable figures of merit that serve as a basis on which requirements on properties and protocols of prospective THz communication systems are derived.
With a focus on applications for THz communications that underlie time-invariant channels, the spatial characterization of the environment with regard to multipath propagation is emphasized.In the following, the channel parameters are presented that take into account the relevant aspects derived from the system functions of the universal description of channels.The parameters are defined based on the formalism of L extracted, discrete MPCs.

1) CHANNEL IMPULSE RESPONSE
The CIR given by is the fundamental description of a linear time-invariant (LTI) system in time domain with the input-output relation with input signal x and output signal y.

2) CHANNEL TRANSFER FUNCTION
The channel transfer function (CTF) is the Fourier transform of the CIR with respect to the delay variable and given by A l e − j 2π fτ l .(6)

3) PATH GAIN AND CHANNEL GAIN
The path gain of the l-th MPC is defined by with A = |A| denoting the absolute value of the complex amplitude.The channel gain corresponds to the maximum of the absolute value of the amplitudes of the MPCs

4) POWER DELAY PROFILE
The PDP describes the path gain of the different MPCs as a function of the delay.The PDP P τ is given by the combination of the square of the amplitudes located at the respective delays Consequently, a superposition of multiple MPCs at the same delay yields the sum of the squared amplitudes.

5) DELAY SPREAD
The RMSDS is a measure of the dispersion of the channel and is given by the square root of the second central moment of the PDP [20] where τ i denotes the delay of the i-th component of the PDP.Here, the sample method based on the detected MPCs is applied [21].

6) COHERENCE BANDWIDTH
The coherence bandwidth is a measure of the frequency range over which the signal can be assumed to pass through the channel with similar or equal gain and linear phase [20].
It is defined on the basis of the correlation function of the Time-Variant Transfer Function where (•) * denotes the complex conjugate and H(t, f) denotes the cut through the complex stochastic process of the timevariant transfer function at time t and frequency f.The common assumption of wide-sense stationary uncorrelated scattering (WSSUS) assumes that the fading processes of the time-variant MPCs are wide-sense stationary with respect to the time variable and uncorrelated with respect to the delay variable.These properties yield that the general dependency of the correlation function to the absolute time instances and frequencies can be reduced to a dependency on the difference of time t and difference of frequency f resulting in Ψ HH ( t, f).In this case, the coherence bandwidth is given by the frequency difference at which the correlation function drops under a certain threshold γ CB with For an easy access to the coherence bandwidth, the literature provides different approximations for the relation between the RMSDS and the coherence bandwidth that base on the WSSUS assumption.Popular approximations are given by for a correlation level of γ CB = 0.9 and for γ CB = 0.5 [20].
In the present case of channel measurements in complex scenarios in a static environment, the WSSUS assumption does not hold as a whole.On the one hand, the static scenario does not depend on the absolute time variable and thus the Doppler effect does not matter.The scenario could be interpreted as wide-sense stationary with regard to the time variable since the stochastic process that always provides the same realization has a constant ensemble expectation in time.However, this observation is trivial and of little significance.More important is the characterization of scatterers since the MPCs may be correlated.For uncorrelated scatterers, the correlation function of the time-variant transfer function depends only on the frequency offset f = f 1 − f 2 rather than the two absolute frequency variables f 1 , f 2 .To verify the assumption on uncorrelated scatterers, we calculate the more general case and check if the condition on the frequency dependency holds.
The correlation function is generally calculated by [22] where p(H 1 , f 1 ; H 2 , f 2 ) denotes the joint probability density function.If the measured MPCs are assumed to be deterministic and the only possible realization of the stochastic process, the integral results in and the frequency dependent coherence bandwidth can be calculated by The calculation of the correlation function shows that for a constant frequency offset its absolute value is frequency dependent.Hence, the scatterers must be assumed to be correlated.However, the relation between the correlation values for a constant frequency offset at different frequencies is constant within the frequency interval of interest from 300.2 GHz to 308.2 GHz.Therefore, the coherence bandwidth yields the same value for all frequencies within the frequency band of interest and will be reported as a frequency independent value in the course of this paper.

7) RICIAN K-FACTOR
The Rician K-factor is defined as the ratio of the gain of the strongest path in relation to the gain of the remaining MPCs

8) MAXIMUM EXCESS DELAY
The maximum excess delay (MED) is defined as the time difference of the delay of the MPC with the maximum delay τ L and the first MPC τ 1

9) POWER ANGULAR PROFILE
As a counterpart of the PDP in the angular domain, the PAP describes the path gain of the different MPCs in the angular domain.The PAP can be described as a function of the AoD or AoA, both for azimuth and elevation, or combinations of those with where φ ∈ {ϕ AoD , ϕ AoA , θ AoD , θ AoA } and φ l denotes the respective angle of the l-th MPC.Similarly to the PDP, a superposition of multiple MPCs at the same angle yields the sum of the squared amplitudes.For double-directional measurements in the azimuth plane that are discussed in this paper, the PAP can be expressed as [23] P ϕ (ϕ AoD , ϕ AoA ) The AS describes the standard deviation of the angular distribution of the MPCs.In order to avoid an ambiguity due to the periodicity of the angular domain and the dependency on the orientation of the coordination system, the AS is defined by [24] σ φ = l e j φ l −μ φ 2 with Similar to the PAP, the AS can also be defined for the AoD and the elevation plane, respectively.Because of the lack of measurement data for the elevation plane, the evaluation is limited to the angular spread of arrival (ASoA) σ ϕ,AoA and the angular spread of departure (ASoD) σ ϕ,AoD in the azimuth plane.

III. DOUBLE-DIRECTIONAL PROPAGATION CHANNEL
The propagation channel is the fundamental description of the wave propagation in space in the context of communications [25].It reflects the different propagation phenomena that are relevant for the application under investigation and characterizes the radio environment.The following section presents the measurement setups that capture the propagation channel of wireless links in a data center and analyzes temporal profiles, spatial profiles, and channel parameters.

A. MEASUREMENT SETUP
In the presented measurement campaign, nine doubledirectional measurements are conducted.Due to limited output power at THz frequencies [25], TX and RX are each equipped with a standard gain horn antenna with 26.5 dBi and a half-power beam width (HPBW) of 8.5 For each measurement, TX and RX are initially aligned towards each other marking the AoD ϕ AoD = 0 • and AoA ϕ AoA = 0 • in the measurement related coordinate system.Then, the rotation units turn clockwise recording all combinations of AoD and AoA.The resulting spurious-free dynamic range (SFDR) of a single CIR yields 30 dB for MH and 23 dB for ToR scenarios.The measurements of the MH links are labeled MH1 -MH4 and the ToR measurements are labeled ToR1 -ToR5 accordingly.
MH1 -MH3 are placed in a cold aisle with a temperature of 26.6 • C. In MH1, TX and RX are separated by a long distance of 12.52 m.The x-coordinate of TX and RX is varied in order to avoid blockage by the measurement equipment.The distance is reduced to 3.66 m in MH2 and MH3 that record both vertical (V) and horizontal (H) polarization, respectively.The measurement setup is shown in Fig. 4a.In MH4, TX and RX are moved to the adjacent hot aisle with a temperature of 30.4 • C. In the hot aisle, the back side of the racks are facing the aisle and the ends of the hot aisles are covered by the plastic curtains that are also visible on the floor plan in Fig. 2.
In ToR1, TX and RX are positioned in the same aisle as for MH1 -MH3 but shifted to a height of 2.11 m that is above the majority of racks in the data center.The TX is moved to the adjacent aisle in ToR2 and ToR3 recording V and H polarization, respectively.Fig. 4b shows the corresponding measurement setup.Note that the curtain that closes the hot aisles during the measurement was moved to the left while taking the photo.In ToR4 and ToR5, the TX is moved to the next to adjacent aisle recording again V and H polarization.All MH scenarios and ToR1 are placed in a LoS scenario that is characterized by a clear direct path between TX and RX with no physical obstructions.In contrast, ToR2 -ToR5 are placed in an OLoS scenario that is defined as a scenario where the direct path is still present but partially obstructed by the transmission through at least one boundary layer.Table 1 summarizes the resulting distance between TX and RX d, the temperature in the corresponding aisle ϑ TX , ϑ RX , and the polarization   on the transmitted wave.The exact positions of TX and RX, r TX , r RX , are provided in the research data set [15].The TX's and RX's positions are also depicted in a detail of the floor plan in Fig. 5 where ×, ⊗, +, and ⊕ denote the position of MH TX, MH RX, ToR TX, and ToR RX, respectively.

B. CHANNEL CHARACTERIZATION
As a first confirmation of the underlying assumptions and deployed signal processing, the analysis of the measurements shows that the channel is time-invariant [9].Illustrated in Fig. 6, the path gain G c of the direct path of the LoS scenarios agrees well with free-space path loss where λ denotes the wavelength.One row of curtains in the OLoS scenarios ToR2 and ToR3 adds an additional attenuation of approximately 6 dB and two rows of curtains in ToR4 and ToR5 add an additional attenuation of approximately 16 dB.The differences between the measured values and the theoretical values of the free-space path loss (FSPL) of 1.6 dB lie within the measurement accuracy of the channel sounder.The values above the theoretical FSPL can physically be explained by an additional attenuation introduced by the extender waveguide in the back-to-back (B2B) calibration measurement that lead to lower values in the measurement.

1) SPATIAL AND TEMPORAL PROFILES
The PAPs that are visualized in Fig. 7(c) show the spatial distribution of the MPCs in the scenarios MH1 and MH2.
The color and the size of the MPCs vary as a function of the path gain.Each PAP is presented in a relative coordinate system with ϕ oriented clockwise so that the direct paths are located at an AoD of ϕ AoD = 0 • and AoA of ϕ AoA = 0 • , respectively.This way, the PAPs are presented from the point of view of the communication system and it is easier to compare them with each other.For a longer distance between TX and RX in MH1, the propagation paths are mainly distributed with small AoDs along the axis ϕ AoD = 0 • and concentrated at small angles around the direct path.The strongest MPC beside the direct path has an approximately equal AoD and AoA but with opposite sign.Some weaker MPCs are spotted at ϕ AoD = −180 • , ϕ AoA = 0 • .The PDPs in Fig. 8 show the related temporal distribution of the MPCs.The majority of MPCs are located in an interval up to an excess delay of 15 ns.After that, three groups around a delay of 70 ns, 90 ns, and 125 ns with 5 to 9 MPCs The comparison of the PAPs shows a correlation between the geometric environment in the data center and the measurement results.The setup of MH1 is a stretched version of the setup of MH2.Hence, reflections at the racks are expected to be at smaller angles compared to MH2.Similar spatial distributions along the axis ϕ AoD = 0 • are observed in an aircraft cabin [27] and a train wagon [28] that also have a corridor-like form.In that data center, strong reflections occur on the racks, whereas some weaker reflections with larger delay can be traced back to the end of the aisle.The deviation of the AoD and AoA are linked to the offset of Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.The ToR scenarios present a sparser PAP compared to the MH setups.Comparing ToR1 with MH2 that have the same (x, y)-coordinates and just a different height, the strong MPC is visible in both scenarios.For MH2, the MPC is likely reflected by the front of the racks whereas the MPC is reflected by the plastic curtains for ToR1.However, the other strong MPCs in MH2 in the region around ϕ AoD = −16 • , ϕ AoA = 40 • are not present in ToR1 showing the different impact of the plastic curtains and the influence of the environment and the material properties.In contrast, the cluster at 95 ns is also present in both scenarios which suggests a fix, height-independent influence of the room.

2) GAIN DISTRIBUTION OF MULTIPATH COMPONENTS
The influence of multipath propagation and the possible property of a sparse channel are important characteristics for the design of a communication system with its high complexity.Therefore, the impact of the MPCs in terms of their gain is presented and analyzed in Fig. 11.All extracted MPCs of each double-directional measurement are normalized with regard to their respective maximum and sorted in a descending order.After the normalization, the strongest path corresponds to an MPC index of 0 and a magnitude of 0 dB.The residual MPCs follow based on their path gain whereas the delay, the AoD, and the AoA do not have an impact on the order.
The designed measurement scenarios show different channel conditions of the variety of channels that can occur in a data center.The following observations and conclusions can be made based on the gain distributions in Fig. 11.The strongest normalized MPC beside the direct path is detected in ToR1 with an additional attenuation of 3.2 dB.The MPCs decrease exponentially in logarithmic scale and flattens around the MPC index of 10, while some consecutive MPCs keep the same level (e.g., MH3, MH4, ToR4).As of an MPC index of 3, the MH scenarios have generally stronger MPCs than the ToR scenarios.Thus, the characteristics of the environment clearly have an impact on the propagation.Longer distances between TX and RX at MH (MH1 and MH4) have a stronger multipath propagation compared to the shorter distance in MH2 and MH3 since the relative path length between the MPCs decreases.A similar behavior was observed in a vehicular scenario investigating the side reflection from a vehicle on the neighboring lane [29].

3) EVALUATION OF CHANNEL PARAMETERS
In general, the design approach of a communication system entails different requirements on the system parameters.In this context, the required SNR is an important figure of merit.The communication system of interest does not perceive MPCs that lie below the noise.Consequently, the channel characteristics change and depend on the specifications of the communication system of interest.In contrast to a global, fixed channel parameter [21], the channel parameters that are seen by the communication system of interest are therefore a function of a threshold c th,dB .This threshold c th,dB specifies the maximum acceptable additional path loss of the MPCs with respect to the strongest MPC of the channel.Only MPCs that have a lower additional path loss than the given threshold are considered in the calculation of the respective channel parameters.
Analyzing the ToR scenarios among each other for a threshold greater than 15 dB, the RMSDS presented in Fig. 12 is first comparatively low for ToR1, then increases for ToR2 and ToR3 and finally decreases for ToR4 and ToR5.This behavior can be explained as follows: For short distances the direct path is strong and only MPCs with a high gain and an inherent, relatively short delay have an impact.For medium distances under OLoS conditions, MPCs with a larger delay gain influence due to the lower relative gain.For long distances finally, the number of MPCs decreases so that even an attenuated direct path is unchallenged.
The coherence bandwidth has to be interpreted in the context of the envisioned bandwidth of the communication system that is in the order of magnitude of multiple GHz for THz communication systems.In the majority of cases, the coherence bandwidth that is calculated with a correlation level of γ CB = 0.8 significantly drops below 500 MHz once a first residual MPC occurs.The inversely proportional dependency between the RMSDS and the coherence bandwidth [20] is clearly visible in Fig. 12.Note that the RMSDS and the coherence bandwidth can naturally also decrease and increase, respectively, with increasing threshold.
The ToR scenarios with the same setup but with different polarizations (ToR2 and ToR3, ToR4 and ToR5) behave similar for the RMSDS and the coherence bandwidth whereas the comparison at MH with MH2 and MH3 provide different results.For vertical polarization, MH2 has a higher RMSDS than MH3 with horizontal polarization up to a threshold of 22 dB.For a higher threshold, the relationship turns around and MH2 has a higher coherence bandwidth.ToR4 and ToR5 allow for the highest coherence bandwidth up to a threshold of 30 dB.
With values below 20 • , the ASoD presented in Fig. 12(c) is significantly lower than the ASoA in Fig. 12(d).The highest ASoD is provided by ToR1 with 19 • whereas the highest ASoA is found in MH1 with 66 • .It is notable that the ASoA is dominated by the strong MPCs and does not change significantly once the first residual path occurs.Also the K-factor presented in Fig. 12(e) shows a comparable behavior.Once the first residual MPC is received, the Kfactor only drops in the order of 2 dB.It has to be kept in mind that the K-factor is a figure of merit that only depends on the gain of the MPCs and does not account for spatial or temporal characteristics.
In contrast, the MED depending only on the delay does not account for gain and spacial characteristics.Fig. 12(f) shows that MPCs with a significant delay occur at a threshold of 15 dB.Below this threshold, all propagation channels have an MED below 3 ns that corresponds to an additional path length of 0.9 m.The MED for a threshold greater than 37 dB in ToR1 amounts to 190 ns.
The measured scenarios cover a wide range of possible channels in the context of wireless links in a data center.They show the variety of channel characteristics and the dependency on the environment in terms of geometry and material properties.Measurements at MH show higher multipath propagation compared to the ToR measurements.The influence of multipath propagation in ToR setups first increases for medium distances and then decreases for greater distances between TX and RX.

IV. DOUBLE-DIRECTIONAL RADIO CHANNEL
The examination of the radio channel that incorporates the antennas of a communication system is essential for a successful system, protocol and algorithm design.The propagation channel that was measured in the azimuth plane can be interpreted as a radio channel of a communication system that transmits and receives with omni-directional antennas.Since multipath propagation drastically increases the RMSDS and reduces the K-factor with increasing HPBW of the antenna [30], the propagation channel represents a kind of worst-case scenario with regard to beneficial channel conditions for high-data rate applications.However, misalignment of a directional antenna could reduce the Kfactor even more and worsen the channel characteristics, so that efficient device detection and beam tracking become more important.The latter case will not be considered and proper antenna alignment is assumed for the following part of the paper.

4) METHODOLOGY
In this section, a realistic antenna diagram is applied to the discrete MPC at the RX and the resulting radio channel that a THz communication system would effectively face is evaluated.The antenna development for THz communications is a challenging task because of the small dimensions and the high gain that is required to compensate for high channel losses.In addition, beam steering capabilities are crucial for P2P links in scenarios with user mobility or reconfigurable links.The non-ideal characteristics of the THz antennas will affect the THz communication systems if strong MPCs or interferes are received via a sidelobe of the antenna.To evaluate the impact of THz antenna arrays, a simulated antenna pattern of a realistic 16 × 16 patch antenna array filters the MPCs at the RX.The antenna pattern of the array with a steering angle of 0 • is provided in [18].The main lobe is oriented towards the strongest path at the respective measurement position.
Compared to the propagation channel as worst-case scenario that can be interpreted as a radio channel with omnidirectional antennas, the selected antenna pattern represents a best-case simulation.Since the angular sampling interval of 8 • almost coincides with the local minima of the antenna pattern, the antenna optimally filters the MPCs.Other steering angles of the antenna array and other antennas will filter the propagation channel in a different way that might result in radio channels lying between this best case and the worst case.

5) GAIN DISTRIBUTION OF MULTIPATH COMPONENTS
The normalized path gain of the MPCs of the radio channel that is visualized in Fig. 13 is significantly reduced compared to the previously presented propagation channel.Only one single residual MPC is above the threshold of −20 dB.It is remarkable that this MPC belongs to OLoS scenario ToR5 and experiences no additional attenuation by the antenna.In contrast, the strongest MPC of ToR1 is additionally attenuated by 21.38 dB compared to the propagation channel.MH1 and MH4 still show the highest multipath propagation for an MPC index greater than 1, although the impact is questionable with a normalized path gain lower than −21 dB and −30 dB, respectively.

6) EVALUATION OF CHANNEL PARAMETERS
The RMSDS in Fig. 14(a) shows that MH1 is the most affected by multipath propagation as of a threshold of 21 dB.All other RMSDSs stay below 1.5 ns.Interestingly, the coherence bandwidth presented in Fig. 14(b) does not decrease before the second residual MPC at a threshold of 26 dB.Changes in the coherence bandwidth of MH4 and ToR5 are visible at a threshold of 36 dB and 37 dB, respectively.Hence, the first five residual MPCs do not affect the coherence bandwidth in MH4.In addition, the coherence bandwidth of the radio channel of all other scenarios is not limited in this context.
With an ASoA that is drastically decreased from up to 68 • in the propagation channel to below 5.2 • in the radio channel, the ASoD and the ASoA lie in the same order of magnitude.This effect is naturally explained by the spatial filtering of the high gain antenna.As long as the AS is equal to 0 • , the MPCs are only radiated from or are incident to the direction of the direct path and is therefore independent of the radiation pattern of the deployed antenna.
The K-factor in Fig. 14(e) demonstrates that all scenarios a channel with one single up to a threshold of 19 dB.Even for a threshold of 40 dB, the K-factor does not fall under 18 dB.It is also remarkable that ToR4 has no residual MPC.Finally, the MED presented in Fig. 14(f) supports the course of the RMSDS since the MPC in MH1 has a delay of 84 ns corresponding to an additional path length of 25.2 m that can be associated with a reflection from the end of the aisle.
It is important to note that the present analysis only considers a P2P communication link and does not account for interference that is likely to occur if multiple THz links are active in a wireless THz network.Interferers that are located close to the RX might compensate the attenuation of the directional antenna.For instance, a link with a distance of 10 m will be strongly impaired by an interfering TX that is located 2 m away from the RX and cancels out the sidelobe level of the antenna of 13.4 dB while benefiting from a lower propagation loss.

V. POINT-TO-POINT RADIO CHANNEL
The setup and the conduct of double-directional and unidirectional measurements are sometimes cumbersome and time-consuming.Alternatively, P2P measurements quickly record the radio channel from the application oriented perspective.This way, a higher number of measurement positions from an application-oriented perspective can be covered in the same time.On the other hand, the measurements inherently contain the antenna diagram that cannot be compensated.For an easy comparison of the radio channels, all measurements are presented with an antenna gain of 15 dBi for the TX and the RX antenna, respectively, in the following section that presents measurements of ToR Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.direct link (DL), ToR RL, and MH links.The measurement positions are drawn in Fig. 15.

A. TOP-OF-RACK LINKS
Two types of P2P measurements are realized in the data center: First, DL measurements recording the radio channel of a link of interest under LoS or OLoS condition with TX and RX being aligned towards each other are performed.Then, reflected link (RL) measurements recording the radio channel of a possible alternative link in a NLoS scenario or an interference link where TX and RX are not aligned towards each other are conducted.A representative measurement setup of each group is presented in Fig. 16.

1) DIRECT LINKS
In the first setup, TX and RX are put on top of two neighboring racks in the same row to emulate a wireless THz link between those racks under LoS condition.The initial distance of 0.772 m is increased from P2P ToR DL1 to P2P ToR DL5 in four steps by integer multiples of the rack width up to 9.828 m.The antennas are located at a height of 2.10 m that is 0.20 m above the top of the rack.Fig. 16(a) illustrates the measurement setup.Due to the initial short distances and possible saturation of the RX, the P2P measurements on the racks are conducted with standard gain horn antennas with a gain of 15 dBi and an HPBW of approximately 40 • and 36 • in E-plane and H-plane, respectively, that are employed with vertical polarization.The SFDR of the measurements amounts to 23 dB.
The PDPs of the ToR links on the same row presented in Fig. 17 show that the path gain of the direct path agrees well with FSPL.Note that for distances greater than 2 m, a possible reflection from the top of the racks will fall in the same delay bin as the direct path due to the short additional path length that lies below the resolution of the channel sounder.In addition, a reflection can only be expected for a distance greater than 1.1 m based on the HPBW of the antenna.Also, a reflection at the ceiling that is located at a height of 3.00 m can be expected for a distance greater than 4.95 m with an additional path length of 0.32 m corresponding to an excess delay of 1.06 ns.The first two distances have a single component whereas the other three distances show two or three MPCs.The additional MPCs occur with an excess delay of around 1 ns and around 4.2 ns corresponding to an additional path length of 0.3 m and 1.26 m, respectively.The earlier MPCs for the distances of 6.016 m and 9.828 m can be linked to the reflection at the ceiling with an additional path loss of 9.0 dB and 12.9 dB with respect to the direct path.The later MPCs cannot be directly linked to any geometrical object in the environment.Although the presence of device clutter cannot be excluded for certain, the excess delay of the additional MPC varies slightly for different measurement positions which is a reasonable argument for the authenticity of the MPCs.In addition, the weaker MPC does not seem to have a notable impact since the coherence bandwidth of the third distance is not limited in contrast to the fourth fifth distance with 98 MHz and 357 MHz, respectively.
When wireless links connect racks from different rows, the plastic curtains cause an OLoS scenario.The doubledirectional ToR measurements discussed in Section III show an additional path loss of 6 dB and 16 dB with respect to FSPL for one and two rows of curtains and an angle of incidence of 45 • and 30 • , respectively.The following two setups examine the impact of the curtains for a perpendicular angle of incidence.First, TX and RX are positioned in P2P ToR DL6 at a distance of 5.962 m with one row of racks and two rows of curtains in between.The PDP of the radio channel measurement shows four MPCs with a path gain of −76.8 dB, −99.1 dB, −96 dB, −99.1 dB and a delay of 19.45 ns, 20.33 ns, 20.97 ns, and 34.9 ns, respectively.The direct path has an additional path loss of 9.3 dB with respect to FSPL.The second MPC corresponds in terms of delay to a reflection at the ceiling, the third MPC might be caused by a reflection on the top of a rack and the fourth MPC by a multiple reflection between curtains.The multipath propagation results in an RMSDS of 1.19 ns and a coherence bandwidth of 155 MHz.
The second setup P2P ToR DL7 measures the largest possible distance of 10.86 m at ToR level that is restricted by the length of the multi-function cables of the channel sounder.Here, four rows of curtains are located between TX and RX that are oriented to the curtains' normal again in a parallel manner.TX and RX are both shifted to a height of 2.55 m in order to top the highest racks lying in between.The measured MPC with a gain of −89.4 dB and a delay of 36.44 ns shows an additional path loss of 16.6 dB with respect to FSPL.Another detected MPC with an additional path loss of 24.8 dB with respect to the direct path and an excess delay of 12.72 ns is traced back to a path with two additional reflections at the curtains in one of the intermediate aisles.Although the RMSDS amounts to 0.73 ns, the coherence bandwidth is not limited by this residual MPC.An overview of the DL measurements is presented in Table 2.

2) REFLECTED LINKS
The influence of the plastic curtains on the propagation is further investigated in dedicated reflection measurements at ToR level.First, TX and RX point in the same direction and Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
towards the curtains in P2P ToR RL1.This way, they are aligned with the curtain's normal as shown in Fig. 16(b).Based on the revised signal processing, three striking peaks can be spotted with a path gain of −93.7 dB, −98.7 dB, and −97.1 dB and a delay of 6.78 ns, 34.54 ns and 47.35 ns, respectively, in this radar-like setup.The distance between the curtains is measured with a laser distance meter and amounts to 4.114 m and 1.890 m, respectively.This matches the distances derived from the PDP with 4.16 m and 1.91 m and results in a maximum absolute error of 0.046 m.It is noteworthy that the reflected power highly depends on the angle of incidence between the wave vector and the curtain's normal.In the second measurement P2P ToR RL2, TX and RX point towards the specular reflection point of the first row of curtains with an incidence angle of 8 • .The power reflected by the first row of curtains increases by 7.5 dB, whereas the power reflected by the second and the third row of curtains decreases by 14.3 dB and 19.5 dB, respectively.
It is also beneficial to characterize the influence of the surrounding environment, e.g., the room that is mainly represented by the walls, the ceiling and the floor.Reflections might also provide a reliable connection in case of an NLoS scenario.The reflection at the ceiling measured in P2P ToR RL3 with an incident angle of 77.7 • with respect to the ceiling's normal has an additional attenuation of 8.8 dB compared to FSPL.The additional attenuation of a reflection on the floor measured in P2P ToR RL4 with an incident angle of 69.3 • with respect to the floor's normal amounts to 9.8 dB.

B. MEDIUM HEIGHT LINKS
Links at MH might be in the situation that they face an NLoS scenario if they do not want to transmit just to the neighboring row of racks.Therefore, the following P2P measurements investigate the possibility to transmit from a TX to a RX via a reflector.
Three measurements are conducted with a metal plate of size 50 cm × 50 cm that serves as a reflector.The measurement setups are designed in such a way that the angle of incidence and the angle of reflection at the reflector coincide.Future application might also include intelligent reflecting surface (IRS) where this prerequisite becomes dispensable.
The first two measurements investigate the transmission to neighboring aisles via a reflection at the end of the row of racks.additional path loss of G dB,add = 5.2 dB.This might be due to the limited size of the reflector and the influence of the plastic curtains that are positioned as depicted in Fig. 18(a).A direct path between TX and RX is not visible.
In the second setup P2P MH2, RX is moved to r RX = (7.82|− 0.61|1.33)m in the main corridor that is oriented orthogonally to the rows of racks in order to examine the path loss to rows of racks that are further away.The position of TX is kept constant and the distance between TX and the center of the reflector amounts to 2.67 m.The distance between the center of the reflector and RX amounts to 6.21 m.The angle of incidence and the angle of reflection are 45 • with respect to the reflector's normal.Here, the additional path loss with respect to the theoretic FSPL of the reflected path is 9.5 dB measured at a delay of 29.8 ns.The higher additional path loss might be explained with the lower solid angle of the metal plate's projection seen by the TX.
The third measurement P2P MH3 examines the channel between two neighboring racks in the same row that communicate via a reflection at the adjacent row of racks.TX is located at r TX = (13.36|3.84|0.77)m and oriented in ϕ TX = 333  only one MPC at a delay of 8.17 ns.The metal plate has a reflection loss of approximately 3 dB with respect to the FSPL of the reflected path.When the metal plate is removed, the reflected path is shifted to a delay of 8.25 ns and shows an additional path loss of 6.8 dB with respect to FSPL.Moreover, two additional MPCs occur with an excess delay of 0.82 ns and 10.96 ns and an additional path loss of 16.5 dB and 21.8 dB with respect to the first reflected path.The first additional MPC might stem from the rear side of the rack door whereas the second additional MPC might be induced by other components in the rack.Table 3 summarizes the angle of incidence φ i , the additional path loss G dB,add with respect to FSPL, and the delay τ of the strongest MPC.
A reference measurement in a LoS setup without reflection with TX located at the bottom of one rack (the rack that holds the metal plate in Fig. 18

VI. INTRA-RACK LINKS
To gain flexibility, adaptability, and less cabling on a smaller scale, servers within the same rack can be connected by wireless links.The intra-rack channel is investigated by measurements that emulate a likely intra-rack link in server rack with 42U and a height of 1.95 m, a width of 0.603 m and a depth of 1.09 m.The measurements are again conducted with standard gain horn antennas with a gain of 15 dBi and an HPBW of approximately 40 • and 36 • in E-plane and H-plane, respectively.
The origin of the new coordinate system for the intra-rack measurements is defined at the lower right corner of the rack's front.Fig. 19 shows the measurement setup and the coordinate system.TX and RX are placed at different heights inside the rack facing towards each other.Three different transmission distances and a different number of additional servers in the rack are evaluated: The position of RX is kept constant at r RX = (0.36|0.88|1.59)m, ϕ RX = 0 • and θ RX = 13 • whereas TX is moved from the initial position (Fig. 19   respectively.In order to examine the influence of the metal enclosure and of additional severs in the rack, up to six servers are mounted in the initially empty rack.Here, two servers with a height of 2U and four servers with a height of 1U are used which are mounted at the positions 30-31, 27-28, 25, 23, 18, and 15 one after another.The cabling is omitted assuming a proper cable management and a high degree of wireless integration.It is notable that the doors are closed while the measurements are recorded.
All measurements are placed in a LoS scenario.The direct path of the shortest distance shows an additional path loss of 5.7 dB with respect to the theoretical FSPL which might be due to a challenging alignment and positioning of TX and RX in the setup.However, it could also be due to a destructive interference caused by a possible reflection at the server that would fall within the same delay bin and is not resolvable with the available measurement bandwidth.The direct path of the intermediate and the largest distance agree well with the FSPL with an additional path loss of the average path gain of 2.2 dB and 0.9 dB, respectively.
The main components of the large distance measurements vary within a range of 2.7 dB which is lager than the measurement accuracy of the channel sounder and likely due to interference effects.Due to the measurement procedure, some scenarios are measured multiple times.The deviation in those scenarios is much smaller with 0.7 dB that supports the presumption of interference.
Other MPCs occur with an additional path loss with respect to the direct path of 23.3 dB, 22.4 dB, and 17.6 dB for the small, intermediate and large distance, respectively.Fig. 20 shows the PDP of three exemplary measurements with the large distance between TX and RX.One striking MPC occurs in all measurements at an excess delay of approximately 13.2 ns that can be linked to the horizontal strut at the ground of the rack.This MPC is attenuated in a special setup (Empty rack with absorbers with orange diamonds in Fig. 20) where the doors of the rack are open and an absorber is placed on the strut in order to minimize its influence.In this setup, no MPCs are detected in the interval from 5 ns to 11 ns.In a rack without additional servers (Empty rack with blue crosses in Fig. 20), a few MPCs start to occur at 6 ns.When additional servers are mounted in the rack (e.g., Three servers with green circles in Fig. 20), the number of MPC drastically increases.The additional MPC are evenly spread between 5 ns and 16 ns with an additional path loss around 36 dB.However, no direct relation between the number of servers and the number of MPCs is observed.The MPC originating from the ground strut of the rack is also lower in the setups with five and six servers.With four and less servers it shows an additional path loss around 6 dB with respect to FSPL.
In conclusion, the direct path is by far the strongest MPC in the intra-rack scenario.A second MPCs originating from the metal enclosure occurs with an additional path loss of at least 17.6 dB which could lead to intersymbol interference (ISI) for communication systems with high SNR requirements.Many MPCs produced by the server blades arise with an additional path loss of around 36 dB.To reduce the impact of the metal enclosure, the rack could be covered with absorbers or a thin foil absorbing THz frequencies.

VII. CONCLUSION
This paper presents extensive channel sounding measurements in different scenarios for wireless communications at THz frequencies in a data center: • Double-directional measurements and P2P measurements examine the propagation channel and the radio channel in intra-rack, MH inter-rack, and ToR inter-rack scenarios.The possible application scenarios in a data center are structured in scenario-specific use cases and are differentiated based on the expected propagation characteristics.
• The spatial and temporal analysis of the measured channels in the context of the application and the design of a communication system, including system requirements such as the SNR, present the general observations and the influence of the environment and Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.materials.It further reveals relevant effects that have to be considered in channel modeling and system design activities of prospective wireless communication systems.The analysis derives relations between the channel parameters and shows the contrast between the propagation channel and the radio channel.The radio channel is also measured in further P2P setups and the channel parameters are evaluated with direct conclusions for communication systems.As verification, the measured profiles are traced back to the geometry of the environment and theoretical models.• Although THz communication systems will likely use high gain antennas with a strong spatial filtering effect, multipath propagation plays an important role for communication systems with higher SNR requirements.Therefore, the channel parameters are presented as a function of an SNR threshold that shows the resulting channel for the respective communication systems.Generally, the impact of multipath propagation is higher in MH setups than in ToR setups.• The comparison of the double-directional propagation channel as worst case scenario and the simulated double-directional radio channel as best case scenario illustrates the filtering effect of the directive antenna.The antenna likely attenuates many MPCs.However, when a MPC and a sidelobe of an antenna array matches, additional attenuations of only a few dB are possible leading to ISI. • A simplified summary of the propagation effects, focusing mainly on a first-order approximation for ToR propagation, is presented in Table 4. Reflection loss from the plastic curtains in the different measurement setups varied between 3 dB for high angle of incidence and 28 dB for perpendicular incidence so that a firstorder approximation with a constant value is hardly possible.
• The measurement data of the whole measurement campaign is published so that the research community has a collective benefit [15].Based on the measurement results various conclusions on propagation characteristics at THz frequencies can be made and the system design and development of protocols and Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
algorithms can be adapted.The measurements together with the adherent analysis lie the foundation for scenario-specific link-and system level simulations and modeling activities of higher layer protocols.

FIGURE 1 .
FIGURE 1. Illustration of a data center with wireless links.

FIGURE 2 .
FIGURE 2. Floor plan of the data center.

FIGURE 3 .
FIGURE 3. Classification of the wireless links in a data center.

FIGURE 5 .
FIGURE 5. Positions of the double-directional measurements.

FIGURE 6 .
FIGURE 6. Channel gain of the double-directional measurements.

FIGURE 7 .
FIGURE 7. PAP of double-directional channel measurements at MH.

FIGURE 8 .
FIGURE 8. PDP of the double-directional measurements at MH.

FIGURE 10 .
FIGURE 10.PDP of the double-directional ToR measurements.

FIGURE 11 .
FIGURE 11.Normalized MPCs of the propagation channel sorted by their path gain.

FIGURE 12 .
FIGURE 12. Channel parameters of the double-directional propagation channel.

FIGURE 13 .
FIGURE 13.Normalized MPCs of the radio channel sorted by their path gain.

FIGURE 14 .
FIGURE 14. Channel Parameters of the double-directional radio channel.
TX and RX are positioned on tripods at a height of 1.33 m.In the first measurement P2P TX and RX are positioned in neighboring aisles at r TX = (14.03|− 3.28|1.33)m and r RX = (10.37|− 3.28|1.33)m as presented in Fig. 18(a).The distance of both, TX and RX, to the reflector amounts to approximately 3.9 m.The angle of incidence and the angle of reflection are 24 • with respect to the reflector's normal.Due to the size of the tripod, TX and RX are positioned in the middle of the aisle.Compared to the theoretic FSPL of the reflected path at the metal plate, the path measured at a delay of 26.26 ns shows an
• and θ TX = 65 • pointing towards the opposite rack.RX is located at a different height at r RX = (13.36|2.62|1.99)m and oriented in ϕ RX = 27 • and θ RX = 115 • .The point of reflection can be assumed at r refl = (14.73|3.23|1.38)m.Fig. 18(b) shows the described setup.First, the setup is measured with metal reflector that shows (b)) with r TX = (14.35|3.91|0.75)m, ϕ TX = 235 • and θ TX = 53 • and RX located at the top of the opposite rack in the neighboring row (the same position as in Fig. 18(b)) with r RX = (13.36|2.62|1.94)m, ϕ RX = 55 • and θ RX = 128 • agrees with FSPL and shows no other MPCs.

FIGURE 19 .
FIGURE 19.Setup of intra-rack P2P measurements shown from the backside.

FIGURE 20 .
FIGURE 20.PDP of the intra-rack P2P channel measurements shown from the backside.

FIGURE 21 .
FIGURE 21.Additional PDPs and PAPs of MH double-directional measurements.

FIGURE 22 .
FIGURE 22.Additional PDPs and PAPs of ToR double-directional measurements.