RadarVerses in Metaverses: A CPSI-Based Architecture for 6S Radar Systems in CPSS

Metaverses have caused significant changes in the industry and their academic foundation can be traced back to the term cyber–physical–social systems (CPSS), which was proposed in 2010. Radar is an important sensor in sensing systems that are widely applied in many fields, especially, in autonomous driving. To deal with the complex environment, smart radars with real-time information processing capabilities are required. Human factors play a critical role in the operation and management of radar systems, thus, digital twins’ radars in cyber–physical systems (CPS) are unable to achieve intelligence in CPSS due to an incomplete consideration of human involvement. For this consideration, we propose a novel framework of RadarVerses for smart radars in metaverses based on ACP-based parallel intelligence, which is also known as cyber–physical–social intelligence (CPSI). RadarVerses consist of five main parts which are physical radars, descriptive radars, predictive radars, prescriptive radars, and deep radars. To construct RadarVerses at the technical level, we introduce four main technical foundations: 1) communication technology; 2) scenarios engineering; 3) foundation models; and 4) digital workers. In addition, we also provide a case study about LiDARs’ predictive maintenance of accumulated snow in RadarVerses.

role in metaverses that they not only complete the perception of surroundings in the physical world but also serve as a bridge between physical space and cyberspace. Radar is a type of sensor that uses lasers or electromagnetic waves to provide accurate 3-D depth information about the environment. It is initially applied in the military [2], [3] and then plays an increasingly important role in other fields, especially, in autonomous driving [4], [5]. Due to the increasing environmental complexity of autopilots, it is necessary to construct smart radar systems that can perform real-time intelligent adjustments.
Metaverses have introduced human factors into conventional cyber-physical systems (CPS). The academic foundation of metaverses is cyber-physical-social systems (CPSS) proposed by Wang in 2010 [6] and metaverses can be regarded as the specific realization of CPSS. CPSS are complex Morton systems with a self-fulfilling prophecy that human intervention will affect the system's output, while CPS correspond to Newton systems that operate independently of human beings [7], [8]. Digital twins' radars [9], [10], [11], [12] have proven to be an effective tool to construct smart radars in CPS and it has already solved many problems in autonomous driving [13], [14], [15], [16]. However, the operation and maintenance of radar systems are closely related to human involvement. Although metaverses have provided simple human-computer interfaces, they lack not only the timely virtual-real interaction but also the ability to generate deep intelligence. Owning to the incomplete consideration of human factors, digital twins' radars are insufficient to achieve intelligent radars in metaverses. ACP-based parallel intelligence [17], which is also known as cyber-physicalsocial intelligence (CPSI), can address these issues effectively. The ACP method proposed in 2004 [18] is a methodological framework to build smart systems in CPSS: A denotes artificial systems, C corresponds to computational experiments, and P is parallel execution. It has already been widely applied in many fields, including control and management [19], [20], [21], [22], transportation [23], [24], and sensing systems [25], [26], [27], [28]. Based on the ACP method and CPSI, we propose a novel framework of RadarVerses for 6S radars in this article. 6S, including safety, security, sustainability, sensitivity, service, and smartness, is the new evaluation criterion for radar systems [29], [30]. RadarVerses can not only intelligently adjust to dynamic environments through virtual-real interaction in real time but also break the hardware limitations of physical radars. It also provides an efficient This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ human-computer interaction mechanism to achieve knowledge automation [31]. RadarVerses consist of five main parts: 1) physical radars; 2) descriptive radars; 3) predictive radars; 4) prescriptive radars; and 5) deep radars. We present communication technology, scenarios engineering, foundation models, and digital workers to build RadarVerses. Besides, a case study on LiDARs' predictive maintenance for accumulated snow in RadarVerses is provided. This article is organized as follows. Section II introduces the common automotive radars and their limitations. The framework and process of RadarVerses are illustrated in Section III, and Section IV discusses the technical infrastructure of RadarVerses in detail. Section V provides a specific case about LiDARs' predictive maintenance in mines. Section VI concludes this article and prospects for future work.

II. AUTOMOTIVE RADARS
Radars that can measure depth information about the environment are indispensable in autonomous driving. In this section, we will discuss the two most common types of automotive radars: 1) LiDARs and 2) mm-wave radars.

A. LiDARs
Automotive LiDARs can obtain the 3-D structure of the environment with multiple laser beams, which is useful for perception tasks, such as object detection [32], [33], [34] and semantic segmentation [35], [36]. A LiDAR consists of three major components, which are laser sources, a scanning system, and photodetectors [37]. Laser sources are designed to emit laser beams of a certain power, scanning systems are used to cover a large area with the emitted laser beams, and photodetectors can convert the received optical signals into electrical signals via the photoelectric effect. Presently, there are two frequently used classification methods for LiDARs. The first method relies on waveform modulation, which can be further subdivided into the time of flight (ToF) and frequencymodulated continuous waveform (FMCW). Due to the low complexity and cost of pulse signal modulation, ToF LiDARs are widely used for autonomous driving. The second one is categorized according to different scanning systems. As shown in Table I, LiDARs can be classified into four types, which will be discussed further below: mechanical LiDARs, microelectromechanical systems (MEMSs) LiDARs, optical phased array (OPA), and flash LiDARs [5], [37].
1) Mechanical LiDARs: Mechanical LiDARs are currently the most mature and widely applied automotive LiDARs [37]. Multiple laser sources are stacked vertically and the common configurations are 16, 32, and 64 laser beams. The density and spatial resolution of LiDAR point clouds gradually increase as the number of stacked laser sources rises, which is important for high-quality perception tasks. The mechanical rotation systems will spin the stacked laser emitters and complete the overall environment exploration. Although mechanical LiDARs can provide a 360-degree field of view (FoV), their complex internal mechanical structure makes them expensive and bulky, making large-scale deployment difficult.
2) MEMS LiDARs: MEMS LiDAR is a type of semisolid-state LiDAR, which is a transitional product between mechanical and solid-state LiDARs. It introduces advanced MEMS technology for manufacturing, and the most important component in MEMS LiDARs is the MEMS mirror [38]. In MEMS LiDARs, the laser emitter remains fixed during operation, and a constant voltage is exerted on MEMS mirrors to adjust the tilt angle, reflecting laser beams in different directions to achieve plane scanning. It can help MEMS LiDARs reduce the number of laser emitters to save costs and space. Nevertheless, the FoV of MEMS LiDARs is limited due to the physical structure constraints of MEMS mirrors.
3) OPA: OPA LiDAR is a solid-state LiDAR without any mechanical motion component [39]. The OPA is made up of several closely spaced optical transmitting and receiving units, each of which can be controlled independently by the voltage. Enhanced interference in a specific direction can be generated to sense the environment by controlling the phase relationship between multiple units. OPA LiDARs benefit from small size due to simple mechanical structures, but the small FoV is an issue that should be addressed in the future. Furthermore, the size of each unit in OPAs should be less than half a wavelength to ensure normal operation, which is a significant fabrication challenge.

4) Flash:
Flash LiDAR is another type of solid-state LiDAR with a similar operating principle to cameras [40]. Flash LiDARs can emit lasers to cover the entire detection area in a short period of time and then receive the returned light signals with high-sensitivity photodetectors. It is appropriate for mass production and deployment due to its simple mechanical structure and small volume. However, high power is required for Flash LiDARs to illuminate the entire region at once, which may endanger the safety of human eyes. Due to the limitations of laser power in the real application, the detection range of Flash LiDARs is typically less than 100 m.

B. Mm-Wave Radars
Mm-wave radars use centimeter-wavelength electromagnetic waves to sense the environment in autonomous driving. Owing to the great penetration of electromagnetic waves in the atmosphere, mm-wave radars can operate normally in different weather conditions, including rain, fog, and snow. The majority of mm-wave radars use FMCW signals with frequencies of 24 GHz for short range and 77 GHz for long range. With the advantages in volume and resolution, 77-GHz mm-wave radars will be the first choice in the future [41]. Current mm-wave radars, which are also known as 3-D radars, can provide sparse point clouds with information, including  range, azimuth angle, and Doppler velocity. 3-D mm-wave radars can be classified into three types based on the detection range in Table II: 1) short-range radars (SRR); 2) middlerange radars (MRR); and 3) long-range radars (LRR) [42]. In autonomous driving, SRR are primarily used for parking aid, MRR for lane change assistance and blind spot detection, and LRR for forward collision warning and adaptive cruise control.
With the advancement of multiple-input-multiple-output (MIMO) technology [43], 4-D mm-wave radar with additional elevation angle information has been suggested and is already being manufactured on a small scale [44]. It can generate denser point clouds than 3-D mm-wave radar, which is very important for perception tasks. Recently, a new concept of 5-D radar that incorporates micro-Doppler information based on 4-D radar was proposed. It can effectively improve the motion detection performance of small targets and is expected to be widely employed in the future.

C. Limitations
Although evolving radar technologies have achieved great success in autonomous driving, there are still some limitations to constructing intelligent radar systems. First of all, current radars are unable to adjust their operating modes in real time to deal with dynamic external environments, which is a serious problem in the real application. Second, local data processing is adopted by all radars at present. It is a burden on physical radars hardware and advanced cloud computing should be implemented. Finally, there is a lack of interaction between humans and radar systems during operation. Users are only able to analyze the collected data while intervention in the process of data collection is not permitted. Due to the above problems, a new paradigm of smart radar systems in metaverses should be provided.

III. RADARVERSES
In this section, we propose RadarVerses which is a novel technical architecture for constructing 6S radar systems in metaverses. As shown in Fig. 1, RadarVerses consist of five main parts: 1) physical radars; 2) descriptive radars; 3) predictive radars; 4) prescriptive radars; and 5) deep radars. Physical radars perceive and collect data in the physical world, while descriptive radars complete the construction of artificial radar systems in cyberspace. Predictive radars conduct various computational experiments with artificial systems, and prescriptive radars provide indicative feedback to both physical and virtual radars, constituting two closed loops. Deep radars operate as actuators in radar systems to complete parallel execution in physical and cyberspace. RadarVerses follow the principle of "small data to big data to deep intelligence" and provide a platform for virtual-real interaction in real time.

A. Physical Radars
Physical radars refer to the process of sensing the environment and collecting data in the physical world. In autonomous driving, LiDARs are used to acquire dense point clouds, and each point is generally represented by four values: 3-D coordinates in space and signal intensity. RAD tensors and sparse point clouds are two types of data representations for mm-wave radars [45]. RAD tensors are generated by performing three times fast Fourier transforms (FFTs) on ADC signals in the range, angle, and Doppler velocity dimensions, respectively. Each mm-wave radar point has five parameters, including 3-D coordinate, radar cross section (RCS), and Doppler velocity, which is different from a LiDAR point.

B. Descriptive Radars
Descriptive radars are designed to build complete artificial radar systems in cyberspace and each physical radar can correspond to multiple descriptive radars in artificial systems for different applications. Apart from high-fidelity sensor models, descriptive radars also consider scenarios modeling in various weather conditions to model the physical environment more realistically. Besides, it is a pioneering work that takes into account the social environment, including human behaviors, intervention, and thoughts. Due to the complete consideration of human factors, descriptive radars are more closely to the real systems compared with digital twins' radars.
Descriptive radars serve three primary functions in autonomous driving. First, descriptive radars can be used for low-cost simulation experiments to optimize physical radar systems, such as new radar validation [46] and the optimal placement of multiple automotive radars [47]. Second, descriptive radars can help with the long-tail problem of data. It is a costly task to collect data in the physical world and the majority of datasets are normal driving datasets in clear weather without corner cases or severe weather conditions [48], [49]. Thus, models trained on current datasets are incapable of dealing with emergencies effectively. Descriptive radars can generate a large amount of synthetic data to address this issue, and virtual data has already demonstrated significant advantages in object detection [13], segmentation [14], [15], and localization [16]. Finally, descriptive radars can guarantee the safe operation of radar systems during driving. To cope with unexpected physical radar failures, the working condition and position of descriptive radars should be consistent with physical radars in real time. When the physical radar fails, synchronized data generated by virtual radars can be adopted for emergency management to protect drivers.

C. Predictive Radars
Predictive radars conduct computational experiments with artificial systems in cyberspace. The computational experiment is a broad concept that consists of various tasks, including object detection [32], [33], [34], semantic segmentation [35], [36], and trajectory planning [50], [51], [52], [53]. Due to human factors in CPSS, it is impossible to identify the system's optimal strategy using only the collected data. In order to achieve convergent solutions, predictive radars forecast data from future scenarios first and then evaluate different situations [54], [55].
In the context of autonomous driving, predictive radars can forecast not only the external environment but also the internal working conditions of radars. In terms of external surroundings, predictive radars can perform functions such as key area estimation and obstacle warning in blind spots. They first process information about current data using the evolving cooperative perception [56], [57], [58], then complete temporal forecasting, and conduct evaluation for the predicted traffic scenarios. For internal working conditions, predictive radars can realize sensors' predictive maintenance in bad weather which has a significant impact on radar performance. The need for human intervention can be determined in real time, reducing human workload and enabling more efficient radar system management. Furthermore, predictive radars can predict keyframes of data to ease the burden of communication transmission, requiring only keyframes to be transmitted between physical space and cyberspace.

D. Prescriptive Radars
Prescriptive radars can provide indicative control for physical and virtual radar systems. It achieves the transition from predictive knowledge to the feedback of current systems, resulting in double closed loops in both physical and cyberspace. Prescriptive radars redefine radars with the software system [59] and generate prescriptive intelligence that can be fed back to controllers or humans for final decisions [60], [61].
In autonomous driving, prescriptive radars calculate the specific feedback scheme based on the optimal strategy obtained through computational experiments [62], [63], [64], [65], [66]. For example, prescriptive radars can adjust important parameters in radar systems such as scanning frequency and distribution of laser emitters to focus on the key area during operation [67]. Besides, they also can provide users with an appropriate maintenance plan based on weather conditions.

E. Deep Radars
Deep radars, which can be regarded as actuators and controllers of radars, complete parallel execution in both physical space and cyberspace based on the generated descriptive, predictive, and prescriptive intelligence. Deep radars enable virtual-real interaction in real time, allowing RadarVerses to form a combination of physical, artificial, and mental worlds.
The evolving digital signal technology promotes the development of deep radars at the technical level. Conventional automotive radars generate waveforms using an analog modulation method, which is difficult to adjust during operation, whereas digital modulation methods, such as OFDM [68], and PMCW [69] can address this issue efficiently. They can alter the physical parameters of radar systems in real time [70], and even adjust the waveform type according to the environment. Besides, deep radars are closely related to the development of solid-state LiDAR technology that can avoid the manipulation of complex mechanical structures to facilitate software-to-hardware feedback control.

IV. TECHNICAL INFRASTRUCTURE
To construct RadarVerses in metaverses, advanced techniques from various fields, such as communication technology and artificial intelligence, are required. As shown in Fig. 2, we will introduce four main technical foundations in this section, which are communication technology, scenarios engineering, foundation models, and digital workers.

A. Communication Technology
With the rapid development of the Internet, Internet of Things (IoT) which denotes the connection of all physical objects via Internet [71], [72], [73], [74], [75], [76] has gained popularity. The sensing capability of a single automotive radar is limited, necessitating the use of vehicle-to-everything (V2X) [77], [78] to connect multiple radar systems to form sensing networks. V2X, which includes vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) [79], [80], can effectively obtain comprehensive information about local areas and improve perceptual performance. In order to realize 6S radars in RadarVerses, it is also necessary to achieve low-latency interaction between physical and virtual radars. Advanced communication technology, such as 5G [81], provides technical support for real-time interaction in RadarVerses.
Although the rapid development of communication technology has sped up data transmission, we propose the following improvements to make better use of communication resources in RadarVerses. First, only the models and deep knowledge will be shared among radar systems that can reduce data transmission amount significantly. Besides, physical radars will operate on an intermittent basis, with time intervals dynamically adjusted according to dynamic weather and road conditions. Virtual radars provide guidance for the operation of physical radars by predicting keyframes and nonkey frames are directly generated by virtual radars for compensation.

B. Scenarios Engineering
Radar systems primarily use features extracted from public datasets to train models for various perception tasks. However, it is impossible to cover all road and weather conditions in public datasets and the extracted features cannot be guaranteed to be valid in all scenarios. Therefore, we propose a transformation from feature engineering to scenarios engineering for radars to achieve trustworthy AI [30].
Scenarios engineering is known as a collection of scenarios and activities within a specific time and space. RadarVerses divide traffic scenes into small regions to build multiple scenarios in autonomous driving. For each scenario, users can generate virtual data and perform computational experiments on different road conditions in this region. Researchers can create leaderboards for different downstream tasks and everyone can upload their models to leaderboards for comparison, which is similar to the operation of current public datasets. The best model for each scenario is transferred to the model library in the cloud for sharing with all users. When the vehicle enters a new scenario, the corresponding model is downloaded from the model library to serve as the initial model and is updated using the collected and predicted data. Scenarios engineering can reduce the burden of data collection and make more efficient use of computing resources by sharing models.

C. Foundation Models
With the rapid development of artificial intelligence, automotive radars are increasingly being applied for a lot of downstream tasks, resulting in the emergence of a large number of small models. Fragmented models complicate unified deployment in practical applications, whereas foundation models can efficiently solve this problem. Foundation models use a large amount of data as input, which is pretrained at first and then fine-tuned for different tasks [82]. They not only have the advantage of good versatility but they can also outperform small models, which has already been demonstrated. To jointly realize intelligent radar systems, we will deploy three types of foundation models in RadarVerses: 1) vision foundation models; 2) language foundation models; and 3) multimodal foundation models [83]. Vision foundation models are primarily applied for the intelligent operation and management of radar systems. When a rough blueprint is entered by the manager, vision models can generate specific flowcharts to efficiently organize multimodal and language models to achieve the goal. Language foundation models that are trained on human speech data are responsible for processing speech signals in the application. They can invoke the relevant multimodal models to fulfill the requirements. Multimodal foundation models, which have received extensive attention recently, train with data from multiple modalities, such as point clouds, RAD tensors, and trajectories, and then fine-tune for specific tasks.
In the real application, a large model library is constructed to store various multimodal and language foundation models in the cloud [84]. Based on the flowcharts from vision models, models in the model library will be transferred and deployed to the edge. The edge foundation models in autonomous driving focus on local, short-term tasks and upload the acquired knowledge to the cloud to solve global, long-term problems.

D. Digital Workers
We are currently living in an era of an intelligence explosion, with a large number of algorithms and foundation models  emerging that far exceed the capabilities of the human brain. To efficiently manage virtual radars, we introduce digital workers [85] to liberate human workers' intellectual work, just as robotic workers in the industrial age liberated human workers' physical work.
Digital workers not only serve as the interactive interface [86], [87], [88] but they can also complete tasks in cyberspace automatically. Digital workers enable the conversion of complex decision-generation problems into simple selection problems in RadarVerses. When a human worker proposes a specific task, he can give commands to digital workers as interfaces directly. Following the processing of input data, it will direct other digital workers to select the appropriate models, construct the method framework, and return the optimal strategy to human workers. In the future RadarVerses, human, robotic, and digital workers will collaborate to realize the efficient management of radar systems. Human workers with intuitive rationality account for 5% of the total and are only responsible for leadership and organization. 15% are robotic workers with adaptive rationality that focus on physical labor, while the rest 80% are digital workers with computational rationality for mental labor in cyberspace.

V. CASE STUDY
With the complete consideration of human factors, we provide a case study of LiDARs' predictive maintenance [89], [90], [91] based on the framework of RadarVerses. It takes point cloud data as input and guides operators to maintain physical radar systems through computational experiments. Apart from regular urban roads, the autopilot can also be applied to some specific areas, such as mines and ports. Extreme weather conditions, i.e., snowstorms and sandstorms, are significant challenges to safe autonomous driving in mines [92]. Snowstorms in Fig. 3 not only interfere with laser beams' propagation [49] but also accumulate on the surface of LiDARs, reducing LiDARs' perception performance seriously. To keep LiDARs' normal operation, it is necessary to conduct manual snow cleaning on a regular basis. However,  the human judgment of whether maintenance is required is inaccurate and the commonly used regular cleaning method is incapable of dealing with various emergencies.
We propose to use a deep-learning-based method to realize accurate predictive maintenance and provide real-time guidance to operators. It also allows workers to monitor multiple mining cars remotely, significantly reducing labor costs. Due to the lack of relevant datasets, a new dataset was collected inside the campus using an experimental vehicle equipped with a 64-line Hesai LIDAR, as shown in Fig. 4. This dataset contains 400 frames of point cloud data from five indoor and five outdoor scenes in Fig. 5, with 80% serving as the training set and 20% for the test set. In each scenario, artificial snow is applied to simulate different levels of accumulated snow on the surface of LiDARs as shown in Fig. 6. In Fig. 7, the horizontal axis is the time that is positively correlated with snow accumulation, and the vertical axis represents the number of points in each frame. We observe that the number of point clouds has a negative correlation with accumulated snow from Fig. 7. When the number of point clouds drops to 80% of the normal condition, the distortion becomes severe and manual intervention should be introduced. According to the number of points, we divide the collected data into two categories of whether or not they should be maintained. PointNet++ [93] is trained for this classification task and the epoch number is set as 100. The Adam optimizer is used and the learning rate is 0.0005. The trained PointNet++ model achieves 72.5% accuracy on the test set, allowing managers to receive accurate information in real time.
Although the proposed method is proven to be efficient in LiDARs' predictive maintenance, the size of our dataset is the main problem. Besides, our dataset was only collected in the static campus environment, as opposed to dynamic mining scenes. In future work, we will collect data in real mines to expand our dataset and introduce the temporal features of multiframe data to conduct prediction more precisely.

VI. CONCLUSION
To construct 6S radar systems in metaverses, the novel framework of RadarVerses is proposed in this article. It not only provides a mechanism to achieve knowledge automation in CPSS but also constitutes a closed loop between physical space and cyberspace with physical radars, descriptive radars, predictive radars, prescriptive radars, and deep radars. We also introduce four gordian techniques to construct RadarVerses at the technical level. And a case study about LiDARs' predictive maintenance in RadarVerses is provided. In future work, we will apply the architecture of RadarVerses to investigate more intelligent operations of radar systems in metaverses.