IEEE 1588 for Clock Synchronization in Industrial IoT and Related Applications: A Review on Contributing Technologies, Protocols and Enhancement Methodologies

Precise time synchronization becomes a vital constituent due to the rigorous needs of several time-sensitive applications. The clock synchronization protocol is one of the fundamental factors that can define the quality of communication. Our study starts with a brief discussion on the application domain of precise time synchronization and comes with an in-depth study of the synchronization with the main focus on the IEEE 1588 Precision Time Protocol (PTP). We have compared the well-known synchronization techniques and conclude that the PTP is the most appropriate answer to robust clock synchronization though challenges are there that requires thoughtful efforts and modification in the current version. The working mechanism and main components of the PTP network are discussed. We have established a testbench using commercially available devices and development boards to evaluate the PTP performance under different configurations. Major sources of synchronization error and other aspects contributing to precision are examined. This paper discussed numerous approaches that could enhance the performance of the PTP protocol. Structures for PTP based wireless clock synchronization required by advanced applications has also been discussed. In the end, paper focuses on the main industrial application areas in which PTP plays an important role, including WLAN, optical data centers, Smart grid, IEC 61850, etc. We conclude the paper by identifying the future trends and research directions for PTP based clock synchronization.


I. INTRODUCTION
Time-dependent applications have become essential as technologies like industrial automation, robotics, and Internet of Things (IoT) have grown by many folds [1]. The rise of cloud computing and high-speed networks requires highly precise time synchronization [2]. Cheap oscillators or quartz crystals properties vary with power, aging, and heat. Therefore, two similar crystals are not guaranteed to oscillate at the The associate editor coordinating the review of this manuscript and approving it for publication was Rashid Mehmood .
identical time/frequency. These limitations put oscillators to run slightly different than the others. Replacement of computer's built-in cheap clocks with an expensive one is not practicable for large infrastructure. Consequently, a vigorous and effective way to synchronize clocks of a disseminated structure is essential [3]. Network Time Protocol (NTP) was the most diffused solution for clock synchronization. Lately, a more precise solution, known as PTP, has been proven to be more beneficial for clock synchronization [4]. Although the PTP communication algorithm is analogous to NTP's, PTP is different in its accurate hardware-assisted time record of the event named as time stamping [5]. PTP is one of the conceivable solutions for highly precise clock synchronization. However, there are still certain challenges that must be considered.
This paper conducted a comprehensive study and analysis of IEEE 1588 PTP based synchronization solutions for IoT and industrial applications. We have illustrated several aspects that might enhance the performance of PTP. Despite of the development that is made with these practices, highly precise clock synchronization is still an open issue, especially in wireless network communication. Other contents of the paper are ordered as follows. Section II provides a detailed background of the synchronization and its application in different emerging areas. Clock synchronization standards and comparison of the most popular synchronization techniques are also presented in this section. Section III discusses in detail the PTP algorithm, its versions, and implementation methods. Source of time error and enhancement approaches along with commercially available solutions are summarized in Section IV. Wireless PTP that is an extension to PTP is discussed in Section V. A thorough summary of the recent literature discussing software and hardware tools for PTP evaluations and different deployment techniques are listed in Section VI. Section VII presents the synchronization requirements and the role of the PTP in the area of IoT and related industrial applications including smart grid, optical data centers, and PTP based synchronization for WLAN with a specific attention on the infrastructure mode which is utmost applicable to industrial scenarios. Paper is concluded with challenges and future needs in Section VIII.

II. BACKGROUND
Coordinated Universal Time (UTC) is the mutual base for the worldwide public time. Two main modules are used to define it.
1. The International Atomic Time: the time gauge that integrates the output of greatly precise atomic clocks globally. 2. Universal Time (UT1) astronomic time is synchronized with UT1 to keep the definite length of a day [6]. UTC high precision is delivered by atomic clocks. Though, such reference clocks are very expensive and are not feasible for common customers as they may not require high accuracy. However, even a simple sensor may own time reference [7], normally they cannot offer essential time constancy in the long term due to time drift. Consequently, it is desirable to synchronize such clocks with an accurate time reference. Following subsections discuss the synchronization types, applications, and the existing standards.

A. SYNCHRONIZATION TYPES
In broad-spectrum, synchronization can be described in numerous ways. Clock synchronization could refer to time, frequency, or phase. It can be defined as frequency synchronization if dissimilar clocks drive to the identical state in the equivalent time interludes such as pulse per second (PPS) output. In another way, a system might utilize signals of unlike frequencies with an organized phase [7]. If a system requires clear information of time then it is called clock synchronization [6]. There are two main distributed time synchronization structures based on the continuity/discontinuity of the time. In Continuous-Time (CT) methodology transmission start/end times are interpreted constantly for the network. A highly accurate clock is essential to provide time info steadily. While Discrete-Time (DT) methods, described by the time incoherence, it defines the time slots of a fixed interval [5]. CT methods only rely on a precise clock synchronization method that makes CT approach most appropriate for high-speed networking requirements [8]- [10].

B. APPLICATION AREA
Timing demands on networking and control systems have become progressively challenging, particularly for real-time applications like nuclear fusion control, mobile communication, substation automation, and, modern manufacturing plant. The real-time market data require latency of one millisecond to pass by the cables, routers and, switches with the timestamp. Smart grid applications require time harmonization to permit many parts of the grid to attach or detach without disturbance. In IoT case, for distributed networks sensor data want to be gathered in real-time. In a softwaredefined network, highly accurate regulator clocks aid to attain accurate network apprises with a small packet loss rate [11]. Clock synchronization is one of the critical features that might control the quality of the communication. Synchronization is an essential part of the various application areas as discussed below.

1) SMART GRIDS
The smart grid is an extension of the legacy power grid supporting the smart bidirectional flow of energy and data to enhance the grid efficiency and reliability. Currently, power grids have several measurement and control data that requires accuracy from 100ms to 1µs or more [12]. The power grids have distributed and complex system with following processes that needs synchronization [13]: • Event reconstruction.
• System deviation control for time and frequency. • Time tagging in estimation and error discovery systems.

2) TELECOMMUNICATIONS
lots of possessions in advance and forthcoming wireless networks desire accurate synchronization, counting but not restricted to [13], [14].

3) IOT AND INDUSTRIAL APPLICATIONS
The info gathered from diverse types of sensors has an important role for manufacturing and business operations in the industrial internet. Alignment in time of the data coming from different geographical areas is very important. Algorithms, employed for the investigation of the big data, must have the precise sequence of the events [1], [15]. An incongruity in computing from a specific cluster of sensors could end in the wrong alarms and judgments, needless expenditures, or even damage the equipment [14].

C. IMPACT OF SYNCHRONIZATION ERROR
Precise clock synchronization is mandatory for coordinated real-time chores. Desynchronized clocks in time-sensitive applications can degrade the system performance and damage the infrastructure [16]. Let us discuss the case of a smart grid that installed smart meter and other intelligent electronic devices (IEDs) at the substation, to monitor and control the instruments they need sub-millisecond clock synchronization accuracy. The absence of accurate synchronization will result in lose control and compromise the grid's safety. In the manufacturing industry if the robots working in a group on the same product lose the synchronization can result in clashes of their acts and upset the production pipeline. Time synchronization is challenging in wireless sensor networks (WSNs) for distributed systems like structural health monitoring (SHM). Time shift causes error among the raw data from sensor nodes that affect the data integrity and can result in the incorrect mode shape identification of the structure under observation. In [17] authors illustrate the influence of time synchronization error (TSE) on the modal analysis and damage detection and concluded that TSE has a strong impression on the mode shape detection, damage localization, and damage discovery. [18] analyses the effect of clock synchronization loss on the synchro phasorsbases applications. Experimental evaluations demonstrate that a phase angle monitoring system expresses an erroneous power system state and operating time of an anti-islanding protection application also gets disturbed due to loss of time-synchronization signal input to PMUs.
The industrial networks use deterministic data transmission and reception with a predefined node location [19]. Poor synchronization among the nodes affects deterministic data exchange and destroy the whole control system. Time desynchronization in an Industrial Internet of Things (IIoT) network can affect the system performance or may result in safety compromises. clock synchronization is necessary for coordinated real-time operation, in IIoT where connectivity is the key concern hence secure clock synchronization is an imperative research issue.

D. TIME SYNCHRONIZATION STANDARDS AND SOLUTIONS
To fulfill the synchronization needs in the modern networks many international standards have been established.
The literature highlights mainly two classes of synchronization schemes: one is hardwired and the other is protocolbased. Hard-wired practices do not need to interchange the info over the network, and are extensively accepted as they fulfill even the severe requirements [20]. However, the lack of flexibility adversely affects the overall operational costs of the system [21].
1) GPS-BASED TIME SYNCHRONIZATION Despite of the high accuracy it has numerous disadvantages as it entails that all the participating nodes are furnished with a GPS receiver. Outdoor GPS antennas are obligatory in many scenarios; these outdoor antennas are susceptible to vandalism and cannot be installed in underground substations [20], [22].
2) 1PPS OR PULSE PER MINUTE (PPM) sources are capable to deliver the utmost precision of the frequency synchronization, no additional info is communicated.
3) NETWORK SYNCHRONIZATION PROTOCOLS NTP is the most popular protocol and is realized by numerous communication stacks. It goes through many versions since 1985. NTP version 3 was stated in the Internet Engineering Task Force (IETF) (RFC)-1305 and the newest version NTPv4 was suggested in RFC-5905 [6], [23]. Classically, NTP synchronization accuracy is highly related to the performance of the underlying network [20], [24], [25]. 4) SIMPLE NETWORK TIME PROTOCOL (SNTP) is a slightly less complex implementation of NTP that does not realize a few of the difficult time-tracking procedures [26]. SNTP is appropriate for embedded devices as it needs few resources as compare to the NTP at the cost of lower synchronization accuracy [27]. 5) PTP recognized as IEEE 1588, it is another standard for precise time synchronization. This protocol provides hardware-level time precision by a customary network links, using devoted timing hardware to every port in an Ethernet environment [28]. This protocol allows attaining synchronization in the range of nanoseconds provided that all the network components like switches and routers are PTP enabled [29]. 6) WHITE RABBIT (WR) White Rabbit (WR) protocol, also known as PTP version 3 was initiated and used at CERN (European Organization for Nuclear Research) to align the clocks of their accelerator complex. WR network achieves sub nanoseconds timing accuracy using existing standards [30]. 7) THE INTER RANGE INSTRUMENTATION GROUP (IRIG): This approach uses codes to competently route the timing system to the consumer. Specifically, atomic frequency and GPS receivers are usually armed with an IRIG outcome [31]. The IEEE 1344 addition to IRIG-B Time codes succeeded by IEEE C37.118 describes synchronized phasor capacity for power systems. It describes the usage of PTP in power systems. ITU-T recommendation G.8265 Designates the placement of PTP standard in telecom networks for backup the essential timing precision and transmission of frequency [32], [33].  8) TIME-SENSITIVE NETWORKING (TSN) TSN working group developed a group of standards IEEE 802.1 for data broadcast over Ethernet. IEEE 802.1AS is defined to carry accurate timing and perform the synchronization in bridged LANs. G.8261, G.8262, and G.8264 make the synchronous Ethernet (SyncE) group of standards. It postulates the architecture, clocks, and messaging over the Ethernet physical layer [34], Uses includes cellular networks and access technologies. Though, the majority the time synchronization designs consider the centralized time circulation.

E. COMPARISON OF THE WELL-KNOWN SYNCHRONIZATION MECHANISMS
In this section, most popular time synchronization technologies: GPS time synchronization, NTP, PTP, and WR are compared in terms of their architecture and other relevant properties. Table 1 list and compare these protocols in term of synchronization precision, scalability, hardware requirements, cost, and other main features.
GPS-based time synchronization is the finest way to attain nanosecond accuracy. A GPS satellite contains atomic clock information and GPS receivers obtain precise time information [35]. This info is dispersed to all the associated nodes as shown in Figure 1. Furthermore, inside the data centers, GPS signals reception is difficult which makes it an unattractive approach.
NTP was established for packet-switched networks targeting millisecond synchronization accuracy. Time messages were exchanged using the User Datagram Protocol (UDP). A hierarchical architecture is shown in Figure 2 where the layer 1 time provider is synchronized to an atomic clock and passes down the timing data to the next level [5]. The distribution of the time information to multiple layers causes' loss of accuracy. Furthermore, each layer adds additional delay which continues to aggregates till the last layer. Consequently, NTP is not well suited for current delay-sensitive data traffic.
PTP is realized on the similar requirements that inspired NTP, but they mainly diverge in their implementation mechanism. Many factors make PTP more accurate than NTP. The first of them is that PTP has hardware-assisted time stamping which reduces the distance to physical receive/transmit ports of the reference clock. Another aspect is that PTP has many types of clock to support its synchronization accuracy these clocks include boundary clock, and transparent clock. In PTP, a grandmaster synchronizes entire slave nodes and this function eliminates all the additional delays that are practiced due to redistribution in NTP. Also, PTP uses active synchronization to compensate network introduce delays as discussed in section III.
WR is a low latency, Ethernet-based time distribution method for disseminated sensing systems. WR synchronization scheme uses the existing standards to achieve sub-nanosecond accuracy, these include Ethernet (IEEE 802.3), SyncE, PTPv2 along with precise phase measurement VOLUME 8, 2020 technique [36], [37]. Just like PTP, WR is a complementary solution to GPS mainly when the sky is not accessible or ethernet based telecom infrastructure is pre-installed.
This project is open source and achieves high accuracy up to sub-nanoseconds with a reliable and deterministic data delivery [38].
These properties make WR a favorable choice for smart grid application, though the accuracy is high for synchrophasor needs. The main bottleneck for WR protocol in the synchro phaser network is the absence of fiber optic cable in the electric grid, installing the fiber link in the feeder may not be a cost-efficient solution though the stability of WR over fiber optic is 950km [30]. A typical WR Network is consists of WR nodes and WR switches interconnected by fiber links. Figure 3 shows a WR network which is a standard Ethernet switched network foe data (no hierarchy) while for time synchronization it is a hierarchical model that goes from WR master down to other WR switches. In addition to standard ethernet switch properties, WR switch is also able to distribute the WR master clock using a precise phase measurement technique.

III. PRECISION TIME PROTOCOL (PTP)
PTP is based on the IEEE 1588 standard [39], a synchronization procedure for the dispersed packet-switched network, delivering accuracy around nanosecond. Data communication and synchronization are done using common cabling that results in easy deployment and permits the use of the standard network cabling [40], [41]. Cable distribution delays are not easy to measure simply by cable lengths. In a network data packet route changes dynamically and ultimately needs the correct estimation of network time delay for each packet. The additional delay could be added by other components present in the network like routers and switches and this added delay could be higher as compare to cabling delay. PTP presents a way that detect and compensate such time delays automatically [5].  A. PTP ALGORITHM PTP uses a message exchange algorithm along with a precise time source to periodically synchronize and look over all the clocks present in the network. It starts with the Best Master Clock (BMC) algorithm to recognize the master (the highly precise clock present in the network) and slave clocks. Seeing the grandmaster clock as the source, slave clocks obtained the time data by swapping request-response with it [4], [33]. Functionalities of the basic messages exchanged among the grand master and slave clocks are listed in Table 2 for proper functionality there are few other messages for signaling and management [42]. Figure 4 depicts the PTP algorithm, and how the PTP messages are exchanged in a specific sequence [8].
After establishing the master/slave hierarchy, the master clock transmits a Sync message including its own timestamp, tm1 to all the slaves in the network. This timestamp can be sent by a single-step or a two-step process. The master clock reads the time information and instantly sends a Sync message to the slave clock. A supplementary Follow_Up message is transmitted with the Sync message in two-step method which excludes the necessity to accurately read the timestamp and put to protocol message concurrently. On the reception of this Sync message, the slave clock records the time of arrival as t 2 . Delay_Req is time stamped by the slave as t 3 and master receive them at t 4 . At last, the master clock transmits Delay_Resp message to the slave clock time stamped at t 4 . Slave clock computes the offset and synchronizes with master's time, the calculation of the clock offset and average path delay is described in equation 1 and 2 respectively.
According to the clock type used in the PTP network, there are mainly two versions of the PTP, the first version defined in 2002 is known as PTP v1 . This version presented two types of clock, boundary clock (BC) and the ordinary clock (OC) [43]. The OC is the internal clock of each device in the network, while the BC is the clock that appointed to the devices between master and the slaves. Boundary clock enables the midway nodes between master and slave clocks also being synchronized. Before the slave node gets sync to the master, synchronization will be done at every hop among the master and the slave [28]. The modified version of PTP is called PTP v2 (revised in 2008) that offers another type of the clock in addition to the existing clocks. This clock is named as the transparent clock (TC) [44]. TC is one of the numerous enhanced features from ver-sion1 [4] and can resolve the queuing delay issue. Transparent clock passes the PTP messages with an additional feature of measuring their residence time in each intermediate node.
The residence time is the duration from the reception of the message at the node to the time it leaves the node [33], [41]. A modification field is introduced that includes a 48-bit nanosecond share and a 16-bit sub-nanosecond portion, this correction field removes a significant percentage of calculation faults. A short Sync message interval is permissible for varied update rates as compare to 1s Sync messages in PTPv1. Fast reconfigure synchronization hierarchy is empowered in PTP devices that enable the peer-delay mechanism to compute a peer delay for each communication bond. Network resource consumption is minimized by dividing the master selection and timing info into two different but very short messages. In the new message layout, the size of the Sync message payload is 46 B, instead of 165 B [4]. PTP v2 is recently revised in 2019 and come up with a slightly modified version as PTP v2.1 [45]. PTP-2019 improve the PTP-2008 optional features, and it is backward compatible with the older version some of the newly introduces features are as follows [46].
In this mechanism, a slave node periodically sends Delay_Request messages to the master node and the master replies with Delay_Request message. Path delay is estimated by using egress and ingress time stamps of the Delay_Request. Delay is measured for the full path from master to slave and the delay is corrected at the slave node. In Figure 5 complete delay that a sync message experience when it travels from master to slave (Node A to B) is corrected at node B.

2) PEER-TO-PEER (P2P)
In the case of peer-to-peer delay mechanism path delay is estimated between two neighboring PTP nodes irrespective of the type of node i.e. BC, TC OC. In general, TC stays calm in any other traffic but they actively participate in the P2P delay mechanism. Each node in the delay path estimates the delay for its preceding network link and applies correction for all pass-through event messages.

3) E2E VERSUS P2P
Which delay mechanism is better? The answer to this question is not straightforward because it depends upon the network. If a network is fully PTP supported i.e. all the switches or routers are guaranteed to be PTP enabled then P2P is best otherwise E2E is a feasible choice. Figure 5 presents a visual comparison among the two mechanisms. E2E mechanism is more versatile as it can handle non-PTP switches/routers, In E2E slave measure the full distance to its master while the P2P measures the distance to the next neighbor only. This behavior becomes important in a large network where E2E could result in network bandwidth and resource wastage. P2P has another benefit that delays between master and slave is already known when network path changes as all the links measured periodically.

D. PTP NETWORK
IEEE 1588 follows master-slave architecture. It can have more than one network segment with multiple clocks. Generally, the grandmaster clock has the best oscillator with standard time and associated to an external GPS or atomic clock.Each network segment has a master clock that is mainly synchronized to the grandmaster clock. The clocks that are synchronized to the master clock are termed as slaves; they do not pass the time information to further nodes. On the VOLUME 8, 2020 contrary to switches or router, master and slave clocks are end nodes in the network [47]. The worst condition of the queuing delay happens in switches or routers [48]- [50]. Figure 6 represents a complete PTP network employing all kinds of PTP clocks, their port states (Master/Slave), and the positions in the network.

E. PTP EMPLOYMENT TECHNIQUES
The IEEE-1588 does not provide any details about the implementation of the PTP profile. Two approaches have been accepted for the PTP over Ethernet known as PTP with software timestamps and PTP with hardware timestamps [51]. Commonly used configurations have been discussed below.

1) GRANDMASTER WITH HARDWARE TIME STAMPING
With GPS grandmaster clock could be able to deliver precise nanosecond timestamp resolution and more than 30 nanoseconds accuracy [52]. The reference oscillator is employed with devoted hardware for the accurate timestamp of the received delay demand and outward sync packets [53]. In this case, 1 PPS output from the grandmaster and the slave can be likened via oscilloscope to test the synchronization accuracy.

2) SLAVE WITH HARDWARE TIME STAMPING
Hardware timestamps with a PTP software daemon deliver precise nanosecond timestamp resolution with devoted hardware normally in a PCIe (Peripheral Component Interconnect Express) form factor. The hardware slave got many benefits over the software slave, like a better oscillator, a 1PPS output for comparison with the master, and, devoted hardware that is robust to the latency caused by the operating system. Synchronization accuracy up to 100 nanoseconds could be achieved.

3) SLAVE WITH SOFTWARE TIMESTAMPING
This approach use present computer hardware along with a PTP daemon. The slave software solution is essential to recompense for the internal oscillator on the computer motherboard by software timestamping. Log file statistics are the only way to test the accuracy, due to the absence of 1PPS output. Achievable accuracy lies between 10 to 100µs [54].

F. TEST BENCH
We have setup the PTP test bench as shown in Figure 7, where QULSAR QG2 gateway was configured as grandmaster and P60 chipset and M88 managed clock engine configured as a boundary clock/gateway [55]. Experimentation was performed with the different configuration on the master and slave nodes, we have tested the accuracy achieved by direct connection between master and slaves and observed the effect of inserting non-PTP switch in the PTP path. The performance of M88 was also evaluated as a jitter cleaner. Figure 8 (a) depicted the synchronization drift is about 70ns when Qg2 is directly connected to the slave node at ethernet port 1. Figure 8 (b) there are two slave nodes one is directly connected to master and other is connected via a non-PTP switch, max drift is about 50ns when P60 board connected to Qg2 directly and drift of 1PPS signal is about 200ns when the master and slave are connected via a switch. So, a non-PTP switch degrades the performance.
In Figure 8 (c) there are one master (QG2) and three slave nodes (QG2, QG2, M88), results collected after running the system for about 12 hours. The results clearly present the higher drift with switch and lower drift with the M88. Figure 8(d) shows the performance of M88 as a boundary clock and jitter cleaner, yellow is the 1PPs reference signal from the master, the green is 1PPS from M88 as slave node, and the pink is 1PPS from QG2 as a slave, connected to master via M88 as a boundary cock.
Results show that it has almost the same drift as in the direct case with M88. From the test results, we conclude that a non-PTP switch lowers the PTP performance even with very accurate and expensive PTP supported equipment. M88 has a good impact on removing the jitter and can enhance the synchronization while acting as a boundary clock. All the equipment is very sensitive, and results have significant variations without any change to configurations, reasons for such variations are under observation and testing.

IV. PTP PERFORMANCE ENHANCEMENT AND ERROR SOURCES A. PERFORMANCE LIMITING FACTORS
One of the main sources of time error is propagation delay asymmetry. A PTP slave approximates the prop- agation delay of Sync requests by supposing the bidirectional delay is symmetric, this supposition though, is not usually correct. Asymmetry of the propagation delay causes an error at the physical level [56]. The propagation delay comprises of cable delay, transmit and receive latency. Moreover, this latency considerably varies amongst chip vendors because of dissimilar employment methods.

1) PDV (PACKET DELAY VARIATION)
One important factor that defines the superiority of the clock synchronization process is PVD. The value of the PVD represents the phase (time) difference among the master and the slave. According to [3], PDV components could be divided into some parts as discussed below.

a: QUEUING DELAY
Queuing delay is the time spent by the transmitted message at the intermediate nodes. Estimation of the queuing delay occurred in any intermediate node is difficult to be done either in the master or slave clock as it is highly dependent upon the congestion level or the network. Queuing delay inside the switch covers the noteworthy part of the asymmetry problems, (up to milliseconds).

b: TRANSMISSION DELAY
Is the time required by the message to travel in the medium. This delay is highly dependent upon the medium it travels. In a wired network, the cable category, its material, and length, all are involved in upsetting the transmission delay. While, in a wireless network, channel capacity, signal power, and the distance are the influencing factors.

c: CABLES DELAY
All the infrastructures related delays among the clocks present in the network are compensated with PTP v2 except the one caused by the connecting cable between the GPS antenna and grandmaster clock. Even high frequency antenna cables have high attenuation at the GPS reception frequency that puts limits on the maximum cable length. [57] proposed a solution to this issue by integrating the grandmaster clock VOLUME 8, 2020 directly into the antenna, which eliminates the use of a coaxial antenna cable.

d: PROCESSING DELAY
As the transmission/reception of all the messages is on the physical layer, a small time is required to process the messages to the application layer. Though in the majority of the cases this delay is ignored as its value is very small, still, this delay might reduce the accuracy of the synchronization.

2) QUANTIZATION ERROR
When dealing with analog to digital conversion it is unavoidable to encounter quantization error. Generally, the internal clock of each node yields some analog signal from the clock oscillators, these signals are used in the PTP protocol after an analog to digital conversion. The clock unit notices the signal changeovers on the gigabit media-independent interface built on its system clock to yield timestamps. Every time the PTP message passes a clock edge at the point the signaling and the sampling clock are dissimilar, it causes a non-deterministic error [58].

3) OSCILLATOR STABILITY
This element largely depends on the superiority of the clock itself. Clock drift is one factor that defines the quality of every clock, it is expressed in ppm (part per million). The location of the node affects the drift value, and the clocks with smaller drift are expensive. Regardless that there is a method to increase the stability of the clock with PLL (Phase Locked Loop), still there are some jitter that occurs inside the PLL itself [4].

B. ENHANCEMENT APPROACHES IN IEEE 1588
Lots of ways have been proposed and adopted to enhance the accuracy of PTP based clock synchronization [59]. The main goal of these approaches is to decrease the PDV to the maximum possible extend. Based on the optimization level and working mechanism these enhancements mechanisms have been divided into different categories as discussed below.

1) BOUNDARY CLOCK OPTIMIZATION
The main issue with the boundary clock is the queuing delay, Jaspernite et al. [60] have suggested an idea to replace the boundary clock with a bypass clock. This bypass clock looks similar to the transparent clock, the performance gets enhanced but still not sufficient for many existing and future applications. To alleviate the exponentially accumulated timing error, [61] have advised an approach that combines the frequency compensation algorithm and the periodic offset recompense algorithm, both are optimized to diminish the synchronization error independently. They experimentally shown that the fast jitter is a leading factor that decreases the timing accuracy, and the peak to peak jitter was found to be less than one microsecond for multiple hops network.

2) MODIFICATION IN IEEE 1588 MESSAGE TRANSMISSION
PTPv2 offers an improved timing accuracy with the aid of the transparent clock features and the condition of replacing the intermediate nodes with PTP supported ones. There are many proposals in the literature to resolve the PDV caused by queuing delay in the intermediate nodes without considering PTP v2 supported nodes. An approach has been proposed by Murakami et al. in [62] a technique that modifies the IEEE 1588 PTP transmission in a non-PTP network. Few extra PROBE messages are sent by the master clock, these messages are used as the measurement tools to estimate the queuing delay. This method encourages the offset value, but it could bring another issue, that the intermediate nodes become much busy owing to the increased packet exchange.

3) FILTERING TECHNIQUE
Initially, they were developed with PTPv1 as the target to choose the minimum delay measurement from master to slave and vice versa [63]. Though this technique is still employed in the improvement of PTPv2, the main drive of the filtering in PTPv2 is not to straight alleviating the queuing delay. Hadzic et al. in [64] presented a filtering technique named as EAPF (earliest arrival packet filter) with PTP v2 . The EAPF method do well in a slightly loaded network with a low probability of queuing delay, in a situation of high load it could not perform well. In [65] author recommended alternative improved filtering due to the dynamic network. The network situation is hard to forecast, this technique offered an adaptive filtering method that can change the clock recovery algorithm in the slave node rendering to the network condition. In [66] Kalman filter based a clock servo system is planned with software based PTP for WLAN.

4) HARDWARE TIMESTAMPING
In PTP every message requires traveling from the application layer to the physical layer or vice versa, and this procedure needs time. According to [59] the time difference might disturb the synchronization accuracy, they have discussed that the ordinary offset measurement technique could not deliver a satisfactory result in the wireless network, offered a way to compute the offset among master and slave while bearing in mind diverse transmission speed in every link using hardware timestamping. Exel in [67] put some light on the role of the processing delay in the time synchronization. According to them, this processing delay needs to be considered in the offset calculation; PTP v1 does not deal with the processing delay as the main donor of inaccuracy. The examination of processing delay could lead to some beneficial results that could help to improve clock synchronization performance.

5) INCREASING INTERNAL CLOCK STABILITY
Most of the clock synchronization systems depends upon the internal clock either to take timestamp or to estimate the residence time. PLL has been proved to remove the likelihood of the internal clock shift, though PLL could deliver some errors. Therefore, another control loop is desirable, and that control loop usually is located in the slave node that is trying to synchronize. According to Du et

V. WIRELESS PTP
To be used in industrial measurement and control systems, the devices have to be able to synchronize wirelessly. There are many wireless technologies, the majority of them are radio-based and non-deterministic which makes them challenging for realizing the PTP. The integration of PTP with serial infrared (SIR) has been deliberated in [70]. Particularly the timestamping and encapsulation of PTP frames have been fully examined. The presented outcome shows the sub-microsecond synchronization via a wireless link. [71] advises a novel version of PTP named as wireless precision time protocol (WPTP) for multi-hop wireless networks.
The author exploits the broadcast nature of the wireless communications to overlap the synchronization procedure among adjacent levels. WPTP repeats in levels, in the first round the level-one nodes are synchronized these nodes and become masters for the next level nodes and so on as summarized in Figure 9. Only three packets are involved in the calculation and correction of clock offset, the delay computation is the same as offset, except that the node at an inferior level will send the difference of timestamps. They proposed an analytical model and the results show that the reduction in convergence time of WPTP and the number of packets to transmit does not affect the synchronization accuracy.

A. WIRELESS PTP ISSUES
The main issue when integrating wireless techniques with PTP is the non-deterministic nature of the transmission media (air) that puts a limit on achieving high accuracy. 1) Another main issue is that there are no wireless integrated circuits (ICs) offered yet that are PTP responsive. Without PTP support in the wireless ICs either statistics, filtering, and a lot of computation is obligatory to resolve the determinism issue or custom-built PTP aware hardware need be developed to get sub-microsecond accuracy.
2) Wireless interface brings mobility that could result varying distances amongst the nodes. PTP however depends on knowing the path delay. According to the accuracy requirement and the velocity of a node, path delay measurement cycles require to be run more often.
3) Network topology is also important in such a network that defines the use of either PTP End to End (E2E) delay measurement or PTP Peer to Peer (P2P) delay measurement.

VI. SUMMARY OF THE LITERATURE ON PTP BASED SYNCHRONIZATION
There have been a lot of efforts and proposals in the recent literature to evaluate and enhance the PTP performance and its employment in industrial communication. This section describes the previous work in this regard. Table 3-5 summarizes the literature work proposed to enhance the PTP and evaluate its performance in different configurations. Table 3 describes the summary of the simulation-based work, Table 4 depicts the efforts to enhance and evaluate the PTP performance with different hardware modules and commercially available devices. Table 5 encompasses the theoretical case studies and survey over PTP bases synchronization.

A. SOFTWARE AND HARDWARE TOOLS
OMNET++ has been often used for PTP evaluation because of its per packet discrete-event nature. Table 3 shows the different ways in which OMNET++ is employed for PTP evaluation. The work in [71] uses the Contiki Operating System and Cooja simulator for the soft implementation of wireless PTP and few other use a Linux based System-On-Chip (SoC) Platform for this purpose. The Field Programmable Gate Arrays (FPGAs), and reconfigurable devices are the best applicants to fulfill the flexibility requirements of the emergent Ethernet-based protocols and to implement substation network devices capable to chain numerous protocols [72]. These devices have hardware processing abilities to attain low switching latency and elasticity to adjust the plan to particular consumer desires, protocols updates, and composite protocols combinations. The technology presented by the reconfigurable logic is tending to the next stage of cost-efficient SoC devices [73], other PTP implementations and evaluation in literature used following (but not limited to) classes of FPGA kits/boards [74]- [77].  There are numerous professional PTP based synchronization solutions a few worth mentioning are summarized in Table 6.
The main vendors include Texas Instruments, National Instruments, Qulsar, Microsemi, Xilinx SOC. Their commercial products with main features and applications are briefly discussed, for detailed description reader can check the relevant reference.

VII. SYNCHRONIZATION REQUIREMENTS AND ROLE OF PTP IN THE AREA OF INDUSTRIAL APPLICATIONS A. PTP FOR INDUSTRIAL IOT
The smart factory of Industry 4.0 envisages an industrial ecosystem as a cyber-physical body. Wireless communication has to accomplish rigorous necessities of industrial applications regarding extremely-low latency and very high consistency [13]. As compared to other communication systems industrial networks, usually feats deterministic data transmission/ reception (Tx/Rx) to function in real-time [78], [79]. The data Tx/Rx timings and the node locations within such networks are pre-planned [80]. To preserve a reliable data interchange among the contributing nodes, highly precise clock synchronization among them is of supreme status [19]. Clock synchronization approaches based on NTP and PTP are broadly used to guarantee precise clock synchronization between the contributing nodes [81], [82]. PTP offers better accuracy as compared to other protocols [83], the inferences of PTP have been proved in numerous current research works, with the help of well-synchronized nodes, pre-planned data Tx/Rx timings can be systematized in an industrial network for competent and real-time distribution of the control signals.
Highly precise clock synchronization is crucial for the recognition of emerging IoT real-time applications. The absence of theclocks synchronization amongst the nodes in the industrial IoT can damage system performance and can result in safety compromises. [84] offers a reliable framework for the IoT established within the realm of EPOS (Embedded Parallel Operating System). EPOS timing protocol brings clock time across a wireless sensor network in conformance with the IEEE 1588 standard. EPOS PTP is capable to retain a PAN synchronized of sub-millisecond accuracy. PTP presents tools to measure and take into account the delay asymmetry problem but obliges PTP support at all nodes that increase the system cost by many folds. [85] consider partial on-path PTP support considering the PTPv2, in which a subset of the nodes is PTP unaware they suggest a probing-based mechanism to measure the asymmetry and increase the synchronization performance in a cost-effective way.
Many computing devices with adequate resources can act as edge devices for IoT. Swarm Box is one of the examples which are recommended as a hardware platform in the Terra Swarm project. It functions as a smart gateway for devices that connect to the Internet over the Swarm Box [86]. It has dual Ethernet ports one for the local network is also furnished with hardware support for PTP, permitting nanosecond-scale clock synchronization. In [19] an enhanced PTP has been proposed to enable accurate clock synchronization for industrial wireless sensor network organized for acute control and automation applications.
They showed that including clock drift factor, the precision of clock offset approximation can be meaningfully enhanced. The wireless technologies adopted in industrial IoT scenarios are typically installed at the sensing layer and networking layer to incorporate with wired technologies at higher layers. IEEE 802.11 has become a key communication tool to inaugurate the IoT in industries. IEEE 802.11 Wireless Local Area Networks (WLANs) delivers high throughput and area coverage [112]. This enables flexible communication system designs and the emergence of new amenities such as device localization on the factory floor. There has not been abundant support for synchronized clocks for IoT designs by IEEE 802.11. PTP over WLAN has been the chosen way to establish synchronization among the nodes. Subsection discusses the PTP employment in WLAN.

B. PTP PROTOCOL FOR IEEE 802.11 WLAN
The evolution of network-based wireless communication has offered lots of benefits including cost reduction, flexible network topologies, and mobility. IEEE 802.11 wireless local area network (WLAN) is one of the examples which spreads its usage in the offices and industry [8]. Though the accessibility of synchronized clocks to harmonize and control message is not a default property of WLAN. The infrastructure mode of WLAN, that permits communication among the stations via a centralized access point (AP), is perfectly suitable for practice in centralized designs engaged in applications like smart grids and the industrial automation. Hence, the time-based necessities of the application are important to be considered. [113] states that the different kinds of industrial applications should be reinforced by industrial communication, such as the ones mentioned in Table 7 along with their generic real-time classes and particular temporal requirements as defined in [114], [115]. NTP and IEEE 1588 PTP are two generally recognized CS protocols intended for wired media and also employed for wireless synchronization [51]. The working mechanism VOLUME 8, 2020  of PTP discussed in section 3 shows that the precondition of PTP appreciating precise clock synchronization is that the bidirectional delay among the master and the slave is symmetric.
The prerequisites of PTP are difficult to meet in WLAN as compare to Ethernet due to the existence of asymmetric links [66]. The bidirectional delay in WLAN is asymmetric with large variance, which leads to the restraint of PTP incorporation to WLAN, and the asymmetric delay is primarily caused by the 802.11 back off mechanism. In the PTP algorithm, the Delay_Req is a multicast packet, while in WLANs this packet must be transmitted as unicast to compute the offset and delay among the AP and a wireless station. PTP recognizes the master via the best master clock algorithm (BMCA). Though, the AP can act as the master that synchronizes all stations in the WLAN [8]. As stated in [28], the BMCA can be switched with a modified version of BMCA that selects the AP as the master every time. The software-based PTP employment over WLAN was conducted in [116] for both Windows and Linux OS. The hardware-timestamping board was FPGA-based, and straight integrated to IEEE 802.11b transceivers. With hardware timestamping, a mean clock synchronization error of 1.1ns was attained, while for software-timestamping, the average clock error for the platforms was <10 µs. However, these studies only deliberate a general CS routine and do not examine other factors that might disturb the CS accuracy. Mahmood et al. [117] conducted a study of software-based PTP over WLAN which examine a timestamping delays and jitters for a Linux-based system.

C. PTP BASED TIME SYNCHRONIZATION FOR OPTICAL DATACENTER NETWORKS (ODCN)
A datacenter is an exceedingly distributed multiprocessing system, which requires a precise course of time among a massive volume of devices and machines [118]. In [5] the authors suggested a zero-overhead microsecond-accuracy method of synchronization over an optical network for datacenters. They established the PTP profile to harmonize the server and central controller clock. They showed the microsecond time accuracy via simulation and demonstrated the dependence of the precision on different factors like packet lengths, traffic loads, and traffic distributions. Simulations contemplate a packet-switched optical networks datacenter with a central control [119]. The implementation of PTP in wide area networks (WANs) has been discussed in literature widely, though just a limited number of such studies emphasis on the DCN needs.
Ref. [120], [121] emphasize on the precision obligations, zero overhead, and the continuous-time method of time synchronization. Authors in [122] provide a PTP simulation prototype for the OMNeT++ framework to examine the synchronization precision of the PTP with diverse network situations. [123] puts efforts on modeling the PTP clocks, they conclude that the hardware timestamps reduce the synchronization errors and the clock-frequency skew correction promotes error reduction up to 100 ns. The thorough employment of most of the PTP structures in OMNeT++ was presented in [88]. To create a realistic state, it comprises noise possessions in the clock model. They designed a moveable library for PTP known as LibPTP and LibPLN. In [124] examine numerous formations of the master-slave devices in the power-system network, occurrence of time for each of these formations was compared.

D. ROLE OF PTP IN SMART GRID
Accuracy requirements for smart grid use cases scope from 1 µsec to 100 milli-sec. The utmost challenging tasks at the substation level are sampled values (SV) fault localization, and synchro phasor computing [125], [126]. The 1 µsec accuracy is required for SV and fault identification. An increase in IEDs leads to an increasing interest to distribute synchronization time in a cost-efficient manner, PTP is the optimal choice for such systems. NIST established a testbed to verify the synchronization performance of PTP for power distributed applications. In this platform PPS system record all the clocks simultaneously to calculate the statistics and evaluate the synchronization performance. This testbed can be used to validate current and future smart grid applications. Experimental results show the accuracy of up to ±60ns that meets the emerging requirements. Table 8 [20] lists the time synchronization classes and the respective accuracy requirements along with their applications [98].

1) PTP FOR TRANSMISSION SUBSTATIONS
A substation is an acute fragment of a power transmission system. Its related security and intelligent electronic devices (IEDs) reduce the harm triggered by a fault, permit substituting actions to happen and save the power grid [104]. IEDs can represent the system's current condition, when several IEDs are synchronized to a mutual time reference [16]. Since a large number of IEDs need a time source, it is inevitable to arrange large number of distributed GPS receivers within a power substation [28], [127]. The PTP designed to coexist with IEC 61850 applications and provide sub µs accuracy.
Widespread tests were performed on a PTP timing system to investigate its limits. In PTP based synchronization a substation needs only two/three PTP master clocks in aggregation with a data network [104]. The PTP traffic shares the Ethernet with IEC 61850 sampled value (SV) and generic object-oriented substation event (GOOSE) applications, that permits the usage of a combined network containing PTP clocks, 61850 merging units (MUs) and IEDs [128]. The PTP power profile is well-described by the IEEE C37.238-2011 standard and specifies many static features to achieve interoperability and expected performance. Ingram et al. have done broad examinations over a network via PTP power profile in combination with 61850 SV and GOOSE [92], [129].
The ±150 ns synchronization accuracy even in a heavily loaded data network can be achieved with PTP clocks and 1588 Ethernet switches. This reinforces the employment of a unified IEC 61850 and PTP network for forthcoming substation automation systems. Synchronization with strict ±1 µs precision is essential for digital substation requests and the integration of the local GPS receivers has been extensively accepted for such solutions. A demo platform with an IEEE 1588 Ethernet controller was presented in [130] to imitate smart grid real-time applications. They validated the requirement for time synchronization among two processors liable for sampling voltage signals. In [131], [132], testing was accomplished with commercially accessible PTP clocks and distributing PTP timing over process bus networks. There, accuracy was assessed by determining the time offset of PPS signals from the time source and sink [73].
2) PTP IN EXISTING IEC 61850-BASED SAS [131] analyzes the present state, corresponding diverse employments of the PTP, and its accuracy for typical substation automation system (SAS) applications. They aim the incorporation of PTP with already prevailing systems and the evaluation of different architectures. The outcomes demonstrate that, in ideal situations, the PTP can accomplish synchronization accuracy marginally inferior (80 ns) than legacy IRIG-B systems (15 ns). The combination of PTP over the prevailing IEC 61850 station bus is mainly a compromise among deployment expenses and accuracy as it necessitates the replacement of the previously implemented Ethernet structure [133]. The standard IEC 61850 arranges a substation into three planes known as process, bay, and the station level, each with distinct responsibilities and needs [134].
The practice of PTP to synchronize the SAS system modules is quite smart because the communication network already exists in an IEC 61850-based SAS. Additionally, PTP could empower the synchronization of all the IEC 61850 levels up to 1 µs or less accuracy that is appropriate for synchro phasor measurements and IEC 61850-9 implementation. The PTP based synchronization has numerous benefits like calibration of the signal is not required, and synchronization information can be passed using a station bus. The experimentations reveal that the existing station bus is not appropriate to conveyance the synchronization messages since the switches are not PTP supported, it makes possible only if all the switches/routers are PTP supported [135].

E. PTP DELAY ATTACK
Current industrial systems often use NTP and PTP to synchronize distributed slave nodes to a master node. Though, these protocols are vulnerable to many cybersecurity extortions, particularly to packet delay attacks. Conventional security measures like encryption are not enough to resolve this vulnerability. [80] suggests to utilize the sine voltage waveform of a utility power grid to synchronize nodes connected to the same grid. Digitized instant values of power system measures, including primary currents and voltages are transmitted using SV, and are indispensable for check and control at the substation. To hit the assembly and alignment of SV and the requests depending on the legitimacy of these values, an attacker can custom PTP delay attack to influence the clocks at the measurement devices.
The consequence of malfunctioning can cascade outside these calculations to board all the uses relying on the accessibility of SV. For voltage stability monitoring case a wrong timing at the phasor measurement units could result in dropping the boundary of active delivered power. That can lead to the system's incorrect activities of voltage balance, Details can be discovered in [136]. Detection and prevention mechanisms to avoid such attacks are required to strengthen its security. [98] offers a recognition and alleviation algorithm to beat the identified PTP delay attack. They put on model checking to count the consequence of the delay attack and formally proven the strength of the design.

VIII. CONCLUSION AND FUTURE NEEDS
Time synchronization clamps as one of the trending research issues in the telecommunication and other industrial IoT applications, a robust time synchronization system is compulsory to fulfill the needs of advanced network communication technology. IEEE 1588 PTP is an encouraging clock synchronization protocol that is now well known, but there are still numerous challenges that must be solved in the practice of PTP. These issues mainly include internal clock stability, processing delay and transmission delay and the PTP employment in wireless communications. The timing synchronization accuracy for real-time communication in an industrial network is very difficult. The clocks in the wireless network nodes must be synchronized with a very high precision.
To meet the strict requirement of time-sensitive industrial applications, two essential issues essential to be considered and resolved. First, there is a necessity for an efficient symbol timing synchronization mechanism that allows exact timestamp message recognition and decrypting at slave the nodes. The other point of concern is that PTP scheme should include clock-drift during clock-offset measurement, to improve the accuracy. The influence of the timestamping instance is not negligible; the literature discussion concludes that the timestamps would be taken near to the PHY layer to remove any probable jitter. There is a long chain of challenges to achieve the target synchronization accuracy, PVD is one of them. PTP v2 compensates many of the factors causing PVD as compared to PTP v1 , but few issues still exist which PTP could not satisfy alone. Though there are many solutions proposed and evaluated in the literature for clock synchronization problems, the clock synchronization protocol is still an open issue for researchers. In this paper, we have conducted an inclusive enough survey on the PTP based synchronization schemes its pros and cons along with the application areas specifically in IoT and industrial communication. We believe that this paper would be useful for readers and researchers looking at synchronization solutions for future emerging technologies.
ZEBA IDREES received the bachelor's degree in telecommunication engineering from the Government College University Faisalabad, Pakistan, and the master's degree in electrical engineering from the National University of Sciences and Technology (NUST), Islamabad, Pakistan, in 2012 and 2014, respectively. She is currently pursuing the Ph.D. degree with the School of Information Science and Engineering, Fudan University, China. She possesses professional and research experience of more than five years in academia as well as industry. She was with FAST National University as a Lecturer for one year. She has been serving as a Lecturer at the Electrical Engineering Department, UET Lahore, Faisalabad, since 2015. Her current research interests include electronic circuits, wireless sensors networks and systems for ambient intelligence, cognitive radio networks, and the Internet of Things. LI GONG (Graduate Student Member, IEEE) received the B.S. degree in electronic engineering from Southeast University, Nanjing, China, in 2010, and the M.S. degree in system-on-chip design from the KTH Royal Institute of Technology, Stockholm, Sweden, in 2012. He is currently pursuing the Ph.D. degree with the State Key Laboratory of ASIC and System, Fudan University, Shanghai, China. His current research interests include wearable electronic systems, machine learning, network precise time synchronization, edge computing, and low power wide area network for the Internet-of-Things applications.
ZHUO ZOU (Senior Member, IEEE) received the Ph.D. degree in electronic and computer systems from the KTH Royal Institute of Technology, Sweden, in 2012. He is currently with Fudan University, Shanghai, as a Professor, where he is conducting research on integrated circuits and systems for IoT and ubiquitous intelligence. Prior to joining Fudan, he was an Assistant Director and a Project Leader at the VINN iPack Excellence Center, KTH, where he coordinated the research project on ultra-low-power embedded electronics for wireless sensing. He has been an Adjunct Professor and Docent with the University of Turku, Finland. He is also the Vice Chairman of IFIP WG-8.12.
LIRONG ZHENG (Senior Member, IEEE) received the Ph.D. degree in electronic system design from the KTH Royal Institute of Technology, Stockholm, Sweden, in 2001. He was with the KTH Royal Institute of Technology as a Research Fellow, an Associate Professor, and a Full Professor. He is the Founding Director of the iPack VINN Excellence Center, Stockholm, Sweden. Since 2016, he has been the Chair Professor in media electronics with the KTH Royal Institute of Technology. He has also been with Fudan University, Shanghai, China, as a Guest Professor, since 2008, and a Distinguished Professor, since 2010. He currently holds the Directorship of the Shanghai Institute of Intelligent Electronics and Systems, Fudan University. He has authored more than 400 publications and servers as steering board member of International Conference on the Internet of Things. His current research interest includes electronic circuits, wireless sensors and systems for ambient intelligence, and the Internet of Thing.