Cooperative Robotics and Machine Learning for Smart Manufacturing: Platform Design and Trends Within the Context of Industrial Internet of Things

Internet of Things (IoT) in industrial settings now leads to the development of a new generation of systems designed to improve the operational efficiency of the new paradigm of smart manufacturing plants. Thereby, the current article introduces in detail the definitions, concepts, standards, and other important aspects related to smart manufacturing, cooperative robotics, and machine learning techniques. The paper highlights the opportunities presented by the new paradigm and the challenges faced in effectively implementing it in the industrial context. Especially, the focus is on the challenges associated with the architectures, communications technology, and protocols that enable the integration and deployment of machine learning algorithms to improve the execution of cooperative tasks performed daily by human operators, machines, and robots. The article also provides a systematic review of state-of-the-art research efforts for the fields aforementioned. Finally, an architecture for integrating collaborative robotics and machine learning based on six layers and four hierarchies of the RAMI 4.0 (Reference Architectural Model Industry 4.0) is presented.


I. INTRODUCTION
The challenges faced by industry to maintain or expand their customer base led to the development of smart manufacturing concepts, which, in turn, bring many challenges to traditional manufacturing companies. Internet of things (IoT) is an idea that describes the universal connection to the Internet, turning public devices into connected devices. The main idea behind IoT is to deploy billions or even trillions of smart ''things'' that can sense their neighboring environment, transfer and process the collected data, and then present feedback after data processing [1].
Recognized as a sub-field of IoT, the IIoT covers the fields of machine-to-machine (M2M) and industrial communication technologies for automated demands. IIoT paves the way The associate editor coordinating the review of this manuscript and approving it for publication was Peng-Yong Kong .
for a better perception of the production process, leading to efficient, personalized, and sustainable production [2]. The required scalability and flexibility of communications features are addressed using standard wireless connections. However, in an industrial environment, most applications are developed using ad hoc solutions, i.e., solutions exclusively created for connecting moving parts or hard-to-reach devices. IIoT applications usually require relatively little throughput per node, where capacity is not the main issue. Instead, a large amount of devices need to be connected to the Internet at low costs, with restricted hardware functions and energy sources (such as small batteries), which makes latency, energy efficiency, low-cost, reliability, and security/privacy the most desirable features [3].
Such IIoT deployment enables an astonishing number of applications not even imagined a few years ago. Moreover, smart manufacturing is nowadays becoming a reality [4]. Therefore, there has been an enormous demand for applications of human-robot collaboration techniques supported by machine learning in industry [5], especially in assembly lines. The main objective is to increase the efficiency and productivity of manufacturing systems [5]. In an ideal smart manufacturing scenario, all machines, equipment, human operators, and devices that compose the manufacturing process are interconnected, monitored, and optimized to increase productivity [6]. Therefore, the application of Artificial Intelligence (AI) techniques plays a key role in this process as an effective enabler for addressing the challenges found by contemporary industry [7].
Satisfying the above functionality will bring some key challenges to technological development and, to ensure the large-scale deployment of IIoT technology, it is essential to meet them. In this paper, concepts related to cooperative robotics and machine learning applied to smart manufacturing are clarified and the current trends of their application in manufacturing processes are explored. The opportunities made possible by IIoT and their challenges are discussed. More specifically, the focus is on the challenges associated with the need for standards, architectures, network protocols, and efficiency for data exchange, which can boost the use of collaborative robotics for production processes and machine learning techniques applied to industrial applications in the context of IIoT. The paper also introduces a conceptually designed architecture for smart manufacturing based on the six layers and four hierarchies of the Reference Architectural Model Industry 4.0 (RAMI 4.0) model to integrate cooperative robotics and machine learning applied to the new manufacturing paradigms. This paper begins by defining smart manufacturing, cooperative robotics, and machine learning in the industrial context in Section II. Section III presents an overview of the latest activities regarding architectures, protocols, and standardization efforts applied to the topics aforementioned. Section IV describes future developments and trends that IIoT can offer and it also introduces a feasible architecture design to integrate all the elements applied to smart manufacturing. Finally, Section V presents the conclusions.

II. SMART MANUFACTURING, COOPERATIVE ROBOTICS AND MACHINE LEARNING
This section starts with an introduction of the essential concepts related to smart manufacturing (Section II-A). Cooperative robotics approaches are defined in Section II-B and the use of machine learning applied to manufacture is introduced in Section II-C.

A. SMART MANUFACTURING
Smart manufacturing is a general concept of intelligent manufacturing systems or processes that uses a new paradigm to facilitate the next generation of manufacturing automation [8]. The terminology and initial concepts of smart manufacturing were conceived by the Smart Manufacturing Leadership Coalition. These concepts utilize a collection of manufacturing methods that correspond to new tendencies in network data and information technology designed to redefine the manufacturing field [9].
The main features of smart manufacturing involve: 1) digitalization; 2) service orientation; 3) smart and connected automation equipment; and 4) collaborative manufacturing network to achieve cost-effective and flexible personalized mass production [10]. Moreover, another aspect that distinguishes smart manufacturing from other programs is the prominence of human creativity in the framework. To enhance the benefits of smart manufacturing and increase productivity, humans should not be replaced by AI and cognitive automation in the workplace, but human capabilities should be improved through customized solutions for specific areas. The importance of product and process information/data that support technology and (man-made or machineinherent) knowledge has been generally accepted [11].
Thereby, there are two main changes underway that impact manufacturing automation in this new era of production: the production model is shifting from mass-customized to mass personalized production; and the rapid development and use of advanced algorithms, connectivity, computation methods, among other technologies [12]. Traditionally, the process of automated manufacturing reduces flexibility to increase productivity. In large-scale personalized systems, strictly entirely automatic systems no longer work and automation is required to be flexible while increasing productivity to mass-produce personalized goods at a fair cost [13]. Therefore, to achieve smart manufacturing, future-oriented manufacturing automation must simultaneously become intelligent in two aspects: 1) Manufacturing process automation based on personalized products -the manufacturing process will be integrated and automated for each unique product (not product series) from design to inspection [14]. 2) Networked self-organizing manufacturing systemsthe usual dedicated layered manufacturing pyramid will evolve into an integrated network of self-manufactured goods with self-configuration, selfoptimization, and self-repair functions [15]. To allow the concept of process automation in smart manufacturing, a product line-based manufacturing method configuration and integration is not completely employed. This is so because every product can be unique. Therefore, the product life cycle development tracking, process configuration, and its inspection need to be treated as a single product instance, such as a single product ID [16]. It requires deep integration and a seamless two-way data stream between all design and manufacturing stages of the product development life cycle without data damage or loss. A product digital twin needs to be created to track all relevant information about a single product (for example, design specifications, materials, manufacturing processes, manufacturing facilities, and inspection logs) [17].
The equipment, machines, systems, and human-operators can be connected through M2M communication channels to create a manufacturing network wherein information carriers VOLUME 9, 2021 can exchange equipment data in near real-time. Data-driven distributed intelligence may enable the rapid configuration of the manufacturing network to produce a variety of personalized products with dynamic batch sizes in a cost-effective and efficient way [18]. Thereby, a smart system should have the following functions [19]:

1) CONTEXT-AWARENESS
Smart manufacturing systems can recognize, interpret and analyze the purposes of things, systems, and users connected in the deployment domain, which allows self-awareness of its condition, status, and options for actions to be adopted.

2) MODULARITY
Smart manufacturing systems must work in a modular manner and set up sub-components to create various system configurations, so that they can cost-effectively manufacture new personalized products.

3) SELF-ORGANIZATION
Participants (such as systems, processes, and people) can interact with each other and organize their actions on a purposeful basis, without outside intervention. Self-organization is usually embodied as self-configuration, self-optimization, and self-healing functions.

4) DATA-DRIVEN DECISION-MAKING
Smart manufacturing delivers full use of the insights obtained from large-scale engineering data to create intelligent and adaptive decisions based on shifting external and internal conditions.

B. COOPERATIVE ROBOTICS
Industrial robots are flexible machines equipped with a great variety of sensors and tools to adapt themselves to production tasks [20]. In the past three decades, robotics has been a booming research field and has made great progress [21]. Robots were thought of as substitutes for humans, i.e., they replaced humans, with great efficiency, in executing repetitive and dangerous tasks.
Industrial robots are usually installed far from human workers, in physically separated work areas, where they are continuously monitored by electronic safety systems. However, based on new technological advances, the robotics field is quickly developing, so that humans not only share the same working space with robots but also regard them as useful collaborators at home and work [22]. Furthermore, since some tasks may be too complicated to be entirely performed by robots, or too expensive to be completely automated, human assistance or shared human-robot execution of tasks may be the most flexible and reasonable solution. In this case, Human-Robot Interaction (HRI) and safety are related goals.
HRI can be defined as ''a general term for all forms of interaction between humans and robots'' [23], as well as it can be characterized as ''the process of conveying human intentions and interpreting task descriptions into a sequence of robot motions complying with robot capabilities and working requirements'' [24].
Such interaction happens either by sharing the workspace or teaching by a human showing how the task is performed [25]. All these advancements in robot technology demand distinct levels of interaction and need to be identified based on the following two principles: 1) the degree of autonomy of the robot system; 2) the closeness of humans and robots during the service execution. As a result, the need for more reliable and more natural user interfaces and interactions for robots is growing and it is necessary to enable technological development in the field. Moreover, it must be determined whether there is a need for contact between entities of the manufacturing process, or whether contact must be avoided in various ways.
Among the different types of HRI, Human-Robot Collaboration focuses on humans and robots performing a difficult task together. However, the interaction can be classified in two categories: (i) there is a clear and intentional physical collaboration requiring a force exchange between humans and robots, and by measuring or calculating these forces/torques, the robot can predict the human movement and respond accordingly; or (ii) a contactless collaboration without physical interaction between them, in this case, actions are coordinated through the data exchange by adopting a direct communication (i.e., voice, gestures, among others) or indirect communication (i.e., intent identification, sight direction, facial expressions, etc) [26]. In both cases, human-operators execute tasks that involve dexterity or decision-making, while the robot accomplishes jobs that are not suitable for direct human intervention (i.e., tasks that require repetitive movement or excessive strength, chemical deposition, precise placement, etc) [27].
Nowadays, HRI is available and safe with the popularization of Collaborative Robots, generally known as Cobots, which enable safe interaction between humans and robots during the execution of the tasks. Deploying collaborative robots in manufacturing environments allows robots and humans to work side by side, as collaborators, to perform many types of tasks executed in industry, at different levels.
The main advantage of this interaction is the combination of the intelligence, cognitive skills, and dexterity of humans to act in the face of unexpected events and the abilities of robots, such as high precision, repeatability, and strength.
Industrial collaborative robots have been deployed on different applications in many manufacturing plants. The main goal of these robots is to support easy-to-use integration of robotic systems and humans. Among these robots, the interactive dialogue robot (verbal or non-verbal cues) collects multimodal data and establishes connections with human partners by using a variety of sensors (such as stereo-vision, infrared, or high-resolution auxiliary cameras, miniaturized 6-axis load cells, force sensing in joints, etc.). Due to its ability, this type of robot is the most used in several industrial tasks [28] such as electronic and small parts assembly lines, pick-and-place, Computer Numerical Control (CNC) machine tending, metal stamping & press tending, plastic injection & blow molding, packaging, testing & quality, and inspection [29].

C. MACHINE LEARNING
Machine learning is a branch of AI that enables a system to autonomously learn and improve from experience without explicit programming. Therefore, instead of writing programs that explicitly instruct computers how to carry out tasks, AI systems use machine learning algorithms that access data and use it to learn by themselves [30].
Commonly, the learning process begins with data observation, i.e., direct experience or guidance, to find data patterns based on examples provided, whose purpose is to allow the computer to learn automatically without manual intervention or assistance, adjust operations accordingly, and take decision by itself in the future [31].
Among the different techniques commonly found in the literature, three kinds of machine learning techniques can be highlighted: supervised, unsupervised, and reinforcement learning.

1) SUPERVISED LEARNING
Supervised learning is the technique used in applications with a large amount of available input data (the training set), and wherein experts provide correct responses (labels). The system uses this labeled data for training, and its function is to classify every data point in one or more groups. Then, the system learns how the training data is structured and uses this knowledge to predict in which categories to classify new output data [32]. Therefore, the main goal of the supervised learning process is to produce a model that identifies which class is closer (using different metrics) to an input and that can be used for all given input operational data sets.
The most usual supervised machine learning tasks are classification and regression. For classification, the algorithm must learn how to predict the most probable class, category, or label of discrete output values from one or more input data sets. Similar to classification, regression problems can also use supervised learning techniques. The difference is that the process must foresee and predict a continuous output value.
In smart manufacturing, supervised learning is used in many applications, as illustrated in [33], where a regression model was used to estimate the thickness in a steel plate rolling mill machine with precision, as well as to predict problems in the plant by using data coming from sensors embedded into it.

2) UNSUPERVISED LEARNING
In this learning algorithm category, there are no experts involved in the process, and the evaluation of actions does not rely on them. Unlike supervised learning, it does not learn from labeled data. Instead, it will strive to find patterns in unlabeled data [34].
The task of unsupervised learning is to find the relevant observation groups of the input data, namely clustering. Such grouping is based on conclusions drawn from similar measurements that categorize similar points in distinct classes. The goal then is to use cluster analysis to discover unknown relationships between classes. Another unsupervised learning task is dimensionality reduction [35], i.e., the process of discovering relationships between input data variables for the elimination of possible redundant dimensions. Considering that some problems may contain thousands of input data dimensions, big data problems become impossible to visualize and reduction may assist in better visualization of the results.
Unsupervised algorithms have been deployed in many manufacturing processes, such as the one discussed in [36], where a system for detecting anomalies in smart manufacturing using real data collected from sensing devices of a factory production line was proposed. By using an LSTM-based Auto-Encoder algorithm, the authors developed an unsupervised real-time system for detection of limited and irregular anomaly patterns with accuracy of detection greater than 90%.

3) REINFORCEMENT LEARNING
Reinforcement learning is a machine learning technique focused on learning from experience. According to the report of the Royal Society [37], reinforcement learning algorithms lie between unsupervised and supervised learning techniques as it uses interactions with an environment for learning.
Reinforcement learning algorithms observe the state of the environment, usually using sensors, while executing actions that change the same environment. In manufacturing applications, the sensors observe the status of the production (or of the machines themselves), and sequential decisions are taken in order to maximize future returns, or ''reward''. Therefore, the algorithm does not receive a labeled correct action but tries to find the best actions that will maximize the rewards in the long run.
Reinforcement learning has been used in several applications in smart manufacturing. In [38], a methodology to model human-robot interaction and to reinforce an agent's learning for autonomous decision-making is proposed. The decision-making capability provides the robots with greater adaptability, by enabling their behavior to change based on observed information, both of its environment and human colleagues. The final evaluation indicates that the reinforcement learning technique can effectively learn to make adjustments to the robot behavior based on the knowledge extracted from observed information, and balance the task demands.

III. STATE OF THE ART
In Section III-A, the explanation and characterization of smart manufacturing architectures are provided. Section III-B discusses communication technologies and protocols standards available, as well as which ones are commonly deployed. Finally, Section III-C introduces the algorithms most used in industrial applications. VOLUME 9, 2021 A. REFERENCE ARCHITECTURE An architecture is commonly characterized as the organizational structure of a component or system, including its relationships and guiding principles that conduct its design and evolution. From the system perspective, the architecture should represent the basic structure of the established system from the point of view of its elements, purposes, relationships, interfaces, processes, constraints, behaviors, principles, rules, characteristics, and physical and logical attributes. According to [39], the system architecture establishes a framework for describing its shape, structure, components, and interactions.
In the smart manufacturing context, there are a few architectures and standards found in the literature. However, RAMI 4.0 for Industry 4.0 (I4.0) [40], as illustrated in Fig. 1, is the most accepted and deployed in industry.

1) HIERARCHY LEVELS
Based on the horizontal axis of IEC 62264 (the international standard for enterprise control system integration), the hierarchy levels were four initially, called ''Enterprise'', ''Work Center'', ''Station'' and ''Control Device'' (from top to bottom). However, three more layers have been added to RAMI 4.0 to support smart factories. At the bottom are ''Field Device'' (controlling machines or systems in a smart way, such as smart sensors), and ''Product'' (or work-pieces) (taking into account the homogeneity of the products to be manufactured and interdependent production facilities). A ''Connected World'' layer was also added at the top, through which factories can transcend boundaries and connect with external partners through a collaborative service network. These layers represent the basic aspects of the I4.0 organization.

2) LIFE CYCLE AND VALUE STREAM
This axis represents the life cycle of entities, such as artifacts, products, and facilities; it is based on IEC 62890 (the standard for life cycle management).

3) LAYERS
The vertical axis describes the breakdown of machines and physical things in a way to allow its virtual mapping. The layers are employed to describe perspectives, such as data maps, functional descriptions, communications behavior, hardware/assets, or business processes, i.e., they determine a structure of Information and Communication Technolo-  RAMI 4.0 breaks down complex processes into software packages to make them easy to understand. It also includes design data privacy and IT security. It answers questions about the semantics, functions, communication standards, internationalization, and cooperation of smart factories. Furthermore, RAMI 4.0 puts the ideas of horizontal integration, vertical integration, end-to-end engineering, and life cycle together (Fig. 1), which includes IIoT and Cyber-Physical Systems (CPS), focusing on manufacturing and including several special functions, whose structure is compatible with IIoT methods. RAMI 4.0 emphasizes the organization, functionality, and interactivity of the components included in the architecture. A main disadvantage of the architecture is that it does not display detailed information about the communication between the machine and the ''thing'' to guide the product in the production process based on the operations required for the product. Moreover, recently the architecture has been used in many projects, such as shown in [41] and [42], whose main goal is always to make the deployment of smart manufacturing concepts feasible.

B. COMMUNICATION TECHNOLOGY AND PROTOCOLS
M2M communication is an important part of smart manufacturing, thus, in the industrial context, this becomes the core of IIoT. Smart manufacturing involves coordination between different devices that are physically far apart in geographical terms and also across different industries. The communication requirements in IIoT are focused on a variety of factors, such as reliability, latency, and the life of the communication device instead of throughput, which is the common focus of human-centric communication. For smart manufacturing, the main requirements for service accomplishments and data transmission are cycle time, reliability, battery longevity, and data rate. The target values for each requirement are shown in Table 1. Currently, the majority of the devices in factories are connected based on a wired infrastructure that works using industrial protocols. Nevertheless, wireless technologies are increasingly playing the role of wired technology. The factors to accelerate the selection of wireless technology in industrial environments include ease of deployment and scalability, notably for extensive coverage areas. Wireless technology is also suitable for hard-to-reach locations/places, which would require extensive wiring. Dynamic topology networks, mobility and effective coordination between different geographic places play a key role in the selection of wireless technologies. Moreover, with the development of new technologies, wireless networks are expected to become ubiquitous in industrial environments.
Industrial Wireless Network (IWN) technology is characterized by a group of distributed wireless devices that perform wireless communication to measure, monitor, and control industrial physical plants, enabling the effective deployment of smart manufacturing architectures. IWN communication systems are usually divided into four main components: smart entities, inter-IWNs, beyond IWNs, display devices, and servers [44]. In IWN, smart things with wireless transceivers (such as workers, Automated Guided Vehicles (AGVs), machines, common sensors) can be assigned as wireless nodes. In addition to IWN, access point nodes and gateways also build bridges to other industrial networks as well as cable and cellular networks. Ultimately, due to higher-level data purposes, the system involves data servers, management, and controllers.
Since nowadays there are many wireless connectivity technologies available, it is not possible to cover all the standards and types of communications schemes. But, the most used wireless connectivity standards for smart manufacturing satisfying the main requirements, shown in Table 1, are Zigbee, Bluetooth, Wi-Fi, Passive Optical Networks (PONs), LoRaWAN, and mobile networks. They are covered in the next sections.

1) ZIGBEE
It is a wireless network protocol created for automated control and sensor networks based on the IEEE 802.15.4 standard. It was designed for applications involving low data rate and low power consumption. The ZigBee specification is provided by the ZigBee Alliance and it can contain up to 65,000 nodes connected to a single control network [45].

2) BLUETOOTH
It is a low-power short-range wireless communication system that can provide connectivity to many electronic devices. It was created by Ericsson, but runs under the auspices of the Bluetooth Special Interest Group (SIG) that developed the Bluetooth standard. Its development aims to provide faster speed, greater flexibility, and more features. Bluetooth has been confirmed as an IEEE 802.15.1 standard and has been expanded to provide applications such as mesh connectivity for IIoT and M2M communications [44].

3) WI-FI
It is a general term to refer to the IEEE 802.11 wireless communication standard of Wireless Local Area Networks (WLAN) and is applicable to the physical and data link layers. Various variants such as 802.11a/b/g/n or 802.11ad are available, defining distinct types of products. Through the release of updated variants, the overall technology has been able to meet the growing request for more data and higher speeds [46]. Furthermore, two other types of WLAN networks can be built using Wi-Fi systems, namely infrastructure networks, and ad-hoc networks.

4) PASSIVE OPTICAL NETWORK
It is a system that can bring fiber optic cabling and signals to the end user in a passive fashion. It consists of an optical line terminal at the communication company's office connected to many network units. Furthermore, since the network is passive, once a signal passes through the network, there are no power requirements or active electronic components for the transmission [47].

5) LORAWAN
It is a long-range low-power wireless standard designed specifically for IoT and M2M applications to provide cellular low data rate communication network capabilities [48].

6) MOBILE NETWORKS
Mobile telecommunication technology is universally used and is based on the idea of repeated use of frequencies over a series of covered cells. Nowadays, cell phones and cellular communication technologies are ubiquitous. A quick evolution from 3G to 4G and then onto 5G has been witnessed worldwide. Each generation of broadband cellular network technology has its own goals and can provide various levels of functionality. There may be several different competing standards in these distinct generations. As the name suggests, mobile technology uses a substantial number of base stations covering a small area, with each of them communicating with a reasonably large number of users [48]. Table 2 summarizes the ranges for the four main technical requirements in the context of smart manufacturing specifications for each one of the wireless technologies described.

7) PROTOCOLS
Protocols are another important element for the feasibility of smart manufacturing systems. Protocols are responsible for the rules and the format of data transmission. In the IIoT context, there is a great demand for efficient protocols and, among the options available on the market, the OPen Communication Unified Architecture (OPC UA) comes up as one of the best options for smart manufacturing. OPC UA is a standardized communication middleware for automation systems and serves as a bridge between off-line-based engineering tasks and the runtime communication of the physical and logical resources of a manufacturing system [49].
OPC UA was developed by the OPC Foundation (IEC 62541). It was created with the intent to provide secure and reliable connectivity to interconnect multiple systems in other communication areas. It is focused on industrial automation and transmission of measurement data as well as event notifications. It enables soft real-time sharing of current and past data and events, including alarm management. OPC UA was selected as the backbone of the German government-driven Industry 4.0 program aimed at increasing the flexibility of production automation [50]. Fig. 2 describes the manufacturing data integration by using the OPC UA protocol. The OPC UA server connects, via Ethernet, the controllers of various devices in the smart factory (Distributed Control System (DCS), Programmable Logic Controller (PLC), Direct Numerical Control(DNC)/CNC, robot/motion controller, smart warehouse/logistics controller). The OPC UA client accesses the OPC UA server through TCP/IP to monitor the interconnection and data integration of the smart devices. Fig. 3 illustrates the flow diagram of reading/write calls from the OPC UA client to the OPC UA server and from the OPC UA server to the device.

C. MACHINE LEARNING ALGORITHMS
As previously mentioned, manufacturing is witnessing an unprecedented expansion in usable sensor data in many formats, and with diverse semantics and structures. Sensory data is collected from various aspects of the manufacturing process, including product lines, equipment, manual activities, and plant manufacturing conditions. Data modeling and analysis is an important part of processing large amounts of massive data and supporting real-time data processing in intelligent manufacturing. These are essential tasks in the effective application of machine learning techniques.
Among the techniques available, only a few have been efficiently used in manufacturing.

1) SUPPORT VECTOR MACHINE (SVM)
This is a supervised learning algorithm that can be used for linear and nonlinear classification and regression. SVM is formed based on the idea of creating a hyperplane or a group of hyperplanes. It divides the high-dimensional or infinite-dimensional vector space into different parts with the largest marginal distance between the training data points of the two largest classes. If the SVM test point is located on one side of the hyperplane, it belongs to one category and, if it is located on the other side, it belongs to the other category. The mapping to the infinite-dimensional vector space is performed by the kernel function, leading to the solution of the problems of classification and prediction. The goal of SVMs is to generate a model based on training data, which can predict data test points related to a subset of training data. To achieve this goal, that is, to achieve maximum accuracy of prediction, SVMs require a large amount of data covering the whole data point space. The biggest disadvantages of SVM are the slow learning speed and the lack of interpretive ability for humans [30].

2) DECISION TREE
Decision Trees are a family of graphical algorithms easy to understand and explain. The main issue in selecting the adequate algorithm is to find the best decision tree type for the training data set. Two types of decision trees are used in IIoT. The first is a classification tree that provides classification output, and the second is a regression tree that provides numerical output. The main disadvantage of the decision tree is that, differently from SVMs, it cannot solve nonlinear problems. However, when compared to SVMs, decision trees learn much faster. Because they score high on the key features of data mining, decision trees have been widely used in exploration and prediction problems [51].

3) K-NEAREST NEIGHBOR (K-NN)
k-NN is a machine learning algorithm used for nonlinear problems (i.e., classification and pattern recognition). The algorithm needs to calculate the distance between data samples, where each input data is performed with its k nearest neighbor samples mark. The k-NN algorithm is very sensitive to missing, noisy, fuzzy, irrelevant and redundant data values, and the classification is very slow. In contrast, k-NN has a high learning speed, making it one of the fastest learning ML algorithms [52].

4) CONVOLUTIONAL NEURAL NETWORK (CNN)
CNN is a multilayer feedforward artificial neural network, which was first proposed for two-dimensional image processing. In CNNs, the featured learning is accomplished by alternating and stacking convolutional layers and merging operations [51]. A convolutional layer uses multiple local kernel filters to convolve with the original input data and generate invariant local features. The subsequent merge layer uses merge operations such as maximum and average merge to extract the most important features on the sliding window of the original input data with a fixed length. Max pooling operations select the maximum value in a region of the feature map as the most important element. Average pooling calculates the average value of an area and uses it as the value of that area. Maximum merging is suitable for extracting sparse features, but the merging operation of all samples may not be the best choice. After all the multi-layer feature learning, fully connected layers convert the two-dimensional feature map into a one-dimensional vector and then feeds it into the softmax function for model construction. This is usually used to minimize the minimum mean square error and train the CNN.

IV. ARCHITECTURE DESIGN AND TRENDS
This section discusses an architecture design that enables the potential for collaborative robotics and machine learning to be deployed in smart manufacturing. The main functions of such systems are: monitoring the production status through collecting data from sensors; other measurement activities (for safety purposes, for example); processing analysis; and controlling the strategies of production over time. All these activities are based on non-centralized control modules (through the use of a distributing data lake), where all the equipment can access data coming from numerous sensors, process the data, and finally provide decision-making actions. Furthermore, all information exchange is done according to production parameters previously established to ensure that the manufacturing process is running according to the strategy adopted.
The proposed architecture is designed in accordance with the RAMI 4.0 layers, consisting of creating a non-centralized network for manufacturing systems, following the implementation of several subsystems: (i) a communication system encompassing all equipment and devices in the production system; (ii) a data lake for data storage [53]; (iii) a machine learning framework; and (iv) decentralized data processing. The tasks must be built in the product instance ''Production'' phase of the RAMI 4.0 life cycle-axis. Fig. 4 illustrates the designed architecture with six layers and four hierarchies according to the RAMI model. The ''Layers''-axis includes the six clustered layers varying from the lowest (''Asset'') to the highest (''Business''). A brief description of the tasks created in each of them is provided below.

A. ASSET LAYER
This is the layer where the components such as machines, collaborative robots, sensors, and other engineering systems are installed. This is also where human-operators are located. These elements represent the primary sources of data and all signals generated in this layer are provided in heterogeneous formats. Therefore, the data need to be integrated through a common information model. Furthermore, the signals are acquired through the IWN and every piece of equipment is set up to work in a predetermined frequency. It must be noticed that, despite the machine learning framework being designed to develop, test, and process data generated in the manufacturing process by using the models embedded on it, other devices also can load data or use their machine learning apparatus. For instance, a collaborative robot can learn how to execute a new task introduced into the production or it can improve the performance of one by using not only the signals acquired by itself, but also a large amount of data available on the data lake to increase the accuracy of its prediction.

B. INTEGRATION LAYER
This is the layer responsible for the transformation from the physical to the virtual world, which includes all resources and infrastructure for capturing and transforming analog/digital signals connected with PLCs and making them available in the IWN in an adequate data format. Moreover, other types of data can be inserted into it to support ML algorithms, e.g., data coming from the Product Lifecycle Management (PLM) systems can provide data before the start of production for learning purposes.

C. COMMUNICATION LAYER
This layer implements communication between the integration and information layers. All the data can be transmitted, collected and transferred using communication protocols (e.g., via Wi-Fi interfaces). The OPC UA uses standard protocols for data communication to assure deterministic data exchange as well as for implementing a full information and security model. The astonishing amount of data generated by devices need to be sent to several receivers, where the data must be stored instantly to avoid been lost. Thereby, a message broker structure minimizes such a problem by providing methods for decoupling communication between data providers and consumers using the publish/subscribe pattern. However, it must be noticed that 5G technology, which is much faster and reliable, may be deployed to increase the data exchange rate with many advantages, as discussed in Section III-B.

D. INFORMATION LAYER
This is the layer that contains the data services (including the data lake) that enable the storage, use (including access), and maintenance of the data produced or altered by the functionalities running on the ''Asset'' layer. The data services also include data persistence, provisioning, integration, and integrity. Moreover, it receives the events and messages coming from physical assets, via lower-level layers, and applies suitable data processing and transformation to support the ''Functional'' layer.

E. FUNCTIONAL LAYER
This layer has two main functionalities, data analysis and data visualization, achieved through the machine learning framework and web application, respectively. One of the goals of the proposed design architecture is to analyze data to uncover hidden patterns and to create machine learning models to improve the performance of the tasks executed. For enhancing the tasks, training data is loaded from data storage and pre-processed before being fed into the machine learning models. Each model is chosen depending on the application and trained by employing the training data. Afterward, the model might be validated or evaluated by using pre-selected test data, providing for a way to improve the model, if needed. These actions are known as model building and are implemented on the model building framework. Once the trained model is satisfactory to address the problem, it is used by the machine learning framework for processing the data stream. Dashboards, for visual interfacing, are crucial to understand difficult concepts or to identify patterns hidden in the process data. Thereby, the visualization module presents the prediction results, enabling experts in the field to add knowledge through the use of semantic annotations to help the analysis. The whole process might involve dashboards, client applications, and software for interactive analyzes.

F. BUSINESS LAYER
This is the level where the task descriptions and challenges to be overcome in the manufacturing process are monitored by the managers. The results of the production analysis might support the engineering team to take decisions to correct undesirable effects or for optimizations of the manufacturing framework.

G. HIERARCHY LEVELS
The elements in the hierarchy ''Hierarchy Levels''-axis illustrate the location of responsibilities and functionalities in the designed system's physical architecture. The arrangement proposed comprises four sectors, from field devices to work centers. The architecture's most basic level encompasses the ''Field Devices'', which comprises the set of data collectors and sensors embedded in elements installed at the ''Asset'' layer. The ''Control Device'' facilitates the automatic control of all the processes accomplished using industrial controllers and computers. The ''Station'' level is responsible for connecting data from the equipment and managing various individual machinery working in the manufacturing system. Finally, the hierarchy's highest level is the ''Work Center'' level, which maintains manufacturing information, defines the production stages, and oversees the renewal of raw materials for the production of personalized products.

H. ARCHITECTURE'S TASKS
The main tasks of the designed architecture differ at different levels. However, three main tasks can be highlighted: 1) monitoring the production process status and behavior; 2) regulating the manufacturing process in real-time; and 3) decision-making on the production process based on ML techniques. Hence, to achieve effective human-machine interfacing, the physical network functions must be properly structured and configured.
In the ''Asset'' layer, the complete physical networks covering the full production process and including all connected equipment along with an IT system (located in the cloud or installed locally) that supports the data acquisition and exchange must be designed and deployed properly. The engineering team should be able to monitor the whole manufacturing situation and behavior using the physical interface. The actuators and sensors of the physical components play a key role in the production asset by sensing the factory-floor and collecting signals at an appropriate frequency.
The ''Integration'' layer connects the IT components to the sensor nodes and gateway. Through network integration, this layer also provides data, such as robots' positions, machines' states, and other parameters, as well as information about human operators' productivity, to be used by other systems. This can be leveraged for the interactions between operators and robots by using human-machine interfaces. In the ''Communication'' layer, a standardized communication interface for all the devices and equipment is supplied. The gateway servers act as signal amplifiers and middleware nodes to the wireless network. A communication protocol (e.g., OPC/TCP and SOA/HTTPS), supported by OPC UA, should be chosen for managing data on the whole manufacturing process.
Decision-making about the production process is achieved based on structured data of acquired parameters delivered by the lower layers. However, the information must be pre-processed before being used by the machine learning algorithms. Thus, in the ''Information'' layer, data is stored for future access and analysis on a data lake used to consolidate and centralize the large amount of unstructured data collected in different formats. Furthermore, this data also can be used for designing customized products with reduced costs and faster time-to-complete production.
The ''Functional'' layer defines the model and services integration of the smart manufacturing by generating decision-making tactics applying the trained machine learning predictive models. The trained models have diverse applications, e.g., by using one machine learning model, a CNC machine can be monitored and the algorithm can recognize machining process failures, leading the system to take decisions to immediately perform automatic control actions for fixing the failure. It can also simultaneously send alert reports about the situation to managers.
Finally, the ''Business'' layer is designed to carry all information concerning the status of the production order, production timeline, and real-time status of every item for a respective client. This information enables a fast view and decision-making process for managers and clients alike.
The architecture introduced overcomes the main shortcomings of applying different technologies to smart manufacturing, summarized as follows: 1) it meets data integration requirements by using unified OPC UA information; 2) the proposed architecture allows the acquisition of data from different protocols used in many types of machines as well as legacy systems; 3) the architecture is created in layers of separate elements that can be connected, making the addition, substitution, and reuse easier, without deleterious cross dependency effects.
However, besides all the opportunities created by the proposed architecture, there are also challenges. One of them is related to the increased connectivity of equipment and systems, making them exposed to cyber-attacks if not protected correctly. Therefore, all layers that compose the architecture must be protected simultaneously from field devices in the ''Asset'' layer to the management system in the ''Business'' layer. Protection against attacks can be deployed by enabling data security techniques, which include authentication, authorization, encryption, and access control of the RAMI 4.0 architecture.

V. CONCLUSION
This paper presented an overview of the definitions, concepts, standards, and other features related to smart manufacturing, cooperative robotics, and machine learning techniques used in the industrial context. Furthermore, a methodology for designing an application of collaborative robotics and machine learning applied to smart manufacturing systems based on IIoT concepts was introduced. The main focus was on the opportunities brought by taking an IIoT perspective, as well as the challenges for the effective use of these techniques in industry. Besides the discussion and literature review, a reference architecture for collaborative robotics, and machine learning in smart manufacturing, compliant with the standards and protocols currently available, was proposed, discussed, and conceptually developed.
The designed architecture must be implemented aiming to test its performance, scalability, adaptability, as well as compatibility with industrial requirements. The designed architecture allows the integration of the concepts at data and devices level for smart manufacturing, where 5G technology will certainly play a key role as a communication framework to make smart manufacture an effective element in the increase of flexibility and productivity in industries around the world.