Towards an Automated Learning Control Architecture for Cyber-Physical Manufacturing Systems

Manufacturers are constantly looking to enhance the performance of their manufacturing systems by improving their ability to address disruptions and disturbances, while reducing cost and maximizing quantity and quality. Even though innovative mechanisms for adaptability and flexibility continuously contribute to the smart manufacturing evolutionary process, they generally stop short of providing a capability for coordinated on-line learning. This is especially true when that learning requires exploration outside of established operational boundaries or uses artificial intelligence (as opposed to purely human intelligence) as part of the dynamic implementation of learning. In this work, we provide a vision for the development of an automated learning control architecture to extend the adaptability and flexibility capabilities of manufacturing systems. As part of this vision, we describe a set of requirements and objectives that, if addressed, provide an environment to allow distributed and automated learning across the manufacturing ecosystem. We then provide an example communication and control architecture that meets these requirements and objectives by gathering information, building a dynamic knowledge base, distributing intelligence, making decisions, and adapting the control commands sent to the equipment and people across the manufacturing ecosystem. The example architecture leverages both centralized and distributed control strategies and the ability to switch between the strategies to gather and learn from information in the system. Example case studies are provided illustrating how this architecture can be used to improve manufacturing system performance.


I. INTRODUCTION
The need to design more adaptive and flexible manufacturing systems that perform effectively in uncertain and dynamic environments has been one of the main lessons of the recent COVID-19 pandemic crisis. During the pandemic, manufacturers struggled with fulfilling a number of new product orders and meeting customer requirements caused by a significant shift in demand [6], [7]. As shown in Figure 1, a number of research trends over the past several decades have focused on this objective. Various approaches have been developed to integrate and improve dynamic decision making and learning within the manufacturing system control strategy to achieve more system adaptability and flexibility. However, there is still a significant amount of work required to create fully flexible and adaptable manufacturing systems that learn and make decisions in a dynamic environment. A vision for a control architecture that can move past its predefined capabilities, learn from new information, and VOLUME X, 20XX i FIGURE 1: The effect of the development of dynamic decision making and learning capabilities for manufacturing system adaptability and flexibility. The arrows indicate how recent trends are pushing manufacturing toward increased adaptability and flexibility through dynamic decision making and learning. The dashed, blue arrow shows how the work proposed in this paper contributes to this trend.
improve its dynamic decision making capabilities based on newly obtained information needs to be realized.
Centralized and distributed control approaches are two of the most common methodologies that are used when developing control architectures [8]. In the centralized control architecture, functions are delegated to the system components (e.g., robots, workstations, etc.) to perform individual operations, while the centralized decision maker has the full authority for their initiation, coordination, and monitoring. A centralized control architecture is effective in ensuring that a manufacturing system meets customer and manufacturer requirements and objectives. However, the centralized control architecture often struggles with going past predefined capabilities and learning from new information due to the inherent complexity of using a global view of the manufacturing system (e.g., large time and computational complexity for tasks such as updating models, analyzing information). This type of architecture has been implemented in the majority of existing manufacturing systems [9].
In the distributed control architecture, a complex global control problem is divided into smaller local problems, each linked to an intelligent control unit, i.e., an agent. Due to the presence of multiple decision makers and knowledge bases, a distributed control architecture will often implement algorithms that allow the agents to move past predefined capabilities and learn from information in the manufacturing system. However, these capabilities are not often well defined and the local decisions made by the distributed agents are frequently difficult to verify as being aligned with global manufacturing objectives and constraints. Therefore, there is a need to develop a manufacturing control architecture that can explore beyond its predefined capabilities and learn from gathered information, while still making sure that the overall manufacturing system objectives and constraints are met.
Note that creating the right balance between centralized and distributed control is a challenge that is not unique in manufacturing. For example, network management, recently transformed by the adoption of software defined networking [10], is benefiting from increased programmability of the network controllers. However, how to realize the effect of logically centralized but physically distributed or hierarchical controllers [11], [12] to ensure scalability and responsiveness is still being actively explored.
In this paper, we describe a vision for automated learning in manufacturing systems that would allow the system to go beyond its predefined capabilities and automatically learn from new information. As part of this vision, we define the requirements and objectives for manufacturing systems to enable automated learning. We also propose an example control architecture that can meet the outlined requirements and objectives by enabling dynamic decision making in the manufacturing system. This example automated learning architecture implements a framework in which a centralized controller leverages intelligent, autonomous agents to gain new information from the system and push the limits of the system's predefined capabilities. The proposed automated learning architecture can enable more robust behavior and improve the flexibility of the manufacturing system, while maintaining a desirable system performance.
The rest of the paper is organized as follows. Section II presents a taxonomy to describe dynamic decision making and learning in manufacturing systems. The requirements for a general manufacturing system architecture are stated in Section III. Section IV overviews how some of the relevant existing literature (i.e., various architectures and frameworks) meets the manufacturing system architecture requirements. An example system-level architecture that meets the identified requirements is described in Section V. Some example case studies that will benefit from the proposed control architecture are provided in Section VI. Section VII lays out concluding remarks and future directions.

II. TAXONOMY
This section provides a summary of the main concepts and terminology that are used to develop the requirements and the structure of the proposed system-level control architecture with enhanced dynamic decision making and automated learning capabilities.

A. GENERAL TERMINOLOGY
Manufacturing system -a collection of resources, materials, people, and information arranged and operated to produce value-added physical products [13], [14].
Manufacturing objectives -parameters that have to be satisfied by the manufacturing system [15]. System objectives can vary widely depending on different factors through the manufacturing ecosystem (e.g., supply chain or shop floor requirements). The objectives are usually functions of manufacturing quantity, quality, and cost [16]. Manufacturing objective metrics -indicators that are used to measure whether the manufacturing system is meeting the manufacturing objectives. Some objective metrics are expressed as key performance indicators (KPIs).

B. MANUFACTURING SYSTEM CAPABILITIES
The following terms are used to describe components in a system-level control architecture that enable a manufacturing system to meet its manufacturing objective. Agent -a cyber element that obtains information about the environment (i.e., sense/measure), makes a decision based on its own internal objective (i.e., think/reason/decide), and affects the environment through its actions (i.e., act) [17]. For manufacturing systems, agents can be used to make decisions for individual distributed resources, parts, processes, cells, etc. Note that an agent can obtain information and affect the environment by communicating with the physical system, other agents, or other cyber elements. This communication can be accomplished using various communication protocols, such as [18], [19]. Knowledge base -a repository of information, data, models, etc. that is used by an agent to store and organize information.
For manufacturing systems, this information can include the capabilities of the system, rules for system behavior, and models of the shop floor equipment [20]- [22]. A knowledge base can belong to an individual agent or it can be used by multiple agents to store shared information. Decision maker -a component of an agent that is responsible for reasoning and determining how the agent should act [20]- [22]. The decision making component of an agent leverages the knowledge base of the agent to accomplish this task. Each agent must have an individual decision maker. Note that conflicts, such as conflicting requests or learned information from other agents, in the manufacturing system can be resolved by the decision maker through cooperation [23], [24] or by leveraging higher authority agents (e.g., a central controller) [25].
Agent capability -an action that the agent can perform. For example, the capabilities of a milling machine agent might include sending programming commands, scheduling various operations, and analyzing milling machine data. Capabilities can be set initially and/or could be learned over time. The collection of agent capabilities associated with a particular agent is referred to as an agent capability set.
Capability boundary conditions -the conditions under which an agent has the ability to perform the specified capability. Capability boundary conditions are often expressed with respect to how, where, when, and to what extent the capability exists. If an agent's decisions lead to an action that does not satisfy the capability boundary conditions, it cannot or should not perform any actions associated with that capability. Capability boundary conditions could be associated with manufacturing objective metrics, but could also be associated with other metrics not related to the manufacturing objectives, such as safety, security, and regulations (e.g., maximum carbon footprint). An example of a capability boundary condition would be a minimum temperature for achieving a minimum yield of a product for a given process, or a maximum temperature limit for all processes based on safety regulations. VOLUME X, 20XX iii A capability boundary condition could be directly tied to a particular parameter (e.g., temperature) or associated with a number of parameters (e.g., a mathematical objective function or a Boolean combination of parameters and functions). The agent's observations from the environment could also be used to define boundary conditions (e.g., can only adjust pressure when temperature is in a certain range). Capability boundary conditions could be established by the agent to define its full set of capabilities in the current environment. These boundary conditions could also be provided by another agent (e.g., a supervising or controlling agent) and may change based on the state of the manufacturing environment. Capability authority -a parameter of an agent's capability that defines whether the agent has been given permission to perform that capability. If an agent does not have the authority over a capability, it is not permitted to perform the actions associated with that capability. The authority of a capability is usually granted along with a capability boundary condition tuple representing limitations to the granted authority. An agent may gain or lose capability authority over time. The authority can be managed by the agent itself or some governing entity (e.g., a supervising agent). Capability boundary conditions tuple -the capability and its associated boundary conditions. Capability boundary conditions set -a set of capability boundary condition tuples.

C. SYSTEM-LEVEL MANUFACTURING CONTROL TERMINOLOGY
Cyber-Physical Manufacturing System (CPMS) / Cyber-Physical Production System (CPPS) -a manufacturing system that contains a collection of physical, cyber, and cyberphysical agents associated with a defined set of objectives and a defined capability boundary conditions set. The system objectives can vary widely depending on the system and ecosystem environment. Note that the capability boundary condition set of the system is the superset of the capability boundary conditions set of the agents in that system. Manufacturing control architecture -consists of a collection of agents and their communication infrastructure that acts on the physical environment to achieve the manufacturing objectives. The agents (or agent) control the physical components of the manufacturing system to meet a defined set of objectives. An example architecture for a manufacturing system that uses the terms defined in this section is shown in Figure 2. Centralized Smart Manufacturing System (CSMS)a Manufacturing System in which one controlling agent, termed "Central Controller" (CC) in this paper, determines the capability authority of the system [26]- [28]. This means that the CC either directly or indirectly dictates or establishes the capability boundary conditions of all the agents in the CSMS (agents may choose to further restrict boundary conditions). Oftentimes, the CC sits atop a multi-layer hierarchy of authority, e.g., an ISA-95 structured system [9].

D. ADAPTATION AND LEARNING TERMINOLOGY
Adaptation capability -the ability of the agent to execute actions within the capability boundary conditions set of the agent. Adaptation capability authority -is the parameter associated with the adaptation capability of an agent that defines whether the agent has been given permission to perform adaptation. This capability can be allowed/restricted by an entity such as a supervising agent or SME that has the power and capability to control that authority. Manufacturing agent adaptation -refers to the action of an agent that results in changes of an agent's behavior within defined bounds of the agent capability boundary conditions set. The purpose of this action is to achieve an objective. The adaptation is usually accomplished through some combination of data collection and analysis such as machine learning, and actuation such as updating control parameters. It could also be accomplished as a result of bounded implemented learning (see definition below). The objective of the adapting agent can be anything and does not necessarily align with the higher-level objectives of the manufacturing system. Learning capability -the ability to acquire knowledge or skills through experience, study, or by being taught. Learning is usually accomplished through some combination of data collection and analysis and some input and assimilation of intelligence, which would necessarily include some elements of Artificial Intelligence or Natural Intelligence, i.e., subject matter expert (SME) input. Note that individual agents in a CPMS could learn from information provided by other agents (e.g., a central controller updates its knowledge or decisions based on information provided by other agents) or could have the capability of self-learning (e.g., agent updates its own models and decisions based on new information from the physical system). Agent learning may result in knowledge updates for the agent, lead to new decisions based on these updates, or affect the behavior of the agent and the system. Learning capability authority -is the parameter associated with the learning capability of an agent that defines whether the agent has been given permission to perform learning. This capability can be allowed/restricted by an entity such as a supervising agent or SME that has the power and capability to control that authority. Manufacturing agent implemented learning -the execution of learning that results in changes in the agent's knowledge base. This execution may be accomplished purely by adaptation (e.g., an agent implementing a new method for setting a parameter within established bounds based on learning) or by an agent moving outside established capability boundary conditions (e.g., actuating a set-point outside a bound). Manufacturing agent bounded implemented learningimplemented learning in which the implementation results in the agent staying inside established bounds dictated by its adaptation capability. An example of bounded implemented learning is improved granularity/fidelity of models of an agent that are used during adaptation. Manufacturing agent exploratory implemented learning -implemented learning in which the implementation results in 1) the agent moving outside established bounds dictated by its adaptation capability (i.e., boundary violation) or 2) the agent having to adjust established bounds that results in a boundary set that is not fully contained in the previous boundary set (e.g., to "wider" or "shifted" bounds). This is often termed "relaxing boundary conditions". An agent often achieves this learning by both (1) relaxing the limits in the boundary condition set and (2) having incentives that allow the agent to explore beyond its existing limits. Relaxing boundary conditions happens in coordination with a supervising agent or SME.

E. ILLUSTRATION VIA A MANUFACTURING SYSTEM EXAMPLE
We will use the proposed taxonomy to describe an example manufacturing system: the System-level Manufacturing and Automation Research Testbed (SMART) at the University of Michigan [29]. An overview of the testbed is shown in Figure 3. SMART has four computer numerical control (CNC) milling machines, two conveyors, and three industrial robots with an integrated industrial control system. Workblocks with RFID (radio-frequency identification) tags are placed in the system at Cell 3. The workblocks are moved to specific CNCs, where a manufacturing process creates a product feature on this workblock. Once all of the required features are created, a workblock travels back to Cell 3 to exit the system. RFID transceivers are located on the conveyors to locate and identify workblocks. The conveyor lines, robots, and CNC machines are able to handle a variety of parts using pallets, interchangeable grippers, and pneumatic clamps. SMART is a CPMS that has been used to develop and test various manufacturing control architectures. For example, in [28], SMART is converted into a CSMS with a single central controller agent to meet system throughput objectives.
SMART has also been used to test a control architecture similar to the architecture shown in Figure 2 [30]. For the developed agent-based control architecture, a central controller agent initializes a set of resource and product agent, where a resource agent controls each manufacturing resource (e.g., CNC machine, conveyor, robot) and a product agent controls each workblock. For example, the CNC machine agent is one of the agents that is initialized by the central controller. This agent would be represented by one of the lower-level agents in Figure 2. The CNC machine agent stores information about the associated CNC machine in its knowledge base and uses the decision maker to send highlevel commands to the lower-level machine controller (e.g., high level controller selects a milling program that the lowerlevel controller will run). The CNC machine agent has a number of capabilities, from sending commands to the lowerlevel controller to scheduling milling tasks to analyzing energy expenditure of the machine. Each of these capabilities is associated with boundary conditions, e.g., the machine agent cannot start a machining task if there is an operator present in the cell (safety), the tasks have a threshold limit for the amount of energy use (regulation), and the program's tool path is within some pre-defined bounds. The lowerlevel agents update the central controller with information about the shop floor. Then, the central controller can send commands regarding the capability authority of each agent, e.g., for a machine agent, the central controller can allow the mill to move beyond existing energy limits to ensure that the system throughput meets the system objective. The different machine agent capabilities and boundary conditions are stored as tuples in the capability boundary conditions set in the agent's dynamic knowledge base.
The machine agent has the adaptation capabilities and authority to change the manufacturing process by changing the tool path of the machine. The new tool path must remain within the pre-defined tool path bounds and must meet the safety and regulation requirements for manufacturing agent adaptation. Bounded implemented learning occurs when the machine agent finds a new tool path that can be used to complete the required product feature within the boundary conditions. If a part requires a completely new feature, the machine agent might have to move beyond previously defined capability bounds and start exploratory implemented learning. In this scenario, a new tool path that is outside the pre-defined tool path bounds and energy requirements might be required to accomplish the desired product feature. If a new tool path is found, the machine agent communicates with a higher authority (e.g., a central controller) to request to move the capability boundary conditions to complete the necessary manufacturing process.
Note that adaptation and learning are not limited to the machine agents and can be done by other resource agents and product agents in the system. However, to enable effective system adaptation and learning, a framework must be developed that allows a central controller and the agents to com-VOLUME X, 20XX v municate and reason about the agents' capability boundary condition sets and capability authorities.

III. REQUIREMENTS DEVELOPMENT
Using the taxonomy developed in Section II, we develop a vision for automated learning in CPMSs, provide the dominant characteristics of CMPSs today, perform a gap analysis to identify gaps that need to be met to achieve the proposed vision, and state the CPMS requirements and objectives to achieve the proposed vision.

A. VISION
As noted in Section I, there is a need to develop a manufacturing control architecture that can explore beyond its defined capability boundaries and learn, while ensuring that the manufacturing system objectives and constraints continue to be met. Restated using the taxonomy developed in Section II, this need can be thought of as a vision for CPMSs to support automated exploratory learning in the manufacturing control architecture. This learning would be accomplished by using the capabilities of the agents to enable manufacturing agent implemented learning. This implemented learning would seek to further optimize or enhance manufacturing objective metrics, while maintaining a capability authority infrastructure across the system to guarantee that all manufacturing requirements are also met. The implementation of this vision requires that CPMSs have a manufacturing communication and control architecture that supports exploratory implemented learning. This architecture would thus have to (1) provide mechanisms to support adjustments to capability authorities and associated capability boundary condition sets across the agent base while (2) maintaining a capability authority of the entire system that ensures manufacturing requirements (including objectives and constraints) are met.
Realizing this vision places specific requirements on the design and implementation of the CPMS. The vision is also associated with objectives that the CPMS must strive to achieve if the capabilities defined in the vision are to be used effectively, e.g., as part of a continuous improvement process. In this section, these requirements and objectives are created by exploring characteristics of typical CPMSs in operation today and conducting gap analysis to identify deficiencies and challenges that prevent the achievement of the vision.

B. CHARACTERISTICS OF A CPMS TODAY
The manufacturing control architectures of today's CPMSs vary widely, depending on a variety of factors including the objectives and requirements of the manufacturing sector and manufacturing instance, and age and location of the facility. Most CPMSs/CPPSs today are CSMSs supporting multi-layer hierarchical control and execution hierarchies with some level of alignment, at least informally, to the ISA-95 model [9]. Additionally, these systems usually have the following properties, resulting in large part from manufacturing objectives and requirements: • Have a central controller (CC) agent that determines the capability authority of the entire system, per the CSMS definition [26]. The manufacturing control architecture might contain distributed agent-based sub-systems dedicated to specific tasks (e.g., maintenance management). However these sub-systems are not allowed to violate the capability boundary conditions established by the CC. The CC could be a fully automated cyber entity or could also be a hybrid of cyber and non-cyber components such as best practices, spreadsheets, and manual executions. • Support various levels of manufacturing agent adaptation as part of the CSMS delegation process across the manufacturing control architecture. The adaptation could consist of fully automated cyber adaptation capabilities (e.g., model-based process control) or be a hybrid of cyber and non-cyber adaptation capabilities (e.g., manual product quality sorting based on machine vision). • Maintain a capability boundary condition set in the CC that can only be updated through human (e.g., SME) involvement. That is, from the perspective of the CC, exploratory implemented learning can only be achieved with human involvement. The level of human involvement can vary widely depending on the application environment and the type/potential impact of learning. For example, the SME might be the source of the learning to be implemented, or the SME might simply be in the verification and validation (V&V) loop of AI sourced knowledge updates [31], [32]. Regardless, any learning that results in a change to the system capability boundary conditions set requires some level of human approval. This process evolved naturally in manufacturing where innovation continues to be a largely human process due to tradition, manufacturing sector requirements (e.g., pharmaceutical industry [33]), general limitations of AI such as lack of Artificial General Intelligence (AGI) capabilities [34], [35], and lack of trust or ability to automatically verify or quantify AI learning prior to implementation. Note that in some systems, agents below the CC might be given the capability authority to conduct exploratory implemented learning as long as the outer capability bounds of the CSMS are not violated. An example here would be permission given to a CNC milling agent to go into a learning mode at the agent's discretion to learn a new path for a part. • Are increasingly becoming an integral part of a manufacturing ecosystem from raw materials to the customer experience [36], [37], but continue to maintain CSMS capability boundary conditions set within the defined CSMS. Note that the defined CSMS could be the "fourwalls" of the manufacturing facility, but could also be an entire integrated manufacturing ecosystem including the entire supply chain and the customer.

C. GAP ANALYSIS
From the perspective of achieving the vision of supporting automated exploratory learning, these are gaps in capabilities of current CPMS systems, as well as issues with AI techniques and human practices. The gaps in current CPMS systems arise largely from a lack of emphasis on quantification and automation of the learning processes. These gaps include: • The procedures for on-line implementation of learning can vary widely and are largely ad hoc. There have been significant strides as part of the smart manufacturing evolution to provide a lifecycle approach for learning development, V&V, deployment and assessment, e.g., as part of the digital twin (DT) implementation process [31]. However, issues such as diversity in learning topics and SMEs often make it difficult to realize consistent detailed procedures. • The connection or linking of boundary conditions to manufacturing requirements is oftentimes not welldocumented or well-understood quantitatively or even qualitatively. For example, many boundary conditions result from supplier documentation (e.g., maintenance practices for a machine), regulations, or simply tradition. Also, boundary conditions are often considered in isolation (e.g., maximum temperature and maximum pressure) and not part of a more complex multivariate set of condition relationships that are more representative of the application environment. • There is little formal categorization or quantification of learning from a risk vs. reward perspective. Exploratory learning processes can be associated with widely different risks and rewards (e.g., expanding a control boundary condition versus a safety boundary condition). While these risks and rewards are often quantified and generally categorized for re-use (e.g., cost-benefit analysis procedures for adjusting a maintenance schedule), the mechanisms vary widely across CPMSs. • There is no formalized process, structure or language for capturing and transferring learning between agents in a CPMS, especially between agents and the CC in a CSMS. As noted earlier, bringing the data together in a CPMS as part of the smart manufacturing evolution provides huge opportunities for learning. However the process for a CC to learn as a result of agent actions and communication of learning is largely non-existent or ad hoc and requires a human in the loop. • Intelligent agents in a CPMS are oftentimes not given the freedom that would allow the system to take advantage of this intelligence. As an example, smart manufacturing systems today oftentimes have highly intelligent edge devices as part of an IIoT infrastructure; this intelligence can result from highly directed AI associated with the edge device application area or access to more detailed information closer to the application environment. CPMSs frequently do not provide the necessary flexibility to allow these "smart" devices to conduct exploratory learning or, as noted above, there generally is not a process for the CC to learn from these agents so as to support a more fruitful CSMS-wide exploratory learning environment. • The available methods to ensure system security, especially data and intellectual property (IP) security are insufficient to maintain security levels in the envisioned CPMS. Data and IP security has been highlighted as the most important security issue in some manufacturing domains [38]. The distribution of information required to support many of the tenets of smart manufacturing is often gated by concerns over IP. This problem will be exacerbated by the need to communicate and share learnings as part of the CPMS vision. • There is a lack of sufficient integration in the ecosystem and associated traceability (e.g., digital thread [39], [40] and blockchain [41], [42]) to support the CPMS vision in areas where supply chain information is a component of implemented learning. Examples include (1) (upstream) learning supplier patterns and implementing associated learning to improve throughput, and (2) (downstream) learning customer buying patterns and optimizing production to customer delivery requirements. • Existing CPMS hardware and software may not be able to easily and effectively integrate new learning approaches. The hardware and software found in existing CPMS were mainly developed to ensure system reliability and may need to be updated to incorporate the computation and reasoning required for enhanced dynamic decision making and learning. • There is no standard communication mechanism to support interoperability, interchangeability, extensibility, reusability, and other "ilities" associated with implemented learning sharing and distribution. This communication solution could be an extension of an existing factory wide data communication solution such an ISA-95 solution provided by the Asset Administration Shell (AAS) [43] or built on existing protocols such as OPC-UA [44]. However enhancements would still be needed to support the requirements of implemented learning communication.
Gaps in AI techniques as they apply to achieving the CPMS vision result largely from the relative infancy of AI application in manufacturing. These include: • AI learning is usually not accompanied by an understanding of the quality of or confidence in the learnings. As with any learning process, there is a possibility of error. More formalized techniques are needed to convey AI learning and associated quality or confidence such as those associated with DT predictions in a DT framework [31]. Quantification could include believability, range of impact, and thresholds on impact such as maximum negative impact on a metric. • AI techniques in manufacturing are primarily narrow VOLUME X, 20XX vii AI applications. Thus the impact of exploratory implemented learning in areas outside of the domain of the AI knowledge base are unknown to the AI system. Narrow AI techniques will need to be formally combined with techniques towards AGI or human intelligence before implemented learning can be trusted [35]. • The use of narrow AI technique also creates a learning gap in that learnings are confined to a limited understanding of aspects of the application environment. This prevents the creative type of learning associated with AGI or the SME often termed "thinking outside of the box" [32]. • There is no formalized process for AI-SME interaction in exploratory implemented learning. Ideally the learning process is a continuum that evolves over time from primarily SME learning to AI assisted learning to AGI and SME learning. Realizing this continuum requires developing a formalized methodology to guide AI-SME interaction throughout the learning process [32], [37]. • AI techniques have been successfully used in specific applications, such as process monitoring, optimization, etc. However, their utilization is limited at the system level of decision making due to the difficulties of data synthesis. The heterogeneous manufacturing data may contain a high degree of irrelevant, redundant, and missing information, which influences the performance and suitability of AI algorithms relative to expected results [45].
Gaps in human practices relate largely to culture, including: • From the manufacturer SME perspective there is oftentimes a general resistance to support AI techniques when they are considered as a replacement for SMEs rather than a tool for enhancing SME capabilities. Recent efforts in this space directed at educating, training, and connecting the manufacturing workforce are meant to address this challenge [46]. • From the AI developer perspective there is oftentimes a belief that data driven solutions will solve problems and SME expertise is not necessary. Recent efforts in this space have begun to address this challenge by illustrating the importance of combining SME and AI knowledge with an objective of "no knowledge left behind" in smart manufacturing [32], [37].

D. REQUIREMENTS AND OBJECTIVES
With an understanding of the CPMS vision, current practices and gaps, CPMS requirements and objectives can be formulated that are necessary to achieve the vision. Leveraging existing works that have defined requirements and objectives for general industrial AI applications [47]- [49], security [50], and distributed CPMS [51], [52], we define the primary requirements and sub-requirements for future CPMS: R1 Exploratory learning must be quantifiable in terms of learning parameters that necessarily include assessment of benefit, risk, quality of (or confidence in) learning, and impact on capabilities, capability boundary conditions, and capability authorities. This necessarily places similar requirements on the intelligence mechanism used in developing the learning. The relationship between learning, learning parameters, and capability boundary conditions must be quantifiable. This includes any interrelationships between boundary conditions related to the learning. R2 A framework must be in place to support the distributed implementation of learning throughout the system. a) The framework must include a mechanism to support the communication of learning among agents within a particular level and between levels in a system. b) The framework must include a mechanism to support the implementation of learning among agents including any adjustments to learning parameters. This mechanism necessarily includes methods for agents to specify, communicate and change capabilities, capability boundary conditions, and capability authority in the CPMS. c) The framework must extend across the entire ecosystem as defined by the CPMS implementation which would likely include elements of the upstream and downstream supply chain. d) The framework must provide a level of re-usability, interoperability (e.g., an abstraction layer or a communication protocol to support different agents, agent types, or agents classes [53], [54]), interchangeability, maintainability, and extensibility to ensure overall benefit. R3 An evolutionary process must be in place to support implementing learning resulting from a continuum of human and AI knowledge.
a) The process must support existing learning mechanisms. This could include isolation of boundaries within a boundary set for adjustment, serial assessment of impact on critical aspects such as safety, learning from the SME (see below), and visualization and explanation of learning for SME empowering. b) The process must support the evolution of the implemented learning mechanisms (as abilities of and confidence in AI systems evolve). This includes the incorporation of AGI with AI, and SME empowering and evolving role definition. c) A re-usable learning development and implementation lifecycle must be developed as part of the process. The lifecycle must include verification and validation of learning prior to deployment, assessment of learning implementation, and feedback to the learning. Specifically for bounded implemented learning, it is important to develop verification and validation algorithms to ensure that the system behaves in a viii VOLUME X, 20XX desired manner. d) The process must account for traceability of the learning in terms of contributors, contributions and knowledge base evolution. e) The process must consider and address conflict resolution in the system. Agent-to-agent (e.g., [23], [24]) and non-agent-to-agent (e.g., human-to-agent [55], [56]) conflict resolution must be developed as part of the process.
The CPMS objectives related to the vision are: O1 Maximize use of existing processes to enable automated exploratory implemented learning a) Enable extension of the adaptation paradigm to support adjustments based on learning for specific bounds to which confidence in learning can be applied and/or possibility of mistakes can be accepted. b) Use off-line and manual processes in existence for implemented learning as input to developing automated on-line learning implementation processes. This could include existing life-cycle processes. O2 Provide incentives for CPMSs and their agents to learn in a manner that is most beneficial to the CPMS. a) Provide quantifiable incentives to improve manufacturing objective metrics and expand/relax capability boundary conditions performance objectives, while establishing appropriate penalties for mistakes as well as missed opportunities for learning. b) Provide incentives to explore outside of the ecosystem boundaries, e.g., to achieve exploratory implemented learning that incorporates supply chain information or learning communication. c) Provide incentives to enhance the re-usability, interoperability, interchangeability, maintainability, and extensibility of the distributed implemented learning framework as part of a continuous improvement process. O3 Provide mechanisms that support human and AI interaction in the learning process with an objective of "no knowledge left behind." a) Provide methods for structured interaction between AI systems and SME networks. Clearly define AI and SME complementary roles (e.g., innovating and approving) and boundaries. Define role evolution paths and conditions for evolution. b) Support and incentivize the empowering and enabling of the SME as a collaborator in CPMS and its evolution (growth), rather than the elimination of the SME. This includes the SME learning from the AI. c) Support the empowering and enabling of the AI system and its evolution. This includes incentivizing the AI system to learn from the SME and the SME to aid in the teaching of the AI system. O4 Ensure that appropriate levels of security, especially data and IP security, are maintained during imple-mented learning. Achieving this goal will likely require a combination of applying existing and emerging capabilities such as homomorphic encryption and federated learning [57], [58].

IV. MAPPING EXISTING WORKS TO REQUIREMENTS AND OBJECTIVES
As described in Section I, there is a need to develop a control architecture that improves system-level dynamic decision making and learning. To achieve this objective, the control architecture must meet the requirements and objectives identified in Section III. A number of existing works have defined and addressed some of these requirements and objectives. Table 1 provides a summary of different works in the manufacturing domain, including centralized control approaches, distributed control approaches, hybrid (centralized and distributed) control approaches, and reference architectures. The rest of this section provides more details about the works identified in Table 1.

A. CENTRALIZED CONTROL APPROACHES
A common approach for control of manufacturing systems is centralized control. For this approach, a central controller has the necessary models, data streams, and control logic to make decisions about the production system. The controller obtains run-time data through the sensors on the plant floor [26], [59]. The data is processed by the controller to run optimization procedures (e.g., production planning, maintenance scheduling), make rule-based decisions (e.g., system reconfiguration, fault detection), or to monitor process KPIs (e.g., system throughput, yield, quality). Due to its centralized structure and access to all components on the plant floor, central controllers can optimize global objectives and implement changes across the entire system. As a result, centralized architectures are often capable of fulfilling R1 since they have a clear understanding of the global view of the CPMS, which they may utilize to define capability boundary conditions. It is important to note that such system-level exploratory learning applications are not extensively studied in the literature due to the issues in computational complexity and scalability. As manufacturing systems get more complex and autonomous, the learning problem becomes increasingly challenging. Hierarchical architectures to enable scalability and modularity are utilized for this purpose (e.g., ISA-95). The main drawback of centralized architectures is the lack of distributed learning capabilities (i.e., requirements related to R3), which limits the applicability of exploratory learning applications in practice. A digital-twin based exploratory scenario for a manufacturing cell is given in [85]. A centralized scheduler is trained using DTs of the CPMS (satisfying R1). While R2 is not explicitly addressed, the offline training of multiple agents is proposed as a solution for distributed learning. Centralized approaches also result in heavy burden on network bandwidths due to the large amounts of data collected frequently that has to be transmitted to the central controller as well as latency and poor response time.   [22], [24], [72] A The values in the table are "-", "D" or "A" , where "-" means the referenced do not mention the Requirement/Objective (R/O); "D" means the references define or state the R/O; "A" means the references propose a method that addresses the R/O. Note that none of the references fully define or address any of the individual requirements or objectives provided in Section III.

B. DISTRIBUTED CONTROL APPROACHES
Another approach for the control of manufacturing systems is distributed control. For most distributed control approaches for manufacturing systems, a number of software agents make high-level decisions for various manufacturing system components (e.g., machines, physical parts, product orders, etc.) [5], [8], [65]- [67]. These decisions determine the performance of the entire manufacturing system [64]. The agents make decisions to take actions based on their own internal objectives and information about the environment [21], [24].
The focus of most existing works is to construct the overall multi-agent architectures to achieve desired manufacturing objectives. PROSA [60] and PABADIS [61] provide a general description of agent knowledge, communication, and decision-making behaviors in their multi-agent architectures. ADMARMS develops the requirements and qualitative design principles for the knowledge, functionality, and communication capabilities of its agents [53]. ADACOR provides a formal specification to describe the behavior of its proposed agents using Petri nets [62], [63]. In [21], [70], the internal architectures that enable agent communication and decisionmaking of two types of manufacturing agents -product agent and resource agent -are developed. However, these architectures do not provide well-defined agent capabilities, and thus cannot support quantifiable implemented learning or the communication of these learnings (R1 and R2).
A few existing works have provided definitions for agent capability, but these definitions are specific to the capabilities of manufacturing resources. In [71], [73], agent capabilities of manufacturing resources are broken down into atomic (simple) and complex (combined) capabilities. These capabilities are described in a semantic manner with several parameters acting as boundary conditions. In [74], agent capabilities are defined by a set of specifications in the form of preconditions and postconditions using propositional logic. A capability can only be executed when its preconditions are true, and the postconditions will hold if the capability ends with success. However, the capability descriptions, combination rules, and condition-based logic of these works are predefined and do not change dynamically. In [22], [24], [72], a finite state machine (FSM) is used to represent the capabilities of each manufacturing resource, which enables agents to update their capabilities dynamically as they receive information from resources in their local manufacturing environment. While introducing quantifiable and dynamic characteristics to agent capabilities, this method only passes information from the physical layer to the agent layer without offering learning mechanisms (R2).
For the learning capability in multi-agent systems, [68] and [69] mentioned explorative knowledge augmentation on a conceptual level. Most existing works apply reinforcement learning to a specific problem (e.g., production scheduling) with specified learning parameters [76]- [78]. However, these works do not contain formal agent capabilities and associated boundary conditions and do not support an evolutionary learning process with dynamic decision-making (R2 and R4). Therefore, in the current multi-agent architectures, the agent capabilities and quantifiable learning parameters are not connected, which limits the ability to support implementing learning behaviors.

C. HYBRID (CENTRALIZED AND DISTRIBUTED) CONTROL APPROACHES
For industrial systems, [79] classifies hybrid (combined centralized and distributed) control architectures into three classes: class I contains typical centralized, hierarchical control, class III contains distributed, heterarchical control, and class II introduces a concept of a hybrid control architecture to bring the advantages of class I and class III together. ORCA-FMS [80] work further classifies class II into four sub-classes based on two criteria: whether the structure can evolve over time (static or dynamic) and whether the control is applied to all entities or not (homogeneous or heterogeneous). Based on this classification, ORCA is a dynamic and heterogeneous architecture in class II. It has both a global controller and many local controllers. The global controller has the state information of the whole system while the local controller only has the local view. It also has two operating modes: normal mode and disruptive mode. In normal mode, the global controller optimizes the global performance and gives orders to the local controller, which will execute these orders. When the local controller detects a perturbation, it switches to disruptive mode and handles the perturbation locally. Our work generalizes such classification by using the definitions in section II (e.g. agent, agent capability, agent adaptation) to describe a class II manufacturing system.

D. REFERENCE ARCHITECTURES
In addition to the various control approaches, a number of reference architectures have also been developed that define some of the requirements and objectives. ISA-95 is a wellknown standard that defines a data model for the integration of enterprise and control systems in manufacturing [9]. Given the established framework of ISA-95, efforts have been focused on infrastructure requirements for enhancing the ISA-95 with the Industrial Internet of Things (IIoT). In Germany, RAMI 4.0 [81] has been proposed as an architecture for the communication between the different system components across the value chain and introduced the idea of asset administration shell acting as a single point of information retrieval. In the US, the Industrial Internet Reference Architecture (IIRA) [82] has been introduced to create a common set of architecture requirements, characteristics, and patterns within and across industries for higher interoperability and enhanced development of Industrial Internet systems. Outside of Germany and the US, other initiatives have been introduced in Japan, China, Korea, and the UK. The Japanese Industrial Value Chain Initiative introduced the Industrial Value Chain Reference Architecture (IVRA) [83]. China has defined its own Intelligent Manufacturing System Architecture (IMSA) composed of three essential elements, namely, lifecycle, system hierarchy, and intelligent functions [84]. The UK has been focusing more on smart services instead of technology within Industry 4.0 [86]. Although the above mentioned reference architectures provide concepts and methods for developing concrete architectures, they are not specific architectures for concrete systems. The concepts and methods have to be applied to realize concrete system architectures and extensions of these reference architectures need to be addressed to realize practical solutions. However, systems developed using these reference architectures will still meet some of the requirement and objectives identified in Section III. Specifically, [81]- [84] state the need for quantifiable learnings and learning parameters (R1). These architectures also provide some methods to support human-AI interaction (O3) and to ensure the security of manufacturing systems (O4).

V. EXAMPLE ARCHITECTURE TO SUPPORT AUTOMATED LEARNING
None of the existing system-level control architectures for manufacturing systems addresses all of the requirements or meets the objectives identified in Section III. Therefore, there is a need to develop an effective automated learning control architecture for manufacturing systems. In this section, we provide a high-level overview of a control architecture that could be extended to meet the requirements and objectives from Section III. A visualization of this example architecture is shown in Figure 2.

A. ARCHITECTURE COMPONENTS
The proposed control architecture utilizes a number of agents to gather information, make decisions, and send commands to the equipment on the shop floor. To achieve this objective, each agent stores its capability boundary conditions set that consists of capability boundary conditions tuples in its knowledge base. Formally, the capability boundary conditions set, Capabilities can be represented as: where capability i is a capability of the agent and conditions i is the associated boundary conditions. Note that the capability boundary conditions set includes the adaptation and learning capabilities of the agent. Each agent also contains a mapping that represents whether the permission to perform that capability has been provided: The capability boundary conditions set and the capability authority are dynamic properties of the agent and will often be altered by other agents in the system, as described in Section V-B. In addition to the boundary conditions set, an agent stores other information in its knowledge base (e.g., production VOLUME X, 20XX xi schedule, dynamic model, etc.). Various other works have explored the information that each agent needs to store to make effective decisions [20], [30], [72]. All of this information is used by agents to decide what commands need to be sent to various resources on the shop floor.

B. STRUCTURE AND COMMUNICATION
To meet the requirements and objectives, we will leverage a CSMS-based approach to the development of the architecture. The proposed architecture contains a central controller (CC) agent that receives updates from the agents in the system, collects data from the shop floor, and receives updates from the manufacturing agents. However, in contrast to existing CSMS, the CC can set parameters for the adaptation capability and the learning capability to enable adaptation, bounded implemented learning, and exploratory implemented learning within the central controller as well as among the agents under its control.
To enable adaptation and learning, the CC gives agents the authority to perform adaptation and learning. In addition, the CC can set the boundary conditions for these capabilities. For example, the CC can enable adaptation, bounded implemented learning, and exploratory implemented learning only for specific agent capabilities. If exploratory implemented learning authority is enabled for a capability, the agent is allowed to relax existing boundary conditions and send commands to the shop floor that violate these conditions (e.g., [23]). In addition, the agent frequently updates the CC with any information that it obtains from the data gathered from the floor. The CC keeps track of this information and monitors the system to ensure that any global manufacturing objectives are satisfied.

C. MEETING REQUIREMENTS AND OBJECTIVES
The proposed architecture can be extended to meet the requirements and objectives identified in Section III. To meet requirement R1, individual agents need to ensure that exploratory learnings are quantifiable. The CC should also ensure that it provides exploratory learning authority to capabilities that can be quantifiable. While the proposed framework provides the means to ensure an the integration of distributed learning (requirement R2), it is still necessary to develop the algorithms required to ensure that learnings are effectively communicated in the system and across the supply chain. To ensure that requirement R3 is met, the implemented learning algorithms used by the individual agents must be able to support existing learning methods and ensure the re-usability of these methods. Further there must be a system in place for human and AI interaction to maintain the capabilities and authority and promote their evolution.
The proposed control architecture can be used to ensure that the CPMS objectives are satisfied. For objective O1, the proposed architecture can be leveraged to ensure existing adaptation paradigms can be extended to create exploratory learning. The CC can provide the agents with initial authority for adaptation and then start to slowly extend the boundary condition for agent learning. Also capabilities can be partitioned into on-line critical-path versus off-line operation so that exploratory learning could be accomplished at lower risk and cost. As part of the communication between the CC and the agents, the CC can provide incentives to the agents to allow for enhanced exploratory implemented learning to satisfy Objective O2. Similar, for Objective O3, there is a need to implement and develop SME-agent interfaces and interactions to ensure that there is "no knowledge left behind" (e.g., [55]). Finally, security protocols need to be added to the proposed architecture to ensure cybersecurity and meet Objective O4. One possible approach to enhance cybersecurity is through the extension of existing communication standards and the development of a security practices that ensure the communication between the various agents meets existing requirements. This would be complemented with a framework to support and encourage secure information and knowledge sharing.

VI. DYNAMIC RECONFIGURATION EXAMPLE
As part of the vision toward automated learning manufacturing systems, we provide a simple example illustrating the automated learning architecture for the additive manufacturing (AM) fleet shown in Figure 4. An AM fleet utilizes multiple AM resources in parallel for high throughput and flexibility [87]. The example AM fleet consists of machines that (1) produce the products, (2) perform post-processing (e.g., support structure removal, part cleaning, material treatment processes, finishing processes, or quality control measurements), and (3) transport the printed parts between other machines. While there may be a number of other machines in industrial AM fleets (e.g., raw material feed to the AM resources, assembly and packaging resources), we limit the conceptual AM fleet to the resources given in Figure 4 for the purpose of presentation.
We have three Fused Deposition Modeling (FDM) machines in the fleet, each producing dedicated products P1, P2, P3, respectively. In the nominal case, we assume that each machine is capable of producing all types of parts, but does not have the authority for reconfiguration. This nominal case describes the situation when each machine produces the corresponding product based on an average demand forecast. Therefore, for this case, each machine has a preset production schedule. The physical layout of the conceptual AM fleet is shown on the right hand side of Figure 4. The fleet has two mobile manipulators (T1, T2) responsible for material transportation between the FDM machines and the post processing stations, with the physical interconnections shown by the dashed arrow in Figure 4. The cyber-physical interconnections and data exchange are also shown in Figure 4. A fleet controller (e.g., the controller described in [26]) is in charge of production management in the system. The production schedules are provided to individual agents, which control the associated process in each machine. Agents use sensor data to control the physical resources and learn new information. Learned information xii VOLUME X, 20XX is shared with the fleet controller for further analysis and learning. The fleet controller shares system information for production boundary conditions, authorities, etc., with the agents, and the agents use their belief models and controllers to send actuator commands on the physical system.
In this conceptual case study, we consider three cases with increasing dynamic decision making, flexibility, and learning capabilities. The three cases of interest presented in Figure 5, are as follows.
• CASE-1: No Learning. In this scenario, the agents in the system have no learning capability or implemented learning authority. Thus, the agents rely on their given models, capabilities, and authorities for production. • CASE-2: Bounded Implemented Learning. The agents in this scenario have the capability to learn from the previous iterations and past process data. The learning is confined to the capability boundaries set by the fleet controller. Therefore an agent can only learn to improve within the predefined boundaries, satisfying the given constraints defined by the fleet controller. This type of learning is common in modern manufacturing systems with learning-based controllers. • CASE-3: Exploratory Implemented Learning. This corresponds to the proposed learning-based adaptation and dynamic reconfiguration scenario. By utilizing the proposed exploratory implemented learning methods from this work, the agents are able to learn beyond their capability boundary conditions. The learned actions by the agents are shared with the fleet controller for further analysis and verification. The fleet controller can choose to allow the agents to implement new control actions accordingly, or overrule the proposed actions of the agents and instead change their capability boundary conditions to ensure desired system performance and constraint satisfaction. We assume that the FDM agents have the capability to communicate with each other in this scenario.
In all cases, we consider an unexpected large order of P1, which is the product produced only by FDM1 in the nominal case. This represents an unexpected disturbance in the production system, and the responses of the AM fleet in the three cases are studied to compare their capability adapting to this unexpected disturbance. The decision making steps taken in each case are illustrated by the vertical flow charts in each corresponding case in Figure 5. In CASE-1, the agents in the system have no learning capability. Therefore, the FDM1 Agent commands the FDM1 machine to produce all the incoming orders. Since all orders are produced by a single machine, this leads to a suboptimal schedule where the order of the incoming products take a long time (compared to an optimized schedule using multiple machines), and FDM1 becomes a bottleneck.
CASE-2 represents resources with limited learning capability. Agents in the system are able to utilize data from the physical system to learn better control actions and policies within the capability boundaries defined by the fleet controller. In this case, the agent of FDM1 learns to improve the process performance by reducing the cycle time. The agent utilizes a data-driven optimization or online learning method to improve process performance while adhering to the constraints set by the fleet controller in terms of e.g., maximum process speed, allowable printing temperature, maximum layer height, and expected part performance as a function of the printing parameters. Under these constraints, the FDM1 Agent employs learning, which now results in another suboptimal schedule (compared to an optimized schedule using multiple machines), but one that is an improvement over the schedule of CASE-1. VOLUME X, 20XX xiii FIGURE 5: The three cases discussed in the case study.
CASE-3 describes the proposed learning-based control scenario decision flow chart. Given the same unexpectedly large order, the FDM agents now communicate with each other to commission the incoming production. This results in a balanced learned suboptimal schedule that is optimized at the level of the FDM machines. The schedule is suboptimal because it does not consider the implications of this new schedule on the downstream resources (transportation and post processing). Note that this learned schedule is outside of the initial capability bopundary of the FDM agents and it is a product of learning between the agents. The FDM agents send a request to the Fleet controller to implement the learned suboptimal schedule. Since the fleet controller has a centralized view of the fleet, it identifies that the learned suboptimal schedule leads to bottlenecks on the downstream since the product P1 can only be post-processed by the PP1 station. Therefore, the fleet controller does not allow the agents to implement the given schedule, but instead uses the learned information and an understanding of the entire system to determine how to best use the resources to improve production. In this case it modifies the capability authority of the FDM agents to give them the authority to communicate with the downstream transport and post process agents to learn a new schedule. After this authorization, all agents communicate to learn an optimal schedule that optimizes the system-wide cycle time and productivity, which is the best learned optimal schedule in this example.
Note that the criteria for whether an agent should or should not use learning, bounded implemented learning, or exploratory implemented learning will be system and situationally dependent. For example, a manufacturer might have to specify that if the agent is doing exploratory implemented learning, it has to be done offline or be verified by another agent or person. However, in general, if exploratory implemented learning is supported, then bounded implemented learning should be supported. Similarly, if bounded implemented learning is not supported, then no learning should be allowed. Future work will look at setting explicit criteria for choosing the different types of learning approaches and extending this approach to automated learning for other applications and case studies.

VII. CONCLUSIONS AND FUTURE WORK
A key to success in advanced manufacturing is the ability of accommodating new capabilities and pushing the boundaries of adaptability and flexibility. The explosion of AI capabilities over the past decade has created an environment where existing manufacturing infrastructure paradigms, often in place for decades, are not sufficient to the task of accommodating these new capabilities. Specifically, while these existing manufacturing paradigms can often be enhanced to support increased levels of adaptability and flexibility, they rarely can accommodate capabilities where the system actually learns and moves beyond established boundary conditions without direct human intervention.
In this paper a system-level architecture has been proposed from a requirements perspective that allows factories to extend adaptability into the domain of implemented learning xiv VOLUME X, 20XX and especially exploratory implemented learning. A taxonomy for implemented learning in CPMSs is presented that allows us to delineate and quantify the capabilities and limitations of agents in a coordinated system; these capabilities are defined with respect to pre-defined or initial capabilities, and capabilities that arise from the ability to adapt or learn. Using this taxonomy a vision for CPMSs is defined to support automated exploratory learning. An audit of existing CMPMs and AI systems reveals many technology gaps that must be addressed to realize this vision including procedures for implementing learning, lack of a quantification of learning and a formal process for transferring learnings between agents, and lack of sufficient guarantees that learning will provide a net benefit without negatively impacting system security and safety. A set of requirements are then developed to achieve the CPMS vision by addressing the defined technology gaps. Key requirements include the quantification of learning, the ability to communicate and make use of learning between agents, and the ability to support the transition along the continuum from pure human implemented learning to AI assisted and AI implemented learning. The vision is also associated with a number of operational objectives including leveraging existing processes, incentivizing and enabling learning through human-AI interaction and goals such as "no knowledge left behind," and maintaining a secure environment throughout the learning process.
Using this requirement set, a hybrid control architecture is proposed that enables dynamic decision making and implemented learning in a CPMS. The architecture addresses the derived requirements as it includes a mechanism for communicating quantified capabilities and capability authorities. An additive manufacturing fleet example illustrates the use of the architecture in scenarios where there is no learning, bounded implemented learning, and exploratory implemented learning. Moving forward, even if the vision of a framework for automated exploratory learning in CPMSs is fully achieved, the learnings will only be implemented if there is trust in these learnings. This includes convincing the human element invested in the process that the learning will provide a net improvement towards manufacturing objectives and will not negatively impact metrics such as safety and security.
Quantification in the learning process (benefit, confidence, et.) as outlined in this paper is an important initial step in providing this trust, however quantification of uncertainty in AI system learning remains as a challenge [88]. One way in which the quantification and trust can be improved is to strive to align the AI learning with the corresponding human learning process. As an example, AI learning is often broken down into identification/perception and reasoning tasks with different techniques (such as neural net and symbolic AI respectively) applied to these categories [89]. The quantification of learning in each of these domains, the interaction between the two domains, and the proactive interaction with human learning as part of the business process will require further exploration [32]. As another example the design and standardization of the human-AI interface to support an inter-active learning process will enhance the interoperability and interchangeability of human and AI elements in the learning process. This interface must be proactive in its engagement of these resources seeking intelligence input rather than just accommodating it. Trust and the human-AI connection represents just one of the key technical challenges in moving towards automated learning in CPMSs. The narrowness of AI and a lack of understanding of its boundaries of narrowness lead to additional issues of applicability and trust; understanding and better specifying AI solution boundaries, widening of AI, and moving towards general AI will all work towards addressing this challenge [34]. V&V of AI evolving systems and AI sourced knowledge updates (as discussed in Section III) will be particularly challenging as these V&V systems in many cases must also "learn." The maintenance of the growing and evolving knowledge base will present a number of challenges. Security and information partitioning in learning environments will present challenges that are already being observed today in supply chain connectivity in the manufacturing ecosystem and other information and knowledge sharing environments such as the cloud. Additionally, while some learnings are absolute, others may be temporal or contextually dependent. The dynamics and context richness typical in CPMSs thus represent additional challenges in knowledge management.
As a final point, a key challenge in moving towards automated learnings in CPMSs might be termed the human exploitation challenge. There must be constant attention to respect the human element in systems. This respect can take many forms including ensuring fairness in learning and learnings sharing, ensuring lack of bias to factors that do not relate to CPMS objectives, and avoiding exploitation of the human element in any way. Regardless of these human-AI interaction challenges, the importance of "no human left behind" can not be understated. Learning at the highest level will always be human guided, with delegation of learning capabilities at lower levels increasingly delegated to AI or AI guided systems. This process of moving through the human to AI continuum towards factories that learn automatically requires strong cooperation from both sides and a corresponding commitment to the value that each side will provide to the ultimate solution.
DAWN TILBURY (Fellow Member, IEEE) received the B.S. degree (summa cum laude) in electrical engineering from the University of Minnesota, in 1989, and the M.S. and Ph.D. degrees in electrical engineering and computer sciences from the University of California, Berkeley, in 1992 and 1994, respectively. In 1995, she joined the Mechanical Engineering Department, University of Michigan, Ann Arbor, where she is currently a Professor, with a joint appointment as a Professor of EECS. In June of 2017, she became the Assistant Director for Engineering at the National Science Foundation, while maintaining her position at the University of Michigan. She has published more than 150 articles in refereed journals and conference proceedings. Her research interests include control systems, including applications to robotics and manufacturing systems.