Process Control
Assembly Line
Multi-agent reinforcement learning for integrated manufacturing system-process control
The increasing complexity, adaptability, and interconnections inherent in modern manufacturing systems have spurred a demand for integrated methodologies to boost productivity, improve quality, and streamline operations across the entire system. This paper introduces a holistic system-process modeling and control approach, utilizing a Multi-Agent Reinforcement Learning (MARL) based integrated control scheme to optimize system yields. The key innovation of this work lies in integrating the theoretical development of manufacturing system-process property understanding with enhanced MARL-based control strategies, thereby improving system dynamics comprehension. This, in turn, enhances informed decision-making and contributes to overall efficiency improvements. In addition, we present two innovative MARL algorithms: the credit-assigned multi-agent actor-attention-critic (C-MAAC) and the physics-guided multi-agent actor-attention-critic (P-MAAC), each designed to capture the individual contributions of agents within the system. C-MAAC extracts global information via parallel-trained attention blocks, whereas P-MAAC embeds system dynamics through permanent production loss (PPL) attribution. Numerical experiments underscore the efficacy of our MARL-based control scheme, particularly highlighting the superior training and execution performance of C-MAAC and P-MAAC. Notably, P-MAAC achieves rapid convergence and exhibits remarkable robustness against environmental variations, validating the proposed approach’s practical relevance and effectiveness.
Manufacturing process optimization for real-time quality control in multi-regime conditions: Tire tread production use case
The high-stake nature of most manufacturing processes empowers the importance of real-time quality control and assurance. In the event of a failure in production, a decision-making process can be time-consuming for the human and prevent timely actions. The agility can be boosted with a decision-support system based on artificial intelligence. Particularly, multi-objective process optimization can be employed to select the optimal control settings in real-time, and thus enhance relevant key performance indicators, concurrently.
Based on process and quality parameters being streamed from the production plant in real-time, the optimizer can act in timely critical and quality-threatening situations and generate immediate corrective actions. The multi-regime operation of the plant and design space dimensionality can impact the convergence rate and add to execution time. Therefore, production regimes recognition and greedy search of suffix tree-based models of the process have been engaged, aiding in a better-focused and faster space search at an early phase of the algorithm run.
Beyond simply reviewing the outputs, the user can leave feedback, which is utilized by the optimizer’s reinforcement learning mechanisms. Validated in this real-world scenario, the solution produced a rise from 81.83% to 90.91% in the tread quality.