With the development and changes of industry and market demand, the personalized customization production mode with small batch and multiple batches has gradually become a new production mode. This makes production environment become more complex and dynamic. However, traditional production workshops cannot effectively adapt to this environment. Combining with new technologies, transforming traditional workshops into intelligent workshop to cope with new production mode become an urgent problem. Therefore, this paper proposes a multi-agent manufacturing system based on IoT for intelligent workshop. Meanwhile, this paper takes flexible job shop scheduling problem (FJSP) as a specific production scenario and establishes relevant mathematics model. To build the agent in intelligent workshop, this paper proposes a data-based with combination of virtual and physical agent (DB-VPA) which has information layer, software layer and physical layer. Then, based on the manufacturing system, this paper designs a dynamic scheduling mechanism with deep reinforcement learning (DRL) for intelligent workshop. This method contains three aspects: (1) Modeling production process based on Markov decision process (MDP). (2) Designing communication mechanism for DB-VPAs. (3) Designing scheduling model combining with improved genetic programming and proximal policy optimization (IGP-PPO) which is a DRL method. Finally, relevant experiments are executed in a prototype experiment platform. The experiments indicate that the proposed method has superiority and generality in solving scheduling problem with dynamic events.

中文翻译：

---

基于多Agent系统架构的深度强化学习方法智能车间动态调度机制

随着行业和市场需求的发展变化，小批量、多批次的个性化定制生产模式逐渐成为一种新的生产模式。这使得生产环境变得更加复杂和动态。然而，传统的生产车间无法有效适应这种环境。结合新技术，将传统车间改造为智能车间以应对新的生产模式成为刻不容缓的问题。因此，本文提出一种基于物联网的智能车间多Agent制造系统。同时，本文以柔性作业车间调度问题（FJSP）为具体的生产场景，建立了相关的数学模型。针对智能车间代理的构建，提出了一种基于数据的虚拟与物理相结合的代理(DB-VPA)，它具有信息层、软件层和物理层。然后，基于制造系统，设计了一种基于深度强化学习（DRL）的智能车间动态调度机制。该方法包含三个方面的内容：(1)基于马尔可夫决策过程(MDP)的生产过程建模。 (2)设计DB-VPA的通信机制。 (3)设计了结合改进遗传规划和近端策略优化的调度模型(IGP-PPO)，这是一种DRL方法。最后，在原型实验平台上进行了相关实验。实验表明，该方法在解决动态事件调度问题上具有优越性和通用性。