Leveraging Task Modularity in Reinforcement Learning for Adaptable Industry 4.0 Automation

The vision of Industry 4.0 is to materialize the notion of a lot-size of one through enhanced adaptability and resilience of manufacturing and logistics operations to dynamic changes or deviations on the shop floor. This article is motivated by the lack of formal methods for efficient transfer of knowledge across different yet interrelated tasks, with special reference to collaborative robotic operations such as material handling, machine tending, assembly, and inspection. We propose a meta reinforcement learning framework to enhance the adaptability of collaborative robots to new tasks through task modularization and efficient transfer of policies from previously learned task modules. Our experiments on the OpenAI Gym Robotics environments Reach, Push, and Pick-and-Place indicate an average 75% reduction in the number of iterations to achieve a 60% success rate as well as a 50%-80% improvement in task completion efficiency, compared to the deep deterministic policy gradient (DDPG) algorithm as a baseline. The significant improvements achieved in the jumpstart and asymptotic performance of the robot create new opportunities for investigating the current limitations of learning robots in industrial settings, associated with sample inefficiency and specialization on one task through modularization and transfer learning.