TY - JOUR
T1 - AGV dispatching algorithm based on deep Q-network in CNC machines environment
AU - Chang, Kyuchang
AU - Park, Seung Hwan
AU - Baek, Jun Geol
N1 - Funding Information:
This work was supported by the Korea Institute for Advancement of Technology [P0008691]; National Research Foundation of Korea [NRF-2019R1A2C2005949]. This research was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (NRF-2019R1A2C2005949). This research was also supported by Korea Institute for Advancement of Technology (KIAT) grant funded by the Korea Government (MOTIE) (P0008691, The Competency Development Program for Industry Specialist).
Funding Information:
This research was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (NRF-2019R1A2C2005949). This research was also supported by Korea Institute for Advancement of Technology (KIAT) grant funded by the Korea Government (MOTIE) (P0008691, The Competency Development Program for Industry Specialist).
Publisher Copyright:
© 2021 Informa UK Limited, trading as Taylor & Francis Group.
PY - 2022
Y1 - 2022
N2 - This research focuses on providing an optimal dispatching algorithm for an automatic guided vehicle (AGV) in a mobile metal board manufacturing facility. The target process comprises multiple computerized numerical control (CNC) machines and an AGV. An AGV feeds materials between two rows of CNC machines for processing metal boards, or conveys a work in process. As it is difficult to derive a mathematically optimal working order owing to the high computational cost, simple dispatching rules have typically been applied in such environments. However, these rules are generally not optimal, and expert knowledge is required to determine which rule to choose. To overcome certain of these disadvantages and increase productivity, a deep reinforcement learning (RL) algorithm is used to learn an AGV’s dispatching algorithm. The target production line as a virtual simulated grid-shaped workspace is modeled to develop a deep Q-network (DQN)-based dispatching algorithm. A convolutional neural network (CNN) is used to input raw pixels and output a value function for estimating future rewards, and an agent is trained to successfully learn the control policies. To create an elaborate dispatching strategy, different hyper-parameters of the DQN are tuned and a reasonable modeling method is experimentally determined. The proposed method automatically develops an optimal dispatching policy without requiring human control or prior expert knowledge. Compared with general heuristic dispatching rules, the results illustrate the improved performance of the proposed methodology.
AB - This research focuses on providing an optimal dispatching algorithm for an automatic guided vehicle (AGV) in a mobile metal board manufacturing facility. The target process comprises multiple computerized numerical control (CNC) machines and an AGV. An AGV feeds materials between two rows of CNC machines for processing metal boards, or conveys a work in process. As it is difficult to derive a mathematically optimal working order owing to the high computational cost, simple dispatching rules have typically been applied in such environments. However, these rules are generally not optimal, and expert knowledge is required to determine which rule to choose. To overcome certain of these disadvantages and increase productivity, a deep reinforcement learning (RL) algorithm is used to learn an AGV’s dispatching algorithm. The target production line as a virtual simulated grid-shaped workspace is modeled to develop a deep Q-network (DQN)-based dispatching algorithm. A convolutional neural network (CNN) is used to input raw pixels and output a value function for estimating future rewards, and an agent is trained to successfully learn the control policies. To create an elaborate dispatching strategy, different hyper-parameters of the DQN are tuned and a reasonable modeling method is experimentally determined. The proposed method automatically develops an optimal dispatching policy without requiring human control or prior expert knowledge. Compared with general heuristic dispatching rules, the results illustrate the improved performance of the proposed methodology.
KW - Automated guided vehicle (AGV)
KW - computerized numerical control (CNC) machines
KW - deep Q-network (DQN)
KW - dispatching algorithm
KW - reinforcement learning (RL)
UR - http://www.scopus.com/inward/record.url?scp=85118282628&partnerID=8YFLogxK
U2 - 10.1080/0951192X.2021.1992669
DO - 10.1080/0951192X.2021.1992669
M3 - Article
AN - SCOPUS:85118282628
SN - 0951-192X
VL - 35
SP - 662
EP - 677
JO - International Journal of Computer Integrated Manufacturing
JF - International Journal of Computer Integrated Manufacturing
IS - 6
ER -