Task Selection by Autonomous Mobile Robots in A Warehouse Using Deep Reinforcement Learning

Maojia P. Li; Prashant Sankaran; Michael E. Kuhl; Raymond Ptucha; Amlan Ganguly; Andres Kwasinski

doi:10.1109/WSC40007.2019.9004792

Task Selection by Autonomous Mobile Robots in A Warehouse Using Deep Reinforcement Learning

Maojia P. Li
, Prashant Sankaran
, Michael E. Kuhl
, Raymond Ptucha
, Amlan Ganguly
, Andres Kwasinski

Department of Industrial and Systems Engineering

University at Buffalo

Rochester Institute of Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

22 Scopus citations

Abstract

We introduce a deep Q-network (DQN) based model that addresses the dispatching and routing problems for autonomous mobile robots. The DQN model is trained to dispatch a small fleet of robots to perform material handling tasks in a virtual, as well as, in an actual warehouse environment. Specifically, the DQN model is trained to dispatch an available robot to the closest task that will avoid or minimize encounters with other robots. Based on a discrete event simulation experiment, the DQN model outperforms the shortest travel distance rule in terms of avoiding traffic conflicts, improving the makespan for completing a set of tasks, and reducing the mean time in system for tasks.

Original language	English
Title of host publication	2019 Winter Simulation Conference, WSC 2019
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	680-689
Number of pages	10
ISBN (Electronic)	9781728132839
DOIs	https://doi.org/10.1109/WSC40007.2019.9004792
State	Published - Dec 2019
Event	2019 Winter Simulation Conference, WSC 2019 - National Harbor, United States Duration: Dec 8 2019 → Dec 11 2019

Publication series

Name	Proceedings - Winter Simulation Conference
Volume	2019-December

Conference

Conference	2019 Winter Simulation Conference, WSC 2019
Country/Territory	United States
City	National Harbor
Period	12/8/19 → 12/11/19

Access to Document

10.1109/WSC40007.2019.9004792

Cite this

Li, M. P., Sankaran, P., Kuhl, M. E., Ptucha, R., Ganguly, A., & Kwasinski, A. (2019). Task Selection by Autonomous Mobile Robots in A Warehouse Using Deep Reinforcement Learning. In 2019 Winter Simulation Conference, WSC 2019 (pp. 680-689). Article 9004792 (Proceedings - Winter Simulation Conference; Vol. 2019-December). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/WSC40007.2019.9004792

@inproceedings{5dd75fc624b84cd5a7ab250ac9902cd7,

title = "Task Selection by Autonomous Mobile Robots in A Warehouse Using Deep Reinforcement Learning",

abstract = "We introduce a deep Q-network (DQN) based model that addresses the dispatching and routing problems for autonomous mobile robots. The DQN model is trained to dispatch a small fleet of robots to perform material handling tasks in a virtual, as well as, in an actual warehouse environment. Specifically, the DQN model is trained to dispatch an available robot to the closest task that will avoid or minimize encounters with other robots. Based on a discrete event simulation experiment, the DQN model outperforms the shortest travel distance rule in terms of avoiding traffic conflicts, improving the makespan for completing a set of tasks, and reducing the mean time in system for tasks.",

author = "Li, \{Maojia P.\} and Prashant Sankaran and Kuhl, \{Michael E.\} and Raymond Ptucha and Amlan Ganguly and Andres Kwasinski",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 2019 Winter Simulation Conference, WSC 2019 ; Conference date: 08-12-2019 Through 11-12-2019",

year = "2019",

month = dec,

doi = "10.1109/WSC40007.2019.9004792",

language = "English",

series = "Proceedings - Winter Simulation Conference",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "680--689",

booktitle = "2019 Winter Simulation Conference, WSC 2019",

address = "United States",

}

Li, MP, Sankaran, P, Kuhl, ME, Ptucha, R, Ganguly, A & Kwasinski, A 2019, Task Selection by Autonomous Mobile Robots in A Warehouse Using Deep Reinforcement Learning. in 2019 Winter Simulation Conference, WSC 2019., 9004792, Proceedings - Winter Simulation Conference, vol. 2019-December, Institute of Electrical and Electronics Engineers Inc., pp. 680-689, 2019 Winter Simulation Conference, WSC 2019, National Harbor, United States, 12/8/19. https://doi.org/10.1109/WSC40007.2019.9004792

Task Selection by Autonomous Mobile Robots in A Warehouse Using Deep Reinforcement Learning. / Li, Maojia P.; Sankaran, Prashant; Kuhl, Michael E. et al.
2019 Winter Simulation Conference, WSC 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 680-689 9004792 (Proceedings - Winter Simulation Conference; Vol. 2019-December).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Task Selection by Autonomous Mobile Robots in A Warehouse Using Deep Reinforcement Learning

AU - Li, Maojia P.

AU - Sankaran, Prashant

AU - Kuhl, Michael E.

AU - Ptucha, Raymond

AU - Ganguly, Amlan

AU - Kwasinski, Andres

PY - 2019/12

Y1 - 2019/12

N2 - We introduce a deep Q-network (DQN) based model that addresses the dispatching and routing problems for autonomous mobile robots. The DQN model is trained to dispatch a small fleet of robots to perform material handling tasks in a virtual, as well as, in an actual warehouse environment. Specifically, the DQN model is trained to dispatch an available robot to the closest task that will avoid or minimize encounters with other robots. Based on a discrete event simulation experiment, the DQN model outperforms the shortest travel distance rule in terms of avoiding traffic conflicts, improving the makespan for completing a set of tasks, and reducing the mean time in system for tasks.

AB - We introduce a deep Q-network (DQN) based model that addresses the dispatching and routing problems for autonomous mobile robots. The DQN model is trained to dispatch a small fleet of robots to perform material handling tasks in a virtual, as well as, in an actual warehouse environment. Specifically, the DQN model is trained to dispatch an available robot to the closest task that will avoid or minimize encounters with other robots. Based on a discrete event simulation experiment, the DQN model outperforms the shortest travel distance rule in terms of avoiding traffic conflicts, improving the makespan for completing a set of tasks, and reducing the mean time in system for tasks.

UR - https://www.scopus.com/pages/publications/85081137505

U2 - 10.1109/WSC40007.2019.9004792

DO - 10.1109/WSC40007.2019.9004792

M3 - Conference contribution

T3 - Proceedings - Winter Simulation Conference

SP - 680

EP - 689

BT - 2019 Winter Simulation Conference, WSC 2019

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2019 Winter Simulation Conference, WSC 2019

Y2 - 8 December 2019 through 11 December 2019

ER -

Task Selection by Autonomous Mobile Robots in A Warehouse Using Deep Reinforcement Learning

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this