Hierarchical Reinforcement Learning using Gaussian Random Trajectory Generation in Autonomous Furniture Assembly

Won Joon Yun; David Mohaisen; Soyi Jung; Jong Kook Kim; Joongheon Kim

doi:10.1145/3511808.3557078

Hierarchical Reinforcement Learning using Gaussian Random Trajectory Generation in Autonomous Furniture Assembly

Won Joon Yun, David Mohaisen, Soyi Jung, Jong Kook Kim, Joongheon Kim

School of Electrical Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

3 Citations (Scopus)

Abstract

In this paper, we propose a Gaussian Random Trajectory guided Hierarchical Reinforcement Learning (GRT-HL) method for autonomous furniture assembly. The furniture assembly problem is formulated as a comprehensive human-like long-horizon manipulation task that requires a long-term planning and a sophisticated control. Our proposed model, GRT-HL, draws inspirations from the semi-supervised adversarial autoencoders, and learns latent representations of the position trajectories of the end-effector. The high-level policy generates an optimal trajectory for furniture assembly, considering the structural limitations of the robotic agents. Given the trajectory drawn from the high-level policy, the low-level policy makes a plan and controls the end-effector. We first evaluate the performance of GRT-HL compared to the state-of-the-art reinforcement learning methods in furniture assembly tasks. We demonstrate that GRT-HL successfully solves the long-horizon problem with extremely sparse rewards by generating the trajectory for planning.

Original language	English
Title of host publication	CIKM 2022 - Proceedings of the 31st ACM International Conference on Information and Knowledge Management
Publisher	Association for Computing Machinery
Pages	3624-3633
Number of pages	10
ISBN (Electronic)	9781450392365
DOIs	https://doi.org/10.1145/3511808.3557078
Publication status	Published - 2022 Oct 17
Event	31st ACM International Conference on Information and Knowledge Management, CIKM 2022 - Atlanta, United States Duration: 2022 Oct 17 → 2022 Oct 21

Publication series

Name	International Conference on Information and Knowledge Management, Proceedings

Conference

Conference	31st ACM International Conference on Information and Knowledge Management, CIKM 2022
Country/Territory	United States
City	Atlanta
Period	22/10/17 → 22/10/21

Bibliographical note

Funding Information:
This work was supported by Samsung Electronics (IO201208-07855-01) and also by MSIT, Korea, under ITRC (IITP-2022-2017-0-01637) supervised by IITP. The authors thank to Mr. MyungJae Shin for his contribution on research initiation, during his master study under the guidance of Prof. Joongheon Kim. Soyi Jung, Jong-Kook Kim, and Joongheon Kim are the corresponding authors.

Publisher Copyright:
© 2022 ACM.

Keywords

assembly control
hierarchical reinforcement learning
reinforcement learning
robotics

ASJC Scopus subject areas

General Business,Management and Accounting
General Decision Sciences

Access to Document

10.1145/3511808.3557078

Cite this

Yun, W. J., Mohaisen, D., Jung, S., Kim, J. K., & Kim, J. (2022). Hierarchical Reinforcement Learning using Gaussian Random Trajectory Generation in Autonomous Furniture Assembly. In CIKM 2022 - Proceedings of the 31st ACM International Conference on Information and Knowledge Management (pp. 3624-3633). (International Conference on Information and Knowledge Management, Proceedings). Association for Computing Machinery. https://doi.org/10.1145/3511808.3557078

Hierarchical Reinforcement Learning using Gaussian Random Trajectory Generation in Autonomous Furniture Assembly. / Yun, Won Joon; Mohaisen, David; Jung, Soyi et al.
CIKM 2022 - Proceedings of the 31st ACM International Conference on Information and Knowledge Management. Association for Computing Machinery, 2022. p. 3624-3633 (International Conference on Information and Knowledge Management, Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Yun, WJ, Mohaisen, D, Jung, S, Kim, JK & Kim, J 2022, Hierarchical Reinforcement Learning using Gaussian Random Trajectory Generation in Autonomous Furniture Assembly. in CIKM 2022 - Proceedings of the 31st ACM International Conference on Information and Knowledge Management. International Conference on Information and Knowledge Management, Proceedings, Association for Computing Machinery, pp. 3624-3633, 31st ACM International Conference on Information and Knowledge Management, CIKM 2022, Atlanta, United States, 22/10/17. https://doi.org/10.1145/3511808.3557078

Yun WJ, Mohaisen D, Jung S, Kim JK, Kim J. Hierarchical Reinforcement Learning using Gaussian Random Trajectory Generation in Autonomous Furniture Assembly. In CIKM 2022 - Proceedings of the 31st ACM International Conference on Information and Knowledge Management. Association for Computing Machinery. 2022. p. 3624-3633. (International Conference on Information and Knowledge Management, Proceedings). doi: 10.1145/3511808.3557078

Yun, Won Joon ; Mohaisen, David ; Jung, Soyi et al. / Hierarchical Reinforcement Learning using Gaussian Random Trajectory Generation in Autonomous Furniture Assembly. CIKM 2022 - Proceedings of the 31st ACM International Conference on Information and Knowledge Management. Association for Computing Machinery, 2022. pp. 3624-3633 (International Conference on Information and Knowledge Management, Proceedings).

@inproceedings{cd97a9bdb81b4013bd5460d0594eb1cf,

title = "Hierarchical Reinforcement Learning using Gaussian Random Trajectory Generation in Autonomous Furniture Assembly",

abstract = "In this paper, we propose a Gaussian Random Trajectory guided Hierarchical Reinforcement Learning (GRT-HL) method for autonomous furniture assembly. The furniture assembly problem is formulated as a comprehensive human-like long-horizon manipulation task that requires a long-term planning and a sophisticated control. Our proposed model, GRT-HL, draws inspirations from the semi-supervised adversarial autoencoders, and learns latent representations of the position trajectories of the end-effector. The high-level policy generates an optimal trajectory for furniture assembly, considering the structural limitations of the robotic agents. Given the trajectory drawn from the high-level policy, the low-level policy makes a plan and controls the end-effector. We first evaluate the performance of GRT-HL compared to the state-of-the-art reinforcement learning methods in furniture assembly tasks. We demonstrate that GRT-HL successfully solves the long-horizon problem with extremely sparse rewards by generating the trajectory for planning.",

keywords = "assembly control, hierarchical reinforcement learning, reinforcement learning, robotics",

author = "Yun, {Won Joon} and David Mohaisen and Soyi Jung and Kim, {Jong Kook} and Joongheon Kim",

note = "Funding Information: This work was supported by Samsung Electronics (IO201208-07855-01) and also by MSIT, Korea, under ITRC (IITP-2022-2017-0-01637) supervised by IITP. The authors thank to Mr. MyungJae Shin for his contribution on research initiation, during his master study under the guidance of Prof. Joongheon Kim. Soyi Jung, Jong-Kook Kim, and Joongheon Kim are the corresponding authors. Publisher Copyright: {\textcopyright} 2022 ACM.; 31st ACM International Conference on Information and Knowledge Management, CIKM 2022 ; Conference date: 17-10-2022 Through 21-10-2022",

year = "2022",

month = oct,

day = "17",

doi = "10.1145/3511808.3557078",

language = "English",

series = "International Conference on Information and Knowledge Management, Proceedings",

publisher = "Association for Computing Machinery",

pages = "3624--3633",

booktitle = "CIKM 2022 - Proceedings of the 31st ACM International Conference on Information and Knowledge Management",

}

TY - GEN

T1 - Hierarchical Reinforcement Learning using Gaussian Random Trajectory Generation in Autonomous Furniture Assembly

AU - Yun, Won Joon

AU - Mohaisen, David

AU - Jung, Soyi

AU - Kim, Jong Kook

AU - Kim, Joongheon

N1 - Funding Information: This work was supported by Samsung Electronics (IO201208-07855-01) and also by MSIT, Korea, under ITRC (IITP-2022-2017-0-01637) supervised by IITP. The authors thank to Mr. MyungJae Shin for his contribution on research initiation, during his master study under the guidance of Prof. Joongheon Kim. Soyi Jung, Jong-Kook Kim, and Joongheon Kim are the corresponding authors. Publisher Copyright: © 2022 ACM.

PY - 2022/10/17

Y1 - 2022/10/17

N2 - In this paper, we propose a Gaussian Random Trajectory guided Hierarchical Reinforcement Learning (GRT-HL) method for autonomous furniture assembly. The furniture assembly problem is formulated as a comprehensive human-like long-horizon manipulation task that requires a long-term planning and a sophisticated control. Our proposed model, GRT-HL, draws inspirations from the semi-supervised adversarial autoencoders, and learns latent representations of the position trajectories of the end-effector. The high-level policy generates an optimal trajectory for furniture assembly, considering the structural limitations of the robotic agents. Given the trajectory drawn from the high-level policy, the low-level policy makes a plan and controls the end-effector. We first evaluate the performance of GRT-HL compared to the state-of-the-art reinforcement learning methods in furniture assembly tasks. We demonstrate that GRT-HL successfully solves the long-horizon problem with extremely sparse rewards by generating the trajectory for planning.

AB - In this paper, we propose a Gaussian Random Trajectory guided Hierarchical Reinforcement Learning (GRT-HL) method for autonomous furniture assembly. The furniture assembly problem is formulated as a comprehensive human-like long-horizon manipulation task that requires a long-term planning and a sophisticated control. Our proposed model, GRT-HL, draws inspirations from the semi-supervised adversarial autoencoders, and learns latent representations of the position trajectories of the end-effector. The high-level policy generates an optimal trajectory for furniture assembly, considering the structural limitations of the robotic agents. Given the trajectory drawn from the high-level policy, the low-level policy makes a plan and controls the end-effector. We first evaluate the performance of GRT-HL compared to the state-of-the-art reinforcement learning methods in furniture assembly tasks. We demonstrate that GRT-HL successfully solves the long-horizon problem with extremely sparse rewards by generating the trajectory for planning.

KW - assembly control

KW - hierarchical reinforcement learning

KW - reinforcement learning

KW - robotics

UR - http://www.scopus.com/inward/record.url?scp=85140836688&partnerID=8YFLogxK

U2 - 10.1145/3511808.3557078

DO - 10.1145/3511808.3557078

M3 - Conference contribution

AN - SCOPUS:85140836688

T3 - International Conference on Information and Knowledge Management, Proceedings

SP - 3624

EP - 3633

BT - CIKM 2022 - Proceedings of the 31st ACM International Conference on Information and Knowledge Management

PB - Association for Computing Machinery

T2 - 31st ACM International Conference on Information and Knowledge Management, CIKM 2022

Y2 - 17 October 2022 through 21 October 2022

ER -

Hierarchical Reinforcement Learning using Gaussian Random Trajectory Generation in Autonomous Furniture Assembly

Abstract

Publication series

Conference

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this