Impedance learning for robotic contact tasks using natural actor-critic algorithm

Byungchan Kim; Jooyoung Park; Shinsuk Park; Sungchul Kang

doi:10.1109/TSMCB.2009.2026289

Impedance learning for robotic contact tasks using natural actor-critic algorithm

Byungchan Kim, Jooyoung Park, Shinsuk Park, Sungchul Kang

Research output: Contribution to journal › Article › peer-review

96 Citations (Scopus)

Abstract

Compared with their robotic counterparts, humans excel at various tasks by using their ability to adaptively modulate arm impedance parameters. This ability allows us to successfully perform contact tasks even in uncertain environments. This paper considers a learning strategy of motor skill for robotic contact tasks based on a human motor control theory and machine learning schemes. Our robot learning method employs impedance control based on the equilibrium point control theory and reinforcement learning to determine the impedance parameters for contact tasks. A recursive least-square filter-based episodic natural actor-critic algorithm is used to find the optimal impedance parameters. The effectiveness of the proposed method was tested through dynamic simulations of various contact tasks. The simulation results demonstrated that the proposed method optimizes the performance of the contact tasks in uncertain conditions of the environment.

Original language	English
Article number	5204203
Pages (from-to)	433-443
Number of pages	11
Journal	IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Volume	40
Issue number	2
DOIs	https://doi.org/10.1109/TSMCB.2009.2026289
Publication status	Published - 2010 Apr

Bibliographical note

Funding Information:
Manuscript received February 13, 2009; revised May 22, 2009. First published August 18, 2009; current version published March 17, 2010. This work was supported by the Korean Institute of Construction and Transportation Technology Evaluation and Planning under Program 06-Unified and Advanced Construction Technology Program-D01. The work of J. Park was supported by the Ministry of Knowledge Economy under the Human Resources Development Program for Convergence Robot Specialists. This paper was recommended by Associate Editor A. Tayebi.

Keywords

Contact task
Equilibrium point control
Reinforcement learning
Robot manipulation

ASJC Scopus subject areas

Control and Systems Engineering
Software
Information Systems
Human-Computer Interaction
Computer Science Applications
Electrical and Electronic Engineering

Access to Document

10.1109/TSMCB.2009.2026289

Cite this

@article{3fe353bde9684d32984cf99403e72033,

title = "Impedance learning for robotic contact tasks using natural actor-critic algorithm",

abstract = "Compared with their robotic counterparts, humans excel at various tasks by using their ability to adaptively modulate arm impedance parameters. This ability allows us to successfully perform contact tasks even in uncertain environments. This paper considers a learning strategy of motor skill for robotic contact tasks based on a human motor control theory and machine learning schemes. Our robot learning method employs impedance control based on the equilibrium point control theory and reinforcement learning to determine the impedance parameters for contact tasks. A recursive least-square filter-based episodic natural actor-critic algorithm is used to find the optimal impedance parameters. The effectiveness of the proposed method was tested through dynamic simulations of various contact tasks. The simulation results demonstrated that the proposed method optimizes the performance of the contact tasks in uncertain conditions of the environment.",

keywords = "Contact task, Equilibrium point control, Reinforcement learning, Robot manipulation",

author = "Byungchan Kim and Jooyoung Park and Shinsuk Park and Sungchul Kang",

note = "Funding Information: Manuscript received February 13, 2009; revised May 22, 2009. First published August 18, 2009; current version published March 17, 2010. This work was supported by the Korean Institute of Construction and Transportation Technology Evaluation and Planning under Program 06-Unified and Advanced Construction Technology Program-D01. The work of J. Park was supported by the Ministry of Knowledge Economy under the Human Resources Development Program for Convergence Robot Specialists. This paper was recommended by Associate Editor A. Tayebi.",

year = "2010",

month = apr,

doi = "10.1109/TSMCB.2009.2026289",

language = "English",

volume = "40",

pages = "433--443",

journal = "IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics",

issn = "1083-4419",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "2",

}

TY - JOUR

T1 - Impedance learning for robotic contact tasks using natural actor-critic algorithm

AU - Kim, Byungchan

AU - Park, Jooyoung

AU - Park, Shinsuk

AU - Kang, Sungchul

N1 - Funding Information: Manuscript received February 13, 2009; revised May 22, 2009. First published August 18, 2009; current version published March 17, 2010. This work was supported by the Korean Institute of Construction and Transportation Technology Evaluation and Planning under Program 06-Unified and Advanced Construction Technology Program-D01. The work of J. Park was supported by the Ministry of Knowledge Economy under the Human Resources Development Program for Convergence Robot Specialists. This paper was recommended by Associate Editor A. Tayebi.

PY - 2010/4

Y1 - 2010/4

N2 - Compared with their robotic counterparts, humans excel at various tasks by using their ability to adaptively modulate arm impedance parameters. This ability allows us to successfully perform contact tasks even in uncertain environments. This paper considers a learning strategy of motor skill for robotic contact tasks based on a human motor control theory and machine learning schemes. Our robot learning method employs impedance control based on the equilibrium point control theory and reinforcement learning to determine the impedance parameters for contact tasks. A recursive least-square filter-based episodic natural actor-critic algorithm is used to find the optimal impedance parameters. The effectiveness of the proposed method was tested through dynamic simulations of various contact tasks. The simulation results demonstrated that the proposed method optimizes the performance of the contact tasks in uncertain conditions of the environment.

AB - Compared with their robotic counterparts, humans excel at various tasks by using their ability to adaptively modulate arm impedance parameters. This ability allows us to successfully perform contact tasks even in uncertain environments. This paper considers a learning strategy of motor skill for robotic contact tasks based on a human motor control theory and machine learning schemes. Our robot learning method employs impedance control based on the equilibrium point control theory and reinforcement learning to determine the impedance parameters for contact tasks. A recursive least-square filter-based episodic natural actor-critic algorithm is used to find the optimal impedance parameters. The effectiveness of the proposed method was tested through dynamic simulations of various contact tasks. The simulation results demonstrated that the proposed method optimizes the performance of the contact tasks in uncertain conditions of the environment.

KW - Contact task

KW - Equilibrium point control

KW - Reinforcement learning

KW - Robot manipulation

UR - http://www.scopus.com/inward/record.url?scp=77949776001&partnerID=8YFLogxK

U2 - 10.1109/TSMCB.2009.2026289

DO - 10.1109/TSMCB.2009.2026289

M3 - Article

C2 - 19696001

AN - SCOPUS:77949776001

SN - 1083-4419

VL - 40

SP - 433

EP - 443

JO - IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

JF - IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

IS - 2

M1 - 5204203

ER -

Impedance learning for robotic contact tasks using natural actor-critic algorithm

Abstract

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this