Deep learning-based object understanding for robotic manipulation

Jong Sul Moon, Hyunjun Jo, Jae Bok Song

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Manipulation of objects by a robot arm requires an understanding of the various properties of the object. The robot needs a lot of information for object manipulation, there are few algorithms to estimate such information simultaneously. In this study, we propose an object understanding network (OUNet) based on deep learning that simultaneously estimates three key properties for robot object manipulation: object state, contact position for object manipulation, and manipulation type. The object state means whether an openable object is open or closed. The contact position and manipulation type for manipulating objects means where and what the robot should do to change the object state. Usingthis information, it is expected that the robot will be able to select the appropriate manipulation for the current situation of the given object. Experiments were conducted to verify the performance of the OUNet, and it was shown that three key properties can be successfully detected.

Original languageEnglish
Title of host publication2020 20th International Conference on Control, Automation and Systems, ICCAS 2020
PublisherIEEE Computer Society
Number of pages5
ISBN (Electronic)9788993215205
Publication statusPublished - 2020 Oct 13
Event20th International Conference on Control, Automation and Systems, ICCAS 2020 - Busan, Korea, Republic of
Duration: 2020 Oct 132020 Oct 16

Publication series

NameInternational Conference on Control, Automation and Systems
ISSN (Print)1598-7833


Conference20th International Conference on Control, Automation and Systems, ICCAS 2020
Country/TerritoryKorea, Republic of

Bibliographical note

Funding Information:
This research was supported by the MOTIE under the Industrial Foundation Technology Development Program supervised by the KEIT (No. 20008613)

Publisher Copyright:
© 2020 Institute of Control, Robotics, and Systems - ICROS.


  • Clustering
  • Deep learning
  • Manipulation
  • Object understanding

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Science Applications
  • Control and Systems Engineering
  • Electrical and Electronic Engineering


Dive into the research topics of 'Deep learning-based object understanding for robotic manipulation'. Together they form a unique fingerprint.

Cite this