Deep reinforcement learning with a critic-value-based branch tree for the inverse design of two-dimensional optical devices

Hyo Seok Hwang, Minhyeok Lee, Junhee Seok

Research output: Contribution to journalArticlepeer-review

7 Citations (Scopus)


In optical engineering, designing a device or a system with the desired property is an important but challenging task. It is relatively straightforward to compute the physical properties of a given design; however, there is no general method for the reverse, i.e., designing with desired properties. To address this problem with a computational method, this paper proposes a deep reinforcement learning-based inverse design framework consisting of two methods: Inverse DEsign Agent (IDEA) and Critic-Value-based Branch Tree (CVBT) algorithm. IDEA is a deep reinforcement learning model based on the Advantage Actor–Critic (A2C) method using a deep learning simulator as a replacement for a real environment, significantly reducing training time compared to conventional methods. The CVBT algorithm suggests several design candidates using the critic values of IDEA, while conventional methods propose only one candidate. In this study, the proposed framework was applied to a two-dimensional optical device design problem. Experimental results with untrained target properties demonstrated that the proposed model, IDEA-CVBT, achieved state-of-the-art performance in terms of accuracy and stability. For instance, in a scenario with a binary type design, IDEA-CVBT exhibited an accuracy of 91.5%, while conventional deep reinforcement models showed 36.1% and 7.4% accuracies. Extensive analyses verified that IDEA-CVBT can be employed as an assistance system for engineers since IDEA-CVBT suggests several appropriate candidates that satisfy target properties.

Original languageEnglish
Article number109386
JournalApplied Soft Computing
Publication statusPublished - 2022 Sept

Bibliographical note

Funding Information:
This work was supported by Samsung Electronics Co., Ltd ( IO201214-08149-01 ) as well as a grant from the National Research Foundation of Korea ( NRF-2022R1A2C2004003 ).

Publisher Copyright:
© 2022 Elsevier B.V.


  • Deep learning
  • Deep reinforcement learning
  • Device design automation
  • Multitask reinforcement learning

ASJC Scopus subject areas

  • Software


Dive into the research topics of 'Deep reinforcement learning with a critic-value-based branch tree for the inverse design of two-dimensional optical devices'. Together they form a unique fingerprint.

Cite this