Tunnel ventilation controller design using an RLS-based natural actor-critic algorithm

Baeksuk Chu, Jooyoung Park, Daehie Hong

Research output: Contribution to journalArticlepeer-review

7 Citations (Scopus)


Tunnel ventilation systems provide drivers with a comfortable and safe driving environment by generating sufficient airflow and by diluting the concentration of noxious contaminants below an acceptable level. For that purpose, tunnel ventilation systems contain mechanical equipment such as jet-fans, blowers and dust collectors. These machines consume large amount of energy, therefore, it is necessary to have an efficient operating algorithm for tunnel ventilation in terms of energy savings and safe driving. In this paper, a new reinforcement learning (RL) method is applied as the control algorithm. In the process of formulating the reward of the tunnel ventilation system, which is a performance index to be maximized in the RL methodology, the following two objectives are of great interest: maintaining an adequate level of pollutants and minimizing power consumption. The RL control algorithm adopted in this research is based on an actor-critic architecture and natural gradient method. Due to its ability to achieve the truly steepest direction of gradients, the natural gradient method can be a promising route to improving the efficacy of the actor module. Also, the recursive least-squares (RLS) method is employed to the critic module in order to improve the efficiency by which data is used. Using actual data collected from an existing tunnel ventilation system, extensive simulation studies were performed. It was confirmed that the suggested algorithm achieved the desired control goals and, when compared to previously developed RL-based control algorithms, improved the performance considerably.

Original languageEnglish
Pages (from-to)829-838
Number of pages10
JournalInternational Journal of Precision Engineering and Manufacturing
Issue number6
Publication statusPublished - 2010 Dec

Bibliographical note

Funding Information:
This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science and Technology (2009-0063175).


  • Actor-critic architecture
  • Natural gradient
  • Recursive least-squares (RLS)
  • Reinforcement learning (RL)
  • Tunnel ventilation control

ASJC Scopus subject areas

  • Mechanical Engineering
  • Industrial and Manufacturing Engineering
  • Electrical and Electronic Engineering


Dive into the research topics of 'Tunnel ventilation controller design using an RLS-based natural actor-critic algorithm'. Together they form a unique fingerprint.

Cite this