Nonlinear gradient denoising: Finding accurate extrema from inaccurate functional derivatives

John C. Snyder, Matthias Rupp, Klaus Robert Müller, Kieron Burke

Research output: Contribution to journalArticlepeer-review

20 Citations (Scopus)


A method for nonlinear optimization with machine learning (ML) models, called nonlinear gradient denoising (NLGD), is developed, and applied with ML approximations to the kinetic energy density functional in an orbital-free density functional theory. Due to systematically inaccurate gradients of ML models, in particular when the data is very high-dimensional, the optimization must be constrained to the data manifold. We use nonlinear kernel principal component analysis (PCA) to locally reconstruct the manifold, enabling a projected gradient descent along it. A thorough analysis of the method is given via a simple model, designed to clarify the concepts presented. Additionally, NLGD is compared with the local PCA method used in previous work. Our method is shown to be superior in cases when the data manifold is highly nonlinear and high dimensional. Further applications of the method in both density functional theory and ML are discussed.

Original languageEnglish
Pages (from-to)1102-1114
Number of pages13
JournalInternational Journal of Quantum Chemistry
Issue number16
Publication statusPublished - 2015 Aug 1


  • density functional theory
  • machine learning
  • nonlinear gradient denoising
  • orbital-free density functional theory

ASJC Scopus subject areas

  • Atomic and Molecular Physics, and Optics
  • Condensed Matter Physics
  • Physical and Theoretical Chemistry


Dive into the research topics of 'Nonlinear gradient denoising: Finding accurate extrema from inaccurate functional derivatives'. Together they form a unique fingerprint.

Cite this