Latent transformations neural network for object view synthesis

Sangpil Kim, Nick Winovich, Hyung Gun Chi, Guang Lin, Karthik Ramani

Research output: Contribution to journalArticlepeer-review

2 Citations (Scopus)


We propose a fully convolutional conditional generative neural network, the latent transformation neural network, capable of rigid and non-rigid object view synthesis using a lightweight architecture suited for real-time applications and embedded systems. In contrast to existing object view synthesis methods which incorporate conditioning information via concatenation, we introduce a dedicated network component, the conditional transformation unit. This unit is designed to learn the latent space transformations corresponding to specified target views. In addition, a consistency loss term is defined to guide the network toward learning the desired latent space mappings, a task-divided decoder is constructed to refine the quality of generated views of objects, and an adaptive discriminator is introduced to improve the adversarial training process. The generalizability of the proposed methodology is demonstrated on a collection of three diverse tasks: multi-view synthesis on real hand depth images, view synthesis of real and synthetic faces, and the rotation of rigid objects. The proposed model is shown to be comparable with the state-of-the-art methods in structural similarity index measure and L1 metrics while simultaneously achieving a 24% reduction in the compute time for inference of novel images.

Original languageEnglish
Pages (from-to)1663-1677
Number of pages15
JournalVisual Computer
Issue number8
Publication statusPublished - 2020 Aug 1
Externally publishedYes


  • Conditional generative model
  • Fully convolutional
  • Latent transformation
  • Object view synthesis

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Computer Graphics and Computer-Aided Design


Dive into the research topics of 'Latent transformations neural network for object view synthesis'. Together they form a unique fingerprint.

Cite this