ssEMnet: Serial-section electron microscopy image registration using a spatial transformer network with learned features

Inwan Yoo; David G.C. Hildebrand; Willie F. Tobin; Wei Chung Allen Lee; Won Ki Jeong

doi:10.1007/978-3-319-67558-9_29

ssEMnet: Serial-section electron microscopy image registration using a spatial transformer network with learned features

Inwan Yoo
, David G.C. Hildebrand
, Willie F. Tobin
, Wei Chung Allen Lee
, Won Ki Jeong^*

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

58 Citations (Scopus)

Abstract

The alignment of serial-section electron microscopy (ssEM) images is critical for efforts in neuroscience that seek to reconstruct neuronal circuits. However, each ssEM plane contains densely packed structures that vary from one section to the next, which makes matching features across images a challenge. Advances in deep learning has resulted in unprecedented performance in similar computer vision problems, but to our knowledge, they have not been successfully applied to ssEM image co-registration. In this paper, we introduce a novel deep network model that combines a spatial transformer for image deformation and a convolutional autoencoder for unsupervised feature learning for robust ssEM image alignment. This results in improved accuracy and robustness while requiring substantially less user intervention than conventional methods. We evaluate our method by comparing registration quality across several datasets.

Original language	English
Title of host publication	Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support - 3rd International Workshop, DLMIA 2017 and 7th International Workshop, ML-CDS 2017 Held in Conjunction with MICCAI 2017, Proceedings
Editors	Tal Arbel, M. Jorge Cardoso
Publisher	Springer Verlag
Pages	249-257
Number of pages	9
ISBN (Print)	9783319675572
DOIs	https://doi.org/10.1007/978-3-319-67558-9_29
Publication status	Published - 2017
Externally published	Yes
Event	3rd International Workshop on Deep Learning in Medical Image Analysis, DLMIA 2017 and 7th International Workshop on Multimodal Learning for Clinical Decision Support, ML-CDS 2017 held in Conjunction with 20th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2017 - Quebec City, Canada Duration: 2017 Sept 14 → 2017 Sept 14

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	10553 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Other

Other	3rd International Workshop on Deep Learning in Medical Image Analysis, DLMIA 2017 and 7th International Workshop on Multimodal Learning for Clinical Decision Support, ML-CDS 2017 held in Conjunction with 20th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2017
Country/Territory	Canada
City	Quebec City
Period	17/9/14 → 17/9/14

Bibliographical note

Publisher Copyright:
© Springer International Publishing AG 2017.

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/978-3-319-67558-9_29

Cite this

Yoo, I., Hildebrand, D. G. C., Tobin, W. F., Lee, W. C. A., & Jeong, W. K. (2017). ssEMnet: Serial-section electron microscopy image registration using a spatial transformer network with learned features. In T. Arbel, & M. J. Cardoso (Eds.), Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support - 3rd International Workshop, DLMIA 2017 and 7th International Workshop, ML-CDS 2017 Held in Conjunction with MICCAI 2017, Proceedings (pp. 249-257). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10553 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-67558-9_29

ssEMnet: Serial-section electron microscopy image registration using a spatial transformer network with learned features. / Yoo, Inwan; Hildebrand, David G.C.; Tobin, Willie F. et al.
Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support - 3rd International Workshop, DLMIA 2017 and 7th International Workshop, ML-CDS 2017 Held in Conjunction with MICCAI 2017, Proceedings. ed. / Tal Arbel; M. Jorge Cardoso. Springer Verlag, 2017. p. 249-257 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10553 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Yoo, I, Hildebrand, DGC, Tobin, WF, Lee, WCA & Jeong, WK 2017, ssEMnet: Serial-section electron microscopy image registration using a spatial transformer network with learned features. in T Arbel & MJ Cardoso (eds), Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support - 3rd International Workshop, DLMIA 2017 and 7th International Workshop, ML-CDS 2017 Held in Conjunction with MICCAI 2017, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10553 LNCS, Springer Verlag, pp. 249-257, 3rd International Workshop on Deep Learning in Medical Image Analysis, DLMIA 2017 and 7th International Workshop on Multimodal Learning for Clinical Decision Support, ML-CDS 2017 held in Conjunction with 20th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2017, Quebec City, Canada, 17/9/14. https://doi.org/10.1007/978-3-319-67558-9_29

Yoo I, Hildebrand DGC, Tobin WF, Lee WCA, Jeong WK. ssEMnet: Serial-section electron microscopy image registration using a spatial transformer network with learned features. In Arbel T, Cardoso MJ, editors, Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support - 3rd International Workshop, DLMIA 2017 and 7th International Workshop, ML-CDS 2017 Held in Conjunction with MICCAI 2017, Proceedings. Springer Verlag. 2017. p. 249-257. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-67558-9_29

Yoo, Inwan ; Hildebrand, David G.C. ; Tobin, Willie F. et al. / ssEMnet : Serial-section electron microscopy image registration using a spatial transformer network with learned features. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support - 3rd International Workshop, DLMIA 2017 and 7th International Workshop, ML-CDS 2017 Held in Conjunction with MICCAI 2017, Proceedings. editor / Tal Arbel ; M. Jorge Cardoso. Springer Verlag, 2017. pp. 249-257 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{41dd3ac3db1d4a9baeebbca98cad8745,

title = "ssEMnet: Serial-section electron microscopy image registration using a spatial transformer network with learned features",

abstract = "The alignment of serial-section electron microscopy (ssEM) images is critical for efforts in neuroscience that seek to reconstruct neuronal circuits. However, each ssEM plane contains densely packed structures that vary from one section to the next, which makes matching features across images a challenge. Advances in deep learning has resulted in unprecedented performance in similar computer vision problems, but to our knowledge, they have not been successfully applied to ssEM image co-registration. In this paper, we introduce a novel deep network model that combines a spatial transformer for image deformation and a convolutional autoencoder for unsupervised feature learning for robust ssEM image alignment. This results in improved accuracy and robustness while requiring substantially less user intervention than conventional methods. We evaluate our method by comparing registration quality across several datasets.",

author = "Inwan Yoo and Hildebrand, \{David G.C.\} and Tobin, \{Willie F.\} and Lee, \{Wei Chung Allen\} and Jeong, \{Won Ki\}",

note = "Publisher Copyright: {\textcopyright} Springer International Publishing AG 2017.; 3rd International Workshop on Deep Learning in Medical Image Analysis, DLMIA 2017 and 7th International Workshop on Multimodal Learning for Clinical Decision Support, ML-CDS 2017 held in Conjunction with 20th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2017 ; Conference date: 14-09-2017 Through 14-09-2017",

year = "2017",

doi = "10.1007/978-3-319-67558-9\_29",

language = "English",

isbn = "9783319675572",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "249--257",

editor = "Tal Arbel and Cardoso, \{M. Jorge\}",

booktitle = "Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support - 3rd International Workshop, DLMIA 2017 and 7th International Workshop, ML-CDS 2017 Held in Conjunction with MICCAI 2017, Proceedings",

}

TY - GEN

T1 - ssEMnet

T2 - 3rd International Workshop on Deep Learning in Medical Image Analysis, DLMIA 2017 and 7th International Workshop on Multimodal Learning for Clinical Decision Support, ML-CDS 2017 held in Conjunction with 20th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2017

AU - Yoo, Inwan

AU - Hildebrand, David G.C.

AU - Tobin, Willie F.

AU - Lee, Wei Chung Allen

AU - Jeong, Won Ki

PY - 2017

Y1 - 2017

N2 - The alignment of serial-section electron microscopy (ssEM) images is critical for efforts in neuroscience that seek to reconstruct neuronal circuits. However, each ssEM plane contains densely packed structures that vary from one section to the next, which makes matching features across images a challenge. Advances in deep learning has resulted in unprecedented performance in similar computer vision problems, but to our knowledge, they have not been successfully applied to ssEM image co-registration. In this paper, we introduce a novel deep network model that combines a spatial transformer for image deformation and a convolutional autoencoder for unsupervised feature learning for robust ssEM image alignment. This results in improved accuracy and robustness while requiring substantially less user intervention than conventional methods. We evaluate our method by comparing registration quality across several datasets.

AB - The alignment of serial-section electron microscopy (ssEM) images is critical for efforts in neuroscience that seek to reconstruct neuronal circuits. However, each ssEM plane contains densely packed structures that vary from one section to the next, which makes matching features across images a challenge. Advances in deep learning has resulted in unprecedented performance in similar computer vision problems, but to our knowledge, they have not been successfully applied to ssEM image co-registration. In this paper, we introduce a novel deep network model that combines a spatial transformer for image deformation and a convolutional autoencoder for unsupervised feature learning for robust ssEM image alignment. This results in improved accuracy and robustness while requiring substantially less user intervention than conventional methods. We evaluate our method by comparing registration quality across several datasets.

UR - https://www.scopus.com/pages/publications/85029818837

U2 - 10.1007/978-3-319-67558-9_29

DO - 10.1007/978-3-319-67558-9_29

M3 - Conference contribution

AN - SCOPUS:85029818837

SN - 9783319675572

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 249

EP - 257

BT - Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support - 3rd International Workshop, DLMIA 2017 and 7th International Workshop, ML-CDS 2017 Held in Conjunction with MICCAI 2017, Proceedings

A2 - Arbel, Tal

A2 - Cardoso, M. Jorge

PB - Springer Verlag

Y2 - 14 September 2017 through 14 September 2017

ER -

ssEMnet: Serial-section electron microscopy image registration using a spatial transformer network with learned features

Abstract

Publication series

Other

Bibliographical note

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this