Accurate bone segmentation and anatomical landmark localization are essential tasks in computer-aided surgical simulation for patients with craniomaxillofacial (CMF) deformities. To leverage the complementarity between the two tasks, we propose an efficient end-to-end deep network, i.e., multi-task dynamic transformer network (DTNet), to concurrently segment CMF bones and localize large-scale landmarks in one-pass from large volumes of cone-beam computed tomography (CBCT) data. Our DTNet was evaluated quantitatively using CBCTs of patients with CMF deformities. The results demonstrated that our method outperforms the other state-of-the-art methods in both tasks of the bony segmentation and the landmark digitization. Our DTNet features three main technical contributions. First, a collaborative two-branch architecture is designed to efficiently capture both fine-grained image details and complete global context for high-resolution volume-to-volume prediction. Second, leveraging anatomical dependencies between landmarks, regionalized dynamic learners (RDLs) are designed in the concept of “learns to learn” to jointly regress large-scale 3D heatmaps of all landmarks under limited computational costs. Third, adaptive transformer modules (ATMs) are designed for the flexible learning of task-specific feature embedding from common feature bases.
|Title of host publication||Medical Image Computing and Computer Assisted Intervention – MICCAI 2020 - 23rd International Conference, Proceedings|
|Editors||Anne L. Martel, Purang Abolmaesumi, Danail Stoyanov, Diana Mateus, Maria A. Zuluaga, S. Kevin Zhou, Daniel Racoceanu, Leo Joskowicz|
|Publisher||Springer Science and Business Media Deutschland GmbH|
|Number of pages||10|
|Publication status||Published - 2020|
|Event||23rd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2020 - Lima, Peru|
Duration: 2020 Oct 4 → 2020 Oct 8
|Name||Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)|
|Conference||23rd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2020|
|Period||20/10/4 → 20/10/8|
Bibliographical noteFunding Information:
Acknowledgements. This work was supported in part by NIH grants (R01 DE022676, R01 DE027251 and R01 DE021863).
© 2020, Springer Nature Switzerland AG.
- Craniomaxillofacial (CMF)
- Landmark localization
- Multi-task learning
ASJC Scopus subject areas
- Theoretical Computer Science
- Computer Science(all)