QuIIL at T3 Challenge: Towards Automation in Life-Saving Intervention Procedures from First-Person View

Trinh T. L. Vuong; Doanh C. Bui; Jin Tae Kwak

doi:10.1007/978-3-031-71626-3_10

QuIIL at T3 Challenge: Towards Automation in Life-Saving Intervention Procedures from First-Person View

Trinh T. L. Vuong
, Doanh C. Bui
, Jin Tae Kwak^*

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

In this paper, we present our solutions for a spectrum of automation tasks in life-saving intervention procedures within the Trauma THOMPSON (T3) Challenge, encompassing action recognition, action anticipation, and Visual Question Answering (VQA). For action recognition and anticipation, we propose a pre-processing strategy that samples and stitches multiple inputs into a single image and then incorporates momentum- and attention-based knowledge distillation to improve the performance of the two tasks. For training, we present an action dictionary-guided design, which consistently yields the most favorable results across our experiments. In the realm of VQA, we leverage object-level features and deploy co-attention networks to train both object and question features. Notably, we introduce a novel frame-question cross-attention mechanism at the network’s core for enhanced performance. Our solutions achieve the rank in action recognition and anticipation tasks and rank in the VQA task. The source code is available at https://github.com/QuIIL/QuIIL_thompson_solution.

Original language	English
Title of host publication	AI for Brain Lesion Detection and Trauma Video Action Recognition - 1st BONBID-HIE Lesion Segmentation Challenge and 1st Trauma Thompson Challenge, Held in Conjunction with MICCAI 2023, Proceedings
Editors	Rina Bao, Ellen Grant, Yangming Ou, Andrew Kirkpatrick, Juan Wachs
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	82-93
Number of pages	12
ISBN (Print)	9783031716256
DOIs	https://doi.org/10.1007/978-3-031-71626-3_10
Publication status	Published - 2025
Event	1st BONBID-HIE Lesion Segmentation Challenge and 1st Trauma Thompson Challenge Held in Conjunction with 26th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2023 - Vancouver, Canada Duration: 2023 Oct 12 → 2023 Oct 16

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	14567 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	1st BONBID-HIE Lesion Segmentation Challenge and 1st Trauma Thompson Challenge Held in Conjunction with 26th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2023
Country/Territory	Canada
City	Vancouver
Period	23/10/12 → 23/10/16

Bibliographical note

Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.

Keywords

VQA
Video classification
co-attention
contrastive learning

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/978-3-031-71626-3_10

Cite this

T. L. Vuong, T., C. Bui, D., & Kwak, J. T. (2025). QuIIL at T3 Challenge: Towards Automation in Life-Saving Intervention Procedures from First-Person View. In R. Bao, E. Grant, Y. Ou, A. Kirkpatrick, & J. Wachs (Eds.), AI for Brain Lesion Detection and Trauma Video Action Recognition - 1st BONBID-HIE Lesion Segmentation Challenge and 1st Trauma Thompson Challenge, Held in Conjunction with MICCAI 2023, Proceedings (pp. 82-93). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14567 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-71626-3_10

QuIIL at T3 Challenge: Towards Automation in Life-Saving Intervention Procedures from First-Person View. / T. L. Vuong, Trinh; C. Bui, Doanh; Kwak, Jin Tae.
AI for Brain Lesion Detection and Trauma Video Action Recognition - 1st BONBID-HIE Lesion Segmentation Challenge and 1st Trauma Thompson Challenge, Held in Conjunction with MICCAI 2023, Proceedings. ed. / Rina Bao; Ellen Grant; Yangming Ou; Andrew Kirkpatrick; Juan Wachs. Springer Science and Business Media Deutschland GmbH, 2025. p. 82-93 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14567 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

T. L. Vuong, T, C. Bui, D & Kwak, JT 2025, QuIIL at T3 Challenge: Towards Automation in Life-Saving Intervention Procedures from First-Person View. in R Bao, E Grant, Y Ou, A Kirkpatrick & J Wachs (eds), AI for Brain Lesion Detection and Trauma Video Action Recognition - 1st BONBID-HIE Lesion Segmentation Challenge and 1st Trauma Thompson Challenge, Held in Conjunction with MICCAI 2023, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14567 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 82-93, 1st BONBID-HIE Lesion Segmentation Challenge and 1st Trauma Thompson Challenge Held in Conjunction with 26th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2023, Vancouver, Canada, 23/10/12. https://doi.org/10.1007/978-3-031-71626-3_10

T. L. Vuong T, C. Bui D, Kwak JT. QuIIL at T3 Challenge: Towards Automation in Life-Saving Intervention Procedures from First-Person View. In Bao R, Grant E, Ou Y, Kirkpatrick A, Wachs J, editors, AI for Brain Lesion Detection and Trauma Video Action Recognition - 1st BONBID-HIE Lesion Segmentation Challenge and 1st Trauma Thompson Challenge, Held in Conjunction with MICCAI 2023, Proceedings. Springer Science and Business Media Deutschland GmbH. 2025. p. 82-93. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-71626-3_10

T. L. Vuong, Trinh ; C. Bui, Doanh ; Kwak, Jin Tae. / QuIIL at T3 Challenge : Towards Automation in Life-Saving Intervention Procedures from First-Person View. AI for Brain Lesion Detection and Trauma Video Action Recognition - 1st BONBID-HIE Lesion Segmentation Challenge and 1st Trauma Thompson Challenge, Held in Conjunction with MICCAI 2023, Proceedings. editor / Rina Bao ; Ellen Grant ; Yangming Ou ; Andrew Kirkpatrick ; Juan Wachs. Springer Science and Business Media Deutschland GmbH, 2025. pp. 82-93 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{f170ae2256ba446fad2e0d64b09b8a7d,

title = "QuIIL at T3 Challenge: Towards Automation in Life-Saving Intervention Procedures from First-Person View",

abstract = "In this paper, we present our solutions for a spectrum of automation tasks in life-saving intervention procedures within the Trauma THOMPSON (T3) Challenge, encompassing action recognition, action anticipation, and Visual Question Answering (VQA). For action recognition and anticipation, we propose a pre-processing strategy that samples and stitches multiple inputs into a single image and then incorporates momentum- and attention-based knowledge distillation to improve the performance of the two tasks. For training, we present an action dictionary-guided design, which consistently yields the most favorable results across our experiments. In the realm of VQA, we leverage object-level features and deploy co-attention networks to train both object and question features. Notably, we introduce a novel frame-question cross-attention mechanism at the network{\textquoteright}s core for enhanced performance. Our solutions achieve the rank in action recognition and anticipation tasks and rank in the VQA task. The source code is available at https://github.com/QuIIL/QuIIL\_thompson\_solution.",

keywords = "VQA, Video classification, co-attention, contrastive learning",

author = "\{T. L. Vuong\}, Trinh and \{C. Bui\}, Doanh and Kwak, \{Jin Tae\}",

note = "Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.; 1st BONBID-HIE Lesion Segmentation Challenge and 1st Trauma Thompson Challenge Held in Conjunction with 26th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2023 ; Conference date: 12-10-2023 Through 16-10-2023",

year = "2025",

doi = "10.1007/978-3-031-71626-3\_10",

language = "English",

isbn = "9783031716256",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "82--93",

editor = "Rina Bao and Ellen Grant and Yangming Ou and Andrew Kirkpatrick and Juan Wachs",

booktitle = "AI for Brain Lesion Detection and Trauma Video Action Recognition - 1st BONBID-HIE Lesion Segmentation Challenge and 1st Trauma Thompson Challenge, Held in Conjunction with MICCAI 2023, Proceedings",

address = "Germany",

}

TY - GEN

T1 - QuIIL at T3 Challenge

T2 - 1st BONBID-HIE Lesion Segmentation Challenge and 1st Trauma Thompson Challenge Held in Conjunction with 26th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2023

AU - T. L. Vuong, Trinh

AU - C. Bui, Doanh

AU - Kwak, Jin Tae

N1 - Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.

PY - 2025

Y1 - 2025

N2 - In this paper, we present our solutions for a spectrum of automation tasks in life-saving intervention procedures within the Trauma THOMPSON (T3) Challenge, encompassing action recognition, action anticipation, and Visual Question Answering (VQA). For action recognition and anticipation, we propose a pre-processing strategy that samples and stitches multiple inputs into a single image and then incorporates momentum- and attention-based knowledge distillation to improve the performance of the two tasks. For training, we present an action dictionary-guided design, which consistently yields the most favorable results across our experiments. In the realm of VQA, we leverage object-level features and deploy co-attention networks to train both object and question features. Notably, we introduce a novel frame-question cross-attention mechanism at the network’s core for enhanced performance. Our solutions achieve the rank in action recognition and anticipation tasks and rank in the VQA task. The source code is available at https://github.com/QuIIL/QuIIL_thompson_solution.

AB - In this paper, we present our solutions for a spectrum of automation tasks in life-saving intervention procedures within the Trauma THOMPSON (T3) Challenge, encompassing action recognition, action anticipation, and Visual Question Answering (VQA). For action recognition and anticipation, we propose a pre-processing strategy that samples and stitches multiple inputs into a single image and then incorporates momentum- and attention-based knowledge distillation to improve the performance of the two tasks. For training, we present an action dictionary-guided design, which consistently yields the most favorable results across our experiments. In the realm of VQA, we leverage object-level features and deploy co-attention networks to train both object and question features. Notably, we introduce a novel frame-question cross-attention mechanism at the network’s core for enhanced performance. Our solutions achieve the rank in action recognition and anticipation tasks and rank in the VQA task. The source code is available at https://github.com/QuIIL/QuIIL_thompson_solution.

KW - VQA

KW - Video classification

KW - co-attention

KW - contrastive learning

UR - https://www.scopus.com/pages/publications/85208435124

U2 - 10.1007/978-3-031-71626-3_10

DO - 10.1007/978-3-031-71626-3_10

M3 - Conference contribution

AN - SCOPUS:85208435124

SN - 9783031716256

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 82

EP - 93

BT - AI for Brain Lesion Detection and Trauma Video Action Recognition - 1st BONBID-HIE Lesion Segmentation Challenge and 1st Trauma Thompson Challenge, Held in Conjunction with MICCAI 2023, Proceedings

A2 - Bao, Rina

A2 - Grant, Ellen

A2 - Ou, Yangming

A2 - Kirkpatrick, Andrew

A2 - Wachs, Juan

PB - Springer Science and Business Media Deutschland GmbH

Y2 - 12 October 2023 through 16 October 2023

ER -