Evaluation of fully automated commercial software for Agatston calcium scoring on non-ECG-gated low-dose chest CT with different slice thickness

Objectives: To evaluate commercial deep learning–based software for fully automated coronary artery calcium (CAC) scoring on non-electrocardiogram (ECG)-gated low-dose CT (LDCT) with different slice thicknesses compared with manual ECG-gated calcium-scoring CT (CSCT). Methods: This retrospective study included 567 patients who underwent both LDCT and CSCT. All LDCT images were reconstructed with a 2.5-mm slice thickness (LDCT_2.5-mm), and 453 LDCT scans were reconstructed with a 1.0-mm slice thickness (LDCT_1.0-mm). Automated CAC scoring was performed on CSCT (CSCT_auto), LDCT_1.0-mm, and LDCT_2.5-mm images. The reliability of CSCT_auto, LDCT_1.0-mm, and LDCT_2.5-mm was compared with manual CSCT scoring (CSCT_manual) using intraclass correlation coefficients (ICCs) and Bland-Altman analysis. Agreement, in CAC severity category, was analyzed using weighted kappa statistics. Diagnostic performance at various Agatston score cutoffs was also calculated. Results: CSCT_auto, LDCT_1.0-mm, and LDCT_2.5-mm demonstrated excellent agreement with CSCT_manual (ICC [95% confidence interval, CI]: 1.000 [1.000, 1.000], 0.937 [0.917, 0.952], and 0.955 [0.946, 0.963], respectively). The mean difference with 95% limits of agreement was lower with LDCT_1.0-mm than with LDCT_2.5-mm (19.94 [95% CI, −244.0, 283.9] vs. 45.26 [−248.2, 338.7]). Regarding CAC severity, LDCT_1.0-mm achieved almost perfect agreement, and LDCT_2.5-mm achieved substantial agreement (kappa [95% CI]: 0.809 [0.776, 0.838], 0.776 [0.740, 0.809], respectively). Diagnostic performance for detecting Agatston score ≥ 400 was also higher with LDCT_1.0-mm than with LDCT_2.5-mm (F1 score, 0.929 vs. 0.855). Conclusions: Fully automated CAC-scoring software with both CSCT and LDCT yielded excellent reliability and agreement with CSCT_manual. LDCT_1.0-mm yielded more accurate Agatston scoring than LDCT_2.5-mm using fully automated commercial software. Key Points: • Total Agatston scores and all vessels of CSCT_auto, LDCT_1.0-mm, and LDCT_2.5-mmdemonstrated excellent agreement with CSCT_manual(all ICC > 0.85). • The diagnostic performance for detecting all Agatston score cutoffs was better with LDCT_1.0-mmthan with LDCT_2.5-mm. • This automated software yielded a lower degree of underestimation compared with methods described in previous studies, and the degree of underestimation was lower with LDCT_1.0-mmthan with LDCT_2.5-mm.