DDW ePoster Library

THE EFFECT OF CHANGING THE SPECIFICITY OF AN ARTIFICIAL INTELLIGENCE-AIDED POLYP DETECTION DEVICE AND ITS IMPACT ON CLINICAL PERFORMANCE
DDW ePoster Library. Shaukat A. 05/23/21; 321061; Su584
Aasma Shaukat
Aasma Shaukat
Contributions
Abstract
Engage with the presenter here during ePoster Session: Endoscopic Technology Poster Session
On Sunday, May 23, 2021 12:15 - 1 p.m. EDT

Number: Su584
THE EFFECT OF CHANGING THE SPECIFICITY OF AN ARTIFICIAL INTELLIGENCE-AIDED POLYP DETECTION DEVICE AND ITS IMPACT ON CLINICAL PERFORMANCE

Society: ASGE
Track: Technologies and Procedural Innovation
Category: Endoscopic Technology 2

Author(s): Aasma Shaukat5,2, Sloane Allebes Phillips1, Neelima Chavali1, Daniel Colucci1, Lavi Erisson1, Jonathan Ng1, Samuel C. Somers3, William R. Brugge41 Iterative Scopes, Cambridge, Massachusetts, United States; 2 University of Minnesota System, Minneapolis, Minnesota, United States; 3 Concord Hospital, Concord, New Hampshire, United States; 4 Mount Auburn Hospital, Cambridge, Massachusetts, United States; 5 Minneapolis VA Health Care System, Minneapolis, Minnesota, United States

Background: Improving adenoma detection rate (ADR) and adenoma per colonoscopy (APC) is important for colonoscopy quality programs, with the goal of reducing colorectal cancer. Several promising AI-aided polyp detection devices (APDD) have been shown to improve ADR and APC. The effect of changing the specificity of an APDD and its impact on clinical performance has yet to be explored. False positive rate (FPR) is inversely proportional to specificity (FPR % = 100 - Specificity %). Lowering FPR leads to increased specificity for the AI model. The aim of our study was to evaluate the pilot performance of a novel APDD during routine colonoscopies in a real-world setting. Namely, the effect of changing an AI model's specificity on clinical efficacy, specifically ADR and APC. Additionally, we assessed the effect that changing specificity had on resection of hyperplastic polyps relative to adenomas and sessile serrated lesions (SSLs) as defined by true histology rate (THR).

Patients and methods: Patients undergoing colonoscopy with our APDD were enrolled in one of two arms. Procedures were completed using an APDD trained on the same data but set to either an FPR of 0.9% or 1.3% (specificity 99.1% and 98.7% respectively). The results were compared with a historical cohort performed by the same endoscopists at the same practices without the use of the APDD. All resected polyps were examined histologically.

Results: 273 patients undergoing colonoscopy were enrolled and outcomes compared with 618 historical control patients. Of these 273 patients, 171 were enrolled with an APDD set to FPR1 = 0.9% FPR, and 102 with an APDD set to FPR2 = 1.3% FPR. Overall, ADR using FPR1, FPR2 and at baseline, was 53.22%, 46.08% and 39.97% respectively. Of these patients, 156 (89 using FPR1, 67 using FPR2) received screening exams with outcomes compared with 363 historical control patients. In screening exams, ADR using FPR1, FPR2 and at baseline, was 47.19%, 40.30% and 34.99% respectively. Overall, APC rate for FPR1, FPR2 and at baseline, was 1.23, 0.92 and 0.79 respectively and 0.96, 0.73, and 0.50 respectively in screening exams. In addition, overall sessile serrated lesion per colonoscopy (SSLPC) with FPR1, FPR2 and at baseline, was 0.18, 0.30 and 0.13 respectively. THR at FPR1, FPR2 and at baseline was 0.74, 0.61 and 0.74 respectively.

Conclusion: A novel APDD increased the ADR and APC in a cohort of patients undergoing screening and surveillance colonoscopy without concomitant increase in hyperplastic polyp resection when set to an FPR of 0.9%. When FPR was set to 1.3% the clinical impact was significantly lower than that of the FPR of 0.9%. We hypothesize that the increased FPR event rate contributed to notification fatigue thereby reducing performance. We recommend further exploration into the impact of FPR on ADR, APC and SSLPC.
Engage with the presenter here during ePoster Session: Endoscopic Technology Poster Session
On Sunday, May 23, 2021 12:15 - 1 p.m. EDT

Number: Su584
THE EFFECT OF CHANGING THE SPECIFICITY OF AN ARTIFICIAL INTELLIGENCE-AIDED POLYP DETECTION DEVICE AND ITS IMPACT ON CLINICAL PERFORMANCE

Society: ASGE
Track: Technologies and Procedural Innovation
Category: Endoscopic Technology 2

Author(s): Aasma Shaukat5,2, Sloane Allebes Phillips1, Neelima Chavali1, Daniel Colucci1, Lavi Erisson1, Jonathan Ng1, Samuel C. Somers3, William R. Brugge41 Iterative Scopes, Cambridge, Massachusetts, United States; 2 University of Minnesota System, Minneapolis, Minnesota, United States; 3 Concord Hospital, Concord, New Hampshire, United States; 4 Mount Auburn Hospital, Cambridge, Massachusetts, United States; 5 Minneapolis VA Health Care System, Minneapolis, Minnesota, United States

Background: Improving adenoma detection rate (ADR) and adenoma per colonoscopy (APC) is important for colonoscopy quality programs, with the goal of reducing colorectal cancer. Several promising AI-aided polyp detection devices (APDD) have been shown to improve ADR and APC. The effect of changing the specificity of an APDD and its impact on clinical performance has yet to be explored. False positive rate (FPR) is inversely proportional to specificity (FPR % = 100 - Specificity %). Lowering FPR leads to increased specificity for the AI model. The aim of our study was to evaluate the pilot performance of a novel APDD during routine colonoscopies in a real-world setting. Namely, the effect of changing an AI model's specificity on clinical efficacy, specifically ADR and APC. Additionally, we assessed the effect that changing specificity had on resection of hyperplastic polyps relative to adenomas and sessile serrated lesions (SSLs) as defined by true histology rate (THR).

Patients and methods: Patients undergoing colonoscopy with our APDD were enrolled in one of two arms. Procedures were completed using an APDD trained on the same data but set to either an FPR of 0.9% or 1.3% (specificity 99.1% and 98.7% respectively). The results were compared with a historical cohort performed by the same endoscopists at the same practices without the use of the APDD. All resected polyps were examined histologically.

Results: 273 patients undergoing colonoscopy were enrolled and outcomes compared with 618 historical control patients. Of these 273 patients, 171 were enrolled with an APDD set to FPR1 = 0.9% FPR, and 102 with an APDD set to FPR2 = 1.3% FPR. Overall, ADR using FPR1, FPR2 and at baseline, was 53.22%, 46.08% and 39.97% respectively. Of these patients, 156 (89 using FPR1, 67 using FPR2) received screening exams with outcomes compared with 363 historical control patients. In screening exams, ADR using FPR1, FPR2 and at baseline, was 47.19%, 40.30% and 34.99% respectively. Overall, APC rate for FPR1, FPR2 and at baseline, was 1.23, 0.92 and 0.79 respectively and 0.96, 0.73, and 0.50 respectively in screening exams. In addition, overall sessile serrated lesion per colonoscopy (SSLPC) with FPR1, FPR2 and at baseline, was 0.18, 0.30 and 0.13 respectively. THR at FPR1, FPR2 and at baseline was 0.74, 0.61 and 0.74 respectively.

Conclusion: A novel APDD increased the ADR and APC in a cohort of patients undergoing screening and surveillance colonoscopy without concomitant increase in hyperplastic polyp resection when set to an FPR of 0.9%. When FPR was set to 1.3% the clinical impact was significantly lower than that of the FPR of 0.9%. We hypothesize that the increased FPR event rate contributed to notification fatigue thereby reducing performance. We recommend further exploration into the impact of FPR on ADR, APC and SSLPC.

By clicking “Accept Terms & all Cookies” or by continuing to browse, you agree to the storing of third-party cookies on your device to enhance your user experience and agree to the user terms and conditions of this learning management system (LMS).

Cookie Settings
Accept Terms & all Cookies