Construct Validity of the BRONCH Mentor Simulator for Essential Bronchoscopic Skills


Nicholas Pastis, MD; Allison Vanderbilt, EdD; Nichole Tanner, MD; Gerard Silvestri, MD; John Huggins, MD; Molly Madden, EdD; Philip Svigals, MD; Ray Shepherd, MD
Chest. 2013;144(4_MeetingAbstracts):581A. doi:10.1378/chest.1702165
PURPOSE: In an era of heightened awareness for patient safety, simulation-based bronchoscopy has emerged as a zero risk training modality, however, its effectiveness remains uncertain. We sought to demonstrate that the simulator could distinguish between 3 groups of bronchoscopists (novice, experienced, and expert), and surveyed their perceptions of simulation.
METHODS: A cohort study was conducted at 2 academic medical centers. Three groups were used to assess construct validity of the Simbionix Bronchoscopy Simulator: a novice group(< 10 bronchoscopies), an experienced group(200-1000 bronchoscopies), and an expert group(>1000 bronchoscopies). Participants were tested in 4 tasks(1-manipulation through the center of airways, 2-anatomic orientation, 3-airway anatomy, and 4-identification of lymph node stations). Participants’ performances were scored by the simulator, and participants were surveyed on their impressions of simulation training for bronchoscopy. Means and Kruskal-Wallis Test among groups were compared by task item(p<0.05).
RESULTS: There were statistically significant differences among the groups for tasks 1 and 3. For task 1, final score, total time, percent time at mid lumen, and total wall hits were most favorable (p=0.006,0.006,0.012, and 0.014, respectively) for expert(n=7), then experienced(n=6), and then novice bronchoscopists(n=7). For task 3, the total time, number of bronchial segments identified on first attempt, number of bronchial segments incorrectly identified after 3 attempts, and bronchials segments skipped were most favorable(p=0.04, 0.012,0.013,0.013, respectively) for expert, then experienced, and then novice bronchoscopists. There was no difference between groups for tasks 2 and 4. All participants agreed that simulation training is helpful and should become part of bronchoscopic training.
CONCLUSIONS: The bronchoscopy simulator demonstrated construct validity in differentiating skill levels in scope manipulation and airway anatomy tasks, however was not validated in differentiating skill levels for anatomic orientation or lymph node station identification. Users of all skill level rated bronchoscopy simulation as helpful.
CLINICAL IMPLICATIONS: Bronchoscopy simulation is a tool that should be considered to enhance teaching of scope manipulation and airway anatomy prior to first real world bronchoscopy.