Competency assessment in focused cardiac ultrasound - can the use of sequential testing help tailor training requirements?

Abstract

Varying numbers of scans are required by different professional bodies before focused cardiac ultrasound (FCU) competence is assumed. It has been suggested that innovation in the assessment of FCU competence is needed and that competency assessment needs to be more individualized. We report our experience of how the use of sequential testing may help personalize the assessment of FCU competence.

Design: This was a planned exploratory reanalysis of previously prospectively collected data. FCU was performed sequentially by an intensive care trainee and expert on the same patient. Assessment of left ventricular (LV) function by the trainee and expert was compared. Sequential testing methods were used in the analysis of this data to see if they could be used to help in the assessment of competence. Each trainee had completed a 38-hour teaching program and a logbook of 30 scans prior to enrollment.

Setting: Tertiary Australian not for profit private academic hospital.

Measurements and main results: Two hundred seventy paired echocardiograms were completed by seven trainees. For trainees to achieve greater than 90% accuracy in correctly assessing LV function when compared with an expert, a variable number of scans were required. This ranged from 13 to 25 (95% CI, 13-25) scans. Over the study period, the ability to correctly identify LV function was maintained, and it appeared there was no degradation in skill.

Conclusions: Using the Sequential Probability Ratio Test demonstrates a variable number of scans were required to show greater than 90% accuracy in the assessment of LV function. As such, the use of sequential testing could help individualize competency assessments in FCU. Additionally, our data suggests that over a 6-month period, echocardiographic skill is maintained without any formal teaching or feedback. Further work assessing the utility of this method based on larger samples is required.