Abstract

Objective: To validate the scores of assessments of basic laparoscopic surgical skills obtained through performance on LAP Mentor.

Design: Cross-sectional validation study. Setting: Oman Medical Specialty Board (OMSB) skills lab, Muscat, Oman.

Participants: Twenty-three surgical residents, registered with OMSB, at different years of residency, underwent assessment of basic laparoscopic surgical skills on the LAP Mentor. A construct validation model was used to validate the scores.

Results: 35% of the candidates, all belonging to the senior group, passed the assessment. Cronbach’s alpha was 0.87 with a standard error of measurement of 1.53 (95% confidence interval). The intra-class coefficient varied from 0.88 to 0.95 for different scales. Factor analysis revealed two underlying constructs, i.e. technical skills and patient safety skills, that explained competency in laparoscopy surgery. A review of the literature supported content validity evidence. Relationships with other variables were documented through convergent and divergent evidence. A correlation for the data revealed that actual residency year and total score achieved were significantly related (r = 0.51, N = 23, P = 0.01, two-tailed). Senior residents scored significantly higher than junior residents on overall performance. Fisher’s exact test showed that more seniors than juniors passed on overall performance using specific criteria determined by factor analysis and Angoff’s method for standard setting. The majority of residents and raters agreed or strongly agreed on feasibility, conducting such tests regularly, meaningful feedback, and objectivity and reuse of the tools.

Conclusions: Assessment on LAP Mentor using different rating scales and construct-based standard setting methods provides meaningful scores. Periodic summative assessment is acceptable to residents.