Education

Framework for Systematic Training and Assessment of Technical Skills

Digital tools like digital box trainers and VR seem promising in delivering safe and tailored practice opportunities outside of the surgical clinic, yet understanding their efficacy and limitations is essential. This study investigated Which digital tools are available to train surgical skills, How these tools are used, How effective they are, and What skills they are intended to teach.

Medline, Embase, and Cochrane libraries were systematically reviewed for randomized trials, evaluating digital skill-training tools based on objective outcomes (skills scores and completion time) in surgical residents. Digital tools effectiveness were compared against controls, wet/dry lab training, and other digital tools. Tool and training factors subgroups were analysed, and studies were assessed on their primary outcomes: technical and/or non-technical.

The 33 included studies involved 927 residents and six digital tools; digital box trainers, (immersive) virtual reality (VR) trainers, robot surgery trainers, coaching and feedback, and serious games. Digital tools outperformed controls in skill scores (SMD 1.66 [1.06, 2.25], P < 0.00001, I² = 83 %) and completion time (SMD -1.05 [−1.72, −0.38], P = 0.0001, I² = 71 %). There were no significant differences between digital tools and lab training, between tools, or in other subgroups. Only two studies focussed on non-technical skills.

While the efficacy of digital tools in enhancing technical surgical skills is evident - especially for VR-trainers -, there is a lack of evidence regarding non-technical skills, and need to improve methodological robustness of research on new (digital) tools before they are implemented in curricula.

This study provides critical insight into the increasing presence of digital tools in surgical training, demonstrating their usefulness while identifying current challenges, especially regarding methodological robustness and inattention to non-technical skills.

Digital training resources show great promise in augmenting traditional surgical education – especially in times of social distancing and limited surgical caseload. Embedding digital resources in surgical curricula is however not current, nor common practice in Dutch hospitals. While the digital world has become part of a resident's everyday life, surprisingly little is known about surgical residents’ experiences and expectations towards use of digital resources for their own surgical education. This study aims to identify digital resources currently used in Dutch surgical curricula and to describe surgical residents’ perspectives towards digital education.

A series of semi-structured interviews with Dutch surgical residents were conducted until data sufficiency occurred. The interviews consisted of two parts: 1) current surgical training and implemented digital resources, and 2) future surgical training and the role of digital resources therein. All interviews were digitally recorded, transcribed verbatim, and thematically analyzed.

Sixteen surgical residents were interviewed – two out of each of the eight educational regions for surgery in the Netherlands. Five digital resource categories were identified and four general educational themes (requirements, advantages, disadvantages, and general education themes), overarching 13 sub-themes. In general, residents were enthusiastic with regard to using digital resources, especially when the perceived advantages supported their autonomy.

Dutch surgical residents indicate that digital resources may support their educational experiences, but state that ideally they must be combined with much appreciated on-the-job training, and be offered to them tailored to their individual needs. No resources are considered to be a “magic bullet” in itself. The specific needs of residents and educators need to be addressed clearly in order to successfully adopt and implement digital resources on a larger scale.

This study aimed to investigate the feasibility of using eye tracking data to identify orthopedic trainees’ technical proficiency in hip arthroscopic procedures during simulation-based training.

A cross sectional study.

A simulation-based training session for hip arthroscopy was conducted. Eye tracking devices were used to record participants’ eye movements while performing simulated operations. The NASA Task Load Index survey was then used to measure subjective opinions on the perceived workload of the training. Statistical analyses were performed to determine the significance of the eye metrics and survey data.

A total of 12 arthroscopic trainees, including resident doctors, junior specialist surgeons, and consultant surgeons from the Department of Orthopedics in five hospitals, participated in this study. They were divided into three subgroups based on their prior clinical experience.

Significant differences, including those for dwell time, number of fixations, number of saccades, saccade duration, peak velocity of the saccade, and pupil entropy, were observed among the three subgroups. Additionally, there were clear trends in the perceived workload of the simulation-based training based on feedback from the participants.

Based on this preliminary study, a correlation was identified between the eye tracking metrics and participants’ experience levels. Hence, it is feasible to apply eye tracking data as a supplementary objective assessment tool to benchmark the technical proficiency of surgical trainees in hip arthroscopy, and enhance simulation-based training.

Chest Tube Insertion (CTI) should be trained in simulated settings prior to patient contact. Feedback and certification is based on valid assessments, especially in simulation-based training. This study aimed to develop a novel assessment tool for CTI and to ensure content validity based on expert opinion collected through a structured Delphi study.

A diverse European expert panel was invited to participate. In round 1, the experts provided at least five procedural steps and three errors involved in CTI. Round 2 evaluated the level of agreement with the inclusion of each item in the assessment tool on a five-point Likert scale. In round 3, experts rated their agreement on inclusion of the procedural step with its descriptive anchors. Consensus was reached when ≥80% of participants agreed on an item's inclusion.

Thirty-six of 105 (34%) invited surgeons (26/75, 35%), pulmonologists (8/23, 35%) and emergency physicians (2/7, 29%) participated. The overall response rate was 81% (29/36): 100% (36/36) in round 1, 83% (30/36) in round 2, and 97% (29/30) in round 3. Round 1 resulted in 23 steps and 44 errors after condensation and removal of duplicates. In round 2 consensus was achieved for 15 steps (65%) and 14 errors (32%). Nineteen steps were adapted into a rating scale with descriptive anchors and a list of 16 errors was presented to the panel. In round 3, experts reached consensus on the inclusion of 17 procedural steps (89%) with descriptive anchors and on all 16 errors.

A multidisciplinary expert panel achieved consensus in the development of the ACTION (Assessment of Competence in Chest Tube Insertion) tool. This procedure-specific rating scale of 17 steps, supplemented with a checklist of 16 errors, requires further research to collect validity evidence.

It is hypothesised that simulation enhances progression along the initial phase of the surgical learning curve.

To evaluate whether residents undergoing additional simulation, compared to conventional training, are able to achieve proficiency sooner with better patient outcomes.

This international, multicentre, randomised controlled trial recruited 94 urology residents with experience of zero to ten procedures and no prior exposure to simulation in ureterorenoscopy, selected as an index procedure.

Participants were randomised to simulation or conventional operating room training, as is the current standard globally, and followed for 25 procedures or over 18 mo.

The number of procedures required to achieve proficiency, defined as achieving a score of ≥28 on the Objective Structured Assessment of Technical Skill (OSATS) scale over three consecutive operations, was measured. Surgical complications were evaluated as a key secondary outcome. This trial is registered at www.isrctn.com as ISCRTN 12260261.

A total of 1140 cases were performed by 65 participants, with proficiency achieved by 21 simulation and 18 conventional participants over a median of eight and nine procedures, respectively (hazard ratio [HR] 1.41, 95% confidence interval [CI] 0.72–2.75). More participants reached proficiency in the simulation arm in flexible ureterorenoscopy, requiring a lower number of procedures (HR 0.89, 95% CI 0.39–2.02). Significant differences were observed in overall comparison of OSATS scores between the groups (mean difference 1.42, 95% CI 0.91–1.92; p < 0.001), with fewer total complications (15 vs 37; p = 0.003) and ureteric injuries (3 vs 9; p < 0.001) in the simulation group.

Although the number of procedures required to reach proficiency was similar, simulation-based training led to higher overall proficiency scores than for conventional training. Fewer procedures were required to achieve proficiency in the complex form of the index procedure, with fewer serious complications overall.

This study investigated the effect of simulation training in junior surgeons and found that it may improve performance in real operating settings and reduce surgical complications for complex procedures.

Surgical simulation is particularly attractive because it allows training in a safe, controlled, and standardized environment. However, the status of surgical simulation among Departments of Surgery (DoS) in the United States is unknown. The objective of this study was to characterize the status of simulation-based training in DoS in the United States.

A Qualtrics online survey was sent to 177 chairs of DoS in the United States in March 2018 regarding the utilization of surgical simulation in their department. Questions in the survey were focused on simulation capacities and activities as well as chairs' perception of the value and purpose of simulation.

A total of 87 of 177 chairs responded to the survey (49% response rate). Most programs had either 20-50 trainees (42 of 87; 48%) or more than 50 trainees (37 of 87; 43%). Most chairs reported having a simulation center in their institution (85 of 87; 98%) or department (60 of 86; 70%) with a formal simulation curriculum for their trainees (83 of 87; 95%). Ninety percent (78 of 87) of DoS had protected time for simulation education for their residents, with most residents engaging in activities weekly or monthly (65 of 85; 76%). Although most chairs felt simulation improves patient safety (72 of 84; 86%) and is useful for practicing surgeons (68 of 84; 81%), only 40% reported that faculty use simulation to maintain technical skills and only 17% reported that faculty use simulation to address high complication rates.

The vast majority of the DoS in the United States have established simulation activities for their trainees. However, engagement of faculty in simulation to maintain or improve their skills remains low.