First steps towards international competency goals for residency training: a qualitative comparison of 3 regional standards in anesthesiology

Article type: Research Article

Keywords: Residency, Curriculum, Clinical competences, Education, Standards, Anesthesia, Competency based education

Authors: Clément Buléon, Reuben Eng, Jenny W. Rudolph, Rebecca D. Minehart

Affiliations: grid.411149.80000 0004 0472 0160Department of Anesthesiology, Intensive Care and Perioperative Medicine, Caen Normandy University Hospital, 6th Floor, Caen, France; grid.412043.00000 0001 2186 4076Medical School, University of Caen Normandy, Caen, France; grid.419998.40000 0004 0452 5971Center for Medical Simulation, Boston, MA USA; grid.460737.10000 0004 0633 4861Department of Anesthesia, Rockyview General Hospital, Calgary, AB Canada; grid.22072.350000 0004 1936 7697Department of Anesthesiology, Perioperative and Pain Medicine, University of Calgary, Calgary, AB Canada; grid.32224.350000 0004 0386 9924Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Boston, MA USA; grid.38142.3c000000041936754XHarvard Medical School, Boston, MA USA

License: © The Author(s) 2021 CC BY 4.0 Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Article links: DOI: 10.1186/s12909-021-03007-w  |  PubMed: 34758815  |  PMC: PMC8582177

Relevance: Moderate: mentioned 3+ times in text

Full text: PDF (1.3 MB)

Background

Competency-based medical education (CBME) has revolutionized approaches to training by making expectations more concrete, visible, and relevant for trainees [ref. 1, ref. 2]. Yet overseeing a residency training program requires program directors, faculty and support staff, and institutions to nimbly adapt curricula to the ever-changing criteria for clinical excellence and competency-based medical education [ref. 1, ref. 3]. The volume of medical knowledge approximately doubles in size every few months [ref. 4], so CBME requirements may change frequently and demand regular updates to account for these advancements [ref. 5]. The work to updating residency requirements is often performed at a national level, to homogenize some aspects of training, but the burden on institutional education programs to translate these guidelines is heavy and investments by volunteer committees may be uneven. The result is incomplete or patchy national and international diffusion of evidence-based practices at the residency training level in any individual country [ref. 6].

By matching the large-scale challenge of adapting curriculum through global processes, we can optimize resource management since all countries need to incorporate advancements on the same medical topics. But to do this, we need to have sufficiently common requirements for our training. This ambitious view mandates that we start with map of the current requirements for training. Outside of cardiac arrest resuscitation guidelines from the International Liaison Committee on Resuscitation [ref. 7, ref. 8], we have few examples of shared competency training goals that are created between countries. A few recent studies have compared international training structures [ref. 9, ref. 10], but none has considered individual competencies.

With this study we aimed to compare different regions’ requirements to create a baseline map of existing anesthesia program requirements. This could be the starting point for creating a shared set of competencies for future internationally agreed-upon standards for anesthesia or other specialties’ training programs. We hope to demonstrate that differences may be mapped in a way that allows for economies of effort and crowdsourcing to accelerate innovative educational design and reduce time wasted reinventing curricula. We used anesthesiology residency as a feasibility test for this conceptual approach.

Methods

This study conforms to standards for reporting qualitative research (SRQR) [ref. 11]. Due to the nature of the study, Institutional Review Board involvement was not required.

Context

During fellowship training of one researcher (CB), three researchers (CB, RE, and RDM) met and began discussions on the similarities and differences in our respective residency programs in EU (CB), Canada (RE), and US (RDM). We developed a collaborative research approach to compare published formal requirements quantitatively and qualitatively within those programs.

Sampling strategy

We first identified each region’s current governance setting standards for anesthesiology residency education, which included: the European Board and Section of Anesthesiology working under the auspices of the European Union of Medical Specialists (UEMS) [ref. 12]; the Accreditation Council for Graduate Medical Education (ACGME) for the United States; and the Royal College of Physicians and Surgeons of Canada. We reviewed available metrics for anesthesiology residency educational assessments published by these governing bodies, following White and Marsh’s recommendations for choosing text to analyze [ref. 13]. Sources for consideration included the European Training Requirement (ETR) in Anesthesiology [ref. 14], the ACGME Program Requirements and the ACGME Milestones for US [ref. 15], and the Anesthesiology Competencies within Competence by Design (CBD) for Canada [ref. 16]. As the goal was to assess and compare published competencies, the ETR, the ACGME Milestones, and the Canadian Anesthesiology Competencies were chosen for comparison because they were most uniform in meeting our predefined criteria.

Repository descriptions

The European training requirement (ETR) [14]

ETR were produced by the European Committee on Education and Professional Development of the Section and Board of Anesthesiology. At the time of this writing, the ETR were intended as a shared repository for all countries in the EU training anesthesiology residents. The latest version was dated February 2018. The scope of ETR was to offer “a comprehensive and robust overall training framework created by medical specialists and based on assembled EU-wide educational and training experience.” [ref. 14] Among the ETR objectives were facilitating professional mobility between European countries and promoting safe and quality care. ETR were not mandatory in EU countries, although the European Board and Section of Anesthesiology supported their adoption [ref. 17]. Nevertheless, repositories and certifications in EU’s countries were based upon or generally approximate ETR, though there were still some heterogeneity between European countries’ programs [ref. 18, ref. 19]. Objectives of the ETR were part of a global framework with four generic roles: clinical expert, professional leader, academic scholar and inspired humanitarian. ETR consist of objectives, organized across 16 headlines (Supplemental Digital Content 1), which themselves belonged to two domains of (i) general competencies and (ii) specific core competencies. Each objective includes goals regarding knowledge, clinical skills and specific attitudes. The clinical skills comprised 165 items and the aimed mastery follows four grades of recommendations: (A) observer level (has knowledge of, describes); (B) performs, manages, demonstrates under direct supervision; (C) performs, manages, demonstrates under distant supervision; and (D) performs, manages, demonstrates independently. Depending on complexity of the skills or items, residents were expected to achieve anywhere from B (e.g. “Management of nerve blocks in pain therapy” or “Management of organ donors in Intensive care and during organ retrieval”), C (e.g. “Performing anesthesiology for kidney transplantation” or “Double lumen tracheal intubation”) or D grades (e.g. “Management of severe peri-partum hemorrhage” and “Management of difficult and delayed extubation after airway interventions”), with concessions given for rarer events.

The United States ACGME milestones [15]

The ACGME Milestones, introduced in 2013 but applied to anesthesiology in 2015, attempted to expand upon the six Core Competencies defined by the ACGME and the American Board of Medical Specialties which were Professionalism, Patient Care, Medical Knowledge, Practice-Based Learning and Improvement, Interpersonal and Communication Skills, and Systems-based Practice (Headlines in Supplemental Digital Content 2) [ref. 20]. The Milestones also intended to formalize the observations expected of residents within each of these six Core Competencies, driving residencies to teach and be assessed by how successfully their trainees met Milestones. Each specialty training programs’ Milestones were developed by experts within their specialty and varied in number of Milestones per specialty. As of 2020, anesthesiology comprised 25 Milestones. Residents attained one of five levels of achievement within each Milestone with clear behavioral definitions and anchors [ref. 15].

The Canadian CBD anesthesiology competencies [Entrustable professional activities (EPAs)] [16]

When this study was undertaken, two documents described the standards of achievement that are expected of Canadian anesthesiology residents when they were conferred fellowship in the Royal College of Physicians and Surgeons of Canada [ref. 16, ref. 21]. The EPA guide summarized which EPAs residents were expected to achieve during their residency training (Headlines in Supplemental Digital Content 3). EPAs were clinical tasks which residents could perform with minimal or no supervision (i.e., the task can be entrusted to them to complete); these were considered the minimal standard of achievement expected at each stage of residency training. The Anesthesiology Competencies document provided a comprehensive description of the specialty of anesthesiology in Canada, and described what residents should aspire to achieve over their residency program [ref. 21]. The Anesthesiology Competencies were categorized in accordance with the CanMEDS competencies (i.e. Medical Expert, Communicator, Collaborator, Leader, Health Advocate, Scholar, Professional). EPAs were assessed with a 5-point scale of entrustability.

Qualitative approach and research paradigm

From April to December 2019, we (CB, RE, and RDM) applied content analysis methodology [ref. 13, ref. 22] and considered each of the training requirement repositories as data sources individually situated within an important cultural context, a distinction highlighted by Ratner [ref. 23] and uniquely suited to qualitative comparison as it allowed interpreting these training competencies through a sociocultural lens (pragmatic paradigm) [ref. 24, ref. 25]. As our goal was to define similarities and differences between training expectations, we defined that each individually-numbered competency within any given repository would be considered the unit of data for comparison, which would be compared with all other competencies within the other two repositories.

Data collection methods, processing and analysis

The three researchers (CB, RE, and RDM) entered each of the ETR, US Milestones, and CBD EPAs into a Microsoft Excel spreadsheet (version 2019) to serve as each region’s repository of competencies. Each competency item was reviewed by the investigator representing that region and compared with the other regions’ competencies to determine congruence. English versions of published competencies were available and were used for comparisons; all authors were fluent in English. As cultural influences were critical for analysis, we together discussed interpretative nuances of our representative region’s requirements and considered how anesthesiologists may have functioned in their scopes of practice in each of EU, US, and Canada. To avoid bias, each round consisted of separate and independent investigator review, followed by comparing and merging results, with discrepancies resolved through discussion and consensus, determining intersubjectivity [ref. 26]. For the first round of analysis, we identified whether each competency item was represented in either of the other repositories. The ETR was chosen as the reference comparison here, given that it had the greatest number of competencies. The second round of analysis sought to determine the relative importance or emphasis of specific skill sets in each country’s repository, using each repository as a reference for the other two to ensure full consideration of all competencies. Competency items were identified as having equal or different levels of importance, or “emphasis” between repositories based on how they were presented within the repository (i.e., competency items which were singled out and treated in-depth as unique competencies were interpreted as more emphasized than ones which were only briefly mentioned). For first round analysis, we sought to include rather than exclude, such that if a competency did not explicitly state an action, yet it could reasonably be included within the scope of the competency, common representation was considered between competencies. However, we applied stricter definitions whether domains were emphasized equally, and cultural context was considered more heavily. We used descriptive statistics to quantitatively present congruency and emphasis of competencies between the repositories.

Researcher characteristics and reflexivity

Three investigators were practicing anesthesiologists having completed training in their respective regions: CB in EU (France), RE in Canada, and RDM in the US. All had domain expertise in education, including educational fellowships and advanced degrees. All three had served on their residency program’s clinical competency committees (either past or current). Two researchers were current or former residency or fellowship program directors (RE, RDM), and all actively taught anesthesiology residents at the time of data analysis.

Techniques to enhance trustworthiness

Rigor was maintained by investigator independence at each stage, coupled with demonstrating intersubjectivity through discussion. Consensus was achieved universally between researchers through these discussions, with minimal instances of disagreement. In addition, external expert reviewers were invited to critique our results prior to submission (see Acknowledgements), lending credibility to our process. Based on their feedback, minor changes and correction were performed.

Results

Round 1: the broad view of shared competencies

Comparisons of ETR, US Milestones, and CBD EPAs competencies for anesthesiology resident training showed congruence of 93% (Fig. 1). All CBD EPAs and US Milestones’ competencies were present in the ETR. ETR competencies were represented in 98% of CBD EPAs and in 95% of US Milestones. CBD EPAs competencies were represented in 98% of US Milestones. US Milestones competencies were present in 96% of CBD EPAs. Table 1 summarizes the main results for the matching between ETR, US Milestones, and CBD EPAs’ competencies. Detailed results of the first round of comparisons of the three repositories’ competencies are presented in Appendix 1.

Differences between repositories

Most of the difference between ETR, US Milestones, and CBD EPAs’ competencies centered on critical emergency medicine (including pre-hospital and emergency medicine). This subject was completely absent from US Milestones. CBD EPAs’ competencies covered a similar field to the ETR for critical emergency medicine, except for “assisting in rescue work” and “declaration of death at the scene of emergency,” which were present only in the ETR. Skills in ultrasound and Perioperative patient positioning avoiding tissue damage were common for CBD EPAs and ETR but were absent from US Milestones. Three competencies were common to ETR and US Milestones and absent from CBD EPAs were: (1) Management of brain death syndrome and donor management including explanation; (2) Positioning of patients with specific pathological conditions; and (3) Responsibility to maintain personal emotional, physical, and mental health.

Round 2: emphasis of certain competencies by ETR, milestones, and CBD

Even if some competencies were cited in every repository, we did not determine all competencies had similar relative importance nor emphasis. Table 2 depicts which competencies were interpreted to have an equal expression among the three repositories, and which were emphasized equally by two or mainly by only one repository. This differential emphasis was demonstrated by comparing unique competencies such as “Promoting safety and well-being of staff” (ETR, 1.6) and “Responsibility to maintain personal, emotional, physical and mental health” (Milestones, P5) with how well-being was represented in CBD, which was dispersed within Core EPA #24, TTP #2, TTP #3, and TTP #5. Based on this deeper analysis, Venn diagrams (Fig. 2) depict the respective emphasis from each perspective (Europe, US, and Canada). More than 30 % of the competencies – whatever repository was taken as reference – had the same importance. ETR had only one (3%) competency that was specifically emphasized, Anaesthesia Non-Technical Skills (ANTS) [ref. 27]. Both CBD EPAs and US Milestones had more unique competencies emphasized. CBD EPAs focused on specific anesthesiology situations highlighting more granular competencies (e.g., point of care ultrasound skills, complex cases). Those competencies were included more generically for ETR and US Milestones, but without a high degree of detail. Professionalism and behavioral practices were emphasized by US Milestones (e.g., analysis of practice, education, communication).

Discussion

This comparison provides a glimpse into common training goals shared among Europe, the US and Canada, using anesthesiology as the example discipline. We identified a high overlap rate of educational objectives (93%). The core competencies appear generally consistent as shown in Table 2 (e.g., perioperative anesthetic plan, management, conduct, and monitoring; assessing, diagnosing and managing critically ill patients in acute care settings; airway management; and honesty, integrity, and ethical behavior, among others).

Differences on specific competencies emphasized are nuanced and likely have historical roots. For example, emergency medicine and prehospital medicine were initiated by anesthesiologists in some European countries, and anesthesiologists still play a large role in caring for prehospital patients despite emergency medicine internationally developing as a separate specialty [ref. 28–ref. 30]. Consideration is being given in the US for anesthesiologists to fill this gap via training in emergency medicine to care for critical patients [ref. 31]. A heavy societal focus on professionalism for US trainees likely has led to more defined behavioral guidelines regarding professional care, generating this unique emphasis in the US-based Milestones [ref. 32]. The large overlap identified in educational objectives with limited differences confirm that training appears very similar between regions and that anesthesiologists can work together, both at educational and professional levels. An important consideration for future work will be the differences in competencies required to practice in lower resourced or austere settings. The skills and knowledge required for clinicians practicing in wealthy, developed countries may not be the ones needed in developing countries. Understanding these differences among residency programs in developed nations highlighted in our study, and exploring the hypotheses stated above, should be topics for future investigation.

There are several reasons why this work has important potential. As countries strive to build better healthcare professionals through competency-based teaching and assessment, cross-cultural dialogue between international educational societies governing these competencies may enable effectively sharing best educational and clinical practices. For example, recently the Royal College of Anaesthetists in the UK adopted a curriculum update, entitled the 2021 Anaesthetics Curriculum, which included cultural values of diversity, inclusion, and respectful interactions between team members [ref. 33], and the Milestones were updated to include assessment of point-of-care ultrasound [ref. 34]. These should serve as exemplars for future updates to all repositories in anesthesiology. If and when this expands to countries with different cultural norms or different levels of technology and resources, other adaptations may need to be made. As regions like the EU, US, and Canada revise their CBME curricula and assessment tools, reviewing other countries’ successes and challenges in adopting changes will allow shared insights to be efficiently incorporated [ref. 33]. Thus, an educational global mindset may increase adaptation of new practices. In addition, best educational practices for adopting new techniques can offer faculty opportunities to refresh or gain new skills without reinventing educational approaches, saving expert time and organizational efforts.

Additionally, these requirements’ complete revision overburdens program directors, providing a substantial source of their burnout [ref. 35, ref. 36]. By focusing on similarities, the workload of national and regional competency assessment efforts could be reduced, freeing program directors and educators to mentor trainees more effectively and adapt requirements for teaching using a regional lens. Engaging in such collaboration may facilitate other large-scale partnerships, such as subspecialty educational groups, to advance the educational sciences of teaching and learning in residency.

Many countries are in the process of developing more healthcare provider training programs or improving existing ones. By highlighting overlaps in competencies between CBME residency programs around the world, an international consensus of disciplines’ competencies may support creation of new training programs. Because best educational practices would be shared through international collaborations, this could facilitate training even in low-resource settings. Established training programs can also benefit from global innovations by identifying and closing their own educational gaps. Working towards a shared repository could be seen as investing time and energy in reciprocal learning, though everyone will gain from such a repository in distinct ways.

Limitations

Our study has limitations that need highlighting. We focused on anesthesiology requirements in three wealthy, developed regions to illustrate our concept as the investigators are anesthesiologists. Generalizing to other specialties requires the existence of formally published competency repositories as a first step of many. Examining the process of creating competency repositories may serve as a preliminary step to unifying existing repositories, which could be aided by shared terminology and competency selection standards. We focused on a restricted area (EU, US, and Canada) as the investigators came from those areas (convenience sample). We would need to overcome difficulties to extend our analysis to other regions’ repositories. First, not every area has an easily accessible recent English version (or other shared language version) of their competencies, introducing language translation and interpretation challenges. Second, integrating and comparing more than three competencies would have been technically arduous. Nevertheless, we offer a robust methodology in comparing CBME training applicable across disciplines in healthcare.

The qualitative characteristic of this study may have limited the objectivity of the comparisons. To reduce this, we have had extensive discussion on the analysis for each item among the three researchers and data are available for consultation (Appendix 1). We also had an external independent review of the results.

The US and Canada have wide national application of their competencies. In EU, despite intentions to harmonize anesthesiology training [ref. 19], ETR adoption in its current form is less clear [ref. 18], and active complementary national ones coexist (e.g., UK or Denmark) [ref. 37, ref. 38]. It is unclear how ETR have influenced the development of other EU national programs, as differences currently exist between length of training in EU countries, which may also lead to differential training, assessment, and regulation within the EU [ref. 18]. Gaps between these programs are not defined and only conjectural, as interpretation of competencies can vary by different program directors, specifically in EU where ETR application may differ from country to country. Furthermore, a final limitation remains that despite significant overlap in competencies, uniting accreditation practices between these three regions is likely to encounter barriers related to political and societal considerations.

Perspectives and future directions

With now-defined competencies of CBME anesthesiology training programs for EU, US, and Canada, we call upon educational leaders from international societies to further develop competency standards. Basic technical skills (e.g. epidural, central line placement) and common competencies from Table 2 “Competencies equally represented for ETR, Milestones, CBD” (e.g. regional anesthesia, airway management) should benefit from developing common training programs which can become international standards. Future work should focus on determining the most effective teaching and assessment methodologies for achieving these competencies. Another exciting area of further development is using shared tools for competency assessment for licensure and certification, enabling anesthesiologists to temporarily practice in other countries during disasters.

Conclusions

With this qualitative comparison, we were able to make a map discerning a baseline of similar competencies between published anesthesiology residency training competencies among EU, US, and Canada. Our approach also highlighted unique regional differences which appeared to be based on importance and approach rather than on fundamental content. Together, these serve as learning opportunities to explore. With over 90% overlap, the anesthesiology regional requirements we compared have enough in common to serve as a springboard to develop a common core of residency requirements in anesthesiology. This conceptual approach is demonstrated to be feasible and may be applied to determine a baseline from which to build an international core of residency competencies required by other disciplines.

References

1. WF Iobst, J Sherbino, OT Cate, DL Richardson, D Dath, SR Swing. Competency-based medical education in postgraduate medical education. Med Teach, 2010. [DOI | PubMed]
2. TJ Nasca, I Philibert, T Brigham, TC Flynn. The next GME accreditation system — rationale and benefits. N Engl J Med, 2012. [DOI | PubMed]
3. TJ Ebert, CA Fox. Competency-based education in anesthesiology: history and challenges. Anesthesiology., 2014. [DOI | PubMed]
4. P Densen. Challenges and opportunities facing medical education. Trans Am Clin Climatol Assoc, 2011. [PubMed]
5. O Ten Cate. Competency-based postgraduate medical education: past, present and future. GMS J Med Educ, 2017. [PubMed]
6. S Kietaibl, A Blank, E De Robertis. Medical training in anaesthesiology: updated European requirements. Eur J Anaesthesiol, 2019. [DOI | PubMed]
7. 7.International Liaison Committee on Resuscitation. ILCOR: Mission & Constitution. https://www.ilcor.org/about/mission-constitution. Accessed 5 May 2021.
8. RM Merchant, AA Topjian, AR Panchal, A Cheng, K Aziz, KM Berg. Part 1: executive summary: 2020 American Heart Association guidelines for cardiopulmonary resuscitation and emergency cardiovascular care. Circulation, 2020. [PubMed]
9. R Vinagre, P Tanaka, MA Tardelli. Competency-based anesthesiology teaching: comparison of programs in Brazil, Canada and the United States. Braz J Anesthesiol (English Edition), 2021. [DOI]
10. 10.Yamamoto S, Tanaka P, Madsen MV, Macario A. Comparing anesthesiology residency training structure and requirements in seven different countries on three continents. Cureus. 9. 10.7759/cureus.1060.
11. BC O’Brien, IB Harris, TJ Beckman, DA Reed, DA Cook. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med, 2014. [DOI | PubMed]
12. 12.European Union of Medical Specialists. UEMS National Member Countries. https://www.eusem.org/images/List_of_UEMS_Members.pdf. Accessed 25 Jun 2020.
13. MD White, EE Marsh. Content analysis: a flexible methodology. Libr Trends, 2006. [DOI]
14. 14.Committee on Education and Professional Development (EPD) of the Section and Board of Anesthesiology. UEMS – European Training Requirement ETR in Anaesthesiology. Union Européenne des Médecins Spécialistes. 2018. https://www.uems.eu/__data/assets/pdf_file/0003/64398/UEMS-2018.17-European-Training-Requirements-in-Anaesthesiology.pdf. Accessed 19 Dec 2018.
15. 15.Accreditation Council for Graduate Medical Education (ACGME), American Board of Anesthesiology (ABA)The anesthesiology milestones project2015
16. 16.Royal College of Physicians and Surgeons of Canada. Requesting Copies of EPA Guides and CBD Observation Templates. Royal College of Physicians and Surgeons of Canada. 2020. http://www.royalcollege.ca/rcsite/documents/cbd/epa-observation-request-e. Accessed 28 Sep 2020.
17. C Carlsson, D Keld, E van Gessel, JPH Fee, H van Aken. Education and training in anaesthesia–revised guidelines by the European Board of Anaesthesiology, reanimation and intensive care. Eur J Anaesthesiol, 2008. [DOI | PubMed]
18. G Jonker, LA Manders, AP Marty, CJ Kalkman, TJ Ten Cate, EF van Gessel. Variations in assessment and certification in postgraduate anaesthesia training: a European survey. Br J Anaesth, 2017. [DOI | PubMed]
19. EF Van Gessel, HT Ostergard, L Niemi-Murola. Harmonisation of anaesthesiology training in Europe. Best Pract Res Clin Anaesthesiol, 2012. [DOI | PubMed]
20. 20.Warm EJ, Edgar L, Kelleher M, Kinnear B, Sall D, Luciano G, et al. A guidebook for implementing and changing assessment in the milestones era. Accreditation Council for Graduate Medical Education; 2020. https://www.acgme.org/Portals/0/Milestones%20Implementation%202020.pdf?ver=2020-05-20-152402-013. Accessed 7 Aug 2020.
21. 21.The Royal College of Physicians and Surgeons of Canada. Anesthesiology Competencies. Royal College of Physicians and Surgeons of Canada. 2017. http://www.royalcollege.ca/rcsite/documents/ibd/anesthesiology-competencies-e. Accessed 28 Sep 2020.
22. K Krippendorff. Content analysis: an introduction to its methodology, 2018
23. 23.Ratner C. Subjectivity and objectivity in qualitative methodology. Forum Qual Sozialforschung / Forum: Qual Soc Res. 2002;3. 10.17169/fqs-3.3.829.
24. C Pope, N Mays. Reaching the parts other methods cannot reach: an introduction to qualitative methods in health and health services research. BMJ., 1995. [DOI | PubMed]
25. C Kivunja, AB Kuyini. Understanding and applying research paradigms in educational contexts. Int J Higher Educ, 2017. [DOI]
26. MP Unger. Intersubjectivity, hermeneutics, and the production of knowledge in qualitative Mennonite scholarship. Int J Qual Methods, 2005. [DOI]
27. R Flin, R Patey, R Glavin, N Maran. Anaesthetists’ non-technical skills. Br J Anaesth, 2010. [DOI | PubMed]
28. SJ Schaller, FP Kappler, C Hofberger, J Sattler, R Wagner, G Schneider. Differences in pain treatment between surgeons and anaesthesiologists in a physician staffed prehospital emergency medical service: a retrospective cohort analysis. BMC Anesthesiol, 2019. [DOI | PubMed]
29. P Hilbert-Carius, MF Struck, V Hofer, J Hinkelbein, L Rognås, J Adler. Mechanical ventilation of patients in helicopter emergency medical service transport: an international survey. Scand J Trauma Resusc Emerg Med, 2020. [DOI | PubMed]
30. E De Robertis, J McAdoo, R Pagni, JTA Knape. Core curriculum in emergency medicine integrated in the specialty of anaesthesiology. Eur J Anaesthesiol, 2007. [DOI | PubMed]
31. M McCunn, RP Dutton, A Dagal, AJ Varon, O Kaslow, CJ Kucik. Trauma, critical care, and emergency care anesthesiology: a new paradigm for the “acute care” anesthesiologist?. Anesth Analg, 2015. [DOI | PubMed]
32. SR Swing. The ACGME outcome project: retrospective and prospective. Med Teach, 2007. [DOI | PubMed]
33. 33.Solymos O, Snyman L, Condon E, Power C, Boland J. Moving beyond the technical skills and promoting professionalism-the experience of the College of Anaesthesiologists of Ireland with incorporating the medical council eight domains of good professional practice into Entrustable professional activities. Ir J Med Sci. 2020;189:1379–89.
34. 34.Accreditation Council for Graduate Medical Education (ACGME). Anesthesiology Milestones. Accreditation Council for Graduate Medical Education (ACGME); 2020. https://www.acgme.org/Portals/0/PDFs/Milestones/AnesthesiologyMilestones2.0.pdf?ver=2020-12-02-125500-287. Accessed 12 Apr 2021.
35. GS De Oliveira, MD Almeida, S Ahmad, PC Fitzgerald, RJ McCarthy. Anesthesiology residency program director burnout. J Clin Anesth, 2011. [DOI | PubMed]
36. KE Fletcher, AB O’Connor, M Kisielewski, LL Willett. Why do residency program directors consider resigning? A mixed-methods analysis of a National Program Director Survey. Am J Med, 2020. [DOI | PubMed]
37. 37.The Royal College of Anaesthetists. 2021 Anaesthetics curriculum. The Royal College of Anaesthetists. 2021. https://rcoa.ac.uk/training-careers/training-anaesthesia/2021-anaesthetics-curriculum. Accessed 30 Aug 2020.
38. 38.Danish Health and Medicines AuthorityThe seven roles of physicians2014