Current approaches to clinical research with respect to esophageal resection: are online clinical datasets the future?

Introduction

What are the modern era options for performing surgical clinical research?

It must be acknowledged that carrying out clinical research in association with a busy surgical practice is time consuming, complex logistically and takes time away from other professional and social activities. It is, however, a recognized venue for contributing to your profession as well as an opportunity to improve the process and outcomes of current surgical therapeutics. It allows surgeons to contribute to regional, national and international meetings and importantly allows individual surgeons to expand professional and academic contacts. Many of the most recognized surgical international centers have established their reputation on the basis of clinical excellence but also by making regular and substantial contributions to the surgical literature.

The avenues for carrying out clinical research studies within surgery are diverse. The hallmark of level I evidence as outlined in the 1988 United States Preventive Services Task Force (1) is the randomized controlled clinical trial (RCT). The advantages of the RCT include a study constructed to minimize bias and provide matched study populations. They routinely require IRB and ethics review as well as study supervision and also typically involve professional study coordinators. They are easy to publish but complex to initiate. Enrolling patients is time intensive and requires a specially constructed infrastructure for adequately informing patients and obtaining informed consent. They are often costly to initiate and many require funding from national agencies or industry.

The realities of randomized control trials are that not all clinical assessments are appropriate and that many trials initiated by industry and pharma are not published when they demonstrate a negative or equivocal outcome. Massarweh (2) highlighted that many RCT’s have exclusion criteria that limit external validity and may create blind spots. In additional, standardization of procedural interventions in randomized trials can be challenging and therefore ultimately may not reflect true clinic practice. As a result, well-designed and conducted observational studies can sometimes provide data that can better characterize the outcome of an intervention when applied to patients in real world conditions.

Other vehicles for clinical research have historically centered on the assessment of longitudinal single institutional datasets. These datasets typically involve high volumes of patients and can represent real world outcomes within individual surgical units. Disadvantages have historically included the fact that single institution datasets are accumulated over long periods of time which typically encompass’ major changes in surgical and oncologic practice as well as surgical service delivery that cannot be routinely factored into outcomes assessment.

In recent years, one of the most common venues for reporting clinical outcomes have been the sub-analysis of national and professional administrative databases. The advantage of this approach to clinical research includes the opportunity to analyze large numbers of patients from preexisting datasets which often provides a reasonable framework for documenting practice patterns and clinical trends. Other advantages would include the fact that these sub-analyses are fairly straightforward to publish in the modern era. Disadvantages of utilizing administrative datasets for surgical research include the reality that these databases were not necessarily developed to provide specific granular answers to specific clinical research questions. Many of these datasets collect data on focused areas of the overall population for example the Surveillance, Epidemiology and End Results Program SEER (3) which reports on approximately 34% of the US population. Other examples include professional or society based datasets such as The Society of Thoracic Surgeons database (4) which reports data predominantly originating in thoracic surgical units. The greatest significant drawback to the majority of publications from administrative datasets is that a large number of these assessments are currently being produced due to increasingly straightforward access to these datasets. Unfortunately, many of these publications produce information which is of intellectual interest but not typically of clinical utility.

Another frequently applied approach to surgical research is the meta-analysis and systematic review. These assessments amalgamate the collective experience within the literature, and when the literature is mature and composed of well-construction randomized or prospective clinical reviews, they will typically produce important statements regarding process and clinical outcomes. The drawbacks to meta-analysis and systematic reviews include the fact that because they are typically fairly easy to publish, many are being produced before the literature is mature and in a position to facilitate a meaningful assessment. These assessments can also be open to bias due to the variability of information within studies on the relevant patient population and the fact that inclusion and exclusion criteria can make these assessments vulnerable to selection bias.

The opportunity for carrying out surgical clinical research are extensive. As outlined above, research has historically involved secure institutional or national datasets. However, with the increasing acceptance that the internet can provide a secure online repository for personal, financial and health-related information, there is now the opportunity to design “made for purpose” datasets which reside on the internet and can be used to confront specific clinical research projects and in fact to follow outcome and process trends over time. An example of one of these online datasets is Esodata.org which was developed to document the outcomes, including complications and process measures, associated with esophageal resection.

In 2011, the Esophageal Complications Consensus Group (ECCG) was formed, bringing together 21 high-volume international esophagectomy units to develop a standardized platform including definitions for assessing perioperative outcomes associated with esophageal resection. This “made for purpose” dataset was produced due to the recognition that outcome reporting for esophageal cancer surgery was heterogenous and inconsistent and that a core outcome set needed to be defined and utilized in future studies and national datasets to facilitate study comparisons and outcome analysis (5). The ECCG produced a standardized basic platform for reporting outcomes including definitions and quality measures that was published in 2015 (6). The Esodata dataset provided an online vehicle to beta test the new platform which led to the benchmarking of outcomes and complications which was published in 2017 (7).

The clear advantage of this approach was that unlike the analysis of national datasets Esodata.org was specifically designed to benchmark the complications platform developed by the ECCG. Because it included significant numbers of high-volume institutions, the reported outcomes reflected international and not just regional outcomes. It also facilitated the accumulation of a large number of patients over a short period of time, making the outcomes assessment contemporary and therefore less likely to reflect process or technical changes over time. The dataset could also be used by individual Esophagectomy units for their institution or national audits and because the dataset resides on the internet, data could be securely entered or reviewed anywhere that provided secure internet access.

Historical overview and current samples of Online Clinical Database

Medical record documentation and data collection have always been a fundamental part of clinical practice and medical knowledge evolution (8). The registry represents one of the earliest information systems and were introduced in ancient civilizations to monitor the population of specific geographical areas (i.e., birth or death registries). A registry is defined as “a file of documents containing uniform information about individual persons, collected in a systematic and comprehensive way, in order to serve a predetermined purpose.” (9). Since disease cannot be considered for registration purposes independent of the affected person, all medical registries which contain personal information regarding patients are named “clinical data registries” or “patient registries”.

A patient registry is defined as an organized system evaluating specific outcomes for a population defined by a particular disease, condition, or exposure, and that serves one or more predetermined scientific, clinical, or governmental policy purposes (10). In more recent times, registries of population or clinical databases have been established to monitor focused demographic populations. These datasets are usually designed with a variety of goals (11,12) but are typically associated with specific clinical societies or based on monitoring selected disease populations with the collected data targeted to address a variety of administrative or clinical objectives.

In 2017, the National Quality Registry Network (NQRN) group, a branch of the Physician Consortium for Performance Improvement convened by American Medical Association (AMA-convened PCPI) (13), conducted a national survey involving all medical specialty and health care professional societies and associations in the United States (152 organizations). This survey was meant to produce a comprehensive list of datasets and to collect information regarding the purpose of the dataset, type of data and collection method (14). This survey demonstrated that 38 organizations (52% of the responding institutions) operated a registry or database with a wide variety of goals and differences in data collection methodology (Table 1). This survey documented the increasing utilization for standard regional and national patient registries and datasets, within virtually every field of medical practice (11).

The most significant event associated with the evolution of medical records has been the introduction of Electronic Health Records (EHR). Theorized in the 1960s by Lawrence Weed MD and then initially developed by Lockheed Corporation, the EHR saw an initial implementation in the 1970s by the U.S. Government in the Department of Veteran Affairs (15). This patient record system eventually became the Veterans Health Information Systems and Technology Architecture (VistA), one of the largest healthcare information dataset amalgamating data from the hospital, ambulatory, pharmacy, and ancillary services for over 8 million U.S. veterans (16). This large dataset provided a unique infrastructure to conduct important quality studies such as the National Department of Veterans Affairs Surgical Risk Study (NVASRS). It also led to the initiation of the National Surgical Quality Improvement Project (NSQIP) by the American College of Surgeons (ACS), an ongoing program for monitoring and improving the quality of surgical care.

The NSQIP database is a voluntary non-disease specific national dataset run by the American College of Surgeons, which has been used as a resource to study many aspects of upper gastrointestinal surgery.

Other examples of national datasets currently utilized for clinical research in upper gastrointestinal disease in the USA include:

• National Inpatient Sample (NIS) (17).
Developed by the Agency for Healthcare Research and Quality (AHRQ) in 1988 and re-designed in 2012 for a more enhanced sampling strategy, NIS dataset is part of Healthcare Cost and Utilization Project (HCUP) and currently collects data from 47 states, and typically represents approximately 20% of inpatient hospitalizations in the United States (18). Data is collected on any disease-related patient discharged from U.S. community hospitals, excluding rehabilitation and long-term acute care hospitals. The primary goal of the dataset was to produce U.S. regional and national estimates of inpatient utilization, access, charges, quality, and outcomes.
• Surveillance, Epidemiology, and End Results (SEER) Database (3).
This database is the result of the SEER program initiated in 1971 after the U.S. Congress passed the National Cancer Act. SEER database collects and publishes data on cancer incidence and survival population-based cancer registries in 19 U.S. geographic areas, including metropolitan regions and special populations whose data are reported to their respective state registries funded by CDC’s National Program of Cancer Registries. Overall, SEER database covers approximately 34.6% of the U.S. population and predominately reports on outcomes in patients over 65 years of age.
• National Cancer Database (NCDB) (19).
Sponsored by the American College of Surgeons and the American Cancer Society, NCDB was established in 1989 and currently captures information on more than 70% of newly diagnosed cancer cases nationwide. Patient registries are collected from more than 1,500 Commission on Cancer (CoC)-accredited hospitals throughout the USA.
• Society of Thoracic Surgeon (STS) National Database (16).
This database was established in 1989 as an STS initiative for quality improvement and patient safety, collecting non disease specific clinical data from Cardiac and General Thoracic surgical units in 50 states in USA and to a limited degree other countries (20). Data on patients with UGI conditions is collected by the General Thoracic surgical database which is a voluntary database and institutions contributing data must pay an annual fee to participate.

The administrative datasets listed above were all developed to achieve different goals and were set up to collect data in different ways. The datasets within the individual database are very different but none of them was initially set up to address focused clinical issues. Computerization of medical records has also affected the methodology by which databases accumulate data. The difference between databases and registries has increased after the application of the digital technology to medicine (also known as e-Health), which has facilitated better systems for accessing, storing and sharing data on-line. This has inevitably led to concerns for protection of personal health information (PHI), and several requirements now are required to preserve patient privacy and data security, such as:

• Elaboration of high-technological systems, including both cryptographic and non-cryptographic approaches guaranteeing appropriate standards and certification criteria (21);
• Development in all western countries of a specific legislations promoting and regulating the information technology in healthcare organizations (22,23);
• Institution of regulatory bodies to oversee patient privacy issues on either a national or international level (24,25).

The opportunity to develop “for purpose” online datasets is a relatively new phenomenon. Some samples of online clinical databases are listed in Table 2. These datasets have been initiated by a variety of clinical societies or professional associations to specifically address focused clinical issues on a national or international level. The increased implementation of online clinical datasets allows the assessment of specific clinical issues by assessing both disease characteristics and treatment outcomes in a targeted group of patients. Nevertheless, similar to the NQRN survey, the wide variety of intended clinical purposes and more importantly the absence of standardized definitions often results in reports highlighting high demographic and geographic diversity.

Previous reviews have described the challenges when using data from large clinical databases, whose main concerns are about data quality but not the goals or the interpretation of the data by researchers mining the datasets (40). Data quality and completeness understandably remains the focus of many standard administrative datasets. However, the type of data required for focused research projects is variable and represents a moving target. The development and maintenance of “for purpose” online datasets facilitating the collection targeted patient information selected specifically for answering particular research questions will be an increasing valuable tool for clinical research moving forward.

Online datasets supporting UGI surgical research: Esodata.org

Esophagectomy has remained an outlier within major oncologic surgical procedures due to its persistently high levels of morbidity and mortality. Complications have been clearly demonstrated to affect virtually every major outcome measure following esophagectomy including length of hospital stay, costs of hospital care, patient-related quality of life and overall survival. In spite of their importance, there was no generally accepted system for documenting incidence and severity of perioperative morbidity associated with esophagectomy and the methodologies which were being applied were heterogeneous and without standardized definitions (5).

Due to this lack of standardization, in 2011 the Esophageal Complications Consensus Group (ECCG) was formed to bring together leading surgeons from high-volume institutions internationally in an attempt to standardize the reporting of outcomes and complications associated with esophageal resection. Through a series of Delphi surveys and face-to-face meetings, a standardized platform for reporting outcomes and complications was finalized. In addition, the ECCG proposed specific definitions for defining the severity and management of the four major surgical complications including anastomotic leak, conduit necrosis, chyle leak and recurrent nerve injury. The ECCG also produced a list of required quality measures that should direct the accumulation of data and highlight critical issues that needed to be collected for standardizing assessment of both clinical and oncologic outcomes (6).

With the completion and publication of the standardized platform in 2015 and working in conjunction with Dr. Madhan Kumar Kuppusamy, the ECCG went on to design and develop Esodata.org. This database was the first prospective online oncologic dataset to collect data on a specific major oncologic procedure using a standardized platform. The 21 international high-volume esophagectomy units representing 14 countries prospectively entered data over a 2-year period with the goal of accruing 1,500 resections over the study period. The efficiency of an online dataset was demonstrated when the ECCG centers enrolled over 2,700 resections during the study period and published a benchmark of outcomes and complications associated with esophagectomy in 2017 (7). This initial publication documented the fact that high-volume esophagectomy units could demonstrate a lower level of perioperative mortality than previously documented with 30- and 90-day mortality being demonstrated at 2.4% and 4.5%, respectively. Results demonstrated that at the time, open resections were more common than minimally invasive procedures (52.1% versus 47.9%, respectively) and that neoadjuvant chemoradiotherapy was more commonly utilized than neoadjuvant chemotherapy (46.1% versus 29.5%). Pertinently, the overall incidence of complications was found to be high, occurring in 59% of patients. The most common individual complications were pneumonia, atrial dysrhythmia and anastomotic leak at 14.6%, 14.5% and 11.4%, respectively. The incidence of severe complications was noteworthy with Clavien-Dindo complications ≥ IIIb documented in 17.2% of patients. The overall quality of the oncologic operations was seen to be high with a R0 resections being achieved in 93.4% of all operations. Readmissions were also seen to be common, occurring at 11.2% of patients with 77.6% of these patients experiencing an in-hospital complication prior to discharge.

The Esodata dataset provided a unique resource as it was designed as a secure online database providing a standardized “user-friendly” web interface including consensus-based data fields which included the definitions developed by the ECCG. This high-performance private web server was dedicated to host the clinical database whose web-based interface was accessible only via authenticated and encrypted secure network connections (SSL Client and Server Certificate with Extended Validation – issued Symantec Corporation). An open-sourced database architecture (MariaDB v10.1.21 by MariaDB Foundation) and an appropriate backup system (Drupal Content Management Software, distribution under the terms of the GNU General Public License) ensured data portability, analytics, modularity and flexibility in content access management (Figure 1). The database as developed also did not require major interactions with individual institutional computer systems. Participation in the ECCG study was voluntary, did not require any payments from contributing institutions and because the information was anonymized, adhered to all currently existing international privacy agreements.

Figure 1 Diagram of Esodata architecture.

Each individual institution was responsible for adhering to regional and institutional IRB and Ethics Committee Rules. The database was also designed so that it did not require any institutional IT support or software updates to institutional hard drives. Potentially most advantageous to contributing institutions is that registered users of the Esodata dataset had access to their institutional data anywhere they had secure internet access worldwide.

The publication of the 2017 ECCG benchmarks for perioperative outcomes and complications associated with esophagectomy provided for the first time a standardized method of comparison for national and institutional outcomes. More importantly, the ECCG standardized system provided a methodology of recording outcomes and complications internationally that would increase the relevance of not only institutional audits, but also national datasets as well as international randomized and prospective clinical trials. Providing this infrastructure also resulted in significant growth in the number of centers contributing to the Esodata dataset. The original 24 centers who formed the original ECCG expanded by the end of 2018 to include 40 centers that had applied for membership and subsequently began prospective entering patients into the Esodata dataset. This group of 40 centers now represents 19 individual countries, making the data collection truly representative of international practice (Figure 2).

Figure 2 Esodata member institutions and relative countries.

During this same period, the ECCG formalized a relationship with the International Society for Diseases of the Esophagus (ISDE) and the Research & Dataset Committee of the ISDE was formed of ECCG members to oversee specific issues involving New Research and Evolution of the Dataset, Publication & Audit of Esodata.org and Membership & Bylaws under individual subcommittees. The governance of the ECCG also evolved with the formation of the International Esodata Study Group (IESG) which became a 501(c)(Jeny3) charitable corporation in the United States to develop an infrastructure to facilitate future fundraising to support the evolution of the dataset and attempt to continue limit costs to the IESG members. The IESG, in conjunction with the ISDE, is currently working to facilitate the expansion of the current Esodata dataset to collect data to support future iterations of AJCC/UICC staging for esophageal cancer.

The ECCG had initially targeted three publications at the time of its formation. The first was to provide the standardized system for reporting outcomes, complications and quality measures as well as the standardized definitions for the key complications which was published in 2015 (6). The second publication was to produce the international benchmark of complications and outcomes that was published in 2017 (7). In 2020, the third of the original ECCG publications was completed, documenting the ability of the Esodata dataset to follow short-term trends in the technical evolution of esophageal resection as well as outcome and complication trends over time. This publication reported on over 6,000 esophagectomies done and entered in Esodata between 2015 and 2018, comparing the original 2,407 esophagectomies in 2015 and 2016 to the additional resections (3,319 esophagectomies) reported in 2017 and 2018.

This report from 40 international centers (Figure 2) highlighted important demographic and outcome trends which included the fact that minimally invasive procedures has become the most common operative approach compared to open procedures (52.8% versus 47.2%, respectively). In addition, 53.1% of the minimally invasive procedures were totally minimally invasive operations. Important demographic changes noted over time included an increase in the application of neoadjuvant chemoradiotherapy from 42.3% to 53.9% of patients. The incidence of pneumonia decreased from 15.3% to 12.8% while the incidence of anastomotic leak increased from 11.7% to 13.1%. Overall, the incidence of complications increased over the 4 years of study from 59.0% to 61.1% but there was a decrease in the number of patients sustaining more than three complications from 18% to 15.3%. The incidence of patients experiencing Clavien-Dindo complications of ≥ IIIb remains stable with a length of hospital stay slightly decreased from 17.3 to 16.7 days.

With respect to ECCG quality measures, 30- and 90-day mortality rates were stable at 2.1% versus 4.6%, respectively. Readmission rates decreased from 10.8% to 8.3% and patients requiring blood transfusions decreased from 14.3% to 10.2%. Patients requiring escalation in their care at some point during their postoperative recovery decreased from 24.5% to 20% with the most common reason for requiring escalation of care being patients experiencing pneumonia 25.9% or atrial dysrhythmia 21.2%. The overall incidence of patients being discharged home following their esophagectomy decreased from 91.4% to 87.8%.

This demonstration of the utility of the Esodata dataset, to not only produce important benchmarks but also to follow demographic and outcome trends over time, highlights its potential clinical relevance. The fact that these outcomes also represent contemporary and international practice patterns make the outcomes more internationally relevant. The Esodata dataset is now embedded in multiple international datasets which provides the potential for international comparisons and audit to occur more routinely in the future. The Publications & Audit Subcommittee of the ISDE has recently awarded two Esodata studies to contributing centers which also demonstrates the potential utility of the dataset for carrying out focused research projects moving forward. The IESG has also taken over responsibility for collecting national esophageal outcomes data for all of Ireland which provides an indication of its ability to reproducibly audit national as well as institutional results associated with esophageal resection.

The main advantages of the IESG and Esodata dataset currently can be summarized as:

• Amalgamating a significant number of high-volume participating centers providing access to a large numbers of contemporary resections over a short period of time, making the outcomes contemporary and therefore relevant to current practice;
• The clearness and simplicity of the web interface contributes to complete data submissions while not overtaxing institutional data managers;
• The application of a data collection system that provides a homogeneous framework for clinical definitions and complications reporting within a “made for purpose” dataset that allows regular international updates and comparisons;
• Clearly demonstrating that international outcomes can be securely collected online in a dataset that facilitates the efficient assessment of clinical issues associated with esophageal resection.

For these reasons, the Esodata database will be a valuable resource for monitoring outcomes and trends in an era of quickly evolving technical change and a period in which process evolution including the introduction of ERAS programs is impacting the delivery systems and outcomes of major oncologic procedures worldwide.

Acknowledgments

Funding: None.

Footnote

Provenance and Peer Review: This article was commissioned by the Guest Editor (Riccardo Rosati) for the series “Current issues on GEJ adenocarcinoma” published in Annals of Esophagus. The article has undergone external peer review.

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at: http://dx.doi.org/10.21037/aoe-20-42). The series “Current issues on GEJ adenocarcinoma” was commissioned by the editorial office without any funding or sponsorship. The authors have no other conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. U.S. Preventive Services Task Force. Guide to clinical preventive services: an assessment of the effectiveness of 169 interventions: report of the U.S. Preventive Services Task Force. Williams & Wilkins: Baltimore; 1989.
2. Massarweh NN, Kaji AH, Itani KMF. Practical Guide to Surgical Data Sets: Veterans Affairs Surgical Quality Improvement Program (VASQIP). JAMA Surg 2018;153:768-9. [Crossref] [PubMed]
3. The Surveillance, Epidemiology, and End Results (SEER) Program. Available online: https://seer.cancer.gov. Accessed on April 2020.
4. Society of Thoracic Surgeon (STS) National Database. Available online: https://www.sts.org/registries-research-center/sts-national-database. Accessed on April 2020.
5. Blencowe NS, Strong S, McNair AG, et al. Reporting of short-term clinical outcomes after esophagectomy: a systematic review. Ann Surg 2012;255:658-66. [Crossref] [PubMed]
6. Low DE, Alderson D, Cecconello I, et al. International Consensus on Standardization of Data Collection for Complications Associated With Esophagectomy: Esophagectomy Complications Consensus Group (ECCG). Ann Surg 2015;262:286-94. [Crossref] [PubMed]
7. Low DE, Kuppusamy MK, Alderson D, et al. Benchmarking Complications Associated with Esophagectomy. Ann Surg 2019;269:291-8. [Crossref] [PubMed]
8. Gillum RF. From papyrus to the electronic tablet: a brief history of the clinical medical record with lessons for the digital age. Am J Med 2013;126:853-7. [Crossref] [PubMed]
9. Brooke EM. The current and future use of registers in health information systems. World Health Organization: Geneva, 1974.
10. AHRQ Methods for Effective Health Care. In: Gliklich RE, Dreyer NA, Leavy MB. Registries for Evaluating Patient Outcomes: A User's Guide. Rockville (MD) Agency for Healthcare Research and Quality (US); 2014.
11. National Institute of Health (NIH) - List of Registries. Available online: https://www.nih.gov/health-information/nih-clinical-research-trials-you/list-registries. Accessed on April 2020.
12. Workman TA. Engaging Patients in Information Sharing and Data Collection: The Role of Patient-Powered Registries and Research Networks. Rockville (MD) 2013.
13. Physician Consortium fo Performance Improvement (PCPI). Available online: http://www.thepcpi.org. Accessed on April 2020.
14. Blumenthal S. The Use of Clinical Registries in the United States: A Landscape Survey. EGEMS (Wash DC) 2017;5:26. [Crossref] [PubMed]
15. Doyle-Lindrud S. The evolution of the electronic health record. Clin J Oncol Nurs 2015;19:153-4. [Crossref] [PubMed]
16. Marshall J, Chahin A, Rush B. Review of Clinical Databases. Secondary Analysis of Electronic Health Records. Cham (CH) 2016.
17. The National (Nationwide) Inpatient Sample (NIS). Available online: https://www.hcup-us.ahrq.gov/db/nation/nis/nisdbdocumentation.jsp. Accessed on April 2020.
18. Khera R, Angraal S, Couch T, et al. Adherence to Methodological Standards in Research Using the National Inpatient Sample. JAMA 2017;318:2011-8. [Crossref] [PubMed]
19. National Cancer Database (NCDB). Available online: https://www.facs.org/quality-programs/cancer/ncdb. Accessed on April 2020.
20. Fernandez FG, Shahian DM, Kormos R, et al. The Society of Thoracic Surgeons National Database 2019 Annual Report. Ann Thorac Surg 2019;108:1625-32. [Crossref] [PubMed]
21. Abbas A, Khan SU. A review on the state-of-the-art privacy-preserving approaches in the e-health clouds. IEEE J Biomed Health Inform 2014;18:1431-41. [Crossref] [PubMed]
22. Burde H. Health Law the hitech act-an overview. Virtual Mentor 2011;13:172-5. [PubMed]
23. European Commission. Overview of the national laws on electronic health records in the EU member states (2016). 2016. Available online: https://ec.europa.eu/health/ehealth/projects/nationallaws_electronichealthrecords_en. Accessed on April 2020.
24. Global Observatory for eHealth. Available online: https://www.who.int/goe/data/en/. Accessed on April 2020.
25. Office of the National Coordinator for Health Information Technology (ONC). Available online: https://www.healthit.gov/topic/about-onc. Accessed on April 2020.
26. American College of Radiology (ACR). Available online: https://nrdr.acr.org/. Accessed on April 2020.
27. European Society of Coloproctology (ESCP). Available online: https://www.escp.eu.com/research/cohort-studies. Accessed on April 2020.
28. Cohort Studies ESCPAudits Committee. The 2017 European Society of Coloproctology (ESCP) international snapshot audit of left colon, sigmoid and rectal resections - study protocol. Colorectal Dis 2018;20:5-12. [Crossref] [PubMed]
29. Van Leersum NJ, Snijders HS, Henneman D, et al. The Dutch surgical colorectal audit. Eur J Surg Oncol 2013;39:1063-70. [Crossref] [PubMed]
30. British Association of Urological Surgeons (BAUS). Available online: https://www.baus.org.uk. Accessed on April 2020.
31. Cashman S, Biers S, Greenwell T, et al. Results of the British Association of Urological Surgeons female stress urinary incontinence procedures outcomes audit 2014-2017. BJU Int 2019;123:149-59. [Crossref] [PubMed]
32. Payne SR, Fowler S, Mundy AR. Analysis of a 7-year national online audit of the management of open reconstructive urethral surgery in men. BJU Int 2020;125:304-13. [Crossref] [PubMed]
33. Ferlisi M, Hocker S, Grade M, et al. Preliminary results of the global audit of treatment of refractory status epilepticus. Epilepsy Behav 2015;49:318-24. [Crossref] [PubMed]
34. Ferlisi M, Hocker S. What can we learn from status epilepticus registries? Epilepsia 2013;54:72-3. [Crossref] [PubMed]
35. Price LC, Lowe D, Hosker HS, et al. UK National COPD Audit 2003: Impact of hospital resources and organisation of care on patient outcome following admission for acute COPD exacerbation. Thorax 2006;61:837-42. [Crossref] [PubMed]
36. British Thoracic Society (BTS). Available online: https://www.brit-thoracic.org.uk. Accessed on April 2020.
37. Heaney LG, Brightling CE, Menzies-Gow A, et al. Refractory asthma in the UK: cross-sectional findings from a UK multicentre registry. Thorax 2010;65:787-94. [Crossref] [PubMed]
38. Bourke BM, Beiles CB, Thomson IA, et al. Development of the Australasian vascular surgical audit. J Vasc Surg 2012;55:164-9. [Crossref] [PubMed]
39. Beiles CB, Bourke BM. Validation of Australian data in the Australasian Vascular Audit. ANZ J Surg 2014;84:624-7. [Crossref] [PubMed]
40. Cook JA, Collins GS. The rise of big clinical databases. Br J Surg 2015;102:e93-e101. [Crossref] [PubMed]