Study design and population
We conducted a retrospective cohort study. The study sample was designed to include hospitals that could provide reliable safety estimates for patients aged 18 years and older at each location. We selected the 11 participating hospitals to represent a mix of both large and small facilities, and they were also part of three different healthcare systems. Among these hospitals, two had fewer than 100 beds, four had 100-200 beds, two had 201-500 beds, and three had more than 700 beds.
Our sampling strategy, along with additional details on its representativeness, is outlined in a previous publication focusing on the overall study population.7 At each participating hospital, a random sample of admissions records was obtained, with oversampling in the smaller hospitals. The target sample from the participating hospitals in Massachusetts included all inpatient admissions with discharges in 2018, excluding those for admissions for hospice or rehabilitation care, psychiatric or addiction treatment, and observation only under the two-midnight rule, which categorises a hospital stay that does not cross two midnights as an observation only encounter. If patients were admitted to hospital from a day procedure owing to an adverse event that occurred in the outpatient setting, these patients were not included in our inpatient surgery sample. A total sample of 2750 admissions (averaging 250 per hospital) was initially calculated. Owing to oversampling of four smaller hospitals, the final sample size increased to 2836 admissions. Of 2809 inpatient admissions with usable charts, we ultimately selected those that involved a surgical procedure. Surgical admissions were primarily identified using surgical discharge diagnosis related groups, which categorize and reimburse hospital inpatient services associated with procedures performed in an operating room setting and carry substantial risk for patients. In addition to surgical procedures, these groups included major interventional cardiovascular and endoscopic procedures. To ensure comprehensive sampling, we subsequently included admissions not initially classified under surgical diagnosis related groups but that involved an adverse event related to a surgical procedure and directly involved surgical specialties during the inpatient stay.
To accurately estimate adverse event rates based on a sufficient sample size, we opted a priori to categorize by single specialty for orthopedic and gastrointestinal surgery, group closely related specialties by organs and surgical outcomes for cardiovascular and thoracic procedures and for urology and gynecology procedures, and combine all other remaining specialties with limited samples.
Record review
Nine trained nurses reviewed the records for adults admitted to hospital to identify possible adverse events, using a detailed manual that outlined the chart review process and specified the data to be collected. In this study, we defined adverse events as unintended physical injury resulting from or contributed to by medical care that required additional monitoring, treatment, or hospital admission, or that resulted in death.9 Medical care encompassed the actions of individual hospital staff as well as the broader systems and care processes, including both acts of omission (such as failure to diagnose or treat) and acts of commission (such as incorrect diagnosis or treatment, or substandard performance). We excluded adverse events that occurred during previous inpatient admissions or outpatient visits, as our focus was on assessing the incidence of adverse events during hospital admissions.
The reviewers were randomly assigned admitted patients across the hospitals. If they discovered information in a chart that warranted further investigation to identify adverse events related to the index admission, they were allowed to review data recorded up to 30 days after the patient’s discharge. To determine if harm was associated with the index admission, we applied no restrictions to reviewing chart information recorded before the index admission. The reviewers adhered to a protocol outlining the sequence of reviewing an admitted patient in Epic, the widely used electronic health records system used by the hospitals. For hospitals using other electronic health records systems, we randomly assigned admissions to reviewers trained in those systems, following a protocol similar to that used for Epic. Eight hospitals used Epic, two used Meditech, and one used a custom made electronic health records system. All data were entered into a data collection tool developed with Microsoft Access, which allowed for live data validation.
As previously detailed and presented in supplementary method S1,7 in addition to reviewing all relevant information documented in the electronic health records to identify adverse events for each patient, the reviewers looked for triggers of potential adverse events related to patient care, drugs, surgical procedures, intensive care, and emergency care. For each patient, the reviewers were allowed to document up to eight possible adverse events. When the reviewers identified an adverse event, they classified it into specific types, such as an event related to a surgical procedure, an adverse drug event, a patient care event related to nursing care (eg, fall or pressure ulcer), a healthcare associated infection, or a blood transfusion reaction. The reviewers also identified the inpatient setting where the medical management leading to the adverse event occurred, along with the most directly involved specialties and professions. Subsequently, the reviewers performed a comprehensive search to identify any signs of errors during care, such as mistakes in diagnosis or failures to follow procedures. Finally, they compiled a narrative summary of the admission, accompanied by a description of each related adverse event.
Eight physicians reviewed the randomly assigned summaries of adverse events and either agreed or disagreed with the classification of adverse event type. If these adjudicators disagreed, the event type was revised. When the adjudicators had questions or thought one adverse event should be counted as several, they sent their queries or comments back to the nurse for further review. In addition, the adjudicators assessed the severity of each event using a general severity scale,10 which categorized events as clinically significant (causing unnecessary harm but leading to a quick recovery), serious (resulting in substantial intervention or prolonged recovery), life threatening (posing a potentially fatal situation that required immediate intervention), or fatal (resulting in death). A major adverse event was defined as serious, life threatening, or fatal (supplementary table S1). The adjudicators also provided assessments of whether the harm was preventable.11 A potentially preventable adverse event encompassed those assessed as definitively, probably, or possibly preventable. A preventable adverse event only included those assessed as definitively or probably preventable (supplementary table S2). Finally, the adjudicators graded their confidence (with the use of a six point ordinal scale) about whether the event was due to healthcare management.12 A confidence score of 4 or higher indicated an adverse event had occurred, aligning with the confidence threshold used in the Harvard Medical Practice Study (supplementary table S3).13 Supplementary method S2 provides additional information about the record review.
Statistical analysis
We employed a sampling design in which some of the smaller hospitals were oversampled. Each sampled patient’s admission record was assigned a weight for the analyses. The weight for each patient sampled was the inverse of the probability that the patient was sampled, which is estimated as the inverse of the proportion of admission records sampled from that hospital. Intuitively, a sampled individual’s weight can be interpreted as the number of patients in each hospital that the sampled individual represents. Applying these weights in all the analyses enabled us to derive estimates of characteristics and outcomes for the population of interest. Along with weighting, all 95% confidence intervals accounted for clustering within a hospital. A generalized estimating equations approach with an exchangeable correlation matrix was used to calculate the marginal probability of an adverse event.1415 We did not adjust confidence intervals for multiplicity, so they should not be used in place of hypothesis testing.
Patient characteristics associated with admissions were reported as numbers and percentages for categorical variables and as means for continuous variables. Weighted adverse event rates were described based on corresponding severity and preventability, stratified by population characteristics, insurance type, and surgical specialty associated with the admission. Additionally, weighted severity and preventability of adverse events were described according to the type of event, setting, and profession involved. Data manipulation and analyses were performed using SAS software version 9.4 (SAS Institute, Cary, NC). Supplementary method S3 provides the SAS code for data preparation and analysis.
link