Learning from Investigation: Reviewing the First Year of MEDCAS

Colloquium at the Human Factors and Ergonomics Society National Conference
Thursday, 19 October 2006, San Francisco, CA 

Richard Cook, MD (co-chair) The University of Chicago
Jay Crowley, US FDA
Meghan Dierks, MD, Beth Israel Deaconess Medical Center/Harvard University
Yoel Donchin, MD, Hadassah Hebrew University Medical Center, Jerusalem
Christopher Nemeth, PhD, (chair) The University of Chicago
Tina Powell, Social and Scientific Systems (SSS)

This proposal built on successful sessions during the previous two HFES national meetings: the 2004 "Afterwords" colloquium, and the HFES 2005 "Above Board" discussion panel. The Medical Event Data Collection and Analysis Service (MEDCAS) is a nation-wide fifteen-month pilot program to validate the methods and procedures to investigate, analyze, and report adverse health care events and provide a 'best practice' model for creating a safe reporting environment. The results of the project since its start in Fall of 2005 provided the basis for this discussion panel.

High hazard sectors such as aviation, ground transportation, the military, and nuclear power routinely confront circumstances that imperil operators and users. The safety challenge to organizations that operate in such settings is to understand the nature of hazards and effectively manage exposure to them. Each of the sectors is similar in a number of ways. Operations are based on a fairly consistent understanding of what is to be done and how it is to be done that is shared across the organization. In fact, the military ensures this through an entire segment of its organization that develops and promulgates doctrine. Equipment and systems are routinely developed to comply with specifications and selection relies on compliance to those specifications. Training is essential to ensure consistent performance among operators. In instances where adverse events threaten operations, a stand down can be ordered to address safety issues. Management is comprised of operators whose career has evolved into a supervisory role. Their operations routinely incorporate reports of events, including those with adverse outcomes. Reports are intended to support the investigation, analysis, and development of measures to prevent or minimize future occurrences.

Healthcare is also a high hazard sector, yet it shares only some its counterparts' traits. The volume of care, type of care that is required, and acuity of need vary widely, and healthcare operations are highly variable to meet that demand. Equipment, staff, supplies and facilities are assembled in unique combinations that are best suited to the singular characteristics of each case. Procedures may be changed while being performed, which may require the assembly of equipment, supplies, and staff to be reconfigured. Management and clinicians are typically separate, as senior physicians typically practice medicine rather than migrate into management. Responsibility for governing healthcare rests at the state level. Accreditation for hospitals is performed by independent agencies, such as the Joint Committee on the Accreditation of Healthcare Organizations (JCAHO). The federal government certifies certain aspects of equipment for use, such as reliability. However, it does not certify other aspects that have a significant bearing on outcome, such as usability.

Healthcare's response to adverse events also differs from other high hazard sectors. For example, the National Transportation Safety Bureau (NTSB) provides objective, independent accident investigation and analysis assistance for aviation, rail, and ground transportation. Healthcare organizations have no such resource. State governments, the Joint Committee on the Accreditation of Healthcare Organizations (JCAHO) and the U.S. Food and Drug Administration (US FDA) have mandated reporting of adverse, or "sentinel," events. In response, healthcare organizations have convened in-house panels to investigate and report on adverse events. Each institution, then, is responsible to develop its own accident investigation and analysis program using resources that are available locally. The safety programs that healthcare organizations create are designed to meet the approval of their accrediting organization when it conducts a review. Panel members routinely lack sufficient training and expertise in how to report events or assist their investigation. Results of inquiries are typically closely held which prevents their own, or other, institutions from learning.

The investigation and analysis of medical accidents is intended to discover information that explains the nature and cause of what occurred in the interest of preventing or minimizing future loss. However, healthcare accidents have features that make post-event investigations particularly difficult. Thorough, objective investigation of medical adverse events rarely happens due to the complexity of the environment, litigation, risk, and socio-political implications. Special concerns can, and do, undermine investigation depth, breadth, and quality.

     Technical-The medical domain's complexity is substantially higher than others in which accident investigation is conducted. This makes the assembly and validation of the precise sequence of the accident and its surrounding context more difficult than in other fields. The variety of technical knowledge in this domain is exceptionally high. Investigation necessarily requires a cadre of investigators who have the necessary technical knowledge as well as specialists whom they can consult for deeper expertise.

     Social-Reports are molded by incentives and sanctions. Any regime that includes sanctions for those who are involved in reported events influences their behavior. Sanctions create a structure of incentives for people to consider in what they report, when they report it, and how they report it. Automatic sanctions can be particularly destructive because they create enormous incentives to risk not reporting an incident, or to misrepresent the incident.  Reporters may attempt to submit reports that misrepresent what happened. They may also deliberately misclassify the incident as a way to decrease the sanctions that they might ultimately face. The shape of the reports that are generated in this environment reflects the structure of the sanctions, as much as the events they are reporting. Regimes that mandate reporting as a prelude to disciplinary actions create further political, professional, and organizational problems for those who submit reports.

     Political-Those who are closest to the sharp (operator) end of the healthcare organization understand the difficulty and uncertainty that underlies their daily activities. Those who are closest to the blunt (management) end are most remote from sharp end operations and are concerned with maintaining the organization. Threats to the organization are minimized by casting adverse events as anomalies. Identifying and removing the event's proximate cause gives the appearance of restoring the organization to "normal" conditions.

     Legal-The potential for possible legal proceedings compels organizations to manage activity that seeks information related to an adverse event. This occurs even through the investigation does not expose the organization any further.

Six panel members who are experienced in adverse event investigation will comprise the panel. Their insights will demonstrate the issues that confront healthcare accident reporting, investigation, and analysis.

Cook  These contentious and politically-charged considerations induce efforts to contain, rather than to understand. Failure to advance the science of adverse events dooms healthcare to repeat them rather than improve on them. Our experience shows that moving from concern over litigation to the discussion of actual events shifts the dynamic to learning about what actually occurs. This is the start toward building the scientific basis for understanding adverse events.

Crowley: Even as new methods are being developed, the surveillance of medical devices still relies on the spontaneous, passive reporting of device-related problems - particularly to identify "new", previously unrecognized problems.  This, in turn, requires users to recognize, understand and report problems they experience.  All of this is predicated on users recognizing that a device was (or could have been) involved in an actual or potential incident - and then analyzing the situation to determine, to a reasonable extent, the actual problem.  However, what is usually reported is what the user "observed" - which often does not connect with the actual problem - and in fact tends to hinder analysis and obscure the real problem.  Users tend to associate a new problem with previous experiences - and so do not necessarily see the event as new or different.  We are developing tools to assist the device reporter to independently analyze a device-related incident to accurately understand and report the problem - and to be able to see "new" problems.  An infusion pump example will be demonstrated.

Patterson   Automated systems such as bar code medication administration (BCMA) have been widely advocated as a remedy to patient safety problems, yet research has shown that they produce unforeseen consequences. Such shortcomings are system level issues that are beyond the perception of the clinicians who are tasked with reporting adverse events. Clinicians have developed their own methods for making their world work every day. Understanding how they do that is essential to understanding acute healthcare. Understanding such systems-level issues requires a combination of both medical as well as human factors expertise. I will discuss the implications for this approach to the way that we learn about, and learn from, adverse events.

Dierks  The most serious/significant medical adverse events (i.e., those from which we have the most to learn) tend to be those that evolve over time, have a poorly defined initiating point and multiple junctures where recovery is possible, but is either not initiated or is inadequate or incomplete. What these complex adverse events really represent are a series of 'micro-states' from which a transition to either recovery or injury may be determined by a wide range of human and system-based factors. Unfortunately, from the eyes of the front-line agents who are involved in the event and who typically report the event, there is a tendency to focus almost exclusively on the end state. During the course of introspection, documentation, discussion and causal analysis, it is rare for most providers to consider more than one or two steps prior to the end states, and the conditions that enabled that final transition. I will discuss our current work, which focuses on a more comprehensive framework to elicit and analyze adverse event data in ways that can help healthcare providers to understand these events in a broader context.

Powell   The US Food and Drug Administration (FDA)-funded Medical Product Safety Network (MedSun) has been collecting reports and special survey data concerning post market problems with medical devices and biologics (including, but not limited to, serious adverse events) from a sample of US hospitals since 2002.  Some of the most important reports, according to FDA analysts, have been those where no patient harm has been involved (at least as yet).  In addition, there is considerable interest in MedSun reports about problems that may be interpreted as "use error" that could indicate the need for changes in product design or labeling.  Ms. Powell will discuss feedback provided to the nation's hospitals from the project, and some of the major actions that FDA and manufacturers have taken to protect patients and/or healthcare providers from harm where MedSun reports have made an important contribution. 

Donchin    There is no substitute for an accident as a warning sign that something is wrong in a system. In order to learn what needs to be known, look into the system's details. Organizations can be their own worst enemy in this regard. Litigation can make event investigation seem too complicated. To avoid that, a formal investigation is postponed and a "second" story is created. We are left to study an adverse event without the benefit of a detailed investigation of the system. Two cases will illustrate this point. In the first, four patients during one shift were injected with 2 ml of Adrenalin instead of Morphine. In the other, the skin of a patient was cleaned before an injection with an acid, rather than with alcohol. This led us to track the way a cleaning solution makes its way from the hospital storage place to the drug cabinet.  

Healthcare's distinct difference from other high hazard sectors requires a unique approach.

Effective healthcare adverse event investigation requires impartial, qualified experts who have detailed domain knowledge, use diverse investigation methods, and remain impartial in the face of stakeholder interests. It also depends on the resolution of issues that span clinical practice, healthcare organization management, and regulatory and governmental agencies. Collaboration among these interested parties will open the way to the scientific collection of data that can serve as a foundation for effective measures to improve patient safety.