Evaluating potential indicators of success.

Cross-sector partners are likely to have differing views of what measures should be used to identify success. Instead of beginning with a discussion of indicators, partners may find it easier to first agree on criteria for evaluating and selecting indicators. Potential criteria the collaboration may want to consider include: How relevant are the indicators to the collaboration’s vision of success? How relevant are the indicators to the facts that have been agreed are applicable to the issue at hand? Are the indicators accessible during the time span of the collaboration? Do the indicators provide insight into the “living experiences” of those affected by the issue the collaboration aims to influence? If collaboration partners are unable to agree on indicators of success, the result can be diminished accountability within the collaboration and limited ability to make claims about the collaboration’s effect on target outcomes.

Assessing capacity to access, evaluate, and manage data related to measuring success.

The collaboration should ensure that it possesses the expertise (e.g. experience using certain methodologies), access (e.g. to data sources or rights to interview), and resources (e.g. human resources or technology) to collect, evaluate, and manage data related to measuring success. If the collaboration determines that it does not have these capabilities, it may enlist additional partners or a third party. If partners fail to assess incapacity in these areas, the result could be delays in data collection, errors in data evaluation, and missteps in data management. Ultimately, this limits the collaboration’s ability to measure and report progress and outcomes.

“Improving Cardiac Arrest Patient Survival in Georgia”

Each year, approximately 300,000 people in the United States suffer a cardiac arrest. Known in the medical community as an out-of-hospital cardiac arrest (OHCA), survival rates have hovered around 8 percent for the last 30 years. When a group of health leaders in Atlanta considered the possibility of installing public-access automatic defibrillators (AEDs) throughout the city to help improve survival rates in 2003, they quickly recognized that their inability to track the progress of OHCA patient outcomes through the medical system — from the ambulance to the hospital to the moment of discharge — would make it impossible to assess the effectiveness of such an investment. Dr. Bryan McNally, Associate Professor of Emergency Medicine at Emory University, worked with Atlanta hospitals, emergency medical technicians, medical software providers, a network of medical experts, and the Centers for Disease Control and Prevention in Atlanta to develop the Cardiac Arrest Registry to Enhance Survival (CARES) — the first database to track key details for OHCAs. Dr. McNally and partners worked closely with Dr. Greg Mears, who had led the development of the National EMS Information System (NEMSIS), to select indicators that would allow the team to track the outcomes of their efforts and to ensure that the data they were collecting was compatible with the nationwide NEMSIS system. They agreed to track survival rate, bystander CPR rate, the rate of AED use, and total transition time between 911 dispatchers and EMS responders, among other indicators, to assess the timeliness and quality of care patients were receiving throughout the response pipeline. The first-year results of the CARES pilot program clearly showed Atlanta’s poor performance in saving the lives of cardiac arrest victims and provided a baseline from which to make improvements. This data contributed to a greater understanding of the barriers that patients and health professionals encountered that led to OHCA fatalities. It also improved collaboration among citizens, emergency responders, and hospital staff, resulting in increased survival rates and support for future funding considerations.