Aug 09 2016 How can partners agree on measures of success? Lessons from Partners for a Competitive Workforce

Through the 2000s, the Ohio-Kentucky-Indiana tri-state area faced a shortage of workers equipped with the skills to match the labor demands of Greater Cincinnati area employers. Unemployment in the region was at 8.6 percent, and 50 percent of the workforce lacked the necessary post-secondary education and training to meet the skills needed by regional employers. This included a mix of hard and soft skills in key industries including advanced manufacturing, information technology, construction, and health care.

In response to this challenge, over 150 organizations from the business, community, non-profit, and public sectors united to create the Greater Cincinnati Workforce Network, now Partners for A Competitive Workforce (PCW), in 2008. This cross-sector collaboration identifies employer needs, connects workers to educational programs that help them develop relevant skills, and facilitates worker readiness — ultimately linking job-seekers with entry-level employment in target industries. Under the leadership of Executive Director Janice Urbanik, Partners for a Competitive Workforce is leading the effort to meet the communities’ Bold Goal for Income which is that 90 percent of the labor force has gainful employment by 2020.

Agree on Measures of Success
A fundamental step in creating career pathways is deciding how to measure progress and track outcomes. First used by the Southwest Ohio Region Workforce Investment Board, PCW partners contribute to a shared database analyzing more than 100,000 records of clients. This data, entered primarily by service providers working in direct contact with job seekers, provides key information for each area in which PCW operates and is used to inform program action and development.

This information provides insight into training completion rates of program participants, program efficacy vs. outcomes, and barriers encountered by participants, as well as employment trends in the region. It is used by more than 50 partner organizations. Additionally, in 2012, PCW participated in a report, “2020 Jobs Outlook,” in partnership with Vision 2015, the Strive Partnership, and Agenda 360, which used statistics from the Bureau of Labor Statistics (BLS) to forecast trends in regional employment. This gave partners a chance to analyze employment trends in the region, as well as identify additional sectors in which to implement career pathways.

This commitment to measuring progress and tracking outcomes is an excellent example of Agreeing on Measures of Success — the identification of indicators to be used in evaluating the progress and results of the collaboration. Consensus among partners on what will define success for the collaboration in the short, mid, and long term creates accountability and helps keep the collaboration on track toward goals.

Cross-sector partners are likely to have differing views of what measures should be used to identify success. Instead of beginning with a discussion of indicators, partners may find it easier to first agree on criteria for evaluating and selecting indicators. Potential criteria the collaboration may want to consider include: How relevant are the indicators to the collaboration’s vision of success? How relevant are the indicators to the facts that have been agreed are applicable to the issue at hand? Are the indicators accessible during the time span of the collaboration? Do the indicators provide insight into the “living experiences” of those affected by the issue the collaboration aims to influence? If collaboration partners are unable to agree on indicators of success, the result can be diminished accountability within the collaboration and limited ability to make claims about the collaboration’s effect on target outcomes.

The Partners for a Competitive Workforce model is an approach increasingly used across the country to address the skills gap. More than 150 partners have come together from the public, private, and non-profit sectors to provide solutions that bridge the skills gap between employer demand and worker preparedness, leveraging more than $40 million in financial support. Since 2008, PCW’s partners have served more than 7,800 individuals. Of those served, 87 percent have completed training, earning over 8,300 credentials, including but not limited to professional certificates. 80 percent have obtained employment, with a 73 percent rate of retention.

When deciding on how to measure success, we suggest that partners reflect on these questions:

• In discussing potential measures of success, how will we ensure that we consider both qualitative and quantitative indicators? Financial and non-financial indicators?
• How will we handle disagreements among partners as to what indicators we should use?
• How will we collect and manage the data we agree upon? Will one or more collaboration partners be responsible for these tasks? Will we enlist a third party?
• How often will we review this data? What will be its role in shaping our actions as a collaboration?

For additional guidance, see these resources:
“Working Group Instructions for Developing Shared Metrics” and “Work Group Reporting Template for Developing Shared Metrics” from FSG
These resources are designed to be used together to assist partnerships with identifying indicators or key data points for shared measurement. While the first resource provides instructions to be given to a working group and reported to a backbone organization, it can easily be adapted for partners to use on their own.

“Creating Objectives” from Community Toolbox
This section provides detailed guidance and activities that walk practitioners through identifying indicators of success (which Community Toolbox refers to as “objectives”). It includes a discussion of differing types of indicators, collecting baseline data, and more. Community Toolbox is an online collection of toolkits and resources for individuals seeking to work collaboratively to bring about social change.

“Partnership Development Toolkit” from the European Commission
Especially see Section 3.5: Indicators Column and Section 3.6: Evidence Column on pp. 29-31 for guidance on identifying indicators and evidence that will be used to monitor the collaboration’s progress. The “Partnership Development Toolkit” is a guide for facilitators of EQUAL Development Partnerships (DPs) but is easily adaptable to partners in a wide variety of issues.