CareerEdge Funders Collaborative was piloting a new approach to workforce development, so it wanted hard evidence of its progress to engage funders and respond to skeptics. The collaborative engaged measurement experts Capital Analytics to help execute a credible, quantitative Big Data analysis of the economic impact of its programs, which was positively received by partners and funding resources.
Many not-for-profits shy away from financially quantifying the impact of their programs. But increasingly, funders are becoming more discerning in how and where they grant resources. To create a strong competitive position and secure continued or additional funding, savvy not-for-profits understand that they must progress beyond basic measurements and demonstrate financial impact from investor dollars.
CareerEdge Funders Collaborative is an innovative, Florida-based partnership of business, civic, and philanthropic organizations that uses public and private dollars to help employers in the Gulf Coast area fill their workforce needs, as well as help employees and job seekers gain new skills.
Through extensive collaboration, CareerEdge and employers identify worker skills gaps and develop training programs to close them. This approach is a shift from traditional state-run workforce development efforts and, therefore, scrutinized by funders and policymakers.
It is in this environment that CareerEdge leadership recognized the critical importance of an objective evaluation of its programs. Within months of its startup, CareerEdge engaged Capital Analytics as a third-party evaluator to qualitatively and quantitatively assess its activities.
Traditionally, funding sources for the workforce development arena evaluated grant performance using metrics such as number of individuals served or training hours delivered. But CareerEdge wanted to assess the economic impact its programs were having on the communities it serves. As a result, it sought out the Big Data expertise of Capital Analytics to develop and execute its Economic Impact Study, a multiyear evaluation plan.
The evaluation needed to examine and inform stakeholders of progress toward CareerEdge's four goals:
* provide workers with the skills they need to keep up with the evolving needs and requirements of their employers and advance in their careers
* move low-wage workers into higher paying jobs
* provide employers with the skilled employees they need
* create system change regarding partnerships between business and community-based organizations, as well as public policy around workforce development.
Unlike many studies on return-on-investment that focus on a single program within a single organization, this one had multiple stakeholders and sought to determine the impact of the CareerEdge grants on workers' lives and on the Gulf Coast regional economy. Using the data collected, along with forecasting methods, the analysis needed to consider both tangible (financial) and intangible (employee and employer confidence) effects on the region.
The project began with a series of meetings with key stakeholders--from employers and educators to funders and social services providers. Each group shared their own vision of success for this new approach to workforce development, and outlined specific metrics for how to measure program achievements.
Next, the Data Collection Toolkit was developed to guide each of the partners on the information they were to collect. The tool kit also outlined quarterly collection deadlines, and discussed employer participation in semistructured qualitative interviews.
Data collection templates were sent to the data-providing partners on a quarterly basis. Individual-level data were aggregated by employer and industry sector, and included wages, raises, promotions, performance, and turnover. Data could be segmented by some 15 characteristics, including such standard demographics as gender, ethnicity, and age, and study-specific factors such as education level and criminal background.
To provide context for the quantitative data, semi-structured interviews were conducted with employers, educators, social services providers, and policymakers. Employee testimonials from employers, educators, and social services providers also were collected.
Lessons learned and results
Since its inception nearly two years ago, CareerEdge has served more than 1,700 individuals through partnerships with 13 area employers. Participants have completed more than 6,600 classes and earned about 4,600 professional certifications. Further, 78 percent of job seekers found employment, more than 1,000 employees earned wage increases, and more than 400 earned promotions.
The tangibles. According to the data collected, about $4.3 million in new and incremental wages into the economy were attributable to CareerEdge programs. Next, researchers applied the economic theory of the "multiplier effect," which examines how new wages are spent and circulate through the economy and add value to the region.
Using a well-adopted multiplier for the region of 1.7, it is believed that the additional $4.3 million in wages will create another $3 million as it is circulated in the regional economy, resulting in a total of about $7.3 million in new income to the Gulf Coast region over four years.
From an investor perspective, the cost of the program was $1.5 million. The goal was to achieve economic gain for the community that could be attributed to their investment in CareerEdge. The projected four-year return of $7.3 million generates an ROI of 472 percent. (At the request of stakeholders, time value of money and inflation were not factored in.)
The intangibles. The benefits of providing job-specific training go beyond pay raises and promotions; participants are increasing their own human capital through education. Not only does this reduce future risks of unemployment, it also increases participants' confidence in their knowledge and skills. Through the qualitative interviews, employers reported how they were witnessing employees with increased levels of confidence after training.
In addition to uncovering the ROI of its programs, CareerEdge learned that there are five key factors that contribute to the success and credibility of this sort of evaluation program.
* Engage stakeholders in establishing measures of success and building a logic model.
* Define the data collection process and requirements right at the start, and begin collecting data.
* Objectively report the evidence using terms and formulas that make sense to stakeholders.
* For any evaluation to be credible, findings--both good and bad--must be transparent.
* Have the confidence that with a solid research design and hard work, you can measure anything.
More important, CareerEdge is able to leverage findings from its Economic Impact Study in discussions with the local economic development corporations, policymakers, employers, and current and prospective investors to secure future investments for the continuation of its mission.
RELATED ARTICLE: Causal Model Illustrates Intangible Impact
Capital Analytics created a causal model to capture the essence of stories from healthcare-sector participants of CareerEdge programs. The model illustrates how training and certification increases employee confidence, which creates positive outcomes for patients, the employee, and ultimately the organization.
* Gene Pease is co-founder and CEO of Capital Analytics; firstname.lastname@example.org.
* Bonnie Beresford is vice president of client services for Capital Analytics; email@example.com.…