Is Meaningful Evaluation of Public Economic Development Efforts Possible?

More rigorous evaluations of local economic development programs and policies are feasible argues a recently released working paper by Timothy Bartik, a senior economist at The W.E. Upjohn Institute for Employment Research. Evaluating the Impacts of Local Economic Development Policies On Local Economic Outcomes: What Has Been Done and What Is Doable? stresses the importance of evaluation in local economic development. The ability to distinguish the true impacts of proposed policy in providing economic development benefits is crucial in having well-functioning state and local governments, the report claims.

Bartik contends that in the U.S., rigorous evaluation does occur, but it is the exception rather than the rule. When these evaluations are completed, Bartik states, they usually show that services to small and medium-sized firms can be effective in advancing the performance of these organizations. Economic development strategies targeting distressed areas tend to be ineffective if they only provide moderate resources, but are more successful if they offer broad resources over an extended time frame, the author suggests.

Many agencies assert that they do, in fact, perform evaluations on their programs; however, they really do not, Bartik explains. It has become commonplace for state and local economic development organizations to assemble large quantities of data on program activities such as jobs created. These job creations are then declared as a program impact, which inaccurately presumes that none of the economic activity would have occurred without the program. Also, agencies will detail local economic conditions, such as jobs created during a specific time period or reductions in the unemployment rate, and claim again such progress as program impacts. Once again, by assuming improvements in the local economy are due exclusively to local economic development policies, this approach is flawed.

Bartik stresses the importance of "outcome impact" evaluations. Understanding fully how and why the policy has its estimated impacts are critical to evaluations, he says; therefore, evaluations should include estimates of how program impacts may change with any variation in the structure or administration of the program. A model evaluation, argues Bartik, would not only report on local business activity but also the impact on the economic well-being of local residents — the definitive target of local economic development policies.

Numerous ways in which more rigorous evaluations can be performed are presented in the paper. These include random experimentation, statistical analysis of program users and comparison groups, surveys and focus groups, and linking regional econometric models with fiscal impact and local labor market models. Bartik suggests that encouragement for more rigorous evaluations must come from outside pressure and funding. This outside pressure can come from legislatures, governmental audit bureaus, and higher levels of government.

One of the many reasons that evaluations are not performed enough is the fear of negative evaluations being used as a pretext to terminate a program, Bartik concludes. As a result, economic development agencies may not pursue thorough evaluation. The article states rigorous assessment is more likely to be used in the future if the results are used to improve programs, rather than end them.

The working paper can be viewed at:
http://www.upjohninst.org/publications/wp/0389wp.html

Resources for Evaluating Programs
To encourage more thorough and useful review of technology-based economic development programs, SSTI added several exceptional titles on evaluation to its current publication catalog including:

* The Handbook of Practical Program Evaluation

* Evaluation Models

* The Balanced Scorecard: Translating Strategy into Action

* Evaluation: An Integrated Framework for Understanding, Guiding, and Improving Public and Nonprofit Policies and Programs

* Improving the Economy, Efficiency, and Effectiveness of Not-for-Profits: Conducting Operational Reviews

Ordering information is available in Resources for Building Tech-based Economies, SSTI’s Publication Catalog, which is available online at: http://www.ssti.org/Publications/Onlinepubs/catalog.pdf

**********************

Copyright State Science & Technology Institute 2003. Information in this issue of SSTI Weekly Digest was prepared under a cooperative agreement with the U.S. Department of Commerce, Economic Development Administration. Redistribution to all others interested in tech-based economic development is strongly encouraged — please cite the State Science & Technology Institute whenever portions are reproduced or redirected. Any opinions expressed in the Digest do not necessarily reflect the official position of the U.S. Department of Commerce.