While most of us commonly use the word “evaluation” in our day-to-day business vocabulary, in the world of research it’s used more precisely to categorize different means of assessing both proposed and existing programs, policies, personnel, products, and so forth. (In fact, believe it or not, there is even an American Association of Evaluation.)
Of course, these assessments then drive more intelligent decision-making and performance improvements. Generally speaking, evaluations fall into two main categories: formative and summative.
Formative evaluations facilitate exploration
Formative evaluations are a type of exploratory research designed to better define a problem or help strengthen a particular program. Many associations rely on needs assessment studies to determine if they are providing the services and products deemed meaningful by their constituents. Government agencies often undertake assessments to gauge the feasibility of a proposed program to deliver specific desired outcomes. These are types of formative evaluations.
For example, we recently conducted a study for a major government technology solutions provider designed to identify and prioritize the current and future needs of government agencies that provide social and human services, such as social security, welfare, pensions, and workers compensation. Encompassing the U.S., Canada, and nine countries across Western Europe, the results revealed the commonalties and differences in needs between countries. Using our findings, the client could then effectively tailor and segment its portfolio of offerings based on each geographic market’s agency priorities, funding availability, technology investment strategy, decision-making hierarchies, and so forth.
Summative evaluations measure results
Comparatively, summative evaluations measure outcomes of a particular program. Federal, state, and local government agencies often have mandates to include outcome evaluations or impact evaluations as part of programs or services they deliver. The No Child Left Behind Act, a national program instituted by the Department of Education to measure school performance, is an example. Associations and government agencies often employ summative evaluations, such as program scorecards and cost-benefits analyses, to determine whether objectives are being met on specific initiatives.
Government contractors also make very good use of summative evaluations. For example, contract satisfaction surveys, a subset of summative evaluations, have helped many of our clients uncover previously unidentified problems on very important, large-revenue contracts. By prioritizing and proactively addressing the issues,?? both real and perceived,?? the organizations were able to retain the contracts when they went up for re-competes.