By: Jennifer C. Greene
Evaluation is the social practice of valuing.
Evaluation is the social practice of gathering and using empirical information to make judgments about the merit and worth of a given social or educational intervention somewhere in the world. Both our empirical data and our judgments are grounded in particular values, which are too often unnamed and thereby under-used to advance our culturally responsive agenda of cultural respect and socio-political equity.
Where do values show up in evaluation practice?
First, our data are significantly shaped by the evaluation approach and the key evaluation questions that frame and steer a particular evaluation study, and by the different methodologies that support these different evaluation approaches and questions. A policy-oriented evaluation often asks questions about statistically significant outcome attainment, on the average and sometimes also as disaggregated by targeted subgroups, and uses an experimental methodology to address these questions. A participatory evaluation often asks questions about the quality of program services and their match to priority community needs, and uses a case study or participatory methodology to address these questions. And second, our judgments of program success are anchored in a set of quality criteria, too rarely explicitly named or justified. From a policy evaluation perspective, a “good” program has statistically significant average outcome effects, resting on utilitarian and instrumental understandings of the common good. From a participatory perspective, a “good” program addresses important needs of the targeted community, resting on democratic, pluralistic, and community-oriented understandings of the common good.
In culturally responsive evaluation, how can underlying values importantly and consequentially inform evaluation practices?
Privileged in culturally responsive evaluation (CRE) are the needs and interests of cultural minorities in our society who have yet to attain fair and equitable access to the resources and opportunities needed for the development of each person’s full human potential. The boundary lines defining these minorities are constructed along racial, ethnic, gender, economic, geographic, and identity lines. CRE thus, in theory, advances values of inclusion and pluralism, of contextuality and history, of fairness and equity, and of socio-political justice. In practice, CRE work can more consequentially advance these values by clearly naming and justifying them as integral to this evaluation approach, its key questions and audiences, and its accompanying designs and methods. Even more powerfully, the commitments of CRE can be advanced with a clear articulation and justification of – in tandem with a conversation about – the values that constitute the criteria by which a program is judged as worthy or not.
References
Boyce, A.S., DeStefano, L., & Greene, J.C. (September 2015). Values-Engaged, Educative Evaluation
Approach. MSPnet.
Greene, J.C., Boyce, A., & Ahn, J. (2011). A values-engaged educative approach for evaluating education programs: A guidebook for practice. University of Illinois at Urbana-Champaign, IL. (http://comm.eval.org/EVAL/Resources/ViewDocument/?DocumentKey=f3c734c0-8166-4ba4-9808-a07e05294583)
Greene, J.C. (2006). Toward a methodology of mixed methods social inquiry. Research in the Schools. Special Issue: New Directions in Mixed Methods Research, 13(1), 93-99.
Hall, J.N., Greene, J.C., & Ahn, J. (2012). Values-engagement in evaluation: Ideas, implications, and illlustrations. American Journal of Evaluation, 33(2), 195-207.
Julnes, G. (Ed.) (2013). Promoting valuation in the public interest: Informing policies for judging value in evaluation. New Directions for Evaluation no. 133.
Evaluation is the social practice of valuing.
Evaluation is the social practice of gathering and using empirical information to make judgments about the merit and worth of a given social or educational intervention somewhere in the world. Both our empirical data and our judgments are grounded in particular values, which are too often unnamed and thereby under-used to advance our culturally responsive agenda of cultural respect and socio-political equity.
Where do values show up in evaluation practice?
First, our data are significantly shaped by the evaluation approach and the key evaluation questions that frame and steer a particular evaluation study, and by the different methodologies that support these different evaluation approaches and questions. A policy-oriented evaluation often asks questions about statistically significant outcome attainment, on the average and sometimes also as disaggregated by targeted subgroups, and uses an experimental methodology to address these questions. A participatory evaluation often asks questions about the quality of program services and their match to priority community needs, and uses a case study or participatory methodology to address these questions. And second, our judgments of program success are anchored in a set of quality criteria, too rarely explicitly named or justified. From a policy evaluation perspective, a “good” program has statistically significant average outcome effects, resting on utilitarian and instrumental understandings of the common good. From a participatory perspective, a “good” program addresses important needs of the targeted community, resting on democratic, pluralistic, and community-oriented understandings of the common good.
In culturally responsive evaluation, how can underlying values importantly and consequentially inform evaluation practices?
Privileged in culturally responsive evaluation (CRE) are the needs and interests of cultural minorities in our society who have yet to attain fair and equitable access to the resources and opportunities needed for the development of each person’s full human potential. The boundary lines defining these minorities are constructed along racial, ethnic, gender, economic, geographic, and identity lines. CRE thus, in theory, advances values of inclusion and pluralism, of contextuality and history, of fairness and equity, and of socio-political justice. In practice, CRE work can more consequentially advance these values by clearly naming and justifying them as integral to this evaluation approach, its key questions and audiences, and its accompanying designs and methods. Even more powerfully, the commitments of CRE can be advanced with a clear articulation and justification of – in tandem with a conversation about – the values that constitute the criteria by which a program is judged as worthy or not.
References
Boyce, A.S., DeStefano, L., & Greene, J.C. (September 2015). Values-Engaged, Educative Evaluation
Approach. MSPnet.
Greene, J.C., Boyce, A., & Ahn, J. (2011). A values-engaged educative approach for evaluating education programs: A guidebook for practice. University of Illinois at Urbana-Champaign, IL. (http://comm.eval.org/EVAL/Resources/ViewDocument/?DocumentKey=f3c734c0-8166-4ba4-9808-a07e05294583)
Greene, J.C. (2006). Toward a methodology of mixed methods social inquiry. Research in the Schools. Special Issue: New Directions in Mixed Methods Research, 13(1), 93-99.
Hall, J.N., Greene, J.C., & Ahn, J. (2012). Values-engagement in evaluation: Ideas, implications, and illlustrations. American Journal of Evaluation, 33(2), 195-207.
Julnes, G. (Ed.) (2013). Promoting valuation in the public interest: Informing policies for judging value in evaluation. New Directions for Evaluation no. 133.
Author
Jennifer C. Greene is a professor at the University of Illinois at Urbana-Champaign and a CREA-UIUC Affiliate
RSS Feed
