{"id":413,"date":"2015-10-26T11:16:08","date_gmt":"2015-10-26T08:16:08","guid":{"rendered":"https:\/\/levin-rozalis.com\/?page_id=413"},"modified":"2015-10-26T11:16:33","modified_gmt":"2015-10-26T08:16:33","slug":"the-profession-of-program-evaluation","status":"publish","type":"page","link":"https:\/\/levin-rozalis.com\/the-profession-of-program-evaluation\/","title":{"rendered":"The Profession of Program Evaluation"},"content":{"rendered":"

Program evaluation<\/strong>\u00a0is a methodical process for answering questions about specific programs, projects, policies, initiatives and\u00a0organizations. Evaluators use systematic methods and almost every known methodology for collecting, processing, organizing and analyzing data from all sources of information, in order to create useful knowledge for different stakeholders and interested parties.<\/p>\n

Usually, the aim of evaluation is to improve a process, but sometimes there are hidden agendas such as control or mastering power.<\/p>\n

There are many approaches\u2014and even ideologies\u2014among evaluators and evaluation customers as to what is the best way to conduct an evaluation. Two major epistemological or even ontological approaches exist: the Human Agency and the System Agency, as shown below.[1]<\/sup><\/sup><\/a><\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n
Criteria <\/strong><\/td>\nSystem Agency<\/strong><\/td>\nHuman Agency<\/strong><\/td>\n<\/tr>\n
Function<\/strong><\/td>\nSupervision, control, command<\/td>\nLearning<\/td>\n<\/tr>\n
Goal<\/strong><\/td>\nStandardization<\/td>\nRecognition and examination of otherness, diversity, difference<\/td>\n<\/tr>\n
Frame<\/strong><\/td>\nSystemic, structural<\/td>\nDiagnostic: diagnosing differences within and between<\/td>\n<\/tr>\n
Target<\/strong><\/td>\nProducts<\/td>\nProcesses<\/td>\n<\/tr>\n
Benefit<\/strong><\/td>\nClassification<\/td>\nReinforcement, development, enhancement, advancement<\/td>\n<\/tr>\n
Outcome (educational)<\/strong><\/td>\nKnowledge<\/td>\nSkills, learning processes<\/td>\n<\/tr>\n
Research approach<\/strong><\/td>\nAnalytical<\/td>\nMainly holistic<\/td>\n<\/tr>\n
Methodology<\/strong><\/td>\nExperimental<\/td>\nResponsive<\/td>\n<\/tr>\n
Focus<\/strong><\/td>\nExternal to the evaluee<\/td>\nInternal to the evaluee<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

These are two completely different perceptions of the nature of the world and of the role of program evaluation. One, on the left, sees permanence in the world, distributions that can be predicted, and tries to understand them in order to obtain better control of the system. The second sees the world as a mosaic of otherness that is relative and context-dependent, where it is difficult to infer anything about one from what is observed in another. It deals with complexities that are ever-changing and impossible to predict. Some evaluators will swear on one, and some, on the other. I myself believe that it depends on the question we need to evaluate, but the orthodox might claim that even the questions of one kind or another are not the right questions.<\/p>\n

We can add to that the conflict between the RCT (Randomized Controlled Trials) party versus the qualitative party versus the mixed-methods group, which can be violent at times.<\/p>\n

My claim is that all these paradigms and methodological disagreements among the practitioners of program evaluation are futile and lead nowhere. I believe that the role of evaluation as a profession is to convert information into knowledge,<\/u> and that can be done using any methodology that does not misrepresent reality.<\/p>\n

The emphasis in most evaluation courses and training is on research methods, which we cannot do without. But all the research methods, all the ontological and epistemological approaches, all the ideological approaches are part of a whole that gives them meaning: the mission of the profession, which is to create new knowledge. What\u2019s happening at present is that parts of that whole are fighting among themselves for supremacy. The moment it becomes clear that they are part of something, that an analytical method is a means to an end, that an evaluation approach serves a process greater than itself, they will no longer be the be-all and end-all, but a school of thought within a unifying professional entity.<\/p>\n

This does not change anything about what evaluators need to know; it changes only the emphasis. It means that, apart from the basics, evaluators must be fully conversant with what\u2019s happening in their professional world\u2014evaluation theories, fields of knowledge, the various schools of thought\u2014so that they know to ask for help; they know that in this matter I need an evaluator who knows A, B, or C (like a medical doctor knows that for that matter I need to consult a cardiologist or an orthopedist). Additionally, they need to be educated in multidimensional critical thinking. They must possess the ability to constantly ask questions about everything\u2014even about their own work\u2014not to take anything for granted. And then their need for viewpoints other than their own\u2014for collaboration with schools of thought or work methods different from their own\u2014will grow.<\/p>\n

 <\/p>\n

Having said that, it is important for me to emphasize the integral aspect of evaluation, which is that it is a purpose-driven act that has a direct influence on people\u2019s lives. The research objects in evaluation are people. Always. Even when the evaluation\u2019s objective is to examine whether a work approach is successful, the research objects will be people and the program in question will be for people. The inevitable conclusion of this is that evaluators must take into account where their evaluation is leading. We, as evaluators, have to address the implications of the evaluation and the effect it might have in the broadest possible sense, far beyond the question of the nature and quality of the program or its expansion or termination.<\/p>\n

It is no less important to remember that the most significant criterion for examining the implications and effects of an evaluation is not our personal worldview, what we believe or don\u2019t, nor that of the people involved in the intervention. The significant criterion is in the effect of the evaluation on the life and worldview of those who will be affected by those implications and effects: the evaluees and their environment.<\/p>\n

And this is a very great responsibility. As evaluators, we bear this social responsibility: to look broadly, to look forward, to be responsible for our actions, and as far as possible, to be sure that the purpose is indeed for the benefit of those who will ultimately be affected by our work.<\/p>\n

This is the compass that should lead all the professional decisions we make while conducting an evaluation.<\/p>\n

To conclude, our aim is to create knowledge\u2014knowledge that is meant to lead actions in a specific program, project, organization, or intervention\u2014actions that will potentially make a difference in people\u2019s lives. And this we must not forget. It is our professional responsibility to make sure that this is ultimately for the benefit of those who will be affected by it.<\/p>\n

 <\/p>\n

[1]<\/a> Levin-Rozalis, M. (2014) Let\u2019s Talk Program Evaluation in Theory and Practice. Samuel Wactman\u2019s Sons Inc.: Monterey, CA.<\/p>\n","protected":false},"excerpt":{"rendered":"

Program evaluation\u00a0is a methodical process for answering questions about specific programs, projects, policies, initiatives and\u00a0organizations. Evaluators use systematic methods and almost every known methodology for collecting, processing, organizing and analyzing data from all sources of information, in order to create useful knowledge for different stakeholders and interested parties. Usually, the aim of evaluation is to […]<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":[],"yoast_head":"\nThe Profession of Program Evaluation - Miri Levin Rozalis<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/levin-rozalis.com\/the-profession-of-program-evaluation\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"The Profession of Program Evaluation - Miri Levin Rozalis\" \/>\n<meta property=\"og:description\" content=\"Program evaluation\u00a0is a methodical process for answering questions about specific programs, projects, policies, initiatives and\u00a0organizations. Evaluators use systematic methods and almost every known methodology for collecting, processing, organizing and analyzing data from all sources of information, in order to create useful knowledge for different stakeholders and interested parties. Usually, the aim of evaluation is to […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/levin-rozalis.com\/the-profession-of-program-evaluation\/\" \/>\n<meta property=\"og:site_name\" content=\"Miri Levin Rozalis\" \/>\n<meta property=\"article:modified_time\" content=\"2015-10-26T08:16:33+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/levin-rozalis.com\/the-profession-of-program-evaluation\/\",\"url\":\"https:\/\/levin-rozalis.com\/the-profession-of-program-evaluation\/\",\"name\":\"The Profession of Program Evaluation - Miri Levin Rozalis\",\"isPartOf\":{\"@id\":\"https:\/\/levin-rozalis.com\/#website\"},\"datePublished\":\"2015-10-26T08:16:08+00:00\",\"dateModified\":\"2015-10-26T08:16:33+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/levin-rozalis.com\/the-profession-of-program-evaluation\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/levin-rozalis.com\/the-profession-of-program-evaluation\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/levin-rozalis.com\/the-profession-of-program-evaluation\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/levin-rozalis.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"The Profession of Program Evaluation\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/levin-rozalis.com\/#website\",\"url\":\"https:\/\/levin-rozalis.com\/\",\"name\":\"Miri Levin Rozalis\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/levin-rozalis.com\/#\/schema\/person\/e395e28400c77853e8b02d4d22785c49\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/levin-rozalis.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\/\/levin-rozalis.com\/#\/schema\/person\/e395e28400c77853e8b02d4d22785c49\",\"name\":\"mirilevin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/levin-rozalis.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/levin-rozalis.com\/wp-content\/uploads\/2015\/05\/miri.jpg\",\"contentUrl\":\"https:\/\/levin-rozalis.com\/wp-content\/uploads\/2015\/05\/miri.jpg\",\"width\":229,\"height\":261,\"caption\":\"mirilevin\"},\"logo\":{\"@id\":\"https:\/\/levin-rozalis.com\/#\/schema\/person\/image\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"The Profession of Program Evaluation - Miri Levin Rozalis","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/levin-rozalis.com\/the-profession-of-program-evaluation\/","og_locale":"en_US","og_type":"article","og_title":"The Profession of Program Evaluation - Miri Levin Rozalis","og_description":"Program evaluation\u00a0is a methodical process for answering questions about specific programs, projects, policies, initiatives and\u00a0organizations. Evaluators use systematic methods and almost every known methodology for collecting, processing, organizing and analyzing data from all sources of information, in order to create useful knowledge for different stakeholders and interested parties. Usually, the aim of evaluation is to […]","og_url":"https:\/\/levin-rozalis.com\/the-profession-of-program-evaluation\/","og_site_name":"Miri Levin Rozalis","article_modified_time":"2015-10-26T08:16:33+00:00","twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/levin-rozalis.com\/the-profession-of-program-evaluation\/","url":"https:\/\/levin-rozalis.com\/the-profession-of-program-evaluation\/","name":"The Profession of Program Evaluation - Miri Levin Rozalis","isPartOf":{"@id":"https:\/\/levin-rozalis.com\/#website"},"datePublished":"2015-10-26T08:16:08+00:00","dateModified":"2015-10-26T08:16:33+00:00","breadcrumb":{"@id":"https:\/\/levin-rozalis.com\/the-profession-of-program-evaluation\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/levin-rozalis.com\/the-profession-of-program-evaluation\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/levin-rozalis.com\/the-profession-of-program-evaluation\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/levin-rozalis.com\/"},{"@type":"ListItem","position":2,"name":"The Profession of Program Evaluation"}]},{"@type":"WebSite","@id":"https:\/\/levin-rozalis.com\/#website","url":"https:\/\/levin-rozalis.com\/","name":"Miri Levin Rozalis","description":"","publisher":{"@id":"https:\/\/levin-rozalis.com\/#\/schema\/person\/e395e28400c77853e8b02d4d22785c49"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/levin-rozalis.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/levin-rozalis.com\/#\/schema\/person\/e395e28400c77853e8b02d4d22785c49","name":"mirilevin","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/levin-rozalis.com\/#\/schema\/person\/image\/","url":"https:\/\/levin-rozalis.com\/wp-content\/uploads\/2015\/05\/miri.jpg","contentUrl":"https:\/\/levin-rozalis.com\/wp-content\/uploads\/2015\/05\/miri.jpg","width":229,"height":261,"caption":"mirilevin"},"logo":{"@id":"https:\/\/levin-rozalis.com\/#\/schema\/person\/image\/"}}]}},"_links":{"self":[{"href":"https:\/\/levin-rozalis.com\/wp-json\/wp\/v2\/pages\/413"}],"collection":[{"href":"https:\/\/levin-rozalis.com\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/levin-rozalis.com\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/levin-rozalis.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/levin-rozalis.com\/wp-json\/wp\/v2\/comments?post=413"}],"version-history":[{"count":2,"href":"https:\/\/levin-rozalis.com\/wp-json\/wp\/v2\/pages\/413\/revisions"}],"predecessor-version":[{"id":415,"href":"https:\/\/levin-rozalis.com\/wp-json\/wp\/v2\/pages\/413\/revisions\/415"}],"wp:attachment":[{"href":"https:\/\/levin-rozalis.com\/wp-json\/wp\/v2\/media?parent=413"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}