Evaluation...More Than Just Assessment!

 In the world of evaluation, professionals play a crucial role in assessing programs and policies to ensure their effectiveness. For my current coursework in LDT 506, I completed a self-reflection exercise on competencies as an evaluator. This process involved taking a closer look at my strengths and areas for improvement in regard to understanding evaluation as a practice.  Using a form provided to me, I looked at each of the five domains presented in the AEA Evaluator Competencies (2018) and used a Likert scale to determine my confidence/knowledge about each aspect.

For the first domain, “Professional Practice,” my average rating was 5.11.  This indicates that I have a solid understanding of the fundamental responsibilities of an evaluator.  For the second domain, “Methodology,” my average rating was 5.07, which indicates that I am very confident in the processes required to conduct an evaluation.  For the third domain, “Context,” my average rating was 5.25.  This is slightly higher than the first two domains, which shows I feel very confident about how to incorporate various factors that influence data into the evaluation process.  In the fourth domain, “Planning and Management,” my average score was 4.5.  This was my lowest-scoring domain, although it is still above the middle score on the Likert scale.  This indicates that I have a conceptual understanding of the practices related to evaluation, but I do not feel as confident about putting them into practice.  The final domain, “Interpersonal,” includes aspects related to effective communication.  My average score across this domain was 5.75 (my highest score) which shows that I feel very confident about my current ability to adhere to these communication standards.

My average across all domains was 5.14, which indicates that I feel I have a strong understanding of both the responsibilities of and strategies required to be an effective evaluator. This is a self-reported value, so there is a bias in the values and realistically I would likely fall a bit lower. Although I have not conducted an evaluation or acted in the professional role of an evaluator, I do believe that I have a strong foundation of knowledge and would be competent in this field.  My primary weakness would be developing skills related to specific technology used in the evaluator role.  I would also need to be trained according to the agency’s preferred methods of collecting data (i.e. surveys versus case studies).

When reviewing the evaluator competencies, I initially felt surprised that so many of them were focused on parts of the job that are not related to actually collecting and communicating data.  My initial assumption was that most or all of the competencies would center on presenting reliable and valid findings; instead, most of them deal with interpersonal considerations like the ability to communicate effectively and incorporate diverse perspectives when developing and reporting evaluation findings.  This was a positive surprise, though, because I feel that the ability to acknowledge inherent biases in a situation should be part of creating evaluations that are able to present equitable data.

Outside of this class, if I were pursuing further training in the field of evaluation, I would focus particularly on learning technology and programs related to this field.  I would also want to educate myself on the types of data collection that evaluation agencies prefer.

 

References

American Evaluation Association. (2018). AEA evaluator competencies. AEA.  https://www.eval.org/About/Competencies-Standards/AEA-Evaluator-Competencies.