Evaluation...More Than Just Assessment!

 In the world of evaluation, professionals play a crucial role in assessing programs and policies to ensure their effectiveness. For my current coursework in LDT 506, I completed a self-reflection exercise on competencies as an evaluator. This process involved taking a closer look at my strengths and areas for improvement in regard to understanding evaluation as a practice.  Using a form provided to me, I looked at each of the five domains presented in the AEA Evaluator Competencies (2018) and used a Likert scale to determine my confidence/knowledge about each aspect.

For the first domain, “Professional Practice,” my average rating was 5.11.  This indicates that I have a solid understanding of the fundamental responsibilities of an evaluator.  For the second domain, “Methodology,” my average rating was 5.07, which indicates that I am very confident in the processes required to conduct an evaluation.  For the third domain, “Context,” my average rating was 5.25.  This is slightly higher than the first two domains, which shows I feel very confident about how to incorporate various factors that influence data into the evaluation process.  In the fourth domain, “Planning and Management,” my average score was 4.5.  This was my lowest-scoring domain, although it is still above the middle score on the Likert scale.  This indicates that I have a conceptual understanding of the practices related to evaluation, but I do not feel as confident about putting them into practice.  The final domain, “Interpersonal,” includes aspects related to effective communication.  My average score across this domain was 5.75 (my highest score) which shows that I feel very confident about my current ability to adhere to these communication standards.

My average across all domains was 5.14, which indicates that I feel I have a strong understanding of both the responsibilities of and strategies required to be an effective evaluator. This is a self-reported value, so there is a bias in the values and realistically I would likely fall a bit lower. Although I have not conducted an evaluation or acted in the professional role of an evaluator, I do believe that I have a strong foundation of knowledge and would be competent in this field.  My primary weakness would be developing skills related to specific technology used in the evaluator role.  I would also need to be trained according to the agency’s preferred methods of collecting data (i.e. surveys versus case studies).

When reviewing the evaluator competencies, I initially felt surprised that so many of them were focused on parts of the job that are not related to actually collecting and communicating data.  My initial assumption was that most or all of the competencies would center on presenting reliable and valid findings; instead, most of them deal with interpersonal considerations like the ability to communicate effectively and incorporate diverse perspectives when developing and reporting evaluation findings.  This was a positive surprise, though, because I feel that the ability to acknowledge inherent biases in a situation should be part of creating evaluations that are able to present equitable data.

Outside of this class, if I were pursuing further training in the field of evaluation, I would focus particularly on learning technology and programs related to this field.  I would also want to educate myself on the types of data collection that evaluation agencies prefer.

 

References

American Evaluation Association. (2018). AEA evaluator competencies. AEA.  https://www.eval.org/About/Competencies-Standards/AEA-Evaluator-Competencies.

Learning About Learning Is Meta

 The word “metacognition” means (in layman’s terms) to think about thinking.  Similarly, learning about the concept of learning is likewise a metacognitive process.  It takes reflection and analysis to not only comprehend the theories presented, but also to synthesize them with the learning you are simultaneously undergoing so that you may have a better understanding of them.

During the EDP 540 course, I was able to engage in this process each week; as new theories were introduced with each new module, I compared them with previous teachings in order to develop a more comprehensive picture of learning design as a whole.  In the field of instructional design, knowledge of these theories creates a foundation which can be used to craft learning materials that are tailored to diverse learners and therefore far more effective.

Behaviorism and Gamification

In this module, I integrated behaviorism and gamification principles into an interactive Desmos escape room. Behaviorism, which emphasizes learning through reinforcement, was applied by structuring the escape room with immediate feedback mechanisms. Correct responses unlocked new clues, reinforcing desired learning behaviors. Gamification elements such as progression and challenges increased engagement and motivation. This approach effectively transformed a boring work training into an immersive experience, making learning both fun and impactful.

 

 Humanism and Motivation

In order to demonstrate Keller’s ARCS (Attention, Relevance, Confidence, and Satisfaction) model, I designed an infographic to engage learners in understanding remote work best practices. Attention was captured through bold visuals and compelling statistics. Relevance was addressed by aligning content with real-world remote work challenges. Confidence-building strategies included clearly stated productivity tips, while satisfaction was reinforced by including information about additional resources. This structured approach ensured that learners remained engaged and motivated to apply the information.

 Cognitivism and Mayer’s 12 Principles of Multimedia Learning

To demonstrate cognitivism, I created a slideshow presentation that followed Mayer’s Multimedia Principles, which emphasize reducing cognitive overload and enhancing learning through audio-visual processing. Stock photos were used to complement text, providing visual representation of key concepts. The coherence principle guided the elimination of extraneous elements, while color consistency enhanced visual appeal and readability. This intentional design improved information retention and learner engagement.

 Constructivism and Community of Inquiry (CoI) Framework

The Community of Inquiry (CoI) framework was applied to foster social, cognitive, and teaching presence in an online learning community. My activity, "Mentor Meetup," paired veteran employees with new hires for virtual mentorship. This initiative promoted social presence through relationship-building, and cognitive presence by facilitating meaningful discussions.  It also provided opportunities to share and learn from diverse backgrounds and created a teaching presence through guided mentorship. 

Sociocultural Learning Theory, Situated Learning Theory, and Communities of Practice (CoP)

My team-building activity named "Secret Ballot" included both Sociocultural learning theory and situated learning. In this exercise, marketing team members anonymously submitted ad concepts, which their peers then attempted to match to the team members who submitted them. This activity fostered collaborative learning, leveraging each participant’s cultural background and unique perspectives. By contextualizing learning within a shared social experience, team members developed a deeper understanding of branding strategies and creative diversity.  Additionally, I included my own personal experiences in the military as a narrative in this activity. I was able to reflect on how the military created opportunities for teams full of diverse people who worked toward the same objectives.

Self-Determination Theory and Moore’s Theory of Transactional Distance

The collaborative project required an application of Self-Determination Theory, Transactional Distance Theory, and generative AI to design an innovative cybersecurity training program. My team developed a quick pitch for a training scenario that would be tailored to the user, but the part that I found to be most impactful was the concept of an AI-powered chatbot that provided on-demand cybersecurity support. Self-determination principles were incorporated by fostering autonomy through self-paced learning, competence via interactive scenarios, and relatedness through team collaboration. We employed transactional distance theory to help conceptualize a training environment that did not feel extremely removed from an in-person learning environment. Late in the development stage, my team realized that revisions were required to align with project requirements.  We communicated effectively as a team and were able to update our pitch to include several key features that it was missing before submitting.

 

Each module demonstrated how learning theories collectively contribute to effective instructional design. Behaviorism and gamification enhanced motivation, while Keller’s ARCS model structured engaging content. Mayer’s principles improved multimedia learning, the CoI framework facilitated community-driven learning, and sociocultural theory fostered collaborative experiences. Integrating these diverse frameworks led to effective, meaningful learning experiences.

Connectivism and Networked Learning

Connectivism and networked learning offer powerful strategies for ongoing professional growth. By engaging with online communities, professional groups, and learning networks, instructional designers can stay updated with emerging trends and technologies. Strategies include participating in Online Forums, joining Professional Organizations, and attending webinars and conferences. By actively engaging in these networked learning opportunities, instructional designers can continuously refine our skills, adapt to new learning technologies, and stay ahead in a rapidly evolving field.