Artifact #1: Evaluation Plan
My Contributions
- Majority of Evaluative Purpose & Part of Stakeholders
- Logic Model
- Document Analysis Table
- Focus Group
Summary
In EDIT 7350E: Evaluation and Analytics in Instructional Design, taught by Dr. Bagdy, my group Evaluation Plan was based on an anonymized pre-existing case study offered by Dr. Bagdy, though the process itself of creating evaluative methods and tools was priceless. The scenario introduced was an e-commerce company who realized that almost half of their employees left for other positions due to lack of growth potential, which the company attempted to solve by implementing training targeted at the leadership skills of their Directors. In the end, they hoped to evaluate the effectiveness of this training, which was where we came in.
We confirmed early on that the training lacked their own evaluations for tracking growth in their employees, as well as lacked any incentives for employees to engage in the material, which directly contradicts the Theory of Motivation—employees need to want to complete a training because it contributes to their personal goals (Sutton, 2024). To evaluate the training, we assessed the existing formative evaluations of the program, outlined all potential stakeholders, and created multiple types of summative evaluations to assess if employee retention and morale improved. I specifically crafted a thorough logic model that broke down each module of the training into resources and inputs, activities, outputs, short-term outcomes, and long-term outcomes. Although I participated in the process of deciding which three summative evaluation tools we should use, I personally created the focus group questions to collect qualitative data from employees who worked under Directors (the population assigned the training) to detect if the supposed changes in Director leadership was effective from the perspective of those they were intended to lead.
Citations
Sutton, J. (2024, February 14). Victor Vroom’s expectancy theory of motivation. PositivePsychology. https://positivepsychology.com/expectancy-theory/
Artifact #2: Accessibility Critique
Summary
In EDIT 7350E: Evaluation and Analytics in Instructional Design, taught by Dr. Bagdy, I’m happy to include this particular assignment in the “Evaluation” theme, as it is personally meaningful to me in multiple ways. The first reason is that I evaluated a video game, the Discovery Mode of Assassin’s Creed: Origins, which is from a series that I’ve played since I was a tween. Unlike the typical gameplay, this setting is intended to be fully instructional (as well as fun and exploratory, of course). The second reason is that I adore ancient history, and the intentions behind Discover Mode is to let the player navigate the beautiful world design of the late Ptolemaic Kingdom of Egypt without stressful enemy threats and to learn a plethora of interesting historical details compiled by real-life historians, so I had an excuse to sit down and read all these facts. The third reason is that this is an accessibility evaluation, which helped jump-start my interest in enhancing accessibility in my school projects, work assignments, and other life matters.
In this process, I utilized the framework developed in “Designing for Disability: Evaluating the State of Accessibility Design in Video games” (Brown & Anderson, 2021), evaluating the game based on auditory, visual, motor, and cognitive difficulty factors. My ultimate score was “accessible but flawed” for the auditory section, “needs minor-to-moderate improvements” for the visual section, “needs major improvements” for the motor section, and “accessible” for the difficulty section.
As I mentioned in my reflection, this process was an excellent practice in end-user empathy. Since I have been playing these games since I was in middle school, I realized that I was perhaps overly familiar with the gameplay and setting—even how the controllers worked. I had to force myself out of my headspace and into the headspace of someone who may have never been exposed to this media before.
Citations
Brown, M., & Anderson, S. L. (2020). Designing for Disability: Evaluating the State of Accessibility Design in Video Games. Games and Culture, 16(6), 702-718. https://doi.org/10.1177/1555412020971500