Evaluator Competencies Reflection

In this post, I am going to address where I find myself as an evaluator of a program rather than one who administers and measures assessment outcomes. The difference between the two are specific terms of art. An “evaluator” is someone who focuses on making a judgment about values, numbers, or performance, aiming to draw conclusions about the value of something. An evaluation is product-oriented and determines the extent to which objectives are met, while an assessment is process-oriented and provides feedback for improvement.

Check list

Based on the self-assessment/survey of rating myself in the five domains of evaluator competencies and related subdomains as defined by the American Evaluation Association (where 1 as being a novice and 6 being an expert) I rate myself as a 4.5-5 as an evaluator. The five domains include the following: professional practice, methodology, context, planning and management, and interpersonal skills. I base this 4.5 rating on my experience by being a law school student and law clerk, serving as a technical subject matter expert (SME), working as a technical methods and procedure manager, an unofficial project manager, and the owner of a national training curriculum.    

Having been out of the organizational environment for 6 years the tools and technology available to evaluators has changed dramatically. With the access to big data analysis, to the use of AI to assist in the work of an evaluator, my knowledge of these technologies is lagging behind those that currently practice using these tools in their day-to-day careers. I often find myself working independently rather than in groups due to the nature of my personality. I would rather do something my way rather accepting an alternative. Many times, this makes it difficult to work with others. Serving as a project manager and running a team of instructional designers has helped me in this regard, but there is definitely room for improvement.

Many of the domains and subdomains find themselves related to my experience as a law clerk, SME, and methods and procedures manager. Lawyers are expected to maintain a high level of ethical standards. In the third year of law school there is even a class named “Professional Responsibility” that covers how to act professionally. In my training as a trial attorney and technical subject matter expert I had to develop skills that allowed me to communicate very complex concepts to a wide range of audiences in a myriad of contexts (be they members of culturally diverse executives at a fortune 15 company or juries). This is a double-edged sword that can expose very quickly how effective you are communicating to a given audience. There is always room for improvement. As a methods and procedures manager, it was my job to determine where process gaps existed and address those problems either through communications or training. All of these roles required the recognizing of inherent cognitive biases or biases within the existing procedural structure.

I was a bit surprised by the question how evaluation can play a role in promoting social justice and the public good. In my experiences, evaluations were always seen through the lens of an individual client or organization. Selfishly, I have always worked to save someone fiscal repercussions or make them money and ignoring the larger more important philanthropic nature of evaluation. Working for private companies it is easy to fall into the mindset that everything is about the bottom line. I am very interested in working on projects that help take me out of this rut and seeing how evaluations are used in the public sector.

I would like to continue to study the technologies used by present day evaluators. The Learning Design and Technologies program has been helpful in catching me up to speed. Conversely, the program has also revealed how much I do not know about the tools used. Some of the technologies are intuitive and surprisingly easy to use. I recently used a statistical analysis tool named Jeffreys's Amazing Statistics Program (JASP) to distill data that I needed to know out of a larger data set with little effort. My goal is to learn more about such tools and how to use them.       

I look forward to learning more about them as the program continues.  Continue to study different organizational structures and how best to communicate the results of an evaluation. Communication skills can always be honed, especially when addressing leadership/stakeholders as an outside and impartial consultant. Your evaluation may involve significant expenditures on the client side. Not everyone will want to hear what you have to say as it may result in proposing a reorganization of a department(s) or replacing employees with those that are better suited for a particular job. I am not great at having hard conversations and need to practice at having them.

I am actively pursuing my Project Management Professional (PMP) certification so that I might better show me value through various ways such as effective communication, planning and organization, risk management, cost control, team leadership, problem-solving, adaptability, stakeholder management, and post-project evaluation. Part of the PMP certification exam addresses how to have difficult conversations, be they with leadership of members of the project team. As I prepare for the exam, I hope to better learn techniques on handling difficult situations.

My goal is to better utilize the industry recognized competencies in my future as a learning designer or intermediary between the learning and design organization and leadership.

References

American Evaluation Association. (2018). The 2018 AEA Evaluator Competencies. Retrieved from https://www.eval.org/Portals/0/Docs/AEA%20Evaluator%20Competencies.pdf