Is evaluation necessary? This is the question I asked myself around May of 2015. At that time, I was responsible for updating an evaluation manual for my customers--which I could have completed within weeks--but something was missing. Customers were still struggling to effectively evaluate training programs and I was having a hard time understanding why. Learning is essential for growth and goal accomplishment. Learning from mistakes is essential in re-calibrating and ensuring the correct goal is accomplished. Sounds simple...so what was the problem? Two primary problems seemed to be lack of program manager knowledge of evaluation methods, lack of funding to perform evaluations, and lack of the agency leader's support and interest in the data. Again, why? If evaluation is as important and useful as the American Evaluation Association, the United Nations Evaluation Group, the Commission on Evidence-Based Policy Making, nonprofits, business, and hundreds of associations, policies, and practitioners say it is...why is evaluating and the decision-making that follows (or should follow) so unappreciated, underfunded and difficult? The "5 whys" and other root cause analysis methods were not providing satisfactory answers.
As a self proclaimed " Evaluation Evangelist", asking "Is evaluation necessary?" is like Neil deGrasse Tyson genuinely asking, "Is science necessary?" First, why ask a question when the answer is obvious and second, why ask a question that threatens the validity of your work and profession? Much of our identity is in the work we do which is why we describe ourselves as the roles we play. Roles have functions and we validate those functions through actions and decisions. When asked, "who are you" we typically respond with, "I am a mother, author, public speaker, evaluator, daughter, mentor, american, scientist etc." I've never responded to "who are you" with a list of my features (e.g. I am 2 eyes, hair, 2 legs, fingers, muscle joints, skin).
I am what I do. I am what I believe. I am what I support and know. Much of the work I do, and love, is the work of evaluating. Logic and reasoning says if that work is not necessary, then I (as the doer of the work) am not necessary. Asking "is evaluation necessary" was very risky...and I was honestly afraid as I contemplated and searched for an answer.
It was my willingness to be unnecessary and to ask questions where answers already exist that led me to discover an evolved problem and create an entirely new and substantially more evolved and useful language, model, practice, and role for learning and evaluation in work and life.
I'll use a presently popular debate to explain how I did this.
The Flat Earth Society is a group of people who find meaning in the idea that the Earth is flat. I'm not a member and I don't know all the details of their beliefs but I do know that just like all humans on this planet, "flat earthers" invest time, attention, physical and mental energy, money, trust and emotion in being, supporting and defending this role.
The idea of a flat earth has a very long history and I'll leave it you to investigate further if you like, but of interest to me is the method used to support a "flat earth" conclusion and the "tone" of the conclusion. Instead of the scientific method, the conclusion of a flat earth rests largely on the the zetetic method which is also a system of scientific inquiry. The word is derived from the Greek verb ζητέω (zeteo), which means "I seek; I examine; I strive for". The zetetic method bases conclusions on experimentation and observation (going from the specific to the general) rather than the scientific method where an initial theory is proved or disproved (general to the specific).
Deductive: Conclusion is valid or invalid (e.g. the conclusion certainly/necessarily follows premises )
Inductive: Conclusion is strong or weak (describes the probability that the conclusion is true & allows a false conclusion even if all premises are true)
There are a list of things I could point out about these two methods but it's the conclusion that is most relevant here. Both conclusions are subject to revision but the Zetetic conclusion, no matter how "strong", openly acknowledges the reality of unknown information and the limits of perspective. The scientific conclusion is more "right until proven wrong". The methods, tools, technology and perspectives used to arrive at the conclusion are relevant to the veracity of the final verdict and I clearly see that but the idea of something being "certain", "right" or "true" crystallizes and hardens the idea into "the answer". "Right" and "true" are the main ingredients of moral frames.
In my book "Firing God" I tell my story of "escaping" a moral frame in my personal life. In my personal life, I found that being "right" was actually having a devastating impact on my overall ability to flourish, grow authentically and experience happiness. Being "right" was the reward for my morally framed actions and decisions. This frame also comes with other "gifts" like guilt, regret, shame, obligation, unworthiness and an exhausting defensive posture.
I discovered that of greater importance than "rightness" is the freedom to independently and contextually assign merit, value and significance to conclusions (whether self-created or imposed) in guiding my decision-making, thinking, and actions. Moral framing is an option, not a necessity.
George Box, a famous statistician, said, "All models are wrong, but some are useful". His point was that we should focus more on whether something can be applied to everyday life in a useful manner rather than debating endlessly if an answer is correct in all cases.
In what ways is the idea of a flat earth useful in everyday life? In what ways is the idea of a round earth useful in everyday life? What benefits in everyday life does one group have over the other? How are the ideas limiting functionality, growth and adaptation physically, socially, emotionally, creatively, etc.
To discover whether an idea is useful for me in everyday life is something that requires me to put in the work. It's primarily through present observation, questioning, and verification with my own eyes, ears, intuition, and other senses that will tell me if something is useful for me. "The earth is flat", is a useful idea for some people. It's an idea that guides their actions, thinking and decisions. A round earth is also a useful idea that guides actions, thinking and decisions. Recognizing that others can have beliefs about the world that are diverging and non-threatening to my value and use is a mental state that develops as we grow and learn.
Until then, information requires security, trust is regulated, defense is a department and "right" is the one true perspective and only lens through which everyone must see to find answers and to ensure permanent functionality and purpose.
To simply ask the question, "Is evaluation necessary" I had to abandon the present answer (account) provided by decades of research, authorities, journals, tradition, established associations, international conferences, certificate programs, educational institutions, and credentialed professionals.
In my investigation to find an answer (account) to my question, I had to first get over my need to be right about evaluation.
And I did.
To reach the conclusions below, I engaged in years of observation, experimentation and research outside the evaluation community. I began gathering my own data, forming my own questions, interviewing and talking to people I felt I should talk to and assigning merit, value and significance using my findings and my own insights. I went to various fields including bio-mimicry, information technology, portfolio management, change management, forensics, mindfulness, agile methodology, design, machine learning, risk management, cyber security and block-chain technology. Clearly other perspectives were absolutely included, considered and weighed...but I did the including, considering assignment of weight.
I also viewed and studied experiments like Edward Tronick's Still Face Experiment and the dodder vine experiment from What Plants Talk About. Monica Gagliano's experiment and paper, "Experience Teaches Plants to Learn Faster and Forget Slower in Environments Where it Matters" and the idea of intelligence without the need of requirement of a human mind were also influential in helping me reach the following conclusions:
Evaluation is a natural process and inherent function of the evaluator.
Learning, evaluation, decision making and accountability are all one process
No matter the organizational structure, the evaluator is the decision maker. It's the mental and physical constraints and controls (the design of the decision environment) that diminish or enhance use of evaluation capacity (which proportionately diminishes and enhances learning, decision-making and accountability).**See the Cynefin Model of Decision Environments
Evaluation is not synonymous with judgment, nor does evaluation lead to a judgment.
Judgment is NOT evaluation...judgment is declaration of investment in and allegiance to external authority (e.g. strategic goals, training objectives, mental models, moral frames, standards, law, policies and procedure, internal controls, cultural norms, etc.). Judgment does not replace the evaluation process.
These conclusions are just the tip of an interesting and exciting iceberg.
So, is evaluation necessary? Yes, but evaluators (as we are now) are not.
The evolved role of evaluators is something that I will address at a later time.
My conclusions are not "right" but they are (and will be) widely shared and exponentially useful in the verifiable:
effective and useful design of learning
evaluation of knowledge gains and risks through learning
mapping of accountability in all decision environments,
enhancement of authentic work functions,
Return on Knowledge (RoK) (appreciation [escalation in value] of knowledge, action, creativity, and innovation)
impact on business, life, and every endeavor in-between
P.S. My personal return on knowledge as a result of this journey is a much more aesthetic view of the world and a deeper understanding of a variety of topics and their relationship to each other. My knowledge has appreciated exponentially. I attained certifications in Prince2 Project Management, Scrum, Six Sigma, User Experience (UX) Design, Digital Marketing and others. I also formed connections with individuals across several disciplines and in varied countries. The process used to come to these conclusions IS the new and evolved process!