Cognitive Biases and Decision Making

The goal of this blog is to talk about how to best teach students how to successfully use applied statistics. During the last few weeks, and for the next few weeks, it is my plan to talk about a specific group of students: Administrators interested in making data driven decisions. During a prior blog, I reviewed the several components that should be part of such a training session. http://wp.me/pYrkk-d3 

From experience, I have found that administrators are best able to learn about how to make data driven decisions only when they first learn about epistemology, how we know what we know, which was covered at this blog http://wp.me/pYrkk-d6, and cognitive biases, innate tendencies that keep us from accurately assessing what is going on around us. Business Insider summarized 56 organizational cognitive biases here, http://www.businessinsider.com/cognitive-biases-2014-6?op=1. Psychology Today reviews how cognitive biases negatively impact businesses, http://www.psychologytoday.com/blog/the-power-prime/201305/cognitive-biases-are-bad-business 

Though there are many cognitive biases, in an organizational setting I like to speak about these 6.

1. Confirmation Bias – we seek out information that matches what we already believe to be true, ignoring all information that contradicts our beliefs.
2. Ingroup Bias – though quite complex, in short, we tend to prefer people we deem to be part of our group. We view them as more varied as people outside of our group. When they make mistakes we tend to be more forgiven or understanding. We tend to exert more energy to help them and protect them from harm.
3. Projection Bias – what we think and feel is what others are thinking and feeling.
4. Gambler’s Fallacy – that the risk we are about to take is going to pay off, especially after a series of bad events, as our luck is bound to change.
5. Status-Quo Bias – most people are simply more comfortable when things stay the same, even if they are less than ideal. Organizational change is not comfortable for most people.
6. Bandwagon Effect – I have heard people use the phrase, “sheeple” people who are following the herd regardless of what information might be saying otherwise.

What each of these cognitive biases have in common is that we are placed into a cognitive state where we ignore the data right in front of our nose, particularly if it is contradicting our firmly held belief. I was once in an administrative meeting after a particularly challenging decision, one for which the faculty were strongly against. And, yet, an administrator remarked that 80% of the faculty were on board with this decision. I didn’t know of one person, let alone 80% who were supportive of this decision outside of the people in the room, but a couple of cognitive biases were taking hold. The top administrators all felt this was a good idea, so with the bandwagon effect (among other pressures), so did the middle level administrators. Then, since they believed it to be a great decision for which they all agreed, they projected their thinking onto the vast majority of the faculty. A quick survey (formal or informal) would have helped them to see what the faculty were actually thinking. That information could have been used to either change their decision, weaken the intensity of the decision, or provide communication/justification as to why such a widely disagreed upon decision had to be implemented.

Properly designed measures and appropriate sampling techniques can yield great data that can be used to help provide insight to administrators to aid them in moving an organization forward.

Certainly, if we stick with our cognitive biases, we’ll feel better about ourselves, but that won’t help an organization become the best it can be, as in the end, an administrator is only as good as the decisions he or she is making.

In training administrators on how to best use applied statistics, start with explaining how data can help them achieving higher quality decisions by by-passing epistemological and cognitive bias limitations.

Advertisement

Epistemology and Decision Making

In a recent post, http://wp.me/pYrkk-d3, I outlined what information should be used in a series of training sessions for administrators to make data driven decisions. I have found that many people are resistant to the benefits of using data to make decisions even though they throw around phrases like, “data driven decisions,” or “business analytics,” or “big data,”  and as such, we have to first help them to understand how do we know what we know, and how does data fit into that set of knowledge?

Epistemology is the study of how we know what we know. Though there are many ways of classifying and characterizing epistemology types, I find that there are 4 different ways that we know:

1. Authority – We know what we know because someone tells us, and we believe it to be true. For example, everything I know about celebrating an event I learned because my mother and grandmothers told me so.
2. Intuition – Our gut tells us what is true. For example, my gut tells me my dog loves me.
3. Empiricism – We know through observations. This is why we collect data, to learn from it through empiricism.
4. Rationalism – The use of logic, both inductive and deductive, will help us know the truth. This classic example characterizes rationalism well. If a tree falls in the woods and no one is there to hear it, does it make a sound? We use rationalism to know that it does make a sound.

High quality data driven decisions are, primarily, a dance between empiricism and rationalism. We make observations (enrollment is down 20% in Sports Management over the last 5 years), create a prediction or explanation as to what we think is or will be going on (maybe students in these majors are not getting jobs), collect data to test the prediction (students are struggling to find jobs), and then based on the results (Make improvements Sports Management that will provide students with the skills needed for future success). And though as a scientist this may seem like a natural and obvious way to go about seeking information, this is not the standard protocol of seeking knowledge in many administrative ranks.

Decisions from intuition reign supreme in many administrative circles. And, it is true that during the past two years in the administrative role I made many gut level decisions. However, given all of our cognitive biases, which will be discussed in a future posting, this often leads us to less than optimal decisions. In a Harvard Business Review article on how good leaders make bad decisions, http://hbr.org/2009/02/why-good-leaders-make-bad-decisions/ar/ you can find examples of leaders who ignored data and relied upon their intuition to discern what was the optimal decision.

In administration, you can also see many sets of truths to be proclaimed simply because the person in authority told someone it was true. I was in a meeting, and asked a question about the justification for an expense. I was expecting some data to support the expense. Instead I was told, “The President says so.” That is authority, in its purest form. And yet, organizations are chock full of people who permitted authority to push a group decision in the wrong direction. Follow this link http://dssresources.com/cases/spaceshuttlechallenger/ for information on the Space Shuttle Challenge disaster, where the organizational culture and its reliance upon authority driven “truths” cost the lives of 7 astronauts in January of 1986.

I have found that before I can help administrators learn about techniques in data analysis to help answer important organizational questions, I have to first get them to think about what they know, how they know it, and recognize that it is through creating a prediction or explanation, then collecting data to evaluate that predication or explanation, and most importantly, letting the data speak as to what is going on, that we can unveil our eyes from our cognitive biases, and get to the bits of information we truly seek that will lead us to great decisions.

Epistemology and Cognitive biases go hand in hand in helping keep us from truly using data accurately for organizational decisions. As a result, after sharing with administrators the ways of knowing, we must also outline standard cognitive biases that keep us from seeing the truth. Common cognitive biases facing an organization will be discussed in a future post.

If you are interested in learning more about epistemology, these sites have detailed information.    http://pantheon.yale.edu/~kd47/e-page.htm or http://plato.stanford.edu/entries/epistemology/

 

Data Driven Decisions … doing it right

Hello All,

I know there has been a delay of two years in the postings at Statistical Sage. For the past two years, I have been serving as an interim administrator. However, I am returning to faculty, straight back to a half a year sabbatical! I am happy to also be returning to posting here about teaching applied statistics.

As I was weighing the cost and benefits of leaving the classroom for two years, it never occurred to me that my time as an administrator would increase my resolve for the importance of the highest quality curriculum and teaching for applied statistics classes, but of all that occurred in the past two years, that is exactly what happened.

As you see, administrators make countless decisions each and every week. Some of those decisions are fairly minor in nature, while others have huge and lasting impacts on the university, college, faculty, students, and in the case of state sponsored universities … the tax payers. Often, though not nearly as often as I would like, data are at the center of those decisions, making it critical to have people in administration with at least some fundamental understanding of the use of statistics in decision making.

Now, before people start getting upset stating that we cannot quantify all that is important in a classroom and university setting, I openly admit that is true.

Though it is true that not everything can be quantified and formally assessed, much can be. I will review the most critical information that needs to be covered in an applied statistics class designed for training college administrators.

Over the next few weeks, I will be covering each of these topics in more detail as to how to best deliver this material.

1. Epistemology, Decision Making, and Statistics
2. Cognitive Biases; How Statistics can be used to get to the Truth
3. Detecting Data Integrity Issues
4. Data Management Protocol
5. Populations vs. Samples
6. Observational Errors: Measurement, Experimental, and Sampling
7. Quality Decisions are Limited by the Quality of Measures
8. Sampling and Quality Decisions
9. Statistics and Sampling Error
10. Parameters and Mathematical Modeling vs. Inferential Statistics (Introduction)
11. Mathematical Modeling, Parameters, and Assumptions
12. Statistical Decision Errors: Type I and Type II

Though these topics will be directly targeted toward how to teach a university administrator how to be a great data driven decision maker, this information is equally useful to anyone in any position to make data driven decisions, and foundational for any class in applied statistics, regardless of the audience.

In the end, quality decisions based on data are only as good as the integrity and the ability of the person making it. Though not every decision in academia should be a data driven decision, the quality of such decisions are limited by the quality of the data, which are limited by the quality of the measure and the quality of the sample. Such decisions are also extremely constrained by the appropriate use of the appropriate statistic. Over the next several weeks, I look forward to reviewing this information in more detail.