Critical Concept: Making Sense of Variability

Last week I introduced the five most critical topics I felt a professor of applied statistics could impart upon her students https://statisticalsage.wordpress.com/2012/06/10/the-most-critical-concepts-in-applied-statistics-treating-students-like-family/.

1. Making Sense of Variability
2. Capturing Variability
3. Normal Distribution
4. Sampling Distribution of the Means and Standard Error
5. Understanding Hypothesis Testing

This week, I would like to go into detail with the first of those five, “Making Sense of Variability.”

As this is the first critical topic, and I typically start teaching on the very first day of class, this is an issue I typically start to lay the foundation for  on the first day of class. I begin by  conduct activities that help students to think about variability.  Thus, it is typically the second day of class when I hit the foundational concept of variability, but before I enter into this, I provide a framework for students about knowledge. How do we know what we know? This is basic epistemology.

What is statistics? It is a way of us to gather knowledge. We are making sense out of variability. That is all that statistics is. Yet, for a student to understand that, he or she must first understand how do we gather knowledge? What are the “Ways of Knowing,” that is foundational epistemology? http://www.acperesearch.net/waysofknowing.pdf 

• Empiricism – we learn through our observations/ perceptions (i.e., what are the data revealing?)
• Rationalism – we learn through the application of logic (i.e., if no one is in the woods when a tree falls, does it still make a sound?)
• Authority – we learn so much from what others tell us (i.e., what is your name? How do you know? Someone must have told you.)
• Intuition – gut instinct often reveals to us knowledge of a different type. (i.e., My dog’s love me. Science doesn’t tell me this, I just know.)

There are strengths and weaknesses to each way of knowing (follow the link above for additional details). By discussing this with students, we can start to understand why science has become a dance between empiricism and rationalism, as by combining the two … we first start with a logically deduced (or induced) hypothesis (rationalism) then we seek out data to test it (empiricism).  This goes so far beyond the steps of the scientific method discussed in middle schools throughout the US, as this progression is far from linear. I refer to it as a dance because there is give and take between these two approaches. Understanding how we know helps frame the context for statistics … it’s making sense of the observations. It’s just one small component of how we know anything.

In class assignment: This is fairly basic and should take no more than 10 – 15 minutes of class.

1. Start with asking students to make a list of what they know to be true.  (4 – 8 statements will work)
2. Have them form into groups of 4, and start to classify each of the statements into different ways of knowing. Any form of classification is fine.
3. Now, have them try to characterize each classification.
4. Bring the groups back together and see what they have found (in common/ different)
5. Now, introduce to them the four ways of knowing, providing an example and definition of each.
6. Have them put their examples into the four categories, and discuss.

Homework: Have a list of around 12 statements people “know,” and have them place them into one of the four categories.

From the four ways of knowing, we enter in into the foundational explanation of the Four Uses of Statistics. This topic is covered in detail in chapter 1 of Kiess and Green’s (2010) Statistical Concept for the Behavioral Sciences, 4/e. (http://www.pearsonhighered.com/educator/product/Statistical-Concepts-for-the-Behavioral-Sciences/9780205626243.page)  

It helps to provide the basis for which most statistics are used, while providing them with concrete examples.

1. Describe samples: what is the number of people who live in dormitories in the classroom? 
2. Draw inferences from samples to populations: If I wanted to get a sense of everyone in the class, but didn’t have the time to ask, I could just ask a sample of five students … how many hours per week to you expect to study for this class, and infer that the mean of the sample will be similar to the mean of the class (the population). Of course, issue’s of sampling error can start to be discussed.
3. Test hypotheses, about the relationship between two or more variables: I typically have students form a hypothesis, and then we get a sample of data to see if it’s correct. If you aren’t comfortable going with a class created option (which often requires you to tweak the hypothesis on the spot), a good hypothesis is … when it rains, people feel down.
4. Find associations among variables: Does where you sit in class impact how well you are going to do in the class?

 Incredibly, within a few examples, many concepts that underlie the use of statistics can be introduced to students, and then, as you progress through the class, they can be reinforced. Now, you may say … how do the uses of statistics relate to making sense of variability?

1. Descriptive statistics capture the variability a sample
2. Inferential statistics capture the variability in the sample and use it to infer what is going on with the population
3. The critical variability in hypothesis testing is variability due to individual differences of the subjects who just happen to be in your sample, that is variability due to sampling error. The statistic estimates the sampling error, pulls it off, and enables us to test the hypothesis.
4. Of course, statistics used to find associations are looking for how variables are covarying (varying together).

Thus, all four uses of statistics are making sense of variability in different ways. They also have different statistics that they use to make sense of the variability.

This is a very term rich lesson, and I often encourage students to make use of flash cards. There also flash card apps and electric flashcards, but my students have told me … writing out their own flashcards on index cards works the best.

I will admit … this foundation is a bit awkward … I liken it to visiting a lot where a huge, never seen before building is going to be built. You are starting from scratch. Students don’t know where they are going, any more than you know what that new building is going to look like, but helping students to think about what they know and how they know it, how statistics are used, and helping them to get used to terms used in research and statistics, they begin to get comfortable with the thought of learning statistics … and it starts with the Critical Concept of Making Sense of Variability.

Advertisement

THE most critical concepts in applied statistics: Treating students like family

There is nothing like having a child preparing to learn statistics that really gets a mother to focus on … what are THE most critical concepts in applied statistics. I’ll be honest; I’m not basing this posting off of research, as sadly, no such research exists. It is, instead, based off of my experience in teaching and research coupled with the reality, I only have a few hours to cover the most important material to my son and sons and daughters of a few of my dearest friends. You see, they are all preparing to take a math statistics class either this summer or this fall. We all want our children to understand math stats in the larger concept of applied statistics.

In this posting, I will cover the outlines of what I have deemed most critical, then over the course of the next few weeks, I will detail the lessons, activities, and homework assignments.  Each session is equivalent to one weeks’ worth of work during a typical semester for the type of students I teach. As with everything … there may be some variability in how much time it takes to cover this material depending on your class size and student type.

#1: Making Sense of Variability

• Introduction to Epistemology — the four ways of knowing, with a focus on the dance between rationalism (forming hypotheses) and empiricism (gathering observations in the form of data).
• 4 Uses of Statistics: Describe, Infer, Test Hypotheses, Find Associations
• Introduction to research methods (just the experiment, and appropriate terms).
• Brief review of mean, median, and mode

Session #2: Capturing Variability

• Conceptually understanding variability (deviation) and the sum of squares
• Finding the Sum of Squares
• Obtaining the average Sum of Squares — the variance
• Understanding why we need the standard deviation (as it makes conceptual sense, where the variance doesn’t)
• Population Variance and Standard Deviation and Sample Variances and Standard Deviations used to infer the population

Session #3: Normal Distribution

• Review population vs. sample/ parameter vs. statistic
• Normal Distribution as a type of a population
• Properties of the Normal Distribution
• Area under the curve of a normal distribution
• Z-scores as a means of identifying location of an observation on the normal distribution

Session #4: Sampling Distribution of the Means and Standard Error

• Conceptually understanding a sampling distribution
• Exploring the variability in sample mean and understanding why
• Sampling Distribution and the Central Limit Theorem
• Standard Error of the Mean (actual and estimated)
• Introduction to the z-test as a means of finding the location of a sample mean on the sampling distribution of the means
• Comparing and Contrasting the Normal Distribution with the Sampling Distribution of the Means

Session #5: Understanding Hypothesis Testing

• Statistical Hypotheses
• Decisions/ Assumptions/ and Consequences (outside of statistics: common examples, selecting a college & deciding to go on a date).
• Steps of Hypothesis Testing: Research Hypothesis; Statistical Hypothesis; Creation of Sampling Distribution of the Means, and identification of rejection region; Gather Data/Calculate Statistic; Make a decision from data; Draw a Conclusion from data
• Errors in Statistical Decision Making

Now, by understanding all of these concepts, I believe my son and my friends’ children will be prepared to learn any calculation in statistic and better understand what is happening, and how they can interpret the results.

My hope for their classes is that the profession teaching the mathematical statistics class informs the students: Where in the formula the sampling error is calculated or estimated; the times when the statistic can and cannot be used; the assumptions underlying the statistic and what happens to the results when they are violated. I would like my son and my friends’ children to learn about basic parametric and nonparametric statistics, and a little about statistical computing.

Over the next few weeks, I will lay out detailed activities and homework assignments that align with these critical concepts.

Please let me know if you feel I missed a critical component or overstated a concept that you feel isn’t as critical.

Another Thought on Diversity

Bonnie’s recent post on diversity of skill sets, knowledge base, and other student attributes in statistics classes made me think of diversity as the raison d’être for descriptive and inferential statistics.

Students beginning a statistics class often say something to the effect “I want to be a counselor, why do I need to take statistics? One of my favorite answers to this question is to ask students to imagine a world where every person is a clone of me. Everyone looks like me, acts like me, in fact, is identical to me in every way, physically, cognitively, and emotionally. Of course this scenario leads to gasps of horror, especially from the women in the class. With a little class discussion, however, the realization suddenly grows that in this world all  behaviors would be normal because there would be no variability among people. There would be no standard deviation for any measure we might obtain. Hence, no counselors would be needed. No one would be handsome,  beautiful, intelligent, arrogant, energetic, or helpful (I’m not implying that any of these adjectives actually describe me). No behaviors would be abnormal, criminal, empathic, altruistic, selfish, or whatever. Such concepts imply a diversity in physical appearance, intellectual functioning, behavioral actions toward others, and so forth. If we want to know anything about such a population, we need measure only one member of the population. A measure taken on one person would describe all other people.  Descriptive statistics wouldn’t be needed in this world.

Students soon realize that diversity is the reason that statistics is a necessary discipline to understand and explain our world. When there is diversity no single term adequately describes everyone. Thus we have had to develop statistics that describe “typical” and the spread around the typical.

A similar discussion can lead to an understanding for the need for statistical hypothesis testing. Think how easy it would be to decide if an independent variable has an effect on behavior if every person’s behavior were identical. If we introduce the independent variable with one person, and it changes that person’s behavior from the state prior to its introduction, then we know it is effective. And it will have the same effect for everyone.

A world without variability wouldn’t require statistics, but it wouldn’t be much fun to live in either.

First Day of Class — starting off right

How do students comes to us on the first day of class? Yes, I can just about hear you mumbling …

(1) They wonder why they even have to take this class … after all they are a [non-quantitative] major. Why does [psychology, sociology, business, education, etc] need statistics?

(2) They may have had really bad math experiences in the past leading to (a) math anxiety (b) poor math attitudes including a low self efficacy and/or (c) weak math skills.

(3) They have heard lots and lots of stories as to how hard or useless or manipulative statistics can be. We have all heard the quote … and so have they … “There are lies, … , and statistics!”

But the first thing I want to let you know is … instructors of applied statistics may be over estimating the negative thinking of their students. Mills (2004) http://findarticles.com/p/articles/mi_m0FCR/is_3_38/ai_n6249218/?tag=content;col1 found that, in general, students attitudes towards statistics tended to be more positive then negative. Now, granted, that could simply be the case at Mill’s school. Yet, it is worth challenging … what do our students really think? How is this impacting their academic success? Is there anything we can do to get them from less adaptive to more adaptive ways of approaching this class?

If you would like to specifically see what attitudes your students have, the Mills (2004) article includes an abstract with a validated and reliable measure of  Survey of Attitudes Toward Statistics (SATS). 

Though I do find that there are some positive attitudes regarding statistics in the students I see, the majority of students are still coming in with negative attitudes and ways of thinking about the class. Almost importantly, my first day of class activities do as much to help the students with positive attitudes as it does for the negative ones.  

In short … I try to start of the first day of my semester differently than any other class students may have experienced.

I often start by asking students to describe what a typical first day of class look like. That is right, I don’t even introduce myself or the class. I walk in and say, “So … what does the first day of class typically look like.” After a few blank stares, and me having to repeat the question. The students start talking.  There are many types of descriptions … for which I then define variability. We often talk about what is the most typical? Students respond by raising their hands (of course, I have to instruct this) High means strongly agree, low means mildly agree or disagree. Moderately high hands mean that you moderately agree. Sitting on one’s hands means you completely disagree (thus everyone has to do something, everyone has to actively participate. The student with his hands on the desk is ask to participate.)

Yes, as we discuss “typical,” I will often tell students my name, hand out the syllabus and tell them to put it away, and briefly describe things like … there will be four exams, as they state that professors tell them about how the class will be graded. So, the students end up getting the “typical” information from me in an atypical manner.

 Then, I ask students’ their personal preferences of the activities they listed, again  having them to respond with their hands.  This activity really helps to focus on the concept of individual differences, which I define. As individual differences is a precursor to sampling error, which will follow weeks from the first day of class, students are already beginning to conceptualize this class.

I then define statistics as a tool to help us to answer questions by making sense of variability, taking into account individual differences of the subjects we are working with.

Yes, I start the class by forcing students to respond verbally and with voting using their hands … From the first minutes of class, students realize that they have to be engaged in this class … there are no other options. From the first minutes of class, they are beginning to understand two of the most critical concepts … variability and individual differences!

So, why start off the class so differently than most? Because, in the event students are not excited about taking an Applied Statistics class, may be down right afraid of this class, and/or think the class is unnecessary, right from the start, they start to recognize this class is unlike any others.   By starting off so differently, I can challenge their preconceived notions regarding classes in general, and statistics in particular.

At this point, I tend to talk to them about what they had heard or how do they feel about statistics, thus making them face their preconceived notions right away. Again, we conduct another hand poll … and I define for them the concepts of data and sample.

This opening exercise involves kinesthetic action, thus, forces student engagement. Moreover, it takes the abstract concept of variability, individual differences, data, and sampling and puts it into a context of something all of your students can understand, the first day of class. It provides students for context in which statistics functions.

I follow this first day of activities up with Assignments and Exercises 1.1, 1.2, 1.3, & 1.4 all requiring students to think about their behavior and attitude for class. Instructors who are using Kiess and Green (2010) as their statistics textbook can access those assignments at http://www.pearsonhighered.com/educator/product/Statistical-Concepts-for-the-Behavioral-Sciences/9780205626243.page.

Will one day be enough? Of course not … but it’s always the first step!

As always, I would love to hear from others regarding their first day activities!

Green, B. A., & Sandry, J. D. (2010) Assignments and Exercises for Students for Statistical Concepts for the Behavioral Sciences, 4/E . Boston: Pearson.

Mills, J. D. (2004). Students’ attitudes toward statistics: implications for the future. College Student Journal. FindArticles.com. 29 Jan, 2011. http://findarticles.com/p/articles/mi_m0FCR/is_3_38/ai_n6249218/

Before the semester starts … I’m playing with pictures!

I am sure I’m not alone in wanting to use the time between semesters to make adjustments to what I am teaching or how I am teaching it. By now, you probably recognize that I am a fan of learning about new pedagogical techniques. I am dedicated to helping students to truly understanding the concepts of statistics. Often, having visuals when you teach is useful for students.

I use the chalk and a board (OK, more like 8 boards that move). I draw a lot of pictures. However, a mathematics professor (who is both a great colleague and friend) has been bugging me about using Mathematic in addition to chalk (a delivery system she also loves).

With Mathematica, it is my hope that I will not only be able to present my students with a visual image of certain concepts during class time (like how a normal distribution changes when the size of the standard deviation gets larger or smaller) but by making these demonstrations available electronically to students for them to explore these concepts on their own, I am hoping students will gain a greater conceptual understanding of critical statistical concepts.

Mathematica is a software package, that among other things, provides demonstrations of statistical concepts. Each demonstration was designed by an instructor. For it to be published, it is my understanding that it goes through a rigorous peer-review process. As such, if it’s printed for use, you know it will work. The down side is that your university would have to pay for a subscription to Mathematica for the demonstrations to be useful. http://www.wolfram.com/solutions/education/higher-education/uses-for-education.html

As I stated last week, in my list of resolutions, my goal is to find five different demonstrations this semester. Why five? It seemed like a reasonable number … not too challenging.

This was really easier than I anticipated. I started by indentifying the concepts that would most benefit from being able to visualize and manipulate variables. Then I visited the Mathematica web site and searched the topics. Each search yielded anywhere from 5 to 25 demonstrations, some were appropriate, others weren’t. I looked through the demonstrations and selected the ones I liked.

Here are the concepts and the demonstrations I identified as being potentually useful this semester.

(1) The Normal Distribution, where students get to input mu and sigma, would make a nice visual demonstration.

http://demonstrations.wolfram.com/TheNormalDistribution/

This Normal Distribution also shows the area under the curve (i.e., you can manipulate the z-score)

http://demonstrations.wolfram.com/AreaOfANormalDistribution/

(2) Another good demonstration would be the Sampling Distribution of the Means, where students can see the impact of changing mu, sigma, or sample size on its shape.

http://demonstrations.wolfram.com/SamplingDistributionOfTheSampleMean/

I’m also going to throw in a demonstration on the Central Limit Theorem, as how can we talk about the Sampling Distribution of the Means without mentioning the Central Limit Theorem?

http://demonstrations.wolfram.com/TheCentralLimitTheorem/

(3) Of course, what changes in the Sampling Distribution of the mean is the standard error, thus showing how a standard error changes due to changes in the sample size and/or variability makes a great deal of sense. I was really hoping that a demonstration on the standard error would already be available, unfortunately, it doesn’t seem to be. A similar concept is the confidence interval, though even with this demonstration the writer of the Mathematica code for this demonstration did not include how variability (i.e., standard deviation) impacts the size of the “margin of error.” However, it still could be a useful demonstration.

http://demonstrations.wolfram.com/ConfidenceIntervalsConfidenceLevelSampleSizeAndMarginOfError/

Though not as clean looking at the one above, this demonstration also includes the size of the standard deviation. http://demonstrations.wolfram.com/ConfidenceIntervalExploration/

I would expect that the two demonstrations would be necessary for student to get a richer understanding of confidence intervals.

That having been said, I believe that two new Mathematica Demonstrations are in order … one dealing with the size of the standard error based on changes in sample size and variability and a possibily a new CI demonstration that merges the best of these two demonstrations.

(4) The effects of the sample size and population variance on hypothesis testing with the t-test seems like a great visual demonstration.

http://demonstrations.wolfram.com/HypothesisTestsAboutAPopulationMean/

(5) How changes in the variables impact correlation’s (depending on how they are calculated) should be useful for my students.

http://demonstrations.wolfram.com/CorrelationAndRegressionExplorer/

(6) Those of you who know me, are probably not surprised that I can’t just stop at 5 examples for this first semester … so here is a great demonstration on Power. Though I can get students to define power, and identify threats to power, I am never fully certain that they truly get the beauty (and hassle) of power. This demonstration may help.

http://demonstrations.wolfram.com/ThePowerOfATestConcerningTheMeanOfANormalPopulation/

Of course, without proper instruction during class time and an accompanying explanation following class instruction, these demonstrations may end up being little more than pretty pictures to students.

In a few weeks, especially after I actually try these demonstrations with my students, I will provide for you the information I attached with the demonstrations as well as feedback as to what worked and what didn’t. After all … anyone who has taught long enough knows, even the best planned lessons and demonstrations some times flop.

Though not specifically having to do with teaching statistics … I found a nice article at Chronicle of Higher Education on Iphones, Blackberries, etc … and apps that could help professors. The attendance and learning students’ names apps look promising. http://chronicle.com/article/College-20-6-Top-Smartphone/125764/

I look forward to hearing from any of you who have used Mathematica Demonstrations (or others) during class and for homework.

Core Statistical Concepts

I have been spending the week thinking about what I consider to be the “core concepts” that need to be covered in an applied statistics class, be it in psychology, health, business, or education. However, before I post my personal thoughts, I felt it necessary to see what other applied statisticians had to say. In my search, I found http://www.statlit.org/pdf/2004McKenzieASA.pdf . This work was conducted by John McKenzie (2004), Conveying the Core Concepts, is from the Proceedings of the ASA Section on Statistical Education, pages 2755-2757.

In reading what  McKenzie, and several other professors of applied statistics identified as the core concepts in statistics, I must say … I concur. Listed below are the core concepts in applied statistics … the information that, in my opinion, simply has to be covered regardless of illness, snow days, or anything else that could interrupt a professors’ teaching schedule.

Variability: Students cannot understand the purpose of statistics unless they get the concept of variability. Within this, we can further talk about variability due to chance and variability due to effect. Including in the discussion of variability should be the difference between systematic and random variability. I would have to say that not a class period goes by without me spending at least a little time on helping students to focus on issues of variability (especially variability due to the individual differences of the subjects who just happen to be in our sample). 

Randomness: Though I would see randomness and variability as being part of the same large concept, McKenzie’s work identified the concept of randomness as not only separate from variability but also critical for students to master.

Sampling Distribution: Along with Hypothesis Testing, the teaching of sampling distribution is considered to be one of the most complicated to teach.  I would concur, which is why I spend an entire class period just on a single activity with M&M’s to demonstrate the concept of sampling distribution. (Please see a prior blog entry for details on this tactile activity).

Hypothesis Testing: The sages and I spent the month of October and much of November discussing whether Hypothesis Testing is critical and if so, how to best tackle the teaching of this complex topic. Not surprising, McKenzie identified the teaching of hypothesis testing as being one of the two most difficult concepts to teach in applied statistics (the other being sampling distribution). Though there may be several published articles on hypothesis testing no longer being a critical concept to teach, the individuals who were surveyed for McKenzie’s work, certainly consider it to be a critical concepts.

Data Collection Methods: Though I have said to my students more times that I can count, “the quality of our statistics is limited by the quality of our sample,” I must admit to being a bit surprised that this was considered critical by others, especially since when I look at many undergraduate statistics textbooks, data collection methods are barely mentioned. Kiess and Green’s (2010) Statistical Concept for the Behavioral Sciences, 4/e, certainly tackles the issue of data collection methods.

Association vs. Causality: This core concept makes me smile, as often when I meet someone for the first time, and they ask me what I do … my response is often met with one of two comments … “Oh, I hated statistics” or “Correlation does not mean causation.” It’s kind of like me recalling how to greet a person in German, a class that I had for three years, and yet recall so little. We, as applied statisticians, certainly engrave this concept into the minds of our students, but I’m sure most of you are like me, hoping student get more than a “pat phrase” out of our classes.

 Significance (Statistical vs. Practical): This is a critical concept in applied statistics and one that is probably not mentioned in theoretical statistics classes. Sure, we delineate a mark in which we have to say … these results are too extreme for us to attribute them to “chance” … but just because we found a statistically significant difference, doesn’t mean it’s a difference that truly matters. In applied statistics, it’s not enough to understand how statistical significance works, but to be able to interpret the results to determine practical difference. I must admit to not covering this core concept to the same extent I cover the others.

As I think of other “critical concepts” they tend to be a bit more specific and fall under the larger concepts listed above (e.g., understanding what a standard deviation can tell us, clearly falls under the concept of variability. I invite all of you, to comment on what concepts, if any, are missing from this list.