Lecture 8 - Null Hypothesis Significance Testing (NHST)

Check-In : Linear Model

Use the “cal_mini” dataset to test ONE of the following hypotheses.

1. People with larger shoe sizes (IV = shoesize) are more likely to love oski (DV = oskilove).
2. People who love cal sports are more likely to love oski
3. People who love butterflies are more likely to love oski.

Make sure to do the following :

1. Load the dataset and check to make sure it loaded correctly.
2. Graph your variables, and make sure the variables look good.
3. Define, graph, and interpret your linear model.
4. Report inferential statistics (with the summary() function).

Agenda & Announcements

1. R Exam Grades Posted.
2. The Final Project
   1. Milestone #4 Due Next Week. Outline your introduction, and draft your method section.
   2. Data Exported? Can keep collecting, but need to STOP collecting once we start analyzing data / doing linear models (next week!)
3. Brain Exam 2 is Next Week in Section
   1. Same format as Brain Exam #1.
   2. Skill : interpreting regression output.
      1. intercept & slope
      2. NHST
4. Chapter 10. On Multiple Regression. More regression! It is chill.
5. THE END IS NEAR.
   1. TODAY : Inferential Stats
   2. NEXT WEEK : Multiple Regression + Review
   3. THEN : Project Workshop (and Holiday?!)
   4. FINALLY : Conclusion to Introduction <3
   5. BUT WAIT, THERE’S MORE : RRR Week Final Project Workshop

We Are Having Fun [SPOILER ALERT]

Model 1

summary(lm(d$oski.love ~ d$shoe.size))

Call:
lm(formula = d$oski.love ~ d$shoe.size)

Residuals:
    Min      1Q  Median      3Q     Max 
-6.5014 -1.6663 -0.1717  3.1689  4.6527 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)   7.4906     1.0351   7.236 1.09e-11 ***
d$shoe.size  -0.1649     0.1247  -1.322    0.188    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 3.154 on 191 degrees of freedom
  (6 observations deleted due to missingness)
Multiple R-squared:  0.009068,  Adjusted R-squared:  0.00388 
F-statistic: 1.748 on 1 and 191 DF,  p-value: 0.1877

Model 2

summary(lm(d$oski.love ~ d$cal.sports))

Call:
lm(formula = d$oski.love ~ d$cal.sports)

Residuals:
    Min      1Q  Median      3Q     Max 
-7.0851 -2.0851 -0.0851  2.9149  4.7549 

Coefficients:
                Estimate Std. Error t value Pr(>|t|)    
(Intercept)       5.2451     0.3028  17.323  < 2e-16 ***
d$cal.sportsYes   1.8400     0.4372   4.208 3.93e-05 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 3.058 on 194 degrees of freedom
  (3 observations deleted due to missingness)
Multiple R-squared:  0.08365,   Adjusted R-squared:  0.07893 
F-statistic: 17.71 on 1 and 194 DF,  p-value: 3.93e-05

Model 3

summary(lm(d$oski.love ~ d$tuhobura))

Call:
lm(formula = d$oski.love ~ d$tuhobura)

Residuals:
    Min      1Q  Median      3Q     Max 
-6.5510 -1.5712 -0.2338  3.4490  5.3684 

Coefficients:
                  Estimate Std. Error t value Pr(>|t|)    
(Intercept)         6.5510     0.4527  14.473   <2e-16 ***
d$tuhoburahorses   -0.3173     0.5790  -0.548   0.5844    
d$tuhoburarats     -1.9194     0.8563  -2.241   0.0261 *  
d$tuhoburaturtles  -0.4334     0.6338  -0.684   0.4950    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 3.169 on 192 degrees of freedom
  (3 observations deleted due to missingness)
Multiple R-squared:  0.02636,   Adjusted R-squared:  0.01114 
F-statistic: 1.732 on 3 and 192 DF,  p-value: 0.1617

A Picture’s Worth…

par(mfrow = c(1,3))
plot(d$oski.love ~ d$shoe.size)
abline(lm(d$oski.love ~ d$shoe.size), col = 'blue', lwd = 2)
plotmeans(d$oski.love ~ d$cal.sports, connect = F, ylim = c(0,10))
plotmeans(d$oski.love ~ d$tuhobura, connect = F, ylim = c(0,10))

Chapter 8 Recap : NHST is Confusing!

Chapter 8 Check-In Results	Correct Answer (& Prof. Comments)
	A Slope is the Direction & Strength of Relationship hard to evaluate strength across models if the units differ too. steeper slope (compared to another slope with similar units) = stronger relationship
	The amount of variation in an effect you might expect to find due to chance if the null hypothesis were “true”.
	Impossible to Tell Significance from the Slope Alone Large slope & sampling error = NOT SIGNIFICANT
	A Significant Effect is NOT necessarily…. large : you can be VERY confident that a small effect is not due to sampling error. important : why does the effect matter? fool-proof : our estimates of sampling error are all made up.
	A p-value of .03 means…there is a 3% chance that this slope (or a stronger slope) would be found due to chance if the true correlation was zero.

Haller, H., & Krauss, S. (2002). Misinterpretations of significance: A problem students share with their teachers. Methods of Psychological Research, 7(1), 1-20.

Some More NHST Interpretation Examples

Model 1: Is there a relationship between narcissism (DV = NPI) and testosterone?

h <- read.csv("~/Dropbox/!WHY STATS/Chapter Datasets/hormone_data.csv", stringsAsFactors = T)
mod1 <- lm(NPI ~ test, data = h)
plot(NPI ~ test, data = h)
abline(mod1, lwd = 5)

summary(mod1)

Call:
lm(formula = NPI ~ test, data = h)

Residuals:
     Min       1Q   Median       3Q      Max 
-1.29855 -0.36531 -0.02762  0.27570  1.36340 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) 2.885533   0.126962  22.727   <2e-16 ***
test        0.003887   0.001502   2.588   0.0114 *  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.5375 on 84 degrees of freedom
  (36 observations deleted due to missingness)
Multiple R-squared:  0.07386,   Adjusted R-squared:  0.06284 
F-statistic: 6.699 on 1 and 84 DF,  p-value: 0.01136

Model 2 : Is there a relationship between narcissism (DV = NPI) and sex?

library(gplots)
mod2 <- lm(NPI ~ sex, data = h)
plotmeans(NPI ~ sex, data = h, connect = F)

summary(mod2)

Call:
lm(formula = NPI ~ sex, data = h)

Residuals:
     Min       1Q   Median       3Q      Max 
-1.47662 -0.34342 -0.03022  0.36298  1.36618 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  3.01382    0.08998  33.494   <2e-16 ***
sexmale      0.26280    0.10741   2.447    0.016 *  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.5247 on 112 degrees of freedom
  (8 observations deleted due to missingness)
Multiple R-squared:  0.05074,   Adjusted R-squared:  0.04226 
F-statistic: 5.986 on 1 and 112 DF,  p-value: 0.01597

Model 3 : Is there a relationship between testosterone (DV = test) and sex?

mod3 <- lm(test ~ sex, data = h)
plotmeans(test ~ sex, data = h, connect = F)

summary(mod3)

Call:
lm(formula = test ~ sex, data = h)

Residuals:
    Min      1Q  Median      3Q     Max 
-60.144 -17.211  -3.365  12.111 137.486 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)   41.396      6.031   6.864 9.00e-10 ***
sexmale       49.288      7.208   6.838 1.01e-09 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 31.34 on 88 degrees of freedom
  (32 observations deleted due to missingness)
Multiple R-squared:  0.347, Adjusted R-squared:  0.3396 
F-statistic: 46.76 on 1 and 88 DF,  p-value: 1.014e-09

Some Pre-Recorded NHST Review Videos

• Note : I used last semester’s dataset for these examples, so you will likely get different results if you try and replicate in this semester’s class; a good example of how NHST doesn’t really tell us whether the results are “truth” or not, or whether they will replicate, etc.

• Example 1 : LOVEWATER ~ smoke.pot

• Examples 2 - 4 : faster explanations!

NEXT TIME : Chapter 10 : Multiple Regression

In the models above, we see….

1. Testosterone is related to narcissism.
2. Sex is related to testosterone.
3. Sex and testosterone are related to each other……
4. so…….

mod4 <- lm(NPI ~ sex + test, data = h)
summary(mod4)

Call:
lm(formula = NPI ~ sex + test, data = h)

Residuals:
     Min       1Q   Median       3Q      Max 
-1.31476 -0.35970 -0.02464  0.27366  1.35483 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) 2.880244   0.129808  22.189   <2e-16 ***
sexmale     0.035287   0.155960   0.226    0.822    
test        0.003636   0.001875   1.939    0.056 .  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.5406 on 83 degrees of freedom
  (36 observations deleted due to missingness)
Multiple R-squared:  0.07443,   Adjusted R-squared:  0.05213 
F-statistic: 3.337 on 2 and 83 DF,  p-value: 0.04036

Milestone #4 : Introduction Outline

Introduction Deconstruction.

Components of an Introduction

The introduction starts broad, but then quickly focuses on the variables in your model so readers can understand a) what your study is about and b) why you’re doing your study. 

Section	Brief Explanation
1. The Opening	Describe the question you have, and explain why this question matters
2. The Review	Describe what past research and theory has to say on the question and your theory. Your goal is to give the reader the background they need to understand why you are doing your study; you don’t need to cover EVERY single issue on your topic..
3. The Critique	Explain why the past research is not “the final truth”, and what other new questions might be important to consider (and why these questions matter). Only point out limitations with past research that you will address in your study; other limitations that you think future research will address should go in the discussion section.
4. The Current Research	Explain what specific questions your study will address. Be clear by stating each idea as a hypothesis with language like, “I predict” or “My first hypothesis”.

Activity : Deconstruct an Introductin

Read an excerpt from the introduction¹; identify (in the margins) each of part of the introduction (“The Opening”, “The Review”, “The Critique”, and “The Current Research”)

¹ Full article here

Introduction Construction

Parts of an Outline

• The DV (the Opening / Past Research) :

  • THE POINT : What is your DV? [Citation]

    • THE EVIDENCE : How have psychologists operationalized or studied this variable in the past? [Citation]

    • WHO CARES : Why should we care about this topic?

• IV1 (Past Research) :

  • THE POINT : What is your independent variable?

    • THE EVIDENCE :

      • How do psychologists operationalize this variable?

      • Summarize at least one past research study on this independent variable, and how this IV is (or might be) related to your DV. [Citation]

    • WHO CARES : Why does this research matter to your topic?

  • ANOTHER POINT YOU COULD MAKE : Are there any limitations with this past research that you will address in your study (this could set up IV2)?

    • THE EVIDENCE : why (research or common sense) might these limitations interfere with our VALID KNOWLEDGE about the DV?

    • WHO CARES : why should we care about these limitations?

• IV2 (The Critique / Past Research) :

  • THE POINT : What is another independent variable that might be related to your dependent variable?

  • THE EVIDENCE : How has this variable been defined / studied in the past? [Citation]

  • WHO CARES : Why might this variable be important to look at / change the relationship between IV1 and the DV? [Citation or Logic Goes Here]

• The Present Research (The Current Study) :

  • THE POINT : What is the goal of your research? Include your specific predictions in a summary table.

  • THE EVIDENCE : What will your research do? You don’t need to fully explain the methods for your study, but instead should set up the broad ideas of what you hope to do.

  • WHO CARES : How does your study advance past research?

Hypothesis	Null Hypothesis	Alternative
Hypothesis 1 (DV ~ IV1)
Hypothesis 2 (DV ~ IV2)
Hypothesis 3 (DV ~ IV1 + IV2)

tsunami and rebirth