T-Test Essentials: Definition, Formula and Calculation

Using the R base function

There are two options for computing the independent t-test depending whether the two groups data are saved either in two different vectors or in a data frame.

Option 1. The data are saved in two different numeric vectors:

# Save the data in two different vector
women_weight <- genderweight %>%
  filter(group == "F") %>%
  pull(weight)
men_weight <- genderweight %>%
  filter(group == "M") %>%
  pull(weight)
# Compute t-test
res <- t.test(women_weight, men_weight)
res

## 
##  Welch Two Sample t-test
## 
## data:  women_weight and men_weight
## t = -20, df = 30, p-value <2e-16
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -24.5 -20.1
## sample estimates:
## mean of x mean of y 
##      63.5      85.8

Option 2. The data are saved in a data frame.

# Compute t-test
res <- t.test(weight ~ group, data = genderweight)
res

## 
##  Welch Two Sample t-test
## 
## data:  weight by group
## t = -20, df = 30, p-value <2e-16
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -24.5 -20.1
## sample estimates:
## mean in group F mean in group M 
##            63.5            85.8

In the result above :

• t is the t-test statistic value (t = -20.79),
• df is the degrees of freedom (df= 26.872),
• p-value is the significance level of the t-test (p-value = 4.29810^{-18}).
• conf.int is the confidence interval of the means difference at 95% (conf.int = [-24.5314, -20.1235]);
• sample estimates is the mean value of the sample (mean = 63.499, 85.826).

Using the rstatix package

We’ll use the pipe-friendly t_test() function [rstatix package], a wrapper around the R base function t.test(). The results can be easily added to a plot using the ggpubr R package.

stat.test <- genderweight %>% 
  t_test(weight ~ group) %>%
  add_significance()
stat.test

## # A tibble: 1 x 9
##   .y.    group1 group2    n1    n2 statistic    df        p p.signif
##   <chr>  <chr>  <chr>  <int> <int>     <dbl> <dbl>    <dbl> <chr>   
## 1 weight F      M         20    20     -20.8  26.9 4.30e-18 ****

If you want to assume the equality of variances (Student t-test), specify the option var.equal = TRUE:

stat.test2 <- genderweight %>%
  t_test(weight ~ group, var.equal = TRUE) %>%
  add_significance()
stat.test2

The results above show the following components:

• .y.: the y variable used in the test.
• group1,group2: the compared groups in the pairwise tests.
• statistic: Test statistic used to compute the p-value.
• df: degrees of freedom.
• p: p-value.

genderweight %>%
  t_test(weight ~ group, detailed = TRUE) %>%
  add_significance()

## # A tibble: 1 x 16
##   estimate estimate1 estimate2 .y.    group1 group2    n1    n2 statistic        p    df conf.low conf.high method alternative p.signif
##      <dbl>     <dbl>     <dbl> <chr>  <chr>  <chr>  <int> <int>     <dbl>    <dbl> <dbl>    <dbl>     <dbl> <chr>  <chr>       <chr>   
## 1    -22.3      63.5      85.8 weight F      M         20    20     -20.8 4.30e-18  26.9    -24.5     -20.1 T-test two.sided   ****

Cohen’s d for Student t-test

There are multiple version of Cohen’s d for Student t-test. The most commonly used version of the Student t-test effect size, comparing two groups (\(A\) and \(B\)), is calculated by dividing the mean difference between the groups by the pooled standard deviation.

Cohen’s d formula:

\[
d = \frac{m_A - m_B}{SD_{pooled}}
\]

where,

• \(m_A\) and \(m_B\) represent the mean value of the group A and B, respectively.
• \(n_A\) and \(n_B\) represent the sizes of the group A and B, respectively.
• \(SD_{pooled}\) is an estimator of the pooled standard deviation of the two groups. It can be calculated as follow :
  \[
  SD_{pooled} = \sqrt{\frac{\sum{(x-m_A)^2}+\sum{(x-m_B)^2}}{n_A+n_B-2}}
  \]

Calculation. If the option var.equal = TRUE, then the pooled SD is used when compting the Cohen’s d.

genderweight %>%  cohens_d(weight ~ group, var.equal = TRUE)

## # A tibble: 1 x 7
##   .y.    group1 group2 effsize    n1    n2 magnitude
## * <chr>  <chr>  <chr>    <dbl> <int> <int> <ord>    
## 1 weight F      M        -6.57    20    20 large

Note that, for small sample size (< 50), the Cohen’s d tends to over-inflate results. There exists a Hedge’s Corrected version of the Cohen’s d (Hedges and Olkin 1985), which reduces effect sizes for small samples by a few percentage points. The correction is introduced by multiplying the usual value of d by (N-3)/(N-2.25) (for unpaired t-test) and by (n1-2)/(n1-1.25) for paired t-test; where N is the total size of the two groups being compared (N = n1 + n2).

Cohen’s d for Welch t-test

The Welch test is a variant of t-test used when the equality of variance can’t be assumed. The effect size can be computed by dividing the mean difference between the groups by the “averaged” standard deviation.

Cohen’s d formula:

\[
d = \frac{m_A - m_B}{\sqrt{(Var_1 + Var_2)/2}}
\]

where,

• \(m_A\) and \(m_B\) represent the mean value of the group A and B, respectively.
• \(Var_1\) and \(Var_2\) are the variance of the two groups.

Calculation:

genderweight %>% cohens_d(weight ~ group, var.equal = FALSE)

## # A tibble: 1 x 7
##   .y.    group1 group2 effsize    n1    n2 magnitude
## * <chr>  <chr>  <chr>    <dbl> <int> <int> <ord>    
## 1 weight F      M        -6.57    20    20 large