Check out a similar explanation using the palmerpenguins data set here.
Sum Coding for Categorical Variables
what is it?
how do we interpret the regression output?
why would we want to do it?
Introduction
We learned about dummy coding / treatment coding before, wherein the levels of a categorical predictor are represented as a series of 0s and 1s. The baseline is 0 and each level is 1, which allows us to use the familiay “one unit increase” when interpreting the output of a regression model.
In this notebook we talk about sum coding, which is a different contrast coding scheme we can apply to the levels of a categorical variable. In comparison to dummy coding, the sum coding is approximately equivalent to centering a continuous variable, as it will set the intercept to a value “between” the levels of a categorical variable.
Load in data
I’m going to use the same data as used in the prior notebook, the smell/taste data used by BW.
library(tidyverse)
── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr 1.1.4 ✔ readr 2.1.5
✔ forcats 1.0.1 ✔ stringr 1.5.2
✔ ggplot2 4.0.0 ✔ tibble 3.3.0
✔ lubridate 1.9.4 ✔ tidyr 1.3.1
✔ purrr 1.1.0
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag() masks stats::lag()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
dat <-read_csv('winter_2016_senses_valence.csv')
Rows: 405 Columns: 6
── Column specification ────────────────────────────────────────────────────────
Delimiter: ","
chr (2): Word, DominantModality
dbl (4): Val, AbsVal, Sent, AbsSent
ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
Filter out just the rows where the rating was for Taste or Smell words
dat2 <- dat %>%filter(DominantModality %in%c("Taste", "Smell"))
Calculate the mean and standard deviation of Val (the dependent variable)
A reminder of the default coding scheme (dummy/treatment)
contrasts(dat2$DominantModality)
Taste
Smell 0
Taste 1
We recall that the levels are replaced with 0s and 1s. It’s relatively easy to apply a new coding scheme to a factor in R. We use the contr.sum() function, and provide the number of levels that are in the factor/categorical variable (in this case, two).
Here is the syntax on how to do this:
contrasts(dat2$DominantModality) <-contr.sum(2)
Check the new coding scheme: the values are now 0 and 1.
Note
What value will be used to set the intercept?
Where is that value in relation to 1 and -1?
contrasts(dat2$DominantModality)
[,1]
Smell 1
Taste -1
fit linear model
m1 <-lm(Val ~ DominantModality, data = dat2)
How do we interpret this output? We now have the coefficient DominantModality1, which tells us the predictor has been sum coded. The intercept also shows us a value of 5.63957 - what does that represent?
summary(m1)
Call:
lm(formula = Val ~ DominantModality, data = dat2)
Residuals:
Min 1Q Median 3Q Max
-0.99315 -0.20870 0.04343 0.19115 0.62788
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 5.63957 0.03897 144.729 < 2e-16 ***
DominantModality1 -0.16856 0.03897 -4.326 4.95e-05 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 0.3148 on 70 degrees of freedom
Multiple R-squared: 0.2109, Adjusted R-squared: 0.1997
F-statistic: 18.71 on 1 and 70 DF, p-value: 4.951e-05
A reminder of the means of each level - neither is set to the intercept:
But what if we take the average of those means? The mean of means!
mean(descriptives$meanValu)
[1] 5.639568
mean(dat2$Val)
[1] 5.691071
What then does the coefficient represent?
The coefficient is half the difference in their mean. Confusing, right?
This is because moving from one group to the other (-1 to +1) represents a 2-unit change in the predictor variable. Since regression coefficients always represent the change in Y per one-unit change in \(X\), the coefficient shows half the total group difference.
m1[["coefficients"]][["DominantModality1"]]
[1] -0.1685562
Using the regression formula to find the actual values
To understand how this words with the contrast codes we chose, we add the intercept to the estimate which is multiplied by the numerical representation for each level (smell = 1, taste = -1)
Each effect can be interpreted as their deviance from the grand mean, and we can use the exact same regression formula logic to get their predicted effects: