Factors and Data Frames

Data structures for data science

STAT 133 with Gaston Sanchez

While you’re waiting

Imagine you collect University ID number and major from 5 friends:

uid <- c("3850278", "5869204", "4728112", "3829651", "3859278")
maj <- c("STAT", "ECON", "STAT", "DATA", "ECON")


  • Conceptually, in what ways is the data in these two vectors similar? In what ways is it different?

  • Imagine instead you asked 100 friends. For each vector, can you think of a more efficient way for the computer to store that data?

Agenda

  1. Factors

  2. Data Frames

Factors

While you’re waiting

Imagine you collect University ID number and major from 5 friends:

uid <- c("3850278", "5869204", "4728112", "3829651", "3859278")
maj <- c("STAT", "ECON", "STAT", "DATA", "ECON")


  • Conceptually, in what ways is the data in these two vectors similar? In what ways is it different?

  • Imagine instead you asked 100 friends. For each vector, can you think of a more efficient way for the computer to store that data?

Factors

A data structure for storing categorical data.

Created with factor().

Stores the values in an integer vector and adds a levels attribute to map integers to character strings.

maj <- factor(x = c("STAT", "ECON", "STAT", "DATA", "ECON"))
maj
[1] STAT ECON STAT DATA ECON
Levels: DATA ECON STAT

maj <- factor(x = c("STAT", "ECON", "STAT", "DATA", "ECON"))
maj
[1] STAT ECON STAT DATA ECON
Levels: DATA ECON STAT


as.integer(maj)
[1] 3 2 3 1 2


class(maj)
[1] "factor"

Attributes

Objects built on top of simpler data structures can have attributes. A factor’s attributes include its class and levels.

attributes(maj)
$levels
[1] "DATA" "ECON" "STAT"

$class
[1] "factor"


levels(maj)
[1] "DATA" "ECON" "STAT"

Treat factors like vectors

maj
[1] STAT ECON STAT DATA ECON
Levels: DATA ECON STAT
length(maj)
[1] 5
maj[1]
[1] STAT
Levels: DATA ECON STAT
maj[-2]
[1] STAT STAT DATA ECON
Levels: DATA ECON STAT

Setting Levels

Quickly change all of the values with a particular level.

maj
[1] STAT ECON STAT DATA ECON
Levels: DATA ECON STAT


levels(maj) <- c("DATA", "ECON", "STATISTICS")
maj
[1] STATISTICS ECON       STATISTICS DATA       ECON      
Levels: DATA ECON STATISTICS

Add a level even if it’s not observed.

levels(maj) <- c("DATA", "ECON", "STATISTICS", "BIO")
maj
[1] STATISTICS ECON       STATISTICS DATA       ECON      
Levels: DATA ECON STATISTICS BIO

Taking advantage of class

Certain functions behaved differently depending on the class of the object that is passed to it.

class(1:4)
[1] "integer"
summary(1:4)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
   1.00    1.75    2.50    2.50    3.25    4.00 
class(maj)
[1] "factor"
summary(maj)
      DATA       ECON STATISTICS        BIO 
         1          2          2          0

Ordering levels

You can create a factor with ordered levels by adding ordered = TRUE.

maj_or <- factor(x = c("STAT", "ECON", "STAT", "DATA", "ECON"),
                 levels = c("STAT", "ECON", "DATA", "BIO"),
                 ordered = TRUE)
maj_or
[1] STAT ECON STAT DATA ECON
Levels: STAT < ECON < DATA < BIO
sort(maj_or)
[1] STAT STAT ECON ECON DATA
Levels: STAT < ECON < DATA < BIO

barplot(summary(maj_or))

barplot(summary(maj))

Factors and Matrices

Both are augmented versions of atomic vectors that are given classes to allow for special behavior.

mat <- matrix(1:4, ncol = 2)
class(mat)
[1] "matrix" "array" 
summary(mat)
       V1             V2      
 Min.   :1.00   Min.   :3.00  
 1st Qu.:1.25   1st Qu.:3.25  
 Median :1.50   Median :3.50  
 Mean   :1.50   Mean   :3.50  
 3rd Qu.:1.75   3rd Qu.:3.75  
 Max.   :2.00   Max.   :4.00

The Limitations of Our Structures

The Limitation of Atomic Vectors

name <- c("Leia", "Luke", "Han")
gender <- factor(c("female", "male", "male"))
height <- c(150, 170, 180)

Question: What is the gender and height of the second individual?

gender[2]
[1] male
Levels: female male
height[2]
[1] 170

😬 This is brittle!

The Limitation of Matrices

mat <- matrix(c(name, gender, height), ncol = 3)
mat
     [,1]   [,2] [,3] 
[1,] "Leia" "1"  "150"
[2,] "Luke" "2"  "170"
[3,] "Han"  "2"  "180"
mat[2,]
[1] "Luke" "2"    "170"

Task: Summarize the distribution of gender and height.

summary(mat[ ,2:3])
      V1                 V2           
 Length:3           Length:3          
 Class :character   Class :character  
 Mode  :character   Mode  :character

The Limitation of Lists

lst <- list("name" = name, "gender" = gender, "height" = height)
summary(lst[["gender"]])
female   male 
     1      2 
summary(lst[["height"]])
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  150.0   160.0   170.0   166.7   175.0   180.0 
lst[["gender"]][2]
[1] male
Levels: female male
lst[["height"]][2]
[1] 170

🤯 Too complicated to subset.

Data Structures in R (recap)

Data Frames

Data Frame

A data frame is a named list of vectors of the same length. Created with data.frame().

star_wars <- data.frame(
  name = c("Anakin", "Padme", "Luke", "Leia"),
  gender = c("male", "female", "male", "female"),
  height = c(1.88, 1.65, 1.70, 1.50),
  weight = c(84, 45, 77, 49)
)

star_wars
    name gender height weight
1 Anakin   male   1.88     84
2  Padme female   1.65     45
3   Luke   male   1.70     77
4   Leia female   1.50     49

Data Frame (cont.)

A data frame holds attributes for (column) names, row.names, and its class, "data.frame".

attributes(star_wars)
$names
[1] "name"   "gender" "height" "weight"

$class
[1] "data.frame"

$row.names
[1] 1 2 3 4

Data Frame Attributes

names(star_wars)
[1] "name"   "gender" "height" "weight"
row.names(star_wars)
[1] "1" "2" "3" "4"
class(star_wars)
[1] "data.frame"

Other useful functions for Data Frames

  • colnames(): equivalent to names()
  • rownames(): equivalent to row.names()
  • summary(): summary of each column
  • str(): structure
  • head(): first n rows
  • tail(): last n rows
  • dim(): dimensions
  • ncol(): number of columns
  • nrow(): number of rows

Question

    name gender height weight
1 Anakin   male   1.88     84
2  Padme female   1.65     45
3   Luke   male   1.70     77
4   Leia female   1.50     49

Using what you know about matrix and list subsetting, write code to subset from star_wars …

  1. The data frame containing only height and weight.
  2. The character vector gender.
  3. The data frame containing only the last row.
  4. The value 1.88
02:30

Subsetting Data Frames

Data frames can be subset in 2 ways:

  1. as a two dimensional array:
    • dat[row, col]
    • dat[row, ]
    • dat[ ,col]
  2. as a one dimensional list:
    • dat[]
    • dat[[]]
    • dat$

The Evoluation of the Data Frame

Tibbles

An updated version of a data frame with convenient behaviors.

library(tibble)
star_wars_tbl <- as_tibble(star_wars)
star_wars_tbl
# A tibble: 4 × 4
  name   gender height weight
  <chr>  <chr>   <dbl>  <dbl>
1 Anakin male     1.88     84
2 Padme  female   1.65     45
3 Luke   male     1.7      77
4 Leia   female   1.5      49

Tibbles

The main difference is that tibbles are lazy and surly: they do less and complain more.

Hadley Wickham

https://tibble.tidyverse.org/