Factors and Data Frames

Data structures for data science

Last Time

1. A function bundles code into a reusable tool.
2. Functions created with function(), saved as objects.
3. Arguments make it flexible; defaults make it quick.
4. The last expression is returned (or use return()).
5. Scoping determines where a function looks for objects.

Key Ideas for Today

1. A factor stores categorical data as integer with levels lookup.
2. An object’s class lets functions behave differently.
3. Ordered factors allow meaningful sorting.
4. A data frame is a named list of equal-length vectors.
5. Data frames subset like a list $, [[]], [] and like a matrix [r,c].

Concept Acquisition

1. A factor encodes categorical data as an integer vector paired with a levels attribute, which is more compact than a character vector and preserves meaning.
2. An object’s class attribute lets generic functions like summary() dispatch different behavior — a first taste of object-oriented programming.
3. Ordered factors (ordered = TRUE) can be sorted meaningfully, not just alphabetically.
4. Atomic vectors, matrices, and lists are each insufficient for storing related, mixed-type, rectangular data.
5. A data frame is a named list of equal-length vectors, giving it both list-like and matrix-like subsetting.

Tool Acquisition

1. factor(), levels(), ordered = TRUE.
2. class(), summary(), sort().
3. data.frame(), names(), row.names(), str(), dim(), nrow(), ncol().
4. as_tibble() from the tibble package.

Concept Application

1. Convert a character vector to a factor and visualize its distribution with barplot().
2. Add a level and reorder a factor sensibly, then re-plot.
3. Construct a data frame using dataframe().
4. Subset a data frame using both matrix- and list-style syntax.

Something to ponder . . .

Imagine you collect University ID number and major from 5 friends:

uid <- c("3850278", "5869204", "4728112", "3829651", "3859278")
maj <- c("statistics", "economics", "statistics", "data", "economics")

• Conceptually, in what ways is the data in these two vectors similar? In what ways is it different?
• Imagine instead you asked 100 friends. For each vector, can you think of a more efficient way for the computer to store that data?

Factors

Imagine you collect University ID number and major from 5 friends:

uid <- c("3850278", "5869204", "4728112", "3829651", "3859278")
maj <- c("STAT", "ECON", "STAT", "DATA", "ECON")

• Conceptually, in what ways is the data in these two vectors similar? In what ways is it different?
• Imagine instead you asked 100 friends. For each vector, can you think of a more efficient way for the computer to store that data?

Factors

Factor

A data structure for storing categorical data. Stores the values in an integer vector and adds a levels attribute to map integers to character strings. Created with factor().

maj <- factor(x = c("STAT", "ECON", "STAT", "DATA", "ECON"))
maj

[1] STAT ECON STAT DATA ECON
Levels: DATA ECON STAT

Draw two versions of the same data: as a character vector as an integer vector coupled with a levels lookup table.

as.integer(maj)

[1] 3 2 3 1 2

c(maj, 2L)

[1] 3 2 3 1 2 2

Attributes

Attributes

Metadata attached to an object that customize its behavior, on top of the simpler data structure it’s built from. A factor’s attributes include its class and levels.

class(maj)

[1] "factor"

levels(maj)

[1] "DATA" "ECON" "STAT"

Setting Levels

Quickly change all of the values with a particular level.

levels(maj) <- c("DATA", "ECON", "STATISTICS")
maj

[1] STATISTICS ECON       STATISTICS DATA       ECON      
Levels: DATA ECON STATISTICS

Add a level even if it’s not observed.

levels(maj) <- c("DATA", "ECON", "STATISTICS", "BIO")
maj

[1] STATISTICS ECON       STATISTICS DATA       ECON      
Levels: DATA ECON STATISTICS BIO

Taking advantage of class

Certain functions behaved differently depending on the class of the object that is passed to it.

class(1:4)

[1] "integer"

summary(1:4)

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
   1.00    1.75    2.50    2.50    3.25    4.00 

class(maj)

[1] "factor"

summary(maj)

      DATA       ECON STATISTICS        BIO 
         1          2          2          0 

live code to compare vectors to matrices in terms of class and typeof

Ordering levels

You can create a factor with ordered levels by adding ordered = TRUE.

maj_or <- factor(x = c("STAT", "ECON", "STAT", "DATA", "ECON"),
                 levels = c("STAT", "ECON", "DATA", "BIO"),
                 ordered = TRUE)

maj_or

[1] STAT ECON STAT DATA ECON
Levels: STAT < ECON < DATA < BIO

sort(maj_or)

[1] STAT STAT ECON ECON DATA
Levels: STAT < ECON < DATA < BIO

barplot(summary(maj_or))

barplot(summary(maj))

Factors and Matrices

Both are augmented versions of atomic vectors that are given classes to allow for special behavior.

mat <- matrix(1:4, ncol = 2)
class(mat)

[1] "matrix" "array" 

summary(mat)

       V1             V2      
 Min.   :1.00   Min.   :3.00  
 1st Qu.:1.25   1st Qu.:3.25  
 Median :1.50   Median :3.50  
 Mean   :1.50   Mean   :3.50  
 3rd Qu.:1.75   3rd Qu.:3.75  
 Max.   :2.00   Max.   :4.00  

Factors and Matrices

Both are augmented versions of atomic vectors that are given classes to allow for special behavior.

mat <- matrix(1:4, ncol = 2)
class(mat)

[1] "matrix" "array" 

This is a form of object oriented programming.

Guess the Metaphor

A factor is like a listing of garment sizes, S < M < L, that can take on values in a known and ordered set.

Factors are everywhere . . .

The Limitations of Our Structures

The Limitation of Atomic Vectors

songs <- factor(c("Golden by Huntrix", "Stay with me", "4me4u", "Golden by Huntrix"))
distances <- c(2000, 800, 100, 10)

Question: What was the song and distance of the second index card?

songs[2]

[1] Stay with me
Levels: 4me4u Golden by Huntrix Stay with me

distances[2]

[1] 800

😬 This is brittle!

The Limitation of Matrices

mat <- matrix(c(songs, distances), ncol = 2)

mat[2,]

[1]   3 800

Question: Summarize the distribution of songs and distances.

summary(mat)

       V1             V2        
 Min.   :1.00   Min.   :  10.0  
 1st Qu.:1.75   1st Qu.:  77.5  
 Median :2.00   Median : 450.0  
 Mean   :2.00   Mean   : 727.5  
 3rd Qu.:2.25   3rd Qu.:1100.0  
 Max.   :3.00   Max.   :2000.0  

🤬 Matrices must contain only one type.

The Limitation of Lists

lst <- list("songs" = songs, "distances" = distances)

summary(lst[["songs"]])

            4me4u Golden by Huntrix      Stay with me 
                1                 2                 1 

summary(lst[["distances"]])

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
   10.0    77.5   450.0   727.5  1100.0  2000.0 

• Lists can hold different types.

The Limitation of Lists

lst <- list("songs" = songs, "distances" = distances)

lst[["songs"]][2]

[1] Stay with me
Levels: 4me4u Golden by Huntrix Stay with me

lst[["distances"]][2]

[1] 800

• Lists can hold different types.
• But are complicated to subset 🤯.

You could also point out that there is nothing enforcing the fact that the two vectors are the same length, so a for loop that iterated over the index might error out.

Data Structures in R

boardwork

Sketch the following diagram.

Data Frames

Data Frame

Data Frame

A named list of vectors of the same length. Holds attributes for (column) names, row.names, and its class, “data.frame”. Created with data.frame().

star_wars <- data.frame(
    name = c("Anakin", "Padme", "Luke", "Leia"),
    gender = c("male", "female", "male", "female"),
    height = c(1.88, 1.65, 1.72, 1.50),
    weight = c(84, 45, 77, 49)
)
star_wars

    name gender height weight
1 Anakin   male   1.88     84
2  Padme female   1.65     45
3   Luke   male   1.72     77
4   Leia female   1.50     49

Data Frame Attributes

names(star_wars)

[1] "name"   "gender" "height" "weight"

row.names(star_wars)

[1] "1" "2" "3" "4"

class(star_wars)

[1] "data.frame"

Other useful functions for Data Frames

• summary()
• str()
• head()
• dim()
• ncol()
• nrow()

star_wars <- data.frame(
    name = c("Anakin", "Padme", "Luke", "Leia"),
    gender = c("male", "female", "male", "female"),
    height = c(1.88, 1.65, 1.72, 1.50),
    weight = c(84, 45, 77, 49)
)

live code

Index Cards as a Data Frame

index_cards <- data.frame(songs, distances)
index_cards

              songs distances
1 Golden by Huntrix      2000
2      Stay with me       800
3             4me4u       100
4 Golden by Huntrix        10

summary(index_cards)

               songs     distances     
 4me4u            :1   Min.   :  10.0  
 Golden by Huntrix:2   1st Qu.:  77.5  
 Stay with me     :1   Median : 450.0  
                       Mean   : 727.5  
                       3rd Qu.:1100.0  
                       Max.   :2000.0  

Index Cards as a Data Frame, cont.

index_cards[2, ]

         songs distances
2 Stay with me       800

👍👍

Guess the Metaphor

A data frame is like a listing of the characteristics of various garments, with one row for each garment, one column for each characteristic (of different types).

  size   color price
1    M scarlet   230
2    S crimson   170
3    L  purple   285
4    S   white   180
5    M   black   210

Question

    name gender height weight
1 Anakin   male   1.88     84
2  Padme female   1.65     45
3   Luke   male   1.72     77
4   Leia female   1.50     49

Using what you know about matrix and list subsetting, write code to subset from star_wars…

1. The data frame containing only height and weight.
2. The character vector gender.
3. The data frame containing only the last row.
4. The value 1.88

02:30

Subsetting Data Frames

Data frames can be subset either as a two dimensional array (matrix [row, col] subsetting) or as a one dimensional list ([], [[]] or $ subsetting)).

The Evolution of the Data Frame

1990: Data Frame (R)

2009: Pandas DF (Py)

2016: Tibble (R)

2020: Polars (R/Py)

Tibbles

Tibble

An updated version of a data frame with convenient behaviors.

library(tibble)
star_wars_tbl <- as_tibble(star_wars)
star_wars_tbl

# A tibble: 4 × 4
  name   gender height weight
  <chr>  <chr>   <dbl>  <dbl>
1 Anakin male     1.88     84
2 Padme  female   1.65     45
3 Luke   male     1.72     77
4 Leia   female   1.5      49

The main difference is that tibbles are lazy and surly: they do less and complain more - Hadley Wickham

Key Ideas for Today

1. A factor stores categorical data as integer with levels lookup.
2. An object’s class lets functions behave differently.
3. Ordered factors allow meaningful sorting.
4. A data frame is a named list of equal-length vectors.
5. Data frames subset like a list $, [[]], [] and like a matrix [r,c].

For Next Time

1. Sign up for Quiz 2.
2. PS 2 due this Saturday night.
3. Come to office hours / tutoring!