Strings II

Anchors, Classes, and Groups

Last Time

1. A string is a sequence of characters in quotes.
2. A backslash escapes a character’s usual meaning.
3. A regular expression describes a text pattern.
4. Metacharacters let one pattern match many strings.

Key Ideas for Today

1. Shortcut classes name common sets like digits.
2. Anchors pin a pattern to the start or end.
3. Grouping sets the scope of a quantifier.
4. Alternation matches one pattern or another.

On the pinned board, continue the regex toolkit from Strings I. Add each new tool as it appears: ranges and negation, shortcut classes, anchors, grouping, alternation.

Concept Acquisition

1. A character class matches any one character from a set; ^ at its start negates it and - expresses a range.
2. Shortcut classes (\d, \s, \w and their capitals) name common sets, and need a second backslash inside an R string.
3. Anchors (^, $) match positions, not characters, so they pin a pattern to the start or end of a string.
4. Parentheses group characters so a quantifier applies to the whole group, and | matches one pattern or another; grouping limits how far an alternation reaches.

Tool Acquisition

1. Regex syntax: ^, $, [...], [^...], [a-z], \\d, \\s, \\w, (), and |.
2. str_view(..., match = NA), str_count(), and str_extract() from stringr.
3. A regex tester such as https://regexr.com/.

Concept Application

1. Write patterns for file names: a webp extension, images starting with p, and a three-digit number.
2. Write patterns for license plates, spellings of “color”, and valid phone numbers.
3. Predict which strings "^cat|dog$" and "^(cat|dog)$" match.
4. Extract the file name from each path with str_extract().

fs <- list.files(recursive = TRUE)
fs

 [1] "_ps.qmd"                          "_quiz.qmd"                       
 [3] "data/133-survey-small.csv"        "images/headache.webp"            
 [5] "images/headwebp.png"              "images/penguins.webp"            
 [7] "images/regex-tester.png"          "images/thieves-guild-banner.webp"
 [9] "slides.qmd"                       "slides.rmarkdown"                

Question: Write the pattern that matches each of the following:

1. Files with a webp extension.
2. Images with a name that starts with the letter p.
3. Files with a three digit number.

02:00

fs

 [1] "_ps.qmd"                          "_quiz.qmd"                       
 [3] "data/133-survey-small.csv"        "images/headache.webp"            
 [5] "images/headwebp.png"              "images/penguins.webp"            
 [7] "images/regex-tester.png"          "images/thieves-guild-banner.webp"
 [9] "slides.qmd"                       "slides.rmarkdown"                

1. Files with a webp extension.

library(tidyverse) # has library(stringr)
str_view(fs, "\\.webp", match = NA)

 [1] │ _ps.qmd
 [2] │ _quiz.qmd
 [3] │ data/133-survey-small.csv
 [4] │ images/headache<.webp>
 [5] │ images/headwebp.png
 [6] │ images/penguins<.webp>
 [7] │ images/regex-tester.png
 [8] │ images/thieves-guild-banner<.webp>
 [9] │ slides.qmd
[10] │ slides.rmarkdown

fs

 [1] "_ps.qmd"                          "_quiz.qmd"                       
 [3] "data/133-survey-small.csv"        "images/headache.webp"            
 [5] "images/headwebp.png"              "images/penguins.webp"            
 [7] "images/regex-tester.png"          "images/thieves-guild-banner.webp"
 [9] "slides.qmd"                       "slides.rmarkdown"                

2. Images with a name that starts with the letter p.

str_view(fs, "images/p", match = NA)

 [1] │ _ps.qmd
 [2] │ _quiz.qmd
 [3] │ data/133-survey-small.csv
 [4] │ images/headache.webp
 [5] │ images/headwebp.png
 [6] │ <images/p>enguins.webp
 [7] │ images/regex-tester.png
 [8] │ images/thieves-guild-banner.webp
 [9] │ slides.qmd
[10] │ slides.rmarkdown

fs

 [1] "_ps.qmd"                          "_quiz.qmd"                       
 [3] "data/133-survey-small.csv"        "images/headache.webp"            
 [5] "images/headwebp.png"              "images/penguins.webp"            
 [7] "images/regex-tester.png"          "images/thieves-guild-banner.webp"
 [9] "slides.qmd"                       "slides.rmarkdown"                

3. Files with a three digit number.

str_view(fs, "[0123456789]{3}")

[3] │ data/<133>-survey-small.csv

Agenda

• Regular expressions
  • Literal Characters
  • Character Classes
    • Shortcut Classes
  • Metacharacters
    • Wildcards
    • Quantifiers
    • Anchors
  • Grouping and Alternation

• Case Studies
  • Extracting a filename
• Appendix
  • Splitting into majors

More Character Classes

Ranges and Negation

• Special characters in a set:
  • ^ at the start negates the set.
  • - expresses a range.

x <- "abcd ABCD 12345 -!@#%."

str_view(x, "[a-z]+")

[1] │ <abcd> ABCD 12345 -!@#%.

str_view(x, "[^a-z0-9]+")

[1] │ abcd< ABCD >12345< -!@#%.>

On the pinned board: “^ inside [ ] = not; ^ outside = start.” Return to it at Anchors.

Ask: is there a shorter way to write the warm-up’s [0123456789]{3}? Hold the answer for the recap at the end of this section.

Shortcut Class

A character class with a built-in name, such as \d for any digit.

• \d matches any digit;
• \D matches anything that isn’t a digit.
• \s matches any whitespace (e.g., space, tab, newline);
• \S matches anything that isn’t whitespace.
• \w matches any “word” character, i.e. letters, numbers, and underscores;
• \W matches any “non-word” character.

Note: These all have to be escaped when forming a string to express the pattern.

Shortcut classes

x <- "abcd ABCD 12345 -!@#%."
str_view(x, "\\d+")

[1] │ abcd ABCD <12345> -!@#%.

str_view(x, "\\D+")

[1] │ <abcd ABCD >12345< -!@#%.>

str_view(x, "\\s+")

[1] │ abcd< >ABCD< >12345< >-!@#%.

str_view(x, "\\S+")

[1] │ <abcd> <ABCD> <12345> <-!@#%.>

Shortcut classes

x

[1] "abcd ABCD 12345 -!@#%."

str_view(x, "\\w+")

[1] │ <abcd> <ABCD> <12345> -!@#%.

str_view(x, "\\W+")

[1] │ abcd< >ABCD< >12345< -!@#%.>

3. Files with a three digit number.

str_view(fs, "[0123456789]{3}")

[3] │ data/<133>-survey-small.csv

str_view(fs, "[0-9]{3}")

[3] │ data/<133>-survey-small.csv

str_view(fs, "\\d{3}")

[3] │ data/<133>-survey-small.csv

Anchors

Anchor

A metacharacter that pins a regular expression to the start (^) or the end ($) of a string.

df <- tibble(fruit = c("apple", "banana"))
df |>
  mutate(start_a = str_view(fruit, "^a", match = NA))

# A tibble: 2 × 2
  fruit  start_a   
  <chr>  <strngr_v>
1 apple  <a>pple   
2 banana banana    

Anchor

A metacharacter that pins a regular expression to the start (^) or the end ($) of a string.

df <- tibble(fruit = c("apple", "banana"))
df |>
  mutate(start_a = str_view(fruit, "^a", match = NA),
         end_a   = str_view(fruit, "a$", match = NA))

# A tibble: 2 × 3
  fruit  start_a    end_a     
  <chr>  <strngr_v> <strngr_v>
1 apple  <a>pple    apple     
2 banana banana     banan<a>  

fs <- c(fs, "my.webp.pic.png")
fs

 [1] "_ps.qmd"                          "_quiz.qmd"                       
 [3] "data/133-survey-small.csv"        "images/headache.webp"            
 [5] "images/headwebp.png"              "images/penguins.webp"            
 [7] "images/regex-tester.png"          "images/thieves-guild-banner.webp"
 [9] "slides.qmd"                       "slides.rmarkdown"                
[11] "my.webp.pic.png"                 

1. Files with a webp extension.

# will catch a file with .webp in the name
str_view(fs, ".webp", match = NA)

 [1] │ _ps.qmd
 [2] │ _quiz.qmd
 [3] │ data/133-survey-small.csv
 [4] │ images/headache<.webp>
 [5] │ images/hea<dwebp>.png
 [6] │ images/penguins<.webp>
 [7] │ images/regex-tester.png
 [8] │ images/thieves-guild-banner<.webp>
 [9] │ slides.qmd
[10] │ slides.rmarkdown
[11] │ my<.webp>.pic.png

# much better
str_view(fs, "\\.webp$", match = NA)

 [1] │ _ps.qmd
 [2] │ _quiz.qmd
 [3] │ data/133-survey-small.csv
 [4] │ images/headache<.webp>
 [5] │ images/headwebp.png
 [6] │ images/penguins<.webp>
 [7] │ images/regex-tester.png
 [8] │ images/thieves-guild-banner<.webp>
 [9] │ slides.qmd
[10] │ slides.rmarkdown
[11] │ my.webp.pic.png

Tip

Regex tester websites are very helpful for understanding how your regex is working https://regexr.com/

plates <- c("7ABC123", "5ZZA113", "9HMB22B")
colors <- c("col", "color", "colour", "colr")
phones <- c("510-384-2988", "483-3309", "1-558-90-3942", "382-APG-3911")

Question: Write the pattern that will match each of the following¹:

1. Any license plate that ends in A, B, or C.
2. Valid full spellings of the word “color”.
3. Strings that contain only valid phone number with area code (no country code).

01:30

1. https://regexr.com/

plates <- c("7ABC123", "5ZZA113", "9HMB22B")

1. Any license plate that ends in A, B, or C.

str_view(plates, "[ABC]$", match = NA)

[1] │ 7ABC123
[2] │ 5ZZA113
[3] │ 9HMB22<B>

colors <- c("col", "color", "colour", "colr")

2. Valid full spellings of the word “color”.

str_view(colors, "colou?r", match = NA)

[1] │ col
[2] │ <color>
[3] │ <colour>
[4] │ colr

phones <- c("510-384-2988", "483-3309", "1-558-90-3942", "382-APG-3911")

3. Strings that contain only valid phone number with area code (no country code).

str_view(phones, "^\\d{3}-\\d{3}-\\d{4}$", match = NA)

[1] │ <510-384-2988>
[2] │ 483-3309
[3] │ 1-558-90-3942
[4] │ 382-APG-3911

Grouping and Alternation

()

Groups characters to change the scope of different operations.

x <- c("hah", "hahaha", "haaa")

str_view(x, "ha+")

[1] │ <ha>h
[2] │ <ha><ha><ha>
[3] │ <haaa>

str_view(x, "(ha)+")

[1] │ <ha>h
[2] │ <hahaha>
[3] │ <ha>aa

()

Groups characters to change the scope of different operations.

x <- c("hah", "hahaha", "haaa")

str_view(x, "ha+")

[1] │ <ha>h
[2] │ <ha><ha><ha>
[3] │ <haaa>

str_view(x, "(ha)+")

[1] │ <ha>h
[2] │ <hahaha>
[3] │ <ha>aa

str_count(x, "ha+")

[1] 1 3 1

str_count(x, "(ha)+")

[1] 1 1 1

|

Separates different possible patterns; reads like “or”.

str_view(df$fruit, "an|ap")

[1] │ <ap>ple
[2] │ b<an><an>a

survey <- read_csv("data/133-survey-small.csv")
str_view(survey$major, "Stat|stat") # same as "[sS]tat"

 [3] │ <Stat>istics
 [4] │ <Stat>istics
 [8] │ <Stat>istics
 [9] │ <Stat>istics + Economics
[10] │ <Stat>istics
[11] │ <Stat>istics, Economics, Political Economy
[12] │ <stat>istics
[16] │ <Stat>istics
[18] │ <Stat>istics + Integrative Biology
[21] │ <Stat>istics
[24] │ <Stat>istics
[25] │ <stat>istics
[26] │ <Stat>istics
[31] │ <Stat>istics
[32] │ Data Science & <Stat>s
[34] │ <Stat>istics
[37] │ <Stat>istics
[38] │ <Stat>istics
[42] │ <stat>istic
[43] │ Data Science, <Stat>istics
... and 66 more

Grouping with Alternation

x <- c("cat", "catcat", "catdog", "dogcat", "dog")
p1 <- "^cat|dog$"

Question: Which strings will this match?

str_view(x, p1)

[1] │ <cat>
[2] │ <cat>cat
[3] │ <cat><dog>
[5] │ <dog>

Grouping with Alternation

x <- c("cat", "catcat", "catdog", "dogcat", "dog")
p1 <- "^cat|dog$"
p2 <- "^(cat|dog)$"

Question: Which strings will this match?

str_view(x, p2)

[1] │ <cat>
[5] │ <dog>

Working with Regex

Working with Regex

Guess the Metaphor

A regular expression is like a wanted poster on the thief guild’s wall: it describes a pattern (red cloak, one or more scars, last seen at the docks). Anchors say where to look, classes list the looks that count.

Case Studies

Extracting file names

How do I extract just the file name from each path?

fs

 [1] "_ps.qmd"                          "_quiz.qmd"                       
 [3] "data/133-survey-small.csv"        "images/headache.webp"            
 [5] "images/headwebp.png"              "images/penguins.webp"            
 [7] "images/regex-tester.png"          "images/thieves-guild-banner.webp"
 [9] "slides.qmd"                       "slides.rmarkdown"                
[11] "my.webp.pic.png"                 

str_view(fs, "[^/]+$")

 [1] │ <_ps.qmd>
 [2] │ <_quiz.qmd>
 [3] │ data/<133-survey-small.csv>
 [4] │ images/<headache.webp>
 [5] │ images/<headwebp.png>
 [6] │ images/<penguins.webp>
 [7] │ images/<regex-tester.png>
 [8] │ images/<thieves-guild-banner.webp>
 [9] │ <slides.qmd>
[10] │ <slides.rmarkdown>
[11] │ <my.webp.pic.png>

Extracting file names

How do I extract just the file name from each path?

fs

 [1] "_ps.qmd"                          "_quiz.qmd"                       
 [3] "data/133-survey-small.csv"        "images/headache.webp"            
 [5] "images/headwebp.png"              "images/penguins.webp"            
 [7] "images/regex-tester.png"          "images/thieves-guild-banner.webp"
 [9] "slides.qmd"                       "slides.rmarkdown"                
[11] "my.webp.pic.png"                 

str_extract(fs, "[^/]+$")

 [1] "_ps.qmd"                   "_quiz.qmd"                
 [3] "133-survey-small.csv"      "headache.webp"            
 [5] "headwebp.png"              "penguins.webp"            
 [7] "regex-tester.png"          "thieves-guild-banner.webp"
 [9] "slides.qmd"                "slides.rmarkdown"         
[11] "my.webp.pic.png"          

How do you learn and keep track of the str_ functions in stringr?

• stringr Documentation Site
• stringr Cheatsheet

Key Ideas for Today

1. Shortcut classes name common sets like digits.
2. Anchors pin a pattern to the start or end.
3. Grouping sets the scope of a quantifier.
4. Alternation matches one pattern or another.

For Next Time

1. Sign up for Quiz 4 (10/19-10/20) on PrairieTest.
2. Work on PS 4, due Saturday 10/17 at 11:59 pm.
3. Keep collecting your personal data exports for Project 3.

Appendix: Splitting off majors

Splitting off majors

How do I systematically keep track of each major a student has?

select(survey, major)

# A tibble: 202 × 1
   major                           
   <chr>                           
 1 Applied mathematics             
 2 Psychology & Data Science       
 3 Statistics                      
 4 Statistics                      
 5 math major + ds minor           
 6 Economics and minor Data Science
 7 Political Science               
 8 Statistics                      
 9 Statistics + Economics          
10 Statistics                      
# ℹ 192 more rows

What separates two majors?

str_view(survey$major, ",|\\+|&|/| and ")

 [2] │ Psychology <&> Data Science
 [5] │ math major <+> ds minor
 [6] │ Economics< and >minor Data Science
 [9] │ Statistics <+> Economics
[11] │ Statistics<,> Economics<,> Political Economy
[18] │ Statistics <+> Integrative Biology
[27] │ Data Science< and >Applied Math
[32] │ Data Science <&> Stats
[39] │ Neuroscience<,> Molecular< and >Cell Biology<,> Minor: Data Science
[41] │ Applied Mathematics <&> Integrative Biology
[43] │ Data Science<,> Statistics
[49] │ Materials Science< and >Engineering
[50] │ Stat <+> Econ
[62] │ Data Science< and >Applied Math
[66] │ Economics< and >Statistics
[67] │ MCB <&> Stats
[69] │ Econ<,> minor data
[70] │ Statistics< and >Economics
[74] │ Cognitive Science< and >Linguistics
[80] │ Engineering Math< and >statistics
... and 26 more

• + is a quantifier, so it needs escaping to match a literal plus sign.
• The spaces in " and " keep it from matching inside words like “Landscape”.

One major per row

seps <- ",|\\+|&|/| and "
majors <- survey |>
  select(major) |>
  separate_longer_delim(major, delim = regex(seps)) |>
  mutate(major = str_trim(major))
majors

# A tibble: 254 × 1
   major              
   <chr>              
 1 Applied mathematics
 2 Psychology         
 3 Data Science       
 4 Statistics         
 5 Statistics         
 6 math major         
 7 ds minor           
 8 Economics          
 9 minor Data Science 
10 Political Science  
# ℹ 244 more rows

• separate_longer_delim() (from tidyr) splits each string at the delimiter and gives each piece its own row; regex() says the delimiter is a pattern, not a literal string.
• str_trim() drops the spaces left at the start and end of each piece.

Counting majors

count(majors, major, sort = TRUE)

# A tibble: 84 × 2
   major                     n
   <chr>                 <int>
 1 Statistics               62
 2 Economics                28
 3 Applied Math             13
 4 Cognitive Science         9
 5 Data Science              8
 6 Econ                      6
 7 Environmental Science     6
 8 statistics                6
 9 Stats                     5
10 Applied Mathematics       4
# ℹ 74 more rows

Are there really only 62 Statistics majors?

Spelling variants

majors |>
  filter(str_detect(major, "^[Ss]tat")) |>
  count(major)

# A tibble: 8 × 2
  major                   n
  <chr>               <int>
1 Stat                    3
2 Statistic               3
3 Statistics             62
4 Stats                   5
5 statistic               3
6 statistics              6
7 stats                   1
8 stats，applied math     1

majors <- majors |>
  mutate(major = if_else(str_detect(major, "^[Ss]tat"), "Statistics", major))
count(majors, major, sort = TRUE)

# A tibble: 77 × 2
   major                     n
   <chr>                 <int>
 1 Statistics               84
 2 Economics                28
 3 Applied Math             13
 4 Cognitive Science         9
 5 Data Science              8
 6 Econ                      6
 7 Environmental Science     6
 8 Applied Mathematics       4
 9 Public Health             4
10 Sociology                 4
# ℹ 67 more rows

Not done yet

Splitting on a pattern is only as good as the pattern.

str_view(survey$major, " and ")

  [6] │ Economics< and >minor Data Science
 [27] │ Data Science< and >Applied Math
 [39] │ Neuroscience, Molecular< and >Cell Biology, Minor: Data Science
 [49] │ Materials Science< and >Engineering
 [62] │ Data Science< and >Applied Math
 [66] │ Economics< and >Statistics
 [70] │ Statistics< and >Economics
 [74] │ Cognitive Science< and >Linguistics
 [80] │ Engineering Math< and >statistics
 [96] │ Statistics & Environmental Economics< and >Policy
 [99] │ Mathematics< and >Statistics
[100] │ Econ< and >PoliSci
[106] │ Economics< and >Statistics
[110] │ Environmental Economics< and >Policy
[111] │ Cognitive science< and >legal studies, data science minor
[116] │ statistics< and >economics
[126] │ Statistics< and >Economics
[127] │ Math< and >Data science
[134] │ Statistics< and >Business Administration
[136] │ Stats< and >Data Science
... and 4 more

• " and " also splits single majors: “Materials Science and Engineering”, “Molecular and Cell Biology”.
• Minors ride along as their own rows: “ds minor”, “Minor: Data Science”.
• One student used a full-width comma (，), so “stats，applied math” never split, and ^[Ss]tat relabeled it as Statistics.