Testing

Test your code

Last Time

1. Helpers automate package chores.
2. Packages can store data.
3. Roxygen comments make help pages.
4. Load, document, check.

Key Ideas for Today

1. Changes break code you already checked.
2. A unit test writes down what you expect.
3. Test typical inputs, edge cases, and errors.
4. Turn every bug into a test.

Concept Acquisition

1. A change can break behavior you already checked (a regression); checking by eye doesn’t scale, but a test runs every time.
2. A unit test pairs an input with the output you expect, so a set of tests is a specification of what a function should do.
3. Good test cases cover typical inputs, edge cases (empty, boundary, unusual characters), and the errors a function should raise.
4. Test-driven development: write the failing test first, make it pass, then clean up. A bug report becomes a test before it becomes a fix.

Tool Acquisition

1. usethis::use_testthat() and use_test().
2. test_that() with expect_equal(), expect_type(), expect_length(), and expect_error().
3. devtools::test(), and reading its failure report.

Concept Application

1. Add tests for strsplit1() to mypkg, find the bug they reveal, and fix it.
2. Turn a bug report into a failing test, then make it pass.

Agenda

1. Why test?
2. testthat
3. Choosing test cases
4. Test-driven development
5. Development cycle
6. Testing in data analysis

Why Test?

A grade calculator

Steps

1. Letter: 90+ is A, 80+ is B, 70+ is C, 60+ is D, otherwise F.

Check: Does 95 get an A? Does 42 get an F?

No laptops: students run the steps in their heads. Frame it: “The registrar keeps changing the rules, and we’ll update our steps each time. After each change, check the new examples.” Thumbs up or down for each check, about 20 seconds per slide. Don’t suggest re-checking earlier examples; the last slide does that.

A grade calculator

Steps

1. Letter: 90+ is A, 80+ is B, 70+ is C, 60+ is D, otherwise F.
2. Add “+” if the last digit is 7 to 9, and “−” if it’s 0 to 2.

Check: Does 98 get an A+? Does 91 get an A−?

Both pass. This step looks harmless, but it silently breaks 100 (last digit 0, so A−), 58 (F+), and 42 (F−). Don’t point that out.

A grade calculator

Steps

1. Round to the nearest whole number.
2. Letter: 90+ is A, 80+ is B, 70+ is C, 60+ is D, otherwise F.
3. Add “+” if the last digit is 7 to 9, and “−” if it’s 0 to 2.

Check: Does 86.5 get a B+?

It passes: 86.5 rounds to 87, a B+. The new step went in at the top, the way people really patch code.

A grade calculator

Steps

1. Round to the nearest whole number.
2. If the score is 1 or less, it’s a proportion: multiply by 100.
3. Letter: 90+ is A, 80+ is B, 70+ is C, 60+ is D, otherwise F.
4. Add “+” if the last digit is 7 to 9, and “−” if it’s 0 to 2.

Check: Does 0.91 get an A−?

It passes, but only by luck: 0.91 rounds to 1, becomes 100, and the last digit 0 gives an A−. The right answer and the wrong reasoning happen to agree. If a student notices, great; save it for the next slide.

Did we keep our promises?

Task: With a partner, run the final steps on these scores.

Score	100	58	42	0.85
Should get	A+	F	F	B
Gets	?	?	?	?

01:00

All four are broken: 100 gets A− (last digit 0), 58 gets F+, 42 gets F−, and 0.85 rounds to 1 before the proportion step, becoming 100 and an A−. Every check passed along the way.

Debrief:

1. Each change passed its own check and broke one we had already checked. We checked each example once and never again.
2. A list of promises like this one is a test suite. Written down after the first slide, it would have been rerun after every change.
3. What should a score of 1 get: F (1 point) or A+ (100%)? The rules contradict each other, and you only notice when you write down what you expect. Tests are a specification.

Bugs come back

• Every check in the activity passed, at the moment you ran it.
• Later changes broke behavior you had already checked.
• Checking by eye happens once. A test runs every time you change the code.

Draw a timeline of the four rule changes. Under each one, write the check that passed. Then circle the earlier checks it broke (100, 58, 42 under the plus/minus step; 0.85 under the proportion step). Label the circles: regressions.

Regression

A bug where a change breaks something that used to work.

Unit test

Code that runs a function on a known input and checks its output against the output you expect.

The promises table on the last activity slide was a set of unit tests written on paper. Today we write them in R and let R run them.

Tests as a specification

• Each test is a promise the function makes: this input gives this output.
• Writing the promise forces a decision. What should a score of 1 get?
• @examples show users how to call a function; tests check that it still does what it promised.

testthat

Setting up: use_testthat() and use_test()

use_testthat()
use_test("strsplit1")

• use_testthat() adds testthat to Suggests in DESCRIPTION and creates tests/.
• use_test() creates a test file named for the code file it tests.

mypkg/
├── DESCRIPTION
├── R/strsplit1.R
└── tests/
    ├── testthat.R
    └── testthat/test-strsplit1.R

Live-code in mypkg from last time: run both lines, then open the two new files. tests/testthat.R runs the tests during check(); leave it alone. test-strsplit1.R starts with a placeholder test (“multiplication works”); delete it.

Anatomy of a test

tests/testthat/test-strsplit1.R

test_that("strsplit1() splits a string on commas", {
  expect_equal(strsplit1("a,b,c", split = ","),
               c("a", "b", "c"))
})

• test_that() takes a description and a block of code.
• The description finishes the sentence “Test that …”.
• Inside are one or more expectations: each one checks a single thing.

Label the three parts: the description, the curly braces holding the test code, and the expectation with its actual (the call) and expected (the vector) arguments.

Expectations

Expectation	Passes when
expect_equal(x, y)	x equals y
expect_type(x, type)	x has type type
expect_length(x, n)	x has length n
expect_true(x)	x is TRUE
expect_error(code, msg)	code errors with msg
expect_warning(code, msg)	code warns with msg

expect_equal() allows tiny numeric differences; expect_identical() doesn’t. msg is matched against the error or warning message, so a different error won’t pass.

Your Turn

Task: Write a test that strsplit1("a,b,c", split = ",") gives a character vector (not a list) with three elements.

02:00

Answer:

test_that("strsplit1() returns a character vector, not a list", {
  out <- strsplit1("a,b,c", split = ",")
  expect_type(out, "character")
  expect_length(out, 3)
})

Paste a student’s version into test-strsplit1.R; it’s the second test for the next slide. If someone uses expect_true(is.character(out)), it works, but expect_type() gives a clearer message when it fails.

Running tests with test()

devtools::test()

✔ |          3 | strsplit1

══ Results ═════════════════════════════════════════════════════════════════════
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 3 ]

• Calls load_all(), then runs every file in tests/testthat/.
• Counts expectations, not tests: two tests, three expectations.
• check() runs the tests too.

Run test() in mypkg. Keyboard shortcut in Positron/RStudio: Ctrl/Cmd + Shift + T.

Reading a failure

Add a test that splits on a period, then run test():

tests/testthat/test-strsplit1.R

test_that("strsplit1() splits on a period", {
  expect_equal(strsplit1("3.14", split = "."), c("3", "14"))
})

Failure ('test-strsplit1.R:12:3'): strsplit1() splits on a period
Expected `strsplit1("3.14", split = ".")` to equal `c("3", "14")`.
`actual`:   ""  ""   "" ""
`expected`: "3" "14"
[ FAIL 1 | WARN 0 | SKIP 0 | PASS 3 ]

Annotate the report: the file and line (12:3), the test’s description, the call, and actual against expected. Then ask the room why we got four empty strings. split is a regular expression (Strings II), and . matches any character, so every character is a delimiter. The fix comes in the TDD section: fixed = TRUE. A test we expected to be boring found a real bug.

Choosing Test Cases

List the following cases you’ll discuss:

1. typical inputs
2. edge cases
3. errors and warnings

Typical inputs

• The inputs you had in mind when you wrote the function.
• For strsplit1(): "a,b,c".
• Easy to write, and they catch the obvious breaks.
• But bugs live where you weren’t thinking.

Edge case

An input at the boundary of what a function handles: e.g. empty, a single value, very large, missing, or containing unusual characters.

For strsplit1:

• an empty string
• a delimiter at the start or end
• two delimiters in a row
• more than one string.

Your Turn

Question: Predict what each call returns. Which results are bugs?

strsplit1("", split = ",")
strsplit1(",a", split = ",")
strsplit1("a,b,", split = ",")
strsplit1(c("a,b", "c,d"), split = ",")

02:00

Run each line after the countdown.

1. character(0): no pieces. Defensible.
2. c("", "a"): a leading empty piece. Defensible.
3. c("a", "b"): the trailing empty piece is dropped, so "a,b," and "a,b" split the same way. Surprising.
4. c("a", "b"): "c,d" silently disappears, because of the [[1]]. A bug: no error, wrong answer.

Whether 1 to 3 are bugs is a decision, not a fact. Writing the test is where you make it.

Errors and warnings

Some inputs should fail. Test that they fail, and for the right reason.

test_that("log() rejects text", {
  expect_error(log("a"), "non-numeric")
})

test_that("as.numeric() warns when it can't convert", {
  expect_warning(as.numeric("a"), "NAs introduced")
})

• The second argument is matched against the message.
• Without it, any error passes, including a typo in your own code.

Test-Driven Development

Red, green, refactor

1. Red: write a test for the behavior you want. Run it; it fails.
2. Green: write the simplest code that makes it pass.
3. Refactor: revise the code. The tests stay green.

Draw the cycle as three arrows: red, green, refactor, back to red. The red step matters: a test that has never failed might not be testing anything.

Turning a bug report into a test

strsplit1(c("a,b", "c,d"), ",") returns "a" "b". Where did "c,d" go?

Red: before fixing anything, write the test.

tests/testthat/test-strsplit1.R

test_that("strsplit1() errors on more than one string", {
  expect_error(strsplit1(c("a,b", "c,d"), split = ","),
               "single string")
})

Add the test and run test(): it fails, because strsplit1() returns a value instead of throwing an error. That’s the red step. Mention the other possible fix: make strsplit1() work on a vector and return a list. Erroring is the choice for a function that promises one string in and one vector out, and the test records that choice.

Make it pass

Green: stop() throws an error with your message.

R/strsplit1.R

strsplit1 <- function(x, split) {
  if (length(x) != 1) {
    stop("`x` must be a single string, not ",
         length(x), " strings.")
  }
  strsplit(x, split = split, fixed = TRUE)[[1]]
}

• The check fixes the bug report; fixed = TRUE fixes the period test.
• The test stays in the package, so this bug can’t come back silently.

Make both changes and run test(): all green. stop() is new; it’s base R’s way to throw an error, and its arguments are pasted together into the message.

Guess the Metaphor

A test suite is like a fortified dam: each test is a stone set where a leak once broke through. Change the code all you like; the old bugs stay behind the wall.

Development Cycle

Load, document, test, check

Command	Does	Shortcut
load_all()	loads code and data	Shift + L
document()	rebuilds help pages	Shift + D
test()	runs the tests	Shift + T
check()	runs CRAN’s checks	Shift + E

Shortcuts are Ctrl/Cmd + Shift + a letter in Positron/RStudio.

Add test() to the development-cycle drawing from last time (https://r-pkgs.org/whole-game.html#fig-package-dev-workflow), between load_all() and check().

Testing in Data Analysis

Expectations in a cleaning script

zev_raw <- read_excel("data/raw/New_ZEV_Sales_Last_updated_07-17-2026_ada.xlsx",
                      sheet = "County")
zev <- zev_raw |>
  rename(year = `Data Year`, quarter = Quarter, county = COUNTY,
         fuel_type = FUEL_TYPE, make = MAKE, model = MODEL,
         n_vehicles = `Number of Vehicles`) |>
  mutate(in_state = county != "Out Of State")

# Nothing dropped; expectations still hold
stopifnot(
  nrow(zev) == nrow(zev_raw),
  all(zev$quarter %in% 1:4),
  all(zev$n_vehicles >= 1)
)

When the data are updated next quarter, a broken expectation fails loudly.

From Data Cleaning (“Decide, Record, Re-check”), trimmed to fit: the comments and the final write_csv() are gone. The stopifnot() block is a test suite for the data: nothing dropped, quarters in 1 to 4, counts at least 1. When next quarter’s file arrives and you rerun the script, it guards against regressions in the data the way test() guards against regressions in the code.

Testing an LLM

Key Ideas for Today

1. Changes break code you already checked.
2. A unit test writes down what you expect.
3. Test typical inputs, edge cases, and errors.
4. Turn every bug into a test.

For Next Time

1. PS 6 released today, due 11/14.