Project: Gerrymandering

The Gerry-mander cartoon from the Salem Gazette, 1813: Massachusetts's Essex South district drawn as a winged monster, labeled with its towns.

Very soon Californians will go to the polls to vote for their representative to the United States Congress. This election will determine whether the Republicans retain control of the House of Representatives. This is the first election held using district maps that are explicitly partisan. One year ago, Californians passed Proposition 50, which adopted a congressional district map drawn by the state legislature (bill AB 604). The purpose of this bill was to increase the partisan advantage of Democrats as a counterweight to a similar district map that was passed by the state of Texas to advantage the Republicans.

Your general research question for this project is:

Will the adoption of the new district map lead to an increase in partisan advantage relative to 2024? If so, by how much?

As always, you will be invited by email to a template repo corresponding to this project. This time, project.qmd is set to format: dashboard, the format from the Dashboards lecture. Each part below is a page of your dashboard. How you lay out each page is up to you, but each should show your results as explanatory displays (plots, tables, value boxes, or a few sentences) along with the code that produced them. The comment at the top of the template’s project.qmd has some tips on how to set this up.

Your dashboard opens on its Results page (Part 6), because that is where a reader starts, but you’ll build that page last: work through the parts in order.

In the assignment below, “On this page” describes what each page of your dashboard should show. The questions are there to prompt you to fully explore each phase of the analysis to be ready to answer questions about it during the Project Studio.

Context

Data

The template’s data/ folder holds:

  1. state_g24_sov_data_by_g24_srprec.csv: Voting results (Statement of Vote) for the 2024 general election by registration precinct (SR precinct), from the Statewide Database (source: https://statewidedatabase.org/d20/g24.html). Its columns are described in the Statewide Database’s codebooks (https://statewidedatabase.org/info/codebooks.html).
  2. g24-results-by-district.xlsx: Voting results at the district level, broken down by county.
  3. g24-candidates-by-district.csv: Names of candidates running in each district race.

In Parts 5 and 6 you’ll download more files and add them to data/ yourself, unzipping any shapefiles into a subdirectory called data/shapefiles. Many of these files are too large for GitHub, so the template’s .gitignore keeps git from committing anything in data/ beyond the three files above. Keep every download inside data/: a file saved anywhere else would be committed. If you save a cleaned version of a data set, which is good practice, write it to data/clean/; that folder is committed.

Part 1: About the Data

Visit the website of the California Secretary of State for their information on last year’s special election and Proposition 50 and learn about the process, the documents that they make available, and the data that they make available.

Next visit the website of the Statewide Database and learn about what data products are available from the 2024 General Election and any extra information they provide about how to interpret them. Also learn about who runs the database, where it is housed, and how it operates.

On this page: provide an introduction to the election and the data for a reader who is new to both: what Proposition 50 changed and why, who publishes the data you use and how it comes to be, the units of geography it is recorded in and how they relate to one another, and a table of the data files you use, each with its source and what one row represents. Add to the table as you go, including the files you download in Parts 5 and 6. This page needs no code.

Some questions to consider:

  1. What did Proposition 50 do, and how long does the AB 604 map stay in effect?
  2. Who drew the AB 604 map, and how is that different from how California’s congressional maps were drawn before?
  3. What documents and data does the Secretary of State publish about the election?
  4. Who runs the Statewide Database, where is it housed, and what is it for?
  5. What data products does it publish for the 2024 General Election?
  6. What do its notes say about the limits of the data?
  7. How do census blocks, SV precincts, SR precincts, congressional districts, and counties relate to one another? Which nest inside which?
  8. Why is an SR precinct always a geographic area, while an SV precinct may not be?
  9. Votes are counted and reported by SV precinct. How does the Statewide Database produce the SR precinct counts you’ll use, and what does that mean for how you should read them?

Part 2: Data Cleaning

Read in the vote count data from the 2024 General Election at the SR precinct level (state_g24_sov_data_by_g24_srprec.csv). You’ll use this same data set for the rest of the project, including re-running the election on the new map in Part 5. Clean the data frame using the audit checklist from the data cleaning lecture to ensure values are recorded correctly for the rest of the analysis.

On this page: the decisions you made in cleaning the data. For each problem you found, show the evidence for it (a table, a plot, or a count), what you did about it, and how many rows it affected, along with the code for each step. Close with the size of your clean data set.

Tips

  • Read the website of the statewide database carefully for disclaimers about the state of the data.
  • Inspect columns to ensure they’re the correct type. If they’re not, investigate why not and correct it.
  • Inspect the values in each column to ensure they have the values that you would expect.
  • Check that columns that you expect to have unique values indeed don’t have replicates.

Part 3: Exploratory Data Analysis

Now that you have a clean data set for precinct-level vote totals from the 2024 election, now is your opportunity to scratch your curious itch and ask a few questions that can be answered with summary statistics or visualizations.

On this page: a check of your clean data against the official district totals, followed by at least three questions of your own. For each question, show the question, an explanatory display that answers it, and your answer in a sentence or two.

Some examples to get you thinking:

  1. When you add up your precincts by district, do your totals agree with those in g24-results-by-district.xlsx? Where they don’t, why not?
  2. How many precincts/counties/districts featured two candidates from the same party?
  3. Which precinct/county/district had the closest race?
  4. What is the range of total sizes of the precincts?
  5. Which races were the closest?
  6. Which races were the biggest blowouts?
  7. What proportion of incumbents were re-elected?
  8. Be creative here!

Part 4: Measuring Partisan Advantage

For the 2024 general election, calculate the mean-median score and efficiency gap associated with the 2024 map of congressional districts. You should have everything you need in your cleaned precinct-level data frame.

On this page: the partisan advantage under the 2024 map: the seats and the share of the two-party vote that each party won, the mean-median score, and the efficiency gap. Show each statistic alongside a display that shows where it comes from, such as the distribution of vote shares across districts or the votes each party wasted.

Tip: it may be helpful to write a function that takes a vector of votes across districts for party A and a vector of votes for party B, and returns a vector of the number of votes wasted by party A in each district.

Part 5: Re-running the 2024 Election

Calculate what the election results would have been had the 2024 election been run using the district map proposed in AB 604. There are two approaches that can be taken. Both start from the SR precinct data you cleaned in Part 2: SR precincts are always geographical, so their votes can be placed on a map.

The data files listed below are not in the template repo; you’ll be downloading them directly. Put csv files in data/ and unzip any shapefiles into data/shapefiles/, not anywhere else in your repo: inside data/, git ignores them, and some are far too large for GitHub (the SR precinct shapefile is over 300 MB). The AB 604 zip unpacks into an AB604/ folder, so its shapefile is data/shapefiles/AB604/AB604.shp, alongside a __MACOSX/ folder you can delete. Its districts are in a Web Mercator projection, so transform them with st_transform(crs = 3310) before combining them with other layers. Some of this code is slow to run, so the template marks this part’s code cell with cache: true: it reruns only when you change it or the Part 2 cleaning it builds on.

Of these two approaches, Approach A is slightly more direct but induces a bit more error than Approach B. You can choose one to pursue. If you end up doing both, I’d be very interested in an analysis that compares the results. Please email me directly if you do this to share your work.

Approach A: Area-weighted Interpolation

Premise: if we assume the voters are spread evenly across their precincts, we can allocate a precinct’s votes to a congressional district in proportion to the percentage of the area of that precinct that is in the district. Along with your clean SR precinct data, you will need two files:

  1. SR Precinct Shapefiles: Shapefiles for the SR precincts (source: https://statewidedatabase.org/d20/g24_geo_conv.html).
  2. Proposed Congressional Map: source of shapefiles for district map proposed in AB 604 (source: https://aelc.assembly.ca.gov/proposed-congressional-map).

General approach:

  1. Join the geometries from the shapefile to your clean SR precinct data.
  2. Load geometries of the proposed congressional map.
  3. Ensure both geometries are valid and use the same projection (I recommend st_transform(crs = 3310)).
  4. Use area-weighted interpolation to populate the vote counts of each district in the proposed map.

Note: the people at the Statewide Database are working to correct some errors in the geometries of the SR precincts. sf can do a quick job of it using the following:

sr_shp <- sr_shp |>
  st_transform(3310) |>          # switch to equal area projection first
  st_set_precision(1) |>         # snap points to 1 m grid
  st_make_valid() |>             # fix self-intersections/bow-ties
  st_collection_extract("POLYGON")

Approach B: Proportional Allocation

Premise: we can improve on the uniform distribution assumption by working with a smaller geographic unit, census blocks. While we don’t know the number of votes for each candidate at the census block level, we do know the proportion of registered voters at each census block. We can allocate the votes from the SR precincts across the blocks relative to the proportion of registered voters that live on that block.

Along with your clean SR precinct data, you will need two files:

  1. SR Precinct to Block Correspondence: Number of registered voters by (partial) block (source: https://statewidedatabase.org/d20/g24_geo_conv.html).
  2. Block membership in AB 604 districts: A list of the census block keys and their new congressional district as of AB 604 (source: https://aelc.assembly.ca.gov/proposed-congressional-map).

General approach:

  1. Working from the census block data, add SR voting data to each (fractional) block in proportion to the percentage of registered voters in the precinct that live on that block.
  2. Identify which new district each block is in.
  3. Tally up the votes for each new district.

Note: there are some census blocks that are split between multiple SR precincts. Accordingly, each row of the SR Precinct - Block correspondence file is actually a potential fractional block: most rows represent an entire block, but some represent the portion of a block that is in a particular precinct. BLKTOTREG is the total number of registered voters in the full block; SRTOTREG is the total number of registered voters in the SR precinct. PCTSRPREC is the percent of the total SR registrants that live on the fractional block. BLKREG is the number of registered voters on the fractional block.

On this page: how you estimated the 2024 votes in each of the new districts: the approach you chose and the assumption it rests on, the estimated votes for each party in each new district, and a check of whether your allocation kept every vote.

Part 6: Results

This is the first page of your dashboard, so take care to present your results in a manner that is clear and makes the most important comparison easy: the 2024 congressional results against a hypothetical re-run of the 2024 election using the new district map. If there is another visualization or statistic that you find insightful and effective, you’re welcome to add it.

To draw the maps, download the 2024 district shapefiles, and the AB 604 shapefiles if you don’t have them yet from Part 5. As in Part 5, unzip them into data/shapefiles/ so that git ignores them:

  1. 2024 Congressional Districts: shapefiles for the congressional districts in effect for the 2024 election (choose California).
  2. Proposed Congressional Map: shapefiles for the district map in AB 604.

On this page: the headline comparison between the 2024 results and your re-run of the 2024 election on the AB 604 map. Show maps of both sets of districts, each district colored by the party that won it (for the AB 604 map, in your re-run), and for both maps, the number of seats won by Democrats, the mean-median score, and the efficiency gap. Close with your answer to the research question and its caveats.