JSON Data, and APIs II

Web Technologies

STAT 133 with Gaston Sanchez

API Example: Open Notify

About Open Notify

Open Notify is an open source project, developed and maintained by Nathan Bergey, to provide a simple application programming interface (API) for some of NASA’s data.

Open Notify’s APIs allow you to make 2 types of requests¹:

Current location of the International Space Station (ISS)
- http://api.open-notify.org/iss-now.json
Number of people in space
- http://api.open-notify.org/astros.json

Number of people in space

http://api.open-notify.org/iss-now.json

Location of ISS

http://api.open-notify.org/iss-now.json

Interacting with an API

Web browser: enter URL with parameters to make request.

Command line: use curl to make request.

R or Python: use HTTP libraries to make request.

Interacting with a Browser

Craft URL from a base URL (endpoint) and parameters

http://api.open-notify.org/iss-now.json

Interacting from Command Line

Most shells have curl command for making HTTP requests.

curl "http://api.open-notify.org/iss-now.json"

{"timestamp": 1776564579, "iss_position": {"latitude": "-66.4643", "longitude": "-47.3791"}, "message": "success"}

Interacting from R (option 1 quick & dirty)¹

iss_loc_json <- "http://api.open-notify.org/iss-now.json"

iss_loc_list <- fromJSON(iss_loc_json)

iss_loc_list

$timestamp
[1] 1776564579

$iss_position
$iss_position$longitude
[1] "-47.3791"

$iss_position$latitude
[1] "-66.4643"

$message
[1] "success"

iss_lat <- as.numeric(iss_loc_list$iss_position$latitude)
iss_lng <- as.numeric(iss_loc_list$iss_position$longitude)

leaflet() |> 
  addTiles() |> 
  setView(lng = iss_lng, iss_lat, zoom = 2) |> 
  addCircles(lng = iss_lng, lat = iss_lat, radius = 70000)

Interacting with APIs through HTTP Libraries

Interacting from R (option 2 more formal)

Use an HTTP library, e.g. "httr2"

Interacting from R (option 2 more formal)

# install.packages("httr2")
library(httr2)

Open Notify’s ISS location:

Create a request object with request(), this define your base request.
Complete the full path, via req_url_path_append(), by appending the JSON file name.
Submit request with req_perform().
Extract data with resp_body_json()

Interacting from R (option 2 more formal)

api_url <- "http://api.open-notify.org"
req <- request(api_url) |> 
  req_url_path_append("iss-now.json") |> 
  req_perform()

req

<httr2_response>
GET http://api.open-notify.org/iss-now.json
Status: 200 OK
Content-Type: application/json
Body: In memory (112 bytes)

names(req)

[1] "method"      "url"         "status_code" "headers"     "body"       
[6] "request"     "cache"

Interacting from R (option 2 more formal)

api_url <- "http://api.open-notify.org"
req <- request(api_url) |> 
  req_url_path_append("iss-now.json") |> 
  req_perform() |> 
  resp_body_json()

req

$timestamp
[1] 1776564579

$iss_position
$iss_position$longitude
[1] "-47.3791"

$iss_position$latitude
[1] "-66.4643"

$message
[1] "success"

"httr2" is the modern standard for API interaction in R. It treats the API request as a conversation.

Pros:

API-First: Built specifically for modern web requirements like OAuth, API keys, and custom headers.
Pipe-friendly: Works with |> (e.g., request(url) |> req_perform()).
Robust: It has built-in features for retrying failed requests, rate-limiting, and pausing between calls.
Informative: It gives you easy access to HTTP status codes.

Cons:

Dependency: Requires installing the package.
Learning curve: Slightly more complex for a simple one-off file download.

Example: BART API

BART Legacy API

https://www.bart.gov/schedules/developers/api

Like many other APIs, BART Legacy API requires an API key.
Usually, you would have to register with the API provider to obtain an API key.
Good news: BART lets you use their Legacy API key without having to register.
Legacy API key: MW9S-E7SL-26DU-VV8V
For those interested in getting their own API key, BART’s API also provides this kind of key.

API’s Documentation

In order to use an API, the first step consists of finding and reading the API’s Documentation.
The Documentation defines what information you can ask for and how to format your request.
Keep in mind that some API’s provide minimal or almost no documentation, while others provide extensive—and sometimes not-user-friendly—documentation.
BART Legacy API provides a fair amount of documentation but it can feel a bit intimidating to a beginner user.

Interacting from R

# set of all available stations
library(httr2)
api_url <- "https://api.bart.gov/api/stn.aspx"
res <- request(api_url)
res

<httr2_request>
GET https://api.bart.gov/api/stn.aspx
Body: empty

Interacting from R

# set of all available stations
library(httr2)
api_url <- "https://api.bart.gov/api/stn.aspx"
res <- request(api_url) |>
  req_url_query(cmd  = "stns") |>
  req_url_query(key  = "MW9S-E7SL-26DU-VV8V") |>
  req_url_query(json = "y") |>
  req_perform()
res

<httr2_response>
GET https://api.bart.gov/api/stn.aspx?cmd=stns&key=MW9S-E7SL-26DU-VV8V&json=y
Status: 200 OK
Content-Type: application/json
Body: In memory (10289 bytes)

Interacting from R

data <- resp_body_json(res, simplifyVector = FALSE)
names(data)

[1] "?xml" "root"

names(data$root)

[1] "uri"      "stations" "message"

names(data$root$stations)

[1] "station"

# number of stations
length(data$root$stations$station)

[1] 50

data$root$stations$station[[1]]

$name
[1] "12th St. Oakland City Center"

$abbr
[1] "12TH"

$gtfs_latitude
[1] "37.803768"

$gtfs_longitude
[1] "-122.271450"

$address
[1] "1245 Broadway"

$city
[1] "Oakland"

$county
[1] "alameda"

$state
[1] "CA"

$zipcode
[1] "94612"

data$root$stations$station[[2]]

$name
[1] "16th St. Mission"

$abbr
[1] "16TH"

$gtfs_latitude
[1] "37.765062"

$gtfs_longitude
[1] "-122.419694"

$address
[1] "2000 Mission Street"

$city
[1] "San Francisco"

$county
[1] "sanfrancisco"

$state
[1] "CA"

$zipcode
[1] "94110"

Detailed information about the specified station

# detailed information about the specified station
api_url <- "https://api.bart.gov/api/stn.aspx"
res2 <- request(api_url) |>
  req_url_query(cmd  = "stninfo") |>
  req_url_query(orig = "DBRK") |>
  req_url_query(key  = "MW9S-E7SL-26DU-VV8V") |>
  req_url_query(json = "y") |>
  req_perform()
res2

<httr2_response>
GET
https://api.bart.gov/api/stn.aspx?cmd=stninfo&orig=DBRK&key=MW9S-E7SL-26DU-VV8V&json=y
Status: 200 OK
Content-Type: application/json
Body: In memory (1810 bytes)

Interacting from R

downtown_berk <- resp_body_json(res2, simplifyVector = FALSE)
downtown_berk

$`?xml`
$`?xml`$`@version`
[1] "1.0"

$`?xml`$`@encoding`
[1] "utf-8"


$root
$root$`@id`
[1] "1"

$root$uri
$root$uri$`#cdata-section`
[1] "http://api.bart.gov/api/stn.aspx?cmd=stninfo&orig=DBRK&json=y"


$root$stations
$root$stations$station
$root$stations$station$name
[1] "Downtown Berkeley"

$root$stations$station$abbr
[1] "DBRK"

$root$stations$station$gtfs_latitude
[1] "37.870104"

$root$stations$station$gtfs_longitude
[1] "-122.268133"

$root$stations$station$address
[1] "2160 Shattuck Avenue"

$root$stations$station$city
[1] "Berkeley"

$root$stations$station$county
[1] "alameda"

$root$stations$station$state
[1] "CA"

$root$stations$station$zipcode
[1] "94704"

$root$stations$station$north_routes
$root$stations$station$north_routes$route
$root$stations$station$north_routes$route[[1]]
[1] "ROUTE 3"

$root$stations$station$north_routes$route[[2]]
[1] "ROUTE 8"



$root$stations$station$south_routes
$root$stations$station$south_routes$route
$root$stations$station$south_routes$route[[1]]
[1] "ROUTE 4"

$root$stations$station$south_routes$route[[2]]
[1] "ROUTE 7"



$root$stations$station$north_platforms
$root$stations$station$north_platforms$platform
$root$stations$station$north_platforms$platform[[1]]
[1] "1"



$root$stations$station$south_platforms
$root$stations$station$south_platforms$platform
$root$stations$station$south_platforms$platform[[1]]
[1] "2"



$root$stations$station$platform_info
[1] ""

$root$stations$station$intro
$root$stations$station$intro$`#cdata-section`
[1] "The Downtown Berkeley BART Station is located on Shattuck Avenue between Allston Way and Addison Street. It is conveniently located close to the University of California campus and to many shops, restaurants, theaters and other attractions, and has valet bike parking. Public restrooms are available.Maps of this stationStation MapTransit StopsTransit RoutesSchedules and Fares"


$root$stations$station$cross_street
$root$stations$station$cross_street$`#cdata-section`
[1] "Between: Center St. & Allston Way"


$root$stations$station$food
$root$stations$station$food$`#cdata-section`
[1] "Nearby restaurant reviews from <a href=\"http://www.yelp.com/search?find_desc=Restaurant+&ns=1&rpp=10&find_loc=2160 Shattuck Avenue Berkeley, CA 94704\">yelp.com</a>"


$root$stations$station$shopping
$root$stations$station$shopping$`#cdata-section`
[1] "Local shopping from <a href=\"http://www.yelp.com/search?find_desc=Shopping+&ns=1&rpp=10&find_loc=2160 Shattuck Avenue Berkeley, CA 94704\">yelp.com</a>"


$root$stations$station$attraction
$root$stations$station$attraction$`#cdata-section`
[1] "More station area attractions from <a href=\"http://www.yelp.com/search?find_desc=+&ns=1&rpp=10&find_loc=2160 Shattuck Avenue Berkeley, CA 94704\">yelp.com</a> and <a href=\"http://www.sfgate.com/cgi-bin/article.cgi?f=/c/a/2008/03/13/NSENVEA1M.DTL&type=travel\">sfgate.com</a>"


$root$stations$station$link
$root$stations$station$link$`#cdata-section`
[1] "http://www.bart.gov/stations/DBRK"




$root$message
[1] ""

lat <- downtown_berk$root$stations$station$gtfs_latitude
lat

[1] "37.870104"

JSON Data, and APIs II

API Example: Open Notify

About Open Notify

Number of people in space

Location of ISS

Interacting with an API

Interacting with a Browser

Interacting from Command Line

Interacting from R (option 1 quick & dirty)1

Interacting with APIs through HTTP Libraries

Interacting from R (option 2 more formal)

Interacting from R (option 2 more formal)

Interacting from R (option 2 more formal)

Interacting from R (option 2 more formal)

Example: BART API

API’s Documentation

Interacting from R

Interacting from R

Interacting from R

Detailed information about the specified station

Interacting from R

Interacting from R (option 1 quick & dirty)¹