req_retry {httr2}R Documentation

Control when a request will retry, and how long it will wait between tries

Description

req_retry() alters req_perform() so that it will automatically retry in the case of failure. To activate it, you must specify either the total number of requests to make with max_tries or the total amount of time to spend with max_seconds. Then req_perform() will retry if the error is "transient", i.e. it's an HTTP error that can be resolved by waiting. By default, 429 and 503 statuses are treated as transient, but if the API you are wrapping has other transient status codes (or conveys transient-ness with some other property of the response), you can override the default with is_transient.

Additionally, if you set retry_on_failure = TRUE, the request will retry if either the HTTP request or HTTP response doesn't complete successfully leading to an error from curl, the lower-level library that httr2 uses to perform HTTP request. This occurs, for example, if your wifi is down.

It's a bad idea to immediately retry a request, so req_perform() will wait a little before trying again:

Usage

req_retry(
  req,
  max_tries = NULL,
  max_seconds = NULL,
  retry_on_failure = FALSE,
  is_transient = NULL,
  backoff = NULL,
  after = NULL
)

Arguments

req

A httr2 request object.

max_tries, max_seconds

Cap the maximum number of attempts with max_tries or the total elapsed time from the first request with max_seconds. If neither option is supplied (the default), req_perform() will not retry.

max_tries is the total number of attempts make, so this should always be greater than one.'

retry_on_failure

Treat low-level failures as if they are transient errors, and can be retried.

is_transient

A predicate function that takes a single argument (the response) and returns TRUE or FALSE specifying whether or not the response represents a transient error.

backoff

A function that takes a single argument (the number of failed attempts so far) and returns the number of seconds to wait.

after

A function that takes a single argument (the response) and returns either a number of seconds to wait or NULL, which indicates that a precise wait time is not available that the backoff strategy should be used instead..

Value

A modified HTTP request.

See Also

req_throttle() if the API has a rate-limit but doesn't expose the limits in the response.

Examples

# google APIs assume that a 500 is also a transient error
request("http://google.com") |>
  req_retry(is_transient = \(resp) resp_status(resp) %in% c(429, 500, 503))

# use a constant 10s delay after every failure
request("http://example.com") |>
  req_retry(backoff = ~10)

# When rate-limited, GitHub's API returns a 403 with
# `X-RateLimit-Remaining: 0` and an Unix time stored in the
# `X-RateLimit-Reset` header. This takes a bit more work to handle:
github_is_transient <- function(resp) {
  resp_status(resp) == 403 &&
    identical(resp_header(resp, "X-RateLimit-Remaining"), "0")
}
github_after <- function(resp) {
  time <- as.numeric(resp_header(resp, "X-RateLimit-Reset"))
  time - unclass(Sys.time())
}
request("http://api.github.com") |>
  req_retry(
    is_transient = github_is_transient,
    after = github_after
  )

[Package httr2 version 1.0.7 Index]