Skip to main content

Understanding algoseek API Response Headers

When you query endpoints in the algoseek Datasets API, the server returns your data alongside hidden metadata known as HTTP response headers.

Because financial datasets are large and available in various formats, the API uses specific headers to manage pagination, file downloads, and request tracing.

This tutorial covers six critical response headers to process in your client application:

  • X-Pagination-Offset
  • X-Pagination-Limit
  • X-Pagination-Has-Next
  • Link
  • X-Request-ID
  • Content-Disposition

Pagination headers​

Endpoints that return lists of data support pagination through the offset and limit query parameters. To help you track your position in a large dataset, the API returns three custom headers:

  • X-Pagination-Offset: Confirms the number of records skipped before the current batch was returned. For example, if you make a request with ?offset=5000, this header returns 5000. Use it to verify the starting point of your data chunk.

  • X-Pagination-Limit: Confirms the maximum number of records the server returns in this response. This validates the size of your current data page. (For JSON output the max is 10,000, for CSV it is 30,000, and for CSV Gzip it is 80,000).

  • X-Pagination-Has-Next: A boolean string (true or false) that indicates whether more records are available beyond the current page. You can use this to create a while loop in your code. Continue fetching data, increasing your offset by your limit each time, until X-Pagination-Has-Next evaluates to false.

While X-Pagination-Has-Next tells you if there is another page, the Link header provides the exact URL to get there.

The Link header is a standard HTTP header that provides pre-constructed URLs to related pages of your dataset. It contains URLs tagged with relational parameters like rel="next", rel="prev", rel="first", or rel="last".

Example format:

<https://api.algoseek.com/v1/data/us-equity/taq/2024-01-15/AAPL?limit=1000&offset=1000>; rel="next"

Instead of manually constructing the next URL by adding your limit to your offset, your HTTP client can parse the Link header, extract the URL assigned to rel="next", and execute the next request.

The X-Request-ID header​

The X-Request-ID header is a globally unique identifier (UUID) assigned to your API call when it reaches the algoseek servers.

Log this ID alongside your responses. If you experience a validation error, an access error, or an unexpected data gap, provide this ID to algoseek support. It helps backend engineers quickly locate your transaction in their server logs.

The Content-Disposition header​

Using the response_format query parameter, you can request data as JSON, raw CSV, or compressed CSV.

The Content-Disposition header tells your client or browser whether to display the response inline or treat it as a downloadable attachment. When you request ?response_format=csv_gzip, the server attaches this header to the response.

Example format:

`Content-Disposition: attachment; filename="us_equity_taq_AAPL_2024-01-15.csv.gz"`

If you write a script to download and save data, you can programmatically extract the filename value from the Content-Disposition header and use it to save the file locally.

Best-practice workflow​

For a robust implementation, follow these steps to download data:

  1. Send a GET request with ?limit=10000&response_format=csv_gzip.
  2. Read the Content-Disposition header to securely save the gzip file to your computer.
  3. Check the X-Pagination-Has-Next header.
  4. If the value is true, parse the Link header for the rel="next" URL and repeat the process.
  5. If the request fails, catch the exception and log the X-Request-ID for debugging.