Skip to main content

Optimizing Query Performance

When querying historical market data, response sizes can grow rapidly—especially with high-resolution tick and intraday datasets. The algoseek Datasets API provides several query parameters designed to help you fetch exactly what you need. By strategically applying these filters, you optimize network transfer times, reduce latency, and speed up your systematic processing.

Selecting a Specific Time Range for Tick-Level Data​

Tick-level endpoints, such as the US Equities Trade and Quote (TAQ) dataset, require the trade_date and identifier directly in the URL path. Because a single trading day can contain millions of ticks for highly liquid assets, pulling the entire day is not always optimal.

To fetch data for a specific intraday time window, utilize the API's Advanced Filtering capabilities on the timestamp column (e.g., EventDateTime).

Example: Fetching AAPL trade and quote updates only during the first 15 minutes of the market open on March 15, 2023.

curl --get \
     'https://api.algoseek.com/v1/data/us-equity/taq/2023-03-15/AAPL' \
     --data-urlencode 'EventDateTime.gt=2023-03-15 09:30:00' \
     --data-urlencode 'EventDateTime.lt=2023-03-15 09:45:00' \
     -H "X-API-KEY: YOUR_API_KEY"

Selecting a Specific Date Range for Intraday Data​

Unlike tick data endpoints that restrict you to a single day in the path, Intraday and Daily endpoints (like Minute Bars) take only the {identifier} in the path. This allows you to query long-term historical trends in a single request.

To prevent querying a large chunk of available history of a ticker, use filter expressions to define a strict date boundary.

Example: Fetching AAPL minute bars for a specific date range in 2023.

curl --get \
     'https://api.algoseek.com/v1/data/us-equity/taq-1min/AAPL' \
     --data-urlencode 'TradeDate.gt=2023-01-01' \
     --data-urlencode 'TradeDate.le=2023-04-30' \
     -H "X-API-KEY: YOUR_API_KEY"

Limiting the Number of Columns​

Many algoseek datasets are extremely wide. For instance, the US Equities Trade and Quote Minute Bar datasets can contain upwards of 50 to 90 calculated metrics per interval. If your research model only requires basic price and volume metrics, downloading all columns wastes bandwidth.

You can explicitly define which fields the API should return using the columns query parameter.

Example: Fetching only the Open, High, Low, Close, and Volume fields for an intraday bar.

curl --get \
     'https://api.algoseek.com/v1/data/us-equity/trades-1min/AAPL' \
     --data-urlencode 'TradeDate.lt=2023-03-15' \
     --data-urlencode 'columns=BarDateTime,HighTradePrice,LowTradePrice,Volume' \
     -H "X-API-KEY: YOUR_API_KEY"

Choosing the Optimal Response Format​

The API supports multiple data delivery formats through the response_format query parameter. For heavy quantitative research and bulk backtesting data extraction, adjusting the response format is one of the most impactful optimizations you can make.

Format OptionBest Used ForDescription
jsonPrototyping & small queriesThe default format. Easy to read and parse, but carries significant payload overhead.
csvData science workflowsStrips out JSON object overhead, drastically reducing the total payload size for tabular data.
csv_gzipBulk historical downloadsReturns a compressed CSV format. This is the most highly optimized choice for large pulls.

Example: Requesting a highly compressed csv_gzip stream and saving it directly to disk using the --output flag to minimize memory footprint.

curl --get \
     'https://api.algoseek.com/v1/data/us-equity/taq-daily/AAPL' \
     --data-urlencode 'response_format=csv_gzip' \
     -H "X-API-KEY: YOUR_API_KEY" \
     --output aapl_daily_taq.csv.gz
Note

The typical compression factor is 5 to 8 times for market data datasets.