Rate Limits

ScrapeBadger rate limits requests per minute to ensure fair usage and reliable performance for all users.

How Rate Limiting Works

Plan-Based Limits

Your rate limit is determined by your plan. All API keys inherit the same limit from your account.

Sliding Window

We use a sliding window algorithm for accurate rate limiting. Limits are calculated per minute with smooth request distribution.

Real-Time Headers

Every response includes rate limit headers so you can monitor usage and implement smart retry logic.

Rate Limits by Plan

Plan	Tier	Requests/Minute
Free (sign-up credits, nothing bought yet)	Free	5
Pay-as-you-go	Basic	60
Starter, Growth	Pro	300
Pro, Scale	Enterprise	1,000
Volume	Enterprise	3,000
Max	Enterprise	5,000
Custom	Enterprise	By agreement

See pricing for what each plan includes.

Rate Limit Headers

Every API response includes these headers to help you track your rate limit usage:

Header	Description	Example
`X-RateLimit-Limit`	Maximum requests allowed per minute for your tier	`300`
`X-RateLimit-Remaining`	Requests remaining in the current window	`287`
`X-RateLimit-Reset`	Unix timestamp when the rate limit window resets	`1703123456`
`Retry-After`	Seconds until you can retry (only on 429 responses)	`45`

Rate Limit Exceeded Response

When you exceed your rate limit, the API returns a 429 Too Many Requests response:

{
  "detail": "Rate limit exceeded",
  "limit": 300,
  "remaining": 0,
  "reset_at": 1703123456,
  "tier": "basic"
}

Handling Rate Limits

# Check rate limit headers in response
curl -i -H "x-api-key: sb_live_xxx" \
  https://scrapebadger.com/v1/twitter/tweets/123

# Response headers:
# X-RateLimit-Limit: 300
# X-RateLimit-Remaining: 299
# X-RateLimit-Reset: 1703123456

async function fetchWithRateLimit(url, apiKey) {
  const response = await fetch(url, {
    headers: { 'x-api-key': apiKey }
  });

  const remaining = response.headers.get('X-RateLimit-Remaining');
  const limit = response.headers.get('X-RateLimit-Limit');

  if (response.status === 429) {
    const retryAfter = response.headers.get('Retry-After');
    console.log(`Rate limited. Retry in ${retryAfter} seconds`);
    await new Promise(r => setTimeout(r, retryAfter * 1000));
    return fetchWithRateLimit(url, apiKey);
  }

  console.log(`Requests remaining: ${remaining}/${limit}`);
  return response.json();
}

import time
import requests

def fetch_with_rate_limit(url: str, api_key: str):
    response = requests.get(url, headers={"x-api-key": api_key})

    remaining = response.headers.get("X-RateLimit-Remaining")
    limit = response.headers.get("X-RateLimit-Limit")

    if response.status_code == 429:
        retry_after = int(response.headers.get("Retry-After", 60))
        print(f"Rate limited. Retry in {retry_after} seconds")
        time.sleep(retry_after)
        return fetch_with_rate_limit(url, api_key)

    print(f"Requests remaining: {remaining}/{limit}")
    return response.json()

Best Practices

Monitor rate limit headers

Check X-RateLimit-Remaining on each response to proactively slow down before hitting limits.

Implement exponential backoff

When rate limited, wait the Retry-After time and consider increasing wait time on repeated limits.

Use request queuing

Queue requests and process them at a controlled rate to stay within your limits.

Cache responses

Cache API responses where appropriate to reduce the number of requests you need to make.

Avoid Rate Limit AbuseRepeatedly hitting rate limits or attempting to bypass them may result in temporary or permanent restrictions on your account.

Don’t create multiple accounts to circumvent limits
Don’t use rotating API keys to bypass rate limiting
Contact support if you need higher limits

FAQ

How are rate limits applied?

Rate limits are applied per API key using a sliding window algorithm. Each API key has its own separate rate limit counter based on your plan.

What happens when I hit the rate limit?

The API returns a 429 Too Many Requests response with details about when you can retry. Use the Retry-After header to know when to try again.

Which calls count against my rate limit?

Every /v1 call, including the free ones: /v1/quote (an agent’s quote_call) and /v1/account/me (get_account_info) count like a scrape. On the Free tier’s 5 requests per minute, quoting before each call halves how many calls you can make. Re-reading a result (/v1/results/{result_id}) and polling an async request (/v1/requests/{request_id}) do not count.

Do failed requests count against my rate limit?

Yes, all requests (successful or not) count against your rate limit. However, rate limits are generous and designed to accommodate normal usage patterns.

Can I have different rate limits for different API keys?

All API keys inherit the rate limit from your plan. To get higher rate limits, upgrade your plan.

How do I check my current rate limit usage?

Every API response includes rate limit headers (X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset) so you can track your usage in real-time.

Need higher rate limits? Contact us at sales@scrapebadger.com.

Getting Started

Account

How Rate Limiting Works

Plan-Based Limits

Sliding Window

Real-Time Headers

Rate Limits by Plan

Rate Limit Headers

Rate Limit Exceeded Response

Handling Rate Limits

Best Practices

Monitor rate limit headers

Implement exponential backoff

Use request queuing

Cache responses

FAQ

​How Rate Limiting Works

Plan-Based Limits

Sliding Window

Real-Time Headers

​Rate Limits by Plan

​Rate Limit Headers

​Rate Limit Exceeded Response

​Handling Rate Limits

​Best Practices

Monitor rate limit headers

Implement exponential backoff

Use request queuing

Cache responses

​FAQ

How Rate Limiting Works

Rate Limits by Plan

Rate Limit Headers

Rate Limit Exceeded Response

Handling Rate Limits

Best Practices

FAQ