Skip to main content

WebSocket Delivery

Connect to the ScrapeBadger WebSocket endpoint to receive real-time tweet events from all your active stream monitors and filter rules.

Connection

Using wscat

Using Python

Using Node.js

Messages

Every message is a JSON object with a type field.

Tweet event

Tweets from a filter rule also carry "source": "filter_rule", rule_id and tag instead of monitor_id.

Connection behavior

  • Authentication is done via the api_key query parameter
  • Keepalive: the server sends {"type": "ping"} every 40 seconds. If your client doesn’t reply with {"type": "pong"} within 30 seconds, the connection is closed with code 1001
  • Restarts: the server closes all connections with code 1001 when it restarts. Reconnect right away
  • All monitors and filter rules for the API key deliver to the same connection
  • Up to 5 connections per API key. Every open connection gets every event

Close codes

Delivery status and missed tweets

WebSocket delivery is live only: tweets aren’t queued or replayed. A tweet detected while none of your connections is open isn’t sent again. Each tweet a monitor detects is recorded in the delivery log (GET /v1/twitter/stream/logs) with a delivery_status: To make sure you receive everything, backfill after every reconnect:
Skip any tweet_id you’ve already processed, and fetch the rest with Get tweets by IDs if you need the full object. The log keeps the most recent 100 entries per monitor, so backfill promptly.

Best practices

  1. Reply to every ping, or the connection is dropped after 70 seconds
  2. Reconnect on every close with exponential backoff, then backfill pending tweets from the delivery log
  3. Process messages asynchronously so a slow handler doesn’t delay your pong
  4. Deduplicate by tweet_id. With several connections open, each one receives every event
  5. Use webhooks when you need guaranteed delivery. Webhooks are retried, WebSocket is not