Pagination, batching and throughput

Cursors beat offsets

Offset pagination asks the database to skip thousands of rows on every request which is inefficient. Message logs are append-heavy, so offsets are exactly the wrong tool for the job. Cursor pagination encodes the position of the last item you saw into an opaque string for the next call. This approach remains performant even when you are millions of messages deep into your logs.

let cursor = null;
do {
  const page = await api.messages.list({ limit: 100, after: cursor });
  for (const m of page.data) await archive(m);
  cursor = page.has_more ? page.next_cursor : null;
} while (cursor);

Batching for Efficiency

Bulk endpoints accept many messages in one request and return per-item results to reduce overhead. Treat the response as a list of independent outcomes because some items will fail while others succeed. Give each item its own idempotency key so partial retries are safe for the entire batch. This is far more efficient than sending a thousand individual POST requests over high-latency connections.

Throughput and rate limits

Rate limits are usually expressed as a token bucket with a refill rate and a burst size. The right client strategy is a local limiter that matches the provider's published rate limits. Without the local limiter you will oscillate between bursting and getting throttled by the server. This oscillation is bad for your application's throughput and can put unnecessary strain on internal systems.

Handling the token bucket

Implementing a local token bucket or leaky bucket algorithm helps smooth out your outgoing traffic. This ensures that you are utilizing your full allocated bandwidth without constantly hitting the ub3r platform's limits. It also makes your application's performance more predictable, which is essential for meeting Service Level Agreements. Consider using a centralized rate-limiting service if you are running multiple instances of your application.

Scaling your Infrastructure

As your messaging volume grows, you will need to scale your worker pool to handle the load. Use horizontal scaling to add more workers that can process messages from your internal queues in parallel. Ensure that each worker respects the global rate limits to avoid being blocked by the ub3r API. A distributed architecture is the best way to handle millions of messages daily with high reliability.