Handling Ad API Errors: Credits, Rate Limits, and Retries

Most API error handling best practices were written for systems where a failed request costs nothing but latency. Ad intelligence pipelines break that assumption. When your script hits a paid ad library API, a search costs a credit, a retry costs another credit, and a naive retry loop wrapped around the wrong error class can spend a week of budget before breakfast. Error handling here is budget protection, and it deserves the same rigor you'd give a payment integration.

TL;DR: Classify every failure before reacting to it. Auth errors (401/403) page a human and never retry. Rate limits (429) honor the Retry-After header exactly. Credit errors (402) must never be blind-retried, because on a paid API a retry is a purchase. Upstream failures (5xx) are the only class where backoff-and-retry is correct. Wrap the whole pipeline in idempotent jobs, a local cache, a circuit breaker for scheduled runs, and an alert that fires on spend anomalies rather than uptime alone.

This guide walks the full taxonomy with working Python you can drop into any competitor ad monitoring stack. The examples target the AdLibrary API, where the billing mechanics are public and concrete, but every pattern transfers to any metered data API.

Why Ad API Errors Are Money-Shaped

A standard web API fails in time-shaped ways. A request errors, you retry, the user waits 400ms longer. The worst case is latency.

Metered ad data APIs fail in money-shaped ways. On the AdLibrary API, a search costs 1 credit and an AI creative analysis costs 1 credit. The pricing is honest and predictable, which is exactly why your error handling has to be deliberate. Three failure shapes matter:

1. The retry that spends. A 402 means your balance can't cover the call. Retrying it on a timer doesn't fix anything, and the moment credits are topped up, your zombie retry loop fires 40 queued searches you no longer want.
2. The retry that double-spends. A timeout on your side doesn't always mean the request failed on theirs. If the search completed and you re-send it, you pay twice for identical data.
3. The silent burn. A cron job misconfigured to run every 10 minutes instead of nightly produces no errors at all. It just spends 144 credits a day on identical queries. Your ad spend dashboards won't catch it, because this is data spend, and almost nobody monitors data spend.

Engineers who internalize this stop treating API error handling as defensive boilerplate. It becomes the code that decides whether your ad intelligence pipeline costs €329 a month or €329 a week. The ad budget planner logic applies to data budgets too: fixed allowance, projected consumption, alert threshold.

API Error Handling Best Practices: A Taxonomy for Ad Data Pipelines

Every documented failure mode of the AdLibrary API falls into one of five classes, and each class demands a different reflex. The public docs list the status codes, and the taxonomy below adds the operational decision.

The most valuable line of code in your integration is the branch that routes each status to its reflex. Most production incidents in ad data pipelines trace back to a catch-all except block that treats a 402 like a 500. The taxonomy says they're opposites. One is free to retry because the credit was refunded. The other charges you for every attempt the moment the account can pay again.

A useful mental model for ad API error handling: 4xx errors are your state being wrong (key, balance, parameters, pacing), and no amount of repetition fixes your own state. 5xx errors are their state being wrong, and repetition is precisely the cure. Memorize that split and half of the API error handling best practices in this guide become mechanical.

Auth Errors: Fail Fast, Page a Human

A 401 from the AdLibrary API means the Bearer token is missing or invalid. Keys use the adl_ prefix, are shown once at creation, and you can hold up to 10 per account. That last detail is the real error-prevention feature. Scope one key per environment and per client, the way the docs recommend, and a leaked key takes down one workflow instead of your whole book of business.

The handling rule is absolute: a 401 never resolves on retry. The key was valid 10 seconds ago and it isn't now, which means a human rotated it, deactivated it, or a deploy shipped the wrong environment variable. Your code should do three things in order:

• Stop the run immediately. Fifty more 401s on the remaining keywords teach you nothing.
• Page a person with the key prefix (never the full key) and the environment name.
• Mark downstream jobs as blocked rather than failed, so your scheduler doesn't auto-restart into the same wall.

This mirrors how Meta's Graph API documentation separates throttling from authorization failures: one is pacing, the other is a configuration event no client-side waiting can repair. Meta's free Ad Library API adds its own auth wrinkle, an access token that expires every 60 days, one of the reasons teams hit walls with it in scheduled pipelines. A bearer key without timed expiry removes a whole error class, but only if you treat the remaining 401s as alarms instead of noise.

Rate Limits: Honor Retry-After or Pay Twice

The AdLibrary API allows 10 requests per minute and 10,000 per day per key, and a 429 carries a Retry-After header stating how long to back off. Those are the only rate-limit numbers you need. A nightly sweep of 30 competitors across all platforms in one call fits in three minutes of paced requests.

Handling 429 rate limit errors comes down to one discipline: read the header, sleep exactly that long, retry once. The Retry-After semantics come from RFC 6585, which defined status 429 precisely so servers could tell clients how long to wait instead of leaving them to guess. A client that guesses anyway, with a fixed 5-second sleep or an immediate hammer, burns daily quota on rejected requests and looks like abuse to the provider.

import time
import requests

def call_with_pacing(url, payload, api_key):
    r = requests.post(
        url,
        headers={"Authorization": f"Bearer {api_key}"},
        json=payload,
        timeout=30,
    )
    if r.status_code == 429:
        wait = int(r.headers.get("Retry-After", "60"))
        time.sleep(wait)
        r = requests.post(
            url,
            headers={"Authorization": f"Bearer {api_key}"},
            json=payload,
            timeout=30,
        )
    return r

Two refinements separate hobby scripts from production automation. First, pace proactively. If the ceiling is 10 per minute, a 6.5-second sleep between calls means you almost never see a 429 at all. Reactive handling is the seatbelt, proactive pacing is the driving. Second, respect the daily quota in your scheduler design. A job that wants 12,000 searches doesn't need a cleverer retry strategy, it needs to be split across two days or two scoped keys. The Python cookbook approach covers paced loops in depth if you're building from scratch.

Credit Errors: The Class You Never Blind-Retry

This section justifies the article. A 402 from the AdLibrary API means the call needs more credits than your balance holds, and the response body includes both numbers. Searches cost 1 credit each, AI enrichment costs 1 per analysis, and plans carry a monthly allowance, 1,000+ on Business. A 402 is never a mystery. It's an accounting statement.

Here is the trap. To a generic retry library, a 402 looks like any other non-200. Wrap your calls in retry(max_attempts=5, backoff=exponential) and the library dutifully re-sends a call that cannot succeed, five times. That's merely wasteful. The dangerous version is the queued retry: your job runner parks failed calls and replays them when the API "recovers." Credit exhaustion isn't an outage, so it never recovers the way the queue expects. Instead, your balance resets at the next billing cycle, every parked call instantly succeeds, and you wake up to a fresh month's allowance partially spent on three-week-old searches you no longer need. The retry loop didn't crash. It shopped.

The correct 402 reflex has three parts:

1. Halt, don't loop. Treat 402 as terminal for the entire run, the same severity as a 401. Every subsequent call in the batch will also fail, and each blocked call you don't send is a credit preserved for whenever you top up.
2. Read the body before deciding anything. The response tells you how many credits the call needed and what your balance is. A balance of 0 two days before reset is a pacing problem. A balance of 0 on day 3 of the cycle is a runaway job, and the fix is in your code, never in your wallet.
3. Re-enter manually, never automatically. After a top-up or cycle reset, a human (or an explicitly approved workflow step) should restart the pipeline against a reviewed queue. Auto-resume is how stale searches spend new budget.

Prevention beats all of it. The API exposes a free /api/credits balance check, so your pipeline can ask "can I afford this run?" before spending anything. A nightly job expecting to use 40 credits should verify the balance covers 40 plus your reserve floor, and skip the run with an alert if it doesn't. That free call converts credit exhaustion from a mid-run failure into a pre-run decision, and pre-run decisions never leave you with half a dataset. Budgeting data calls follows the same spend pacing logic media buyers apply to campaigns: fixed allowance, daily burn rate, alarm threshold well above zero.

Upstream Failures: Where Retries Actually Belong

A 500 from an ad library API usually means a source platform misbehaved mid-search. The provider aggregates Facebook, Instagram, TikTok, Google, LinkedIn, and more behind one endpoint, and occasionally an upstream hiccups. This is the one class where textbook retry advice applies without caveats, and on the AdLibrary API it comes with an unusual safety property: a failed search refunds its credit automatically. A 500 never costs you anything, so retrying it risks time, never money.

The right retry strategy is exponential backoff with jitter. Back off to give the upstream time to recover, add randomness so parallel workers don't synchronize into waves of simultaneous retries. AWS documented the failure mode in its Builders' Library piece on timeouts, retries, and backoff: naive synchronized retries turn a brief blip into a self-inflicted load spike. The full API error handling router looks like this:

import random
import time
import requests

TERMINAL = {400, 401, 402, 403}

class TerminalApiError(Exception):
    pass

def adl_search(payload, api_key, max_attempts=4):
    for attempt in range(max_attempts):
        r = requests.post(
            "https://adlibrary.com/api/search",
            headers={"Authorization": f"Bearer {api_key}"},
            json=payload,
            timeout=30,
        )
        if r.status_code == 200:
            return r.json()
        if r.status_code in TERMINAL:
            # 401 key, 402 credits, 400 bad params: retrying spends or stalls
            raise TerminalApiError(r.status_code, r.text)
        if r.status_code == 429:
            time.sleep(int(r.headers.get("Retry-After", "60")))
            continue
        # 5xx: upstream fault, credit auto-refunded, backoff + jitter
        time.sleep(min(2 ** attempt, 60) + random.uniform(0, 1))
    raise TerminalApiError("max_attempts", payload)

Sixteen lines of logic, and every one encodes a billing decision. The TERMINAL set is the budget firewall, the Retry-After read is quota citizenship, the capped backoff is upstream courtesy. If your stack runs on Node, the Node.js ad intelligence service guide shows the equivalent wrapper.

One subtlety deserves its own paragraph: timeouts. When your 30-second client timeout fires, you don't know whether the search completed server-side. Unlike a clean 500, a timeout is ambiguous, and re-sending might mean paying twice for the same page. That ambiguity is why the next section exists.

Idempotent Job Design Makes Re-Runs Safe

Idempotency means running the same job twice produces the same result, with no double effects. Payment APIs solved this years ago. Stripe's idempotent requests let a client re-send a charge after a network failure without billing the customer twice. The lesson transfers directly to ad data pipelines, where the "charge" is a credit and the "double effect" is paying again for ads you already hold.

You can't attach an idempotency key to a search call, so you build idempotency into the job around it:

• Make jobs resumable, not restartable. Record progress per unit of work: this keyword, this page, done, stored. When a run dies at page 7 of 12, the re-run starts at page 8. A job that restarts from scratch pays for pages 1 through 7 twice.
• Dedupe on ad_key at write time. Every ad carries a stable ad_key identifier, and the docs recommend treating it as the canonical handle. Make it the primary key in your competitor ad database and an accidental double-fetch becomes a no-op upsert instead of duplicate rows poisoning downstream analysis.
• Name runs deterministically. A run keyed 2026-06-14:nike:facebook can check whether it already completed before spending anything. A run keyed by random UUID can't. This habit makes cron retries, GitHub Actions re-runs, and manual "just run it again" all safe by default.
• Separate fetch from process. Store raw responses first, transform later. When the transform has a bug, you re-process from disk for zero credits instead of re-fetching everything because your only copy was the transformed one.

The payoff compounds. Once jobs are idempotent, your retry policy can afford aggression on the 5xx class, because the worst case of an over-eager retry is a wasted second rather than a duplicated spend. Idempotent job design is what makes the rest of your API error handling safe to use.

Caching: The Cheapest of the API Error Handling Best Practices

The cheapest API call is the one you never make, and the most reliable one too. A request served from your own store can't 429, can't 402, and can't time out. Caching usually gets filed under performance, but of all API error handling best practices it prevents the most failures per line of code, since every avoided call is one less opportunity for a money-shaped error.

Three caching rules cover most ad intelligence workloads:

1. Cache by query signature. Hash the full parameter set (keyword, platform, geo, daysBack, sort, page) and store the response against it with a TTL matched to your research cadence. Competitor creatives don't change hourly. For most competitive intelligence workflows a 24-hour TTL is conservative, and it means a re-run, a colleague's duplicate query, and a flaky orchestrator all hit your store instead of your balance.
2. Cache the expensive derivatives forever. An AI enrichment brief describes a specific creative that will never change after the fact. Store it permanently against the ad_key. Re-enriching an ad you already analyzed is the purest waste available in this category.
3. Let total guide pagination. The search response returns total alongside results, so your code knows after page 1 exactly how many pages exist. Fetch what your analysis needs, never "all pages, to be safe." A keyword with 5,000 matches rarely deserves 50 paid pages when the first 3 sorted by -impression answer the question.

Notice what this does to your failure budget. A pipeline with a 70% cache hit rate exposes only 30% of its workload to rate limits and credit charges. The nightly sweep that flirted with the daily quota now runs in a fraction of the window. To price the uncached version of your workload, the ad spend estimator mindset applies: model the worst week, then build the cache that makes the worst week impossible.

Circuit Breakers for Scheduled Jobs

A retry handles one failed call. A circuit breaker handles a failing system. Most roundups of API error handling best practices stop at the first, and scheduled jobs need the second. Consider a nightly n8n monitoring workflow sweeping 40 keywords. If the first 5 all return 401, the remaining 35 will too. Without a breaker, the job grinds through all 40, fails completely, and repeats the performance tomorrow night, and the night after, until somebody intervenes.

The pattern, described canonically by Martin Fowler, wraps calls in a state machine. Closed means traffic flows normally. After a threshold of consecutive failures the breaker opens and every later call fails instantly without hitting the API. After a cooldown, one half-open probe tests whether the fault cleared before full traffic resumes.

For scheduled ad data jobs, tune the breaker to the taxonomy:

• Trip immediately on 401 and 402. One terminal error predicts every later call in the run. Open at the first occurrence, skip the rest of the batch, alert once. Thirty-nine suppressed doomed calls per night is the breaker paying rent.
• Trip on repeated 5xx, around 5 consecutive. A single blip deserves a backoff retry. Five in a row means the upstream is down, and the half-open probe on the next scheduled run reports when it's back.
• Never trip on 429. Rate limiting is the API pacing you, already handled by Retry-After. A breaker that opens on 429 turns normal throttling into false outages.

The breaker state itself becomes a monitoring signal. "Circuit opened at 02:14 on 402" reaching your Slack alert channel is a more actionable message than 40 stack traces, and it arrives with the run mostly unspent. Visual builders deserve the same treatment: a Make.com scenario can implement a poor man's breaker with a data store flag checked at the start of every run.

Monitoring and Alerting on Spend Anomalies

Standard observability watches error rates and latency. Metered pipelines need a third pillar: spend. The failure that hurts most in practice produces no errors at all, just a credit balance emptying twice as fast as forecast. Treat the balance as a first-class metric with the same seriousness as uptime.

The instrumentation is nearly free because the API hands it to you. Every search response includes a _credits object with used and remaining, so your wrapper can log balance trajectory on every call, and the free /api/credits endpoint covers scheduled snapshots between runs. From that one time series, four alerts cover effectively every spend incident:

1. Burn-rate anomaly. Yesterday consumed 38 credits, today consumed 96 by noon. Page someone. This catches runaway loops, mis-scheduled crons, and a teammate's experimental script, the failures nothing else surfaces.
2. Forecast breach. Balance divided by average daily burn predicts your empty date. Alert when that date lands before the cycle reset, while the fix is still "pace down" rather than "top up."
3. Reserve floor. Alert at a fixed threshold, say 15% of the monthly allowance, held for urgent ad-hoc research. Hitting the floor mid-cycle pauses scheduled jobs while humans decide what's worth the remainder.
4. Refund mismatch. Count 500 responses and verify the refunds appear in your balance trajectory. Discrepancies are rare, and catching one early is a support ticket instead of a month-end surprise.

Route these to wherever your team already looks. The end-to-end monitoring guide covers wiring alerts into the research pipeline itself, and if an AI agent operates your key autonomously, give it the burn-rate alert as a hard stop condition rather than a notification. An agent that checks its own balance before each task and halts on anomaly is strictly safer than one that merely logs.

Putting It Together: Budget Guardrails First

There's a build order hiding in all of this. Teams that harden ad data pipelines successfully follow API error handling best practices in guardrails-first order: the status-code router and TERMINAL set on day one, balance checks and burn alerts in week one, idempotent job keys and the cache before scaling past a handful of competitors, and the circuit breaker once jobs run unattended on a schedule. Features come after, because features built on an unguarded pipeline inherit its failure modes and multiply its spend.

The economics justify the order. On the AdLibrary API's Business plan (€329/month, 1,000+ credits, full API access with integration help included), a disciplined pipeline turns the allowance into 30+ monitored competitors with nightly sweeps, enrichment on the winners, and headroom for ad-hoc questions. An undisciplined one burns the same allowance on retries, re-fetches, and zombie crons while delivering a third of the insight. Same plan, same API, radically different cost per answer. The difference is entirely in the error handling, which is the cheap part to get right. If you're weighing the build, the free vs paid API comparison breaks down the cost math, and the Business plan is the tier with API access when you're ready to automate.

Frequently Asked Questions

Should I ever automatically retry a 402 credit error?

No. A 402 means the call needs more credits than your balance holds, and repetition can't change that. Worse, queued retries fire the moment credits return, spending fresh budget on stale requests. Treat 402 as terminal, halt the run, alert a human, and resume manually against a reviewed queue after topping up.

How do I handle 429 rate limit errors on an ad library API?

Read the Retry-After header, sleep exactly that many seconds, then retry once. The AdLibrary API allows 10 requests per minute and 10,000 per day per key, so pacing calls about 6.5 seconds apart prevents most 429s entirely. Never retry on a guessed fixed delay, and never let parallel workers retry simultaneously without jitter.

Is it safe to retry 500 errors on paid API calls?

Yes, and on the AdLibrary API it's explicitly safe for your budget, because a failed search refunds its credit automatically. Use exponential backoff with jitter, cap attempts at around four, and route only 5xx codes to this path. The codes that involve your own state (400, 401, 402) must never reach the retry branch.

What's the cheapest way to prevent ad API errors?

Caching. A response served from your own store can't hit a rate limit or spend a credit. Cache search responses by full query signature with a TTL matched to your research cadence, store AI enrichment results permanently against the ad_key, and use the total field to fetch only the pages your analysis needs.

How do I monitor API credit spend automatically?

Log the _credits.remaining value returned on every search response, and snapshot the free /api/credits endpoint between runs. Alert on burn-rate anomalies (today's consumption far above the daily average), forecast breaches (projected empty date before cycle reset), and a reserve floor around 15% of the monthly allowance.