Enrich 10,000 companies without paying twice

Big enrichment jobs time out, hit rate limits and get retried. How to retry safely, what a failed call costs, and what 10,000 lookups cost.

The HappyFetch team · October 12, 2026

The trouble with retries#

A job that looks up 10,000 companies will meet timeouts, rate limits and the odd failure. The fix is to retry. The risk is paying twice for one answer: the first call worked, the response got lost on its way back, and the retry runs the lookup again.

One key per company, reused on every retry#

Give each company a retry key. Make it once, before the first try, and send the same key with every retry of that company. The code below shows where it goes.

• If the first call went through, the retry gets the stored answer back and isn't charged again.
• If the first call failed on our side with an error you can retry, it was refunded, and the retry runs again for real. Other errors come back exactly as they were the first time, for 24 hours.

import os
import time
import uuid

import requests

API = "https://api.happyfetch.net"
HEADERS = {"Authorization": f"Bearer {os.environ['HAPPYFETCH_API_KEY']}"}


def lookup(domain: str) -> dict:
    key = str(uuid.uuid4())  # once per company, reused on every retry
    for attempt in range(6):
        try:
            res = requests.post(
                f"{API}/v1/domain",
                headers={**HEADERS, "Idempotency-Key": key},
                json={"domain": domain},
                timeout=120,
            )
            body = res.json()
        except (requests.RequestException, ValueError):
            time.sleep(2**attempt)  # a timeout or a dropped connection: try again, same key
            continue
        if res.ok:
            return body["data"]
        error = body["error"]
        if not error["retryable"]:
            return {"error": error["code"]}
        time.sleep(error.get("retry_after_seconds") or 2**attempt)
    return {"error": "gave_up"}

Let the error tell you what to do#

Every error says whether retrying helps, and a rate limit also says how long to wait. Everything else is an answer to keep, not to retry: there's no company at the domain, the value isn't a website, or your balance ran out.

What a failed call costs#

Nothing. A call we can't complete is refunded in the same request, so the error already shows that nothing was charged. Rejected input and rate limits are never charged either.

What 10,000 domain lookups cost#

Every lookup fetched fresh, at 4 credits each. Lookups our cache answers cost half. A confirmed not-found costs the same, and nothing from our cache. Starter: $10 for 20,000 credits. Business: $500 for 2,000,000 credits. Credits never expire.

Run the job again, accepting copies that are a little older, and the companies we already hold come from our cache at half the price. If your list has LinkedIn company URLs instead of domains, look them up by URL: that's half the price of a domain lookup.

The whole script#

Our guide has the complete script, in Python and Node. It reads a CSV of domains, runs eight lookups at a time with these retries, and writes size, industry, headquarters and the company's LinkedIn page back out.

More on this: Enrich a CSV of domains with company data

The HappyFetch team: We build HappyFetch, a company data API.