How to benchmark a proxy provider properly

Every provider publishes a success rate above 99%, ours included. That number measures whether the network delivered a response, which is a fact about our infrastructure and not about whether you got the data you wanted. Benchmark the thing you actually care about.

Measure cost per successful page

One number, and it captures everything: block rate, retry cost, bandwidth efficiency and unit price at once. A provider that is 40% cheaper per gigabyte but blocked twice as often is more expensive on this metric, which is the only one that shows up in your bill.

import time, requests

def benchmark(proxy, urls, price_per_gb):
    ok = bytes_used = 0
    latencies = []

    for url in urls:
        t0 = time.perf_counter()
        try:
            r = requests.get(url, proxies={"https": proxy}, timeout=30)
            latencies.append(time.perf_counter() - t0)
            bytes_used += len(r.content) + 800      # rough header overhead
            if r.status_code == 200 and looks_valid(r.text):
                ok += 1
        except requests.RequestException:
            latencies.append(time.perf_counter() - t0)

    gb = bytes_used / 1e9
    return {
        "success_rate": ok / len(urls),
        "p50_ms": round(sorted(latencies)[len(latencies) // 2] * 1000),
        "p95_ms": round(sorted(latencies)[int(len(latencies) * 0.95)] * 1000),
        "cost_per_1k_ok": round(gb * price_per_gb / max(ok, 1) * 1000, 2),
    }

Use percentiles, not averages

Residential latency has a long tail because some peers are on genuinely slow connections. A mean of 700 ms can hide a p95 of four seconds, and it is the p95 that determines your crawl's throughput once you have set timeouts.

Test what you will actually run

• Your real targets, not example.com or a speed-test endpoint.
• Your real geographies. US performance tells you nothing about Indonesia.
• Your real concurrency. Pools behave differently at 5 and at 500 parallel connections.
• Over at least 24 hours. Residential pool composition follows the sun; a benchmark run at 3am UTC samples a different network from one run at noon.

A benchmark worth keeping

Run it on a schedule against production traffic, not once during evaluation. Provider quality drifts, targets change their defences, and the pool that was ideal in March may not be in September. Teams that measure continuously catch that in a week; teams that measured once catch it in a quarterly bill review.