HTTP Client & Proxy Pools

How it works

Client lifecycle. ClientRegistry.GetOrCreateClient (clients/registry.go:L47-L53) uses a sync.Map keyed on common.UniqueUpstreamKey(ups). The first call creates the client; all subsequent calls for the same upstream return the cached instance. Creation is guarded by sync.Once. The registry also holds a reference to a ProxyPoolRegistry, resolving the proxy pool by ID at creation time.

Scheme dispatch. CreateClient inspects parsedUrl.Scheme. http/https → NewGenericHttpJsonRpcClient. grpc/grpc+bds → GrpcBdsClient. ws/wss → error (not yet implemented). Any other scheme → error (clients/registry.go:L84-L117).

Transport construction. Each GenericHttpJsonRpcClient owns its own http.Transport:

• DialContext via util.DefaultOutboundDialer(): net.Dialer{Timeout: 10s, KeepAlive: 15s}. The 15-second probe interval means wedged TCP flows die in ~45 seconds (3 probes), versus the Linux kernel default of ~7200 seconds.
• MaxConnsPerHost: 0 (unlimited) is intentional. The prior default of 2 capped throughput at ~5 req/s per host with 400ms upstream latency; removing the cap makes the OS TCP stack the only limit. Validated in clients/http_connection_test.go.
• Overall http.Client.Timeout is 60 seconds (includes body read).

Request preparation. Every outbound call gets: Content-Type: application/json, Accept-Encoding: gzip, User-Agent: erpc (<version>/<sha>; Project/<projectId>). Custom headers from jsonRpc.headers are applied via Header.Set after those defaults — they can override Content-Type or User-Agent. OTel W3C traceparent/baggage injection runs last and cannot be overridden.

Gzip. The client always sends Accept-Encoding: gzip and decompresses any gzip response regardless of enableGzip. When enableGzip: true, request bodies are also compressed using pooled gzip.Writer instances. Both reader and writer are pooled via sync.Pool to avoid allocation pressure.

Forwarded headers. Inbound headers matching project.forwardHeaders patterns are forwarded to the upstream only on single (non-batch) requests — the batch path aggregates multiple caller contexts into one outbound call.

Batch aggregation. When supportsBatch: true, SendRequest queues calls into a map keyed by JSON-RPC ID. A time.AfterFunc(batchMaxWait, ...) fires processBatch when the timer elapses; the batch also fires immediately when the queue reaches batchMaxSize. The batch context deadline is set to the earliest deadline among queued requests. Responses are matched back to callers by JSON-RPC id. A single-object response (e.g. a rate-limit error with "id": null) is broadcast to all queued callers. Duplicate IDs within a window flush the pending batch immediately and start fresh.

Proxy pool. createProxyPool builds one http.Client per proxy URL, with identical transport settings to the direct path plus Proxy: http.ProxyURL(proxyURL). On every outbound call, GetClient() selects the next client via atomic.AddUint64(&counter, 1) % len(clients) — a lock-free round-robin. Counter wraps naturally at uint64 overflow; it is never reset. On error (empty pool), the client falls back to the direct transport and logs an error.

Proxy URL validation uses a case-insensitive prefix check, not url.Parse: only http://, https://, and socks5:// are accepted. socks4://, ws://, and any other scheme fail at startup.

Shutdown. A goroutine watches appCtx.Done() and drains the in-flight batch. Already-canceled contexts in the queue are dropped silently.

Test mode. When util.IsTest() is true, the client substitutes http.DefaultTransport to allow gock mock interceptors.

Config schema

proxyPools[] — top-level array (common/config.go:L48 (opens in a new tab))

upstreams[].jsonRpc.* (also upstreamDefaults.jsonRpc.*)

All fields are optional. JsonRpcUpstreamConfig.SetDefaults() is a no-op; pointer-zero fields retain Go zero values. Important: when an upstream has no jsonRpc: block at all, the full upstreamDefaults.jsonRpc struct is shallow-copied. But if any jsonRpc: block is present (even jsonRpc: {}), field-level defaults are not merged — inheritance is all-or-nothing.

Fixed transport parameters (not user-configurable)

Same values apply to both direct and proxy transports (source: clients/http_json_rpc_client.go:L109-128, clients/proxy_pool_registry.go:L82-98).

Worked examples

1. SOCKS5 proxy rotation for IP-diverse egress. Multiple providers rate-limit by source IP. Routing requests through a pool of residential or datacenter proxies distributes the source IP fingerprint and avoids per-IP throttles:

proxyPools:  - id: egress-proxies    urls:      - socks5://proxy1.example.com:1080      - socks5://proxy2.example.com:1080      - socks5://proxy3.example.com:1080
projects:  - id: main    upstreams:      - endpoint: https://mainnet.infura.io/v3/KEY        jsonRpc:          proxyPool: egress-proxies

2. Batch aggregation for high-QPS read workloads. An indexer fires hundreds of eth_getTransactionReceipt calls per second. Batching coalesces them into fewer HTTP round-trips, reducing latency and upstream connection overhead. Set batchMaxSize and batchMaxWait together so batches fill before the timer fires:

upstreams:  - endpoint: https://mainnet.infura.io/v3/KEY    jsonRpc:      supportsBatch: true      batchMaxSize: 50      batchMaxWait: 10ms

3. Gzip + custom API-key header. Enable request compression and inject a bearer token on every upstream call. The token is sent via Header.Set after the built-in defaults, so it cannot accidentally stomp traceparent:

upstreams:  - endpoint: https://rpc.example.com    jsonRpc:      enableGzip: true      headers:        Authorization: "Bearer my-secret-key"

4. Shared defaults across all upstreams. Use upstreamDefaults.jsonRpc to apply batch + gzip to every upstream without repeating config. Note: any upstream that defines its own jsonRpc: block (even empty) does not inherit from upstreamDefaults.jsonRpc:

projects:  - id: main    upstreamDefaults:      jsonRpc:        supportsBatch: true        batchMaxSize: 20        batchMaxWait: 5ms        enableGzip: true    upstreams:      - endpoint: https://rpc-a.example.com      - endpoint: https://rpc-b.example.com

Request/response behavior

Outbound HTTP request shape (single):

POST <upstream endpoint URL>
Content-Type: application/json
Accept-Encoding: gzip
User-Agent: erpc (<ErpcVersion>/<ErpcCommitSha>; Project/<projectId>)
[Content-Encoding: gzip]        — only when enableGzip=true
[<custom headers>]              — from jsonRpc.headers, via Header.Set
[traceparent: ...]              — OTel W3C trace context (injected last)
[baggage: ...]
[<forwarded headers>]           — only on single requests, from project.forwardHeaders

Body: {"jsonrpc":"2.0","method":"...","params":[...],"id":...}

Outbound HTTP request shape (batch): Same headers except forwarded headers (absent). Body is a JSON array of JsonRpcRequest objects.

Batch response matching:

• Array response: each element's id is looked up in the queued request map; matched response returned to caller. Unmatched IDs produce "no response received for request ID: N" (logged as ERROR with full body).
• Single-object response: broadcast to ALL callers. Handles rate-limit errors returned as {"error":{...},"id":null} for the whole batch.
• Non-JSON response: parsed as JSON-RPC error and broadcast; if that fails, all callers receive the parse error.

Error type mapping:

Behavioral invariants:

• ClientRegistry.GetOrCreateClient uses sync.Map + sync.Once; all callers for the same upstream share exactly one client instance. [clients/registry.go:L47-125]
• OTel traceparent + baggage are injected on every request, including retried and batched calls. [clients/http_json_rpc_client.go:L845-847]
• Both single and batch paths decompress gzip responses whenever Content-Encoding: gzip is present. [clients/http_json_rpc_client.go:L852-867]
• At shutdown, appCtx.Done() triggers processBatch(true) to drain in-flight batches; already-canceled contexts in the queue are dropped silently. [clients/http_json_rpc_client.go:L211-217]
• Pre-queue cancellation check. queueRequest inspects req.ctx.Err() before inserting into the batch map. Already-canceled requests are immediately failed with ErrEndpointRequestTimeout or ErrEndpointRequestCanceled without touching the network. [clients/http_json_rpc_client.go:L241-253]
• Post-queue pre-send cancellation drop. Inside processBatch, a pass over the queued requests drops any entry whose ctx.Err() is non-nil before the HTTP call is made. [clients/http_json_rpc_client.go:L349-361]

Best practices

• Set both batchMaxSize and batchMaxWait together. Either alone produces degenerate behavior: batchMaxSize: 0 fires on every first request (size check 1 >= 0); batchMaxWait: 0 fires before other requests can join. Use at minimum batchMaxSize: 10 + batchMaxWait: "10ms" for meaningful coalescing.
• Do not use supportsBatch: true for upstreams receiving forwarded Authorization headers. Forwarded headers are applied only on single requests; batch requests share one outbound call and carry no per-caller headers.
• Prefer upstreamDefaults.jsonRpc to avoid repetition — but know it is all-or-nothing. Any upstream with an explicit jsonRpc: block (even jsonRpc: {}) does not inherit defaults field-by-field. Either use defaults everywhere or configure each upstream fully.
• enableGzip: true is worth enabling for most RPC endpoints. JSON-RPC responses (especially eth_getLogs, debug_traceTransaction) compress well. The reader and writer are pooled, so CPU overhead is minimal.
• Add at least 3 proxy URLs per pool. With a single-URL pool the round-robin is meaningless (always index 0); with 2 the distribution is uneven under any non-even request count. Three or more gives meaningful rotation.
• Proxy scheme validation is strict at startup. Test proxy URLs locally before deploying — socks4:// and other schemes will prevent eRPC from starting with no graceful fallback.
• There is no per-upstream TLS configuration. Custom CA bundles, client certs, and InsecureSkipVerify are not supported. If an upstream requires mutual TLS or a private CA, the connection must be proxied through a sidecar or gateway that handles TLS termination.

Edge cases & gotchas

1. batchMaxSize: 0 with supportsBatch: true fires instantly. The request is added to the queue before the size check (len >= batchMaxSize), so the check evaluates as 1 >= 0 (always true). Use batchMaxSize: 2 or higher. Source: clients/http_json_rpc_client.go:L294
2. batchMaxWait: 0 (default) fires instantly. time.AfterFunc(0, fn) schedules in a new goroutine immediately. Set a non-zero duration to enable coalescing. Source: clients/http_json_rpc_client.go:L291
3. Duplicate JSON-RPC IDs within a batch window flush the pending batch. The in-flight batch fires immediately and the duplicate starts a fresh batch. Clients that reuse IDs will see reduced batch efficiency. Source: clients/http_json_rpc_client.go:L255-261
4. Custom headers overwrite built-in defaults. Header.Set replaces any existing value. A headers: {Content-Type: text/plain} will overwrite application/json. OTel headers are injected after and cannot be overridden. Source: clients/http_json_rpc_client.go:L838-847
5. Forwarded headers are absent on batch requests. Callers relying on Authorization or custom headers forwarded to upstreams must not enable supportsBatch for those upstreams. Source: clients/http_json_rpc_client.go:L736-742
6. upstreamDefaults.jsonRpc is all-or-nothing. If an upstream has any jsonRpc: block (even empty jsonRpc: {}), defaults are not merged field-by-field. Source: common/defaults.go:L1550-1558
7. Proxy URL scheme check is a strings.HasPrefix after strings.ToLower, not url.Parse. socks4://, ws://, ftp://, and other schemes fail at startup regardless of syntactic validity. Only http://, https://, socks5:// are accepted. Source: common/validation.go:L238-244
8. Empty proxy pool falls back to direct transport. If GetClient returns an error (pool created with zero URLs), the client logs an error and uses the direct transport. Startup validation should prevent this in production. Source: clients/proxy_pool_registry.go:L69-71
9. MaxConnsPerHost: 0 is deliberate. Go's prior default of 2 capped throughput at ~5 req/s per host with 400ms latency. Unlimited removes the queue; the OS TCP stack becomes the only cap. Source: clients/http_connection_test.go:L141-414
10. No per-upstream TLS configuration. Custom CA bundles, client certificates, and InsecureSkipVerify are not supported for HTTP upstreams. Go uses the system CA bundle. Source: clients/http_json_rpc_client.go:L109-118
11. HTTP/2 is not explicitly enabled. ForceAttemptHTTP2 is not set. HTTP/2 may negotiate via ALPN on https:// upstreams that offer it, but this is unverified. Proxy transports are HTTP/1.1 tunnels. Source: clients/http_json_rpc_client.go:L109-118
12. Batch single-object response broadcasts to all callers. When an upstream returns {"jsonrpc":"2.0","error":{...},"id":null}, every pending request in the batch receives the same response — intended for upstream-wide rate-limit errors. Source: clients/http_json_rpc_client.go:L606-636
13. Partial batch response leaves unmatched callers with an error. If the array is shorter than queued requests, absent IDs receive "no response received for request ID: N", logged as ERROR with the full response body. Source: clients/http_json_rpc_client.go:L597-605
14. 5ms race-fix window for timeout sentinels in batch. When batchCtx expires with DeadlineExceeded, the per-request context Cause may lag by a few microseconds. The client waits up to 5ms for the cause to stabilize so ErrDynamicTimeoutExceeded sentinels are preserved for the upstream classifier. Source: clients/http_json_rpc_client.go:L471-479
15. ErrUpstreamMalformedResponse is retryable. Raised when a batch response body is neither a JSON array nor object. Not in the non-retryable allowlist — eRPC will try another upstream. If all upstreams return malformed responses, the request ends as ErrUpstreamsExhausted. Source: common/errors.go:L835-858
16. ErrEndpointServerSideException.originalStatusCode zero-value fallback is 500. When originalStatusCode == 0 (e.g. synthetic gRPC status), ErrorStatusCode() returns 500. A 500 from eRPC does not necessarily mean the upstream returned 500. Source: common/errors.go:L1974-1979
17. JsonRpcErrorExtractor is wired once at startup. The EVM extractor is injected at UpstreamsRegistry construction and cannot be changed at runtime. Non-EVM architectures require a code change to swap extractors. Source: upstream/registry.go:L79
18. ErrUpstreamMalformedResponse method-level 400 vs wire-level 200 discrepancy. ErrorStatusCode() returns 400, which appears in eRPC logs and metrics labels. However, the HTTP response to the calling client is sent with wire-level HTTP 200 (standard JSON-RPC transport), and the 400 appears only inside the JSON-RPC error envelope's metadata. The 400 is not visible in the HTTP response line seen by callers — only in logs and erpc_upstream_request_errors_total. Source: common/errors.go:L856-858
19. ErrEndpointServerSideException is retryable toward both upstream and network. The error code is not in the non-retryable allowlist in IsRetryableTowardsUpstream, so eRPC will attempt another upstream. A 500 fallback (when originalStatusCode == 0) does not mean the error is deterministic. Source: common/errors.go:L1951-1979
20. proxyPool is set twice when jsonRpcCfg != nil. NewGenericHttpJsonRpcClient assigns client.proxyPool = proxyPool in the struct literal and then again unconditionally inside the if jsonRpcCfg != nil block. The net effect is always the passed value — harmless but worth noting when reading the code. Source: clients/http_json_rpc_client.go:L91-148

Observability

The HTTP client layer emits no Prometheus metrics directly. All upstream-level metrics are recorded by upstream/upstream.go and health/tracker.go which wrap the client calls. Errors originating in the HTTP client surface under these metrics:

Trace spans. Single requests produce an HttpJsonRpcClient.sendSingleRequest span with network.id, upstream.id, and (at detailed tracing level) request.id and request.method attributes. Batch requests do not create per-call spans; they run under the propagated caller context.

Log messages (key entries):

• DEBUG "sending json rpc POST request (single|batch)" — host, raw request body, headers
• DEBUG "queuing request ... for batch (current batch size: N)"
• DEBUG "processing batch with N requests"
• TRACE "using client from proxy pool" — pool ID, transport pointer, resolved proxy URL
• TRACE "starting batch timer", "setting batch deadline to earliest request deadline", "committing batch to process total of N requests"
• TRACE response body (first 20 KiB + last 20 KiB when response body is oversized)
• ERROR "failed to get client from proxy pool" — when ProxyPool.GetClient returns error (empty pool)
• WARN "unexpected response received without ID" — batch response element has no id field
• WARN "unexpected response received with ID: <id>" — batch response ID not in pending request map
• ERROR "some requests did not receive a response (matching ID)" — with full response body

Source code entry points

• clients/http_json_rpc_client.go:L89-L148 (opens in a new tab) — NewGenericHttpJsonRpcClient: transport construction, field initialization, gzip and batch config
• clients/http_json_rpc_client.go:L786-L951 (opens in a new tab) — prepareRequest, sendSingleRequest, normalizeJsonRpcError: request shape, OTel injection, error classification
• clients/http_json_rpc_client.go:L241-L479 (opens in a new tab) — queueRequest, processBatch: batch aggregation, deadline propagation, context pre-checks
• clients/registry.go:L47-L125 (opens in a new tab) — ClientRegistry.GetOrCreateClient / CreateClient: sync.Map cache, scheme dispatch, proxy pool resolution
• clients/proxy_pool_registry.go:L23-L98 (opens in a new tab) — ProxyPool.GetClient (round-robin), createProxyPool, proxy transport construction
• util/http_dialer.go:L8-L24 (opens in a new tab) — DefaultOutboundDialer: net.Dialer{Timeout: 10s, KeepAlive: 15s}
• common/validation.go:L230-L247 (opens in a new tab) — ProxyPoolConfig.Validate: ID required, URLs non-empty, prefix scheme check
• common/error_extractor.go:L10-L21 (opens in a new tab) — JsonRpcErrorExtractor interface + JsonRpcErrorExtractorFunc adapter
• common/errors.go:L835-L858 (opens in a new tab) — ErrUpstreamMalformedResponse: meaning, ErrorStatusCode returns 400, retryable
• common/errors.go:L1951-L1979 (opens in a new tab) — ErrEndpointServerSideException: originalStatusCode field, 500 fallback
• clients/http_connection_test.go (opens in a new tab) — behavior-locking tests validating MaxConnsPerHost: 0 fix under load
• common/config.go:L1072-L1079 (opens in a new tab) — JsonRpcUpstreamConfig struct definition
• common/config.go:L1985-L1988 (opens in a new tab) — ProxyPoolConfig struct definition
• common/defaults.go:L1700-L1704 (opens in a new tab) — unconditional u.JsonRpc = &JsonRpcUpstreamConfig{} if nil before SetDefaults()

Related pages

• Retry — retry policy wrapping upstream calls; transport errors flow through here.
• Timeout — per-request timeout bounds that feed ErrDynamicTimeoutExceeded into the batch deadline path.
• Rate limiters — cap outbound RPS per upstream or proxy pool to stay within provider budgets.
• Upstreams — full upstream config including endpoint, jsonRpc, and upstreamDefaults.
• Selection policies — how eRPC chooses which upstream (and which proxy-pool-backed client) receives each request.