Server

How it works

Handler chain. NewHttpServer composes the stack innermost to outermost: createRequestHandler → optional gzipHandler (response compression) → custom TimeoutHandler (global deadline = maxTimeout) → optionally an h2c/gRPC mux on IPv4 when gRPC shares the HTTP port. Two independent http.Server instances handle IPv4 and IPv6; only those whose listenV4/listenV6 flag is true are created, and Start() fails if neither is set. gRPC sharing only ever applies to the IPv4 server. Source: erpc/http_server.go:L150-199

Timeout machinery. eRPC does NOT use net/http's built-in TimeoutHandler. Its own implementation buffers the entire response body in a pooled bytes.Buffer and stages headers privately; only when the inner handler finishes within the deadline does it flush to the real connection. On timeout: JSON-RPC -32603 body at HTTP 200 (POST) or 504 (other). On client cancel: "request cancelled by client" at HTTP 200 (POST) or 503 with empty body (other). Source: erpc/http_timeout.go:L20-143

Request and response gzip. enableGzip: true wraps the handler in a conditionalGzipWriter. The first write decides once: if the chunk is smaller than 1024 bytes, no compression is applied; otherwise Content-Length is deleted, Content-Encoding: gzip + Vary: Accept-Encoding are set, and subsequent writes flow through a pooled gzip.Writer. Inbound gzip bodies (Content-Encoding: gzip) are always accepted and decompressed using a pooled reader, regardless of enableGzip. Source: erpc/http_server.go:L1685-1780

gRPC port sharing. When grpcEnabled: true and the gRPC v4 host:port equal the HTTP v4 host:port (the default derivation), the IPv4 HTTP handler multiplexes: HTTP/2 + Content-Type: application/grpc → in-process gRPC server, bypassing TimeoutHandler and gzipHandler entirely. Without TLS the combined handler is wrapped in h2c to accept cleartext HTTP/2. Sharing never applies to IPv6. To run a standalone gRPC server on a different port, set grpcPortV4 to a different value. Source: erpc/grpc_server.go:L42-53

TLS and mTLS. When tls.enabled is true, both listeners use ListenAndServeTLS with TLS 1.2 as the minimum version, and gRPC uses TLS credentials. Setting caFile is the only knob needed to enable mTLS: it populates ClientCAs and sets ClientAuth = RequireAndVerifyClientCert, forcing every client to present a valid certificate. Source: erpc/http_server.go:L1537-1637

Trusted-proxy IP extraction. resolveRealClientIP trusts forwarding headers only when the direct peer is inside trustedIPForwarders (default: loopback only). It walks trustedIPHeaders in order, parses each value XFF-style, strips trailing trusted-proxy entries right-to-left, and returns the nearest untrusted hop. If every hop is trusted, it falls back to the direct peer IP. RFC 7239 Forwarded is not supported. Source: erpc/http_server.go:L1782-1905

Domain-based aliasing. server.aliasing.rules[] maps a request Host header to a pre-selected (project, architecture, chain) so callers can use a bare URL like https://eth.example.com. Rules are evaluated in order; first wildcard match wins. Not every combination of serveProject/serveArchitecture/serveChain is valid — see Edge cases. Source: erpc/http_server.go:L232-257

Graceful shutdown. Two goroutines watch the app context on SIGTERM: one flips a draining flag immediately so GET /healthcheck returns 503 "shutting down" (letting the LB drain traffic); after sleeping waitBeforeShutdown (default 10s), http.Server.Shutdown is called with a hardcoded 30s budget. After the context is done, Init sleeps an additional waitAfterShutdown (default 10s) before process exit, letting telemetry exporters flush. Source: erpc/http_server.go:L209-221

Config schema

All fields live under server. unless noted. Duration accepts Go duration strings ("30s") or bare integers interpreted as milliseconds ("150" = 150 ms, not 150 s). Struct at common/config.go:L130-200.

Hardcoded constants (non-configurable):

• IdleTimeout = 300s on both servers
• MaxHeaderBytes = 1 MiB — the only hard inbound size limit
• No request-body size limit — util.ReadAll copies until EOF with no upper bound; effective limits are readTimeout and process memory
• Graceful shutdown budget = 30s
• Response compression threshold = 1024 bytes on the first write
• TLS MinVersion = TLS 1.2

Worked examples

All patterns below are distilled from real production fleets; comments explain the non-obvious choices.

1. Public edge gateway behind an Anycast load balancer. Many PaaS proxies inject the real client IP into a platform header (e.g. Fly-Client-IP, CF-Connecting-IP); the RFC-1918 trusted-forwarder CIDRs cover the platform mesh. includeErrorDetails: false strips internal stack traces from public responses. waitBeforeShutdown: 30s covers a couple of LB health-check intervals so rolling deploys drain fully before shutdown:

server:  # platform machines typically sit in RFC-1918 space; trust private CIDRs  trustedIPForwarders:    - 10.0.0.0/8    - 172.16.0.0/12    - 192.168.0.0/16  # your platform injects the real client IP here — prefer it over spoofable X-Forwarded-For  trustedIPHeaders:    - CF-Connecting-IP  # strip internal error details from public 5xx responses  includeErrorDetails: false  # 30s covers a couple of LB health-check intervals before Shutdown is called  waitBeforeShutdown: 30s  # keep the process alive long enough for telemetry exporters to flush  waitAfterShutdown: 30s  responseHeaders:    # expose which region handled this request — useful for latency debugging    X-ERPC-Region: ${REGION}    X-ERPC-Machine: ${MACHINE_ID}

2. Internal Kubernetes deployment with gRPC shared on the HTTP port. Internal indexing fleets enable gRPC so indexer workers can use the gRPC connector without a separate endpoint. The default port derivation (grpcPortV4 = httpPortV4) activates the in-process h2c mux — no separate grpcPortV4 needed. Note that gRPC traffic on the shared port bypasses TimeoutHandler and gzip:

server:  # gRPC shares port 4000 via the h2c in-process mux  # (grpcPortV4 defaults to httpPortV4 — no explicit value needed)  grpcEnabled: true  # 30s matches two readiness-probe intervals in most K8s LB configs  waitBeforeShutdown: 30s  waitAfterShutdown: 30s

3. Standalone gRPC on a dedicated port (when TimeoutHandler must cover all traffic). When grpcPortV4 differs from httpPortV4, eRPC starts a fully independent gRPC server instead of muxing. HTTP requests flow through TimeoutHandler and gzip as normal; gRPC has its own listener. Use this when you need maxTimeout enforcement on HTTP while still serving gRPC:

server:  httpPortV4: 4000  grpcEnabled: true  # different port → standalone gRPC server, HTTP keeps its TimeoutHandler  grpcPortV4: 4010

4. Mutual TLS for a private RPC gateway. Adding caFile is the only field needed to flip mTLS on — it sets ClientAuth = RequireAndVerifyClientCert automatically. All clients must present a valid certificate signed by that CA. waitBeforeShutdown: 20s is two readiness-probe intervals for an NLB (10s probe interval × 2):

server:  httpPortV4: 4443  tls:    enabled: true    certFile: /etc/erpc/tls.crt    keyFile: /etc/erpc/tls.key    # setting caFile alone enables mTLS — no other field required    caFile: /etc/erpc/ca.crt  # NLB probe interval is typically 10s; 2× = full drain window  waitBeforeShutdown: 20s  waitAfterShutdown: 5s

5. Domain-based aliasing for a multi-chain gateway. Operators serving Ethereum mainnet and Arbitrum on separate subdomains can pre-route requests by Host header so callers use bare URLs like https://eth.example.com. Specific rules are placed before the wildcard catch-all — first match wins. Combining serveProject + serveChain without serveArchitecture is an error; all three must be present for fully resolved routing:

server:  aliasing:    rules:      # specific rules must come before the wildcard — first match wins      - matchDomain: "eth.example.com"        serveProject: main        serveArchitecture: evm        serveChain: "1"      - matchDomain: "arb.example.com"        serveProject: main        serveArchitecture: evm        serveChain: "42161"      # wildcard catch-all: project pre-selected, chain still from URL path      - matchDomain: "*.example.com"        serveProject: main

Request/response behavior

HTTP status mapping (determineResponseStatusCode and handleErrorResponse, both switch on the same error code set):

Note: writeFatalError (panics, write failures) and the timeout/cancel paths force HTTP 200 for POST regardless of error type. Source: erpc/http_server.go:L1280-1317

Error response shapes:

• Standard JSON-RPC single error: {"jsonrpc":"<ver>","id":<id>,"error":{"code":<int>,"message":"<msg>"[,"data":<any>]}} — includeErrorDetails adds error.data except on eth_call
• Timeout body: {"jsonrpc":"2.0","id":null,"error":{"code":-32603,"message":"http request handling timeout"}} at HTTP 200 (POST) / 504 (other) [erpc/http_timeout.go:L107-122]
• Client-cancel body: {"jsonrpc":"2.0","id":null,"error":{"code":-32603,"message":"request cancelled by client"}} at HTTP 200 (POST) / 503 (other)
• Fatal-path body (panic/write failure): {"jsonrpc":"2.0","error":{"code":-32603,"message":...}} — no id field
• Batch responses are always HTTP 200; each element is written as a streaming JSON array. They carry ONE aggregated X-ERPC-* diagnostic header set (counters summed, X-ERPC-Cache gains PARTIAL:<n>, slowest-sub-call duration, trace capped at 50 segments; no single-winner X-ERPC-Upstream) under the same executionHeaders mode — see directives. Source: erpc/http_server.go:L706-712

Diagnostic response headers (per-response set on singles via setResponseHeaders; aggregated set on batches via writeBatchExecHeaders):

• X-ERPC-Version, X-ERPC-Commit — build-identity headers (ldflags vars, defaults "dev" and "none"); emitted unless executionHeaders: "off". erpc/http_server.go:L275-278
• X-ERPC-Attempts, X-ERPC-Upstream-Attempts/-Retries/-Hedges, X-ERPC-Network-Attempts/-Retries/-Hedges — counters, zero-filled on early errors (not omitted; only mode: off removes them). erpc/http_server.go:L1149-1166
• X-ERPC-Cache-Attempts/-Retries/-Hedges — emitted only when the cache scope was exercised. erpc/http_server.go:L1170-1174
• X-ERPC-Consensus-Slots, X-ERPC-Consensus-Disputes, X-ERPC-Consensus-Low-Participants — emitted only when consensus was exercised and the count is > 0. erpc/http_server.go:L1175-1183
• X-ERPC-Cache: HIT|MISS, X-ERPC-Upstream, X-ERPC-Duration — response metadata
• X-ERPC-Upstreams (mode all only): per-attempt participation log in the form <id>=<reason>:<outcome>:<duration>ms[:won]
• traceparent (+ tracestate) — injected when tracing is enabled. common/tracing_util.go:L58-65

Request headers with server-level effects:

• Host — aliasing match (port stripped)
• Content-Encoding: gzip — inbound body decompression (always, regardless of enableGzip)
• Accept-Encoding: gzip — response compression eligibility
• Content-Type: application/grpc over HTTP/2 — gRPC mux on shared port (bypasses TimeoutHandler and gzip)
• Origin — CORS evaluation; absent Origin bypasses CORS entirely
• X-ERPC-Force-Trace: true|1|yes or ?force-trace=true|1|yes — force-sample the OTel trace for this request. common/tracing_util.go:L106-119
• traceparent / tracestate — W3C trace-context extraction on ingress. common/tracing_util.go:L67-73
• ?eval= — healthcheck strategy override (bypasses healthCheck.defaultEval). erpc/healthcheck.go:L106-112
• Auth headers: ?token= (deprecated), X-ERPC-Secret-Token, Authorization: Basic|Bearer, ?secret=, ?jwt=, ?signature=&message=, X-Siwe-Message/X-Siwe-Signature. auth/http.go:L13

Best practices

• Set maxTimeout as a Go duration string ("30s"), never a bare integer. maxTimeout: 150 means 150 milliseconds, not 150 seconds — this is the most common misconfiguration footgun.
• Size maxTimeout to be greater than your largest network-level failsafe.timeout plus total retry overhead, or retries will be cut short by the server ceiling before they complete. See Timeout.
• In production behind an LB, set waitBeforeShutdown to at least two readiness-probe intervals (often 20s). The default 10s is too short for many Kubernetes setups.
• Never omit trustedIPForwarders when running behind a proxy. Without it, every request appears to come from the LB IP, breaking per-IP rate limits, auth network strategies, and client identity logging.
• Setting tls.caFile enables mTLS implicitly and immediately — all clients must then present a certificate. Verify this is intended before adding the field in production.
• Keep executionHeaders: "summary" or "off" for production traffic if response size is sensitive — "all" mode emits a verbose X-ERPC-Upstreams header that grows linearly with the number of upstream attempts.
• Use responseHeaders for static CORS or security headers, but pin the env vars at deploy time. Values that expand to empty are silently dropped with no error, so a missing env var makes the header vanish invisibly.

Edge cases & gotchas

1. maxTimeout bare integer is milliseconds. maxTimeout: 150 = 150 ms, not 150 s. Always use a duration string like "150s". Source: common/duration.go:L20-31
2. grpcEnabled: true with default ports silently shares the HTTP v4 port. gRPC traffic bypasses maxTimeout and gzip. Source: erpc/http_server.go:L161-177
3. mTLS is implicit. Setting tls.caFile flips ClientAuth = RequireAndVerifyClientCert — all clients must present certs. No other field is needed. Source: erpc/http_server.go:L1621-1633
4. tls.insecureSkipVerify does nothing for inbound connections. It is a client-dialing flag only; Go's TLS server ignores it. Inbound cert verification is controlled by caFile alone. Source: erpc/http_server.go:L1635
5. responseHeaders env vars that expand to empty are silently dropped. No warning is emitted — only a Debug log. A missing env var in production silently removes the header from all responses. Source: erpc/http_server.go:L135-148
6. listenV6 has no default. IPv6 is OFF unless explicitly set to true; it is never auto-enabled. Source: common/defaults.go:L640-650
7. httpPort (deprecated) is silently migrated. No warning is emitted. After SetDefaults, httpPort always equals httpPortV4. New configs must use httpPortV4. Source: common/defaults.go:L652-661
8. No request-body size limit. There is no http.MaxBytesReader; a multi-GB body is fully read into memory. Only readTimeout and process memory provide back-pressure. Source: util/reader.go:L12-32
9. POST requests can still get non-200 status codes. HTTP 400/401/404/429 mappings apply to POST; only the fatal-writer and timeout/cancel paths force 200-for-POST. Source: erpc/http_server.go:L1470-1491
10. Whole responses are buffered before hitting the socket. Large responses consume RAM; pooled buffers above 256 KiB are discarded to GC. writeTimeout only matters at the final socket copy. Source: erpc/http_timeout.go:L43-48
11. Disallowed CORS origins are not blocked for non-OPTIONS requests. The request proceeds without Access-Control-* headers; the browser enforces the block. Source: erpc/http_server.go:L1036-1051
12. Zero-project configs self-alias. A default main project and matchDomain: "*" rule are injected so /evm/1 works without a project segment. Source: common/defaults.go:L100-111
13. serveProject + serveChain without serveArchitecture is always an error. Source: erpc/http_server.go:L985-990
14. Full request bodies are logged at Info level by default. Sensitive params (private keys, tokens in JSON-RPC args) appear in logs. Source: erpc/http_server.go:L398-402
15. X-ERPC-Attempts excludes NetworkAttempts to avoid double-counting rotations. Source: erpc/http_server.go:L1151-1157
16. Batch detection is byte-exact. Leading whitespace before [ makes the body parse as a single (invalid) request. Source: erpc/http_server.go:L405
17. Any non-POST/non-OPTIONS request becomes a healthcheck, including GET /myproject/evm/1. Source: erpc/http_server.go:L1001-1003
18. RFC 7239 Forwarded is dead code — only headers named in trustedIPHeaders are consulted, parsed XFF-style. Source: erpc/http_server.go:L1856-1881
19. CORS metric label mismatch. The label named project is populated with the URL path, not the project ID. Source: erpc/http_server.go:L1020
20. serveArchitecture-only aliasing requires a project in the URL path. Combining serveArchitecture + serveChain without serveProject is always ErrInvalidUrlPath. Source: erpc/http_server.go:L963-990
21. OPTIONS to a project without CORS config falls through to normal request handling. The OPTIONS early-return lives inside the CORS != nil branch; a project with no cors: block sends an empty body OPTIONS through the full JSON-RPC path, returning a JSON-RPC error (not a 204). Source: erpc/http_server.go:L343-347
22. Gzip decision is first-write-only. A response whose first chunk is smaller than 1024 bytes is never compressed, even if subsequent writes are large. Source: erpc/http_server.go:L1708-1717
23. Done/canceled race silently drops the response. If the inner handler finishes but the request context has already errored (e.g. a race between handler return and deadline), nothing is written to the socket. The timeout layer has already responded. Source: erpc/http_timeout.go:L72-80
24. Counter headers are zero-filled on early errors, not omitted. Even URL-parse or project-lookup failures emit X-ERPC-Attempts: 0, X-ERPC-Upstream-Attempts: 0, etc., because writeCounterHeaders runs against a nil-safe snapshot. Only executionHeaders: "off" suppresses them entirely. Source: erpc/http_server.go:L1149-1166
25. ErrUnknown fallback body is not JSON-RPC shaped. When processErrorBody receives an error that survives all unwrapping as neither a *common.BaseError nor common.StandardError, it produces {"code":"ErrUnknown","message":"unexpected server error","cause":{...}} — a struct dump, not {"jsonrpc":"2.0","error":{...}}. Clients parsing response.error.code as an integer will fail; detection must branch on whether the outer object has a code string key vs an error object key. Source: erpc/http_server.go:L1450-1454
26. ErrorStatusCode() on error types is dead code. Every error type in common/errors.go implements ErrorStatusCode() int, but there are no call sites. The wire HTTP status is determined exclusively by the two switch blocks in determineResponseStatusCode and handleErrorResponse using common.HasErrorCode. Reading an error type's ErrorStatusCode() to infer the wire status gives wrong answers for many types (e.g. ErrNetworkInitializing → 503, ErrUpstreamRateLimitRuleExceeded → 429 per the method, but neither appears in the switch). Source: erpc/http_server.go:L1280-1317
27. Sonic encoder writes a trailing newline and disables HTML escaping globally. The early-error path uses encoder.Encode (trailing \n) and sonic's HTML escaping is off (common/sonic.go). JSON field values such as URLs are not HTML-escaped in error bodies. Source: erpc/http_server.go:L229-230

Observability

Trace spans: Http.ReceivedRequest (SpanKind=server; attrs http.method, http.url, http.scheme, http.user_agent); Request.Handle per sub-request; detail spans Http.ReadBody, Http.ParseRequests, HttpServer.WriteResponse; w3c traceparent/tracestate extraction on ingress, injection on egress.

Note: there is no generic http_requests_total-style server metric; request accounting happens at the network/upstream layers.

Notable log lines:

• "received http request" at Info — includes the full raw JSON body; sensitive params appear at default log level
• "entering draining mode → healthcheck will fail" on SIGTERM
• "starting IPv4/IPv6 HTTP server on <addr>" at listen
• "TLS enabled for IPv4/IPv6 HTTP server" when TLS is active
• "http server forced to shutdown" / "http server stopped" on graceful drain completion
• "custom response header configured" / "custom response header skipped (empty value after env expansion)" for responseHeaders
• "invalid CIDR/IP in trusted forwarders; ignoring" for malformed forwarder entries
• "CORS request from disallowed origin..." at Debug when an origin is rejected

Source code entry points

• erpc/http_server.go:L1-L230 (opens in a new tab) — NewHttpServer: handler chain assembly, gzip/timeout wrapping, gRPC mux, IPv4/IPv6 server construction, responseHeaders env expansion, trusted forwarder parsing
• common/defaults.go:L640-L733 (opens in a new tab) — ServerConfig.SetDefaults: all defaults including port derivation, deprecated httpPort migration, zero-project aliasing injection
• erpc/http_timeout.go:L20-L143 (opens in a new tab) — custom TimeoutHandler: buffered response, timeout/cancel body shapes, panic propagation
• erpc/http_server.go:L1537-L1637 (opens in a new tab) — TLS config construction, ListenAndServeTLS, mTLS ClientAuth assignment
• erpc/http_server.go:L1782-L1905 (opens in a new tab) — resolveRealClientIP: trusted forwarder check, XFF right-trim, fallback to peer IP
• erpc/http_server.go:L810-L1006 (opens in a new tab) — parseUrlPath: full aliasing × path-segment case matrix
• common/validation.go:L70-L109 (opens in a new tab) — ServerConfig.Validate: listen host/port pairing, maxTimeout required, forwarder IP/CIDR syntax
• erpc/http_server_test.go (opens in a new tab) — path parsing matrix, CORS, timeouts, aliasing fixtures

Related pages

• Timeout — per-network timeout that must stay below server.maxTimeout or the server ceiling fires first.
• Rate limiters — per-IP rate limits depend on accurate client IP, which requires correct trustedIPForwarders + trustedIPHeaders.
• Auth — authentication strategies consume the resolved client IP from this layer.
• URL structure & aliasing — full path-parsing logic and all valid segment combinations for domain aliasing.
• Deployment overview — where server: fits in a production Kubernetes or Docker deployment.