Healthcheck

HealthCheckConfig fields

Evaluation strategies

defaultEval (and the ?eval= query parameter) accept only the following named strategies — arbitrary expressions are not supported. Passing an unrecognized string returns HTTP 503 with "unknown evaluation strategy: <value>".

Notes:

• Error-rate strategies read from the in-memory score tracker — they are pure memory operations, sub-millisecond.
• eth_chainId strategies fire real RPC calls to each upstream in parallel. Set readinessProbe.timeoutSeconds high enough (5 s is usually safe).
• all:activeUpstreams is the strictest strategy: it fails if any upstream is missing or has been excluded by a selection policy. Use only when your deployment requires all upstreams to be reachable.

Response shapes

simple mode (default)

Healthy:

HTTP 200
OK

Unhealthy:

HTTP 503
{"code":"HealthcheckUnhealthy","message":"...","details":{...}}

networks mode

Returns a JSON object keyed by project ID. Each project entry contains per-network aggregates.

{
  "status": "OK",
  "message": "all systems operational",
  "details": {
    "main": {
      "status": "OK",
      "networks": {
        "evm:1": {
          "networkId": "evm:1",
          "alias": "ethereum",
          "blockTimeMs": 12003,
          "healthy": true,
          "status": "OK"
        }
      }
    }
  }
}

verbose mode

Identical to networks but each network entry also includes a per-upstream breakdown with individual upstream status, error rate, and scoring info.

{
  "status": "OK",
  "message": "all systems operational",
  "details": {
    "main": {
      "status": "OK",
      "message": "3 / 3 upstreams have low error rates",
      "config": {
        "networks": 2,
        "upstreams": 3,
        "providers": 1
      },
      "networks": {
        "evm:1": {
          "networkId": "evm:1",
          "alias": "ethereum",
          "blockTimeMs": 12003,
          "healthy": true,
          "status": "OK",
          "upstreams": {
            "alchemy-eth": { "healthy": true, "errorRate": 0.01 }
          }
        }
      }
    }
  }
}

blockTimeMs is the EMA-estimated block time for each network, derived from on-chain block timestamps. It is null (field omitted) during startup while observations accumulate.

Drain semantics

When eRPC receives SIGTERM it enters a graceful shutdown sequence:

1. The /healthcheck endpoint immediately starts returning 503 (regardless of actual upstream health).
2. server.waitBeforeShutdown — eRPC keeps accepting in-flight requests but stops accepting new ones. The orchestrator's readiness probe fails during this window, removing the pod from the load-balancer rotation.
3. server.waitAfterShutdown — the HTTP listener closes; the process stays alive briefly so open TCP connections can be drained by Envoy / kube-proxy.

Size waitBeforeShutdown to at least readinessProbe.periodSeconds × readinessProbe.failureThreshold + 1s. For the example probe config above (5 s period × 2 failures = 10 s), a safe value is waitBeforeShutdown: 12s.

URL patterns

# Global (checks all projects / all networks)
GET /healthcheck
 
# Project-scoped (checks only the specified network)
GET /<projectId>/evm/<chainId>/healthcheck
GET /<projectId>/evm/<chainId>   # same as above
 
# With custom eval
GET /healthcheck?eval=all:errorRateBelow90
GET /main/evm/1/healthcheck?eval=any:evm:eth_chainId
 
# With auth secret
GET /healthcheck?secret=<token>
# or header: X-ERPC-Secret-Token: <token>

When project or network aliases are configured:

# Project aliased
GET /evm/42161/healthcheck
 
# Project + network arch aliased
GET /42161/healthcheck
 
# Fully aliased (project + arch + chain)
GET /healthcheck   # on eth-rpc.example.com

Auth configuration

healthCheck.auth accepts the same strategy schema as all other auth points in eRPC (secret, network, jwt, siwe). When omitted, the endpoint is open to all callers. Typical production setup: allow from localhost and internal CIDRs via network strategy so orchestrator probes work without a token, while blocking external access.

healthCheck:
  auth:
    strategies:
      - type: network
        network:
          allowLocalhost: true
          allowedCIDRs:
            - "10.0.0.0/8"
            - "172.16.0.0/12"
            - "192.168.0.0/16"

Common pitfalls

• Using all:activeUpstreams as a readiness probe — any temporarily cordoned or slow-to-initialize upstream will make the probe fail and block traffic to a perfectly healthy pod. Prefer any:initializedUpstreams or any:errorRateBelow90 for readiness.
• eth_chainId evals on a busy cluster — each probe fires real RPC calls; multiply by probe frequency and pod count. With 10 upstreams, 5 s probe interval, and 20 pods that is 40 RPC calls per second directed at your upstreams.
• waitBeforeShutdown too short — if the readiness probe doesn't have time to fail failureThreshold times before the listener closes, live traffic will hit the terminating pod. See the drain formula above.
• Auth accidentally blocking orchestrator probes — the kubelet probes run from a node IP, not localhost. If using network.allowLocalhost: true only, probes from node IPs are rejected. Add the node CIDR to allowedCIDRs or remove auth from the healthcheck entirely if the endpoint is only reachable inside the cluster.
• Liveness probe on /healthcheck instead of TCP — the HTTP healthcheck fails during graceful drain (by design). A liveness probe on the same path will restart the pod during every normal shutdown, making rolling updates restart pods twice.

Real-world examples

Minimal (development / single upstream)

healthCheck:
  mode: simple
  defaultEval: "any:initializedUpstreams"

Production multi-upstream with verbose output

healthCheck:
  mode: verbose
  defaultEval: "any:errorRateBelow90"
  auth:
    strategies:
      - type: network
        network:
          allowLocalhost: true
          allowedCIDRs: ["10.0.0.0/8", "172.16.0.0/12"]

Strict all-upstreams-required (e.g. private RPC with SLA)

healthCheck:
  mode: networks
  defaultEval: "all:activeUpstreams"

Note: with this strategy, a single cordoned or unhealthy upstream makes the pod NotReady. Useful when you need every upstream available, but risky in auto-scaling scenarios.