Trust the data

This page sells the outcome; implementation lives in the feature pages. Fetch their machine-readable companions:

• Consensus — participant selection, agreement rules DSL, quotas, punishment, misbehavior export.
• Integrity — highest-block enforcement, getLogs range checks, empty-result directives.
• Block tracking — state poller cadence, served tip, lag semantics (block-number deltas).
• getLogs splitting — split triggers, sub-request failure semantics, merge/dedup rules.
• Method handlers — per-method normalization and edge cases.

Key interaction: consensus and integrity both consume block-tracking state; getLogs splitting runs before failsafe policies so each sub-request is individually retried/hedged. Field-level tables and exact defaults live in each page's agent section.

How it works

Hook dispatch model. Four hook entry-points in architecture/evm/hooks.go (opens in a new tab):

• HandleProjectPreForward — runs at project layer before cache and upstream selection; handles eth_blockNumber, eth_call, eth_chainId, eth_getLogs, trace_filter/arbtrace_filter.
• HandleNetworkPreForward — runs after upstream selection, can short-circuit; handles eth_getLogs, eth_chainId, trace_filter/arbtrace_filter.
• HandleNetworkPostForward — called after response comes back at network layer; handles eth_getBlockByNumber, eth_getLogs, eth_sendRawTransaction, trace_filter/arbtrace_filter.
• HandleUpstreamPostForward — richest hook, called per-upstream; checks MarkEmptyAsErrorMethods, applies per-method validation, marks unexpected empties as ErrEndpointMissingData.

Block tag normalization. NormalizeHttpJsonRpc (opens in a new tab) runs on every incoming JSON-RPC request: caches the numeric block number on the request, and when the tag is "latest" or "finalized" and the network has a known head, replaces the tag with a concrete 0x… hex. Tags "safe" and "pending" are intentionally passed through unchanged. The X-ERPC-Skip-Interpolation directive or skipInterpolation request directive suppresses mutations while still caching the numeric block number.

Block reference extraction. ExtractBlockReferenceFromRequest (opens in a new tab) and ExtractBlockReferenceFromResponse derive a (blockRef, blockNumber) pair. Special blockRef values:

• "*" — wildcard for transaction/receipt hash lookups or composite range methods; uses ConnectorReverseIndex for cache lookups.
• "1" — finalized-forever static data (chain ID, genesis block); cached indefinitely.
• A numeric string (e.g. "19827314") — concrete block; uses ConnectorMainIndex.

Empty-result handling. upstreamPostForward_markUnexpectedEmpty (opens in a new tab) converts a null/empty result to ErrEndpointMissingData when: (1) the method is in markEmptyAsErrorMethods, (2) req.Directives().RetryEmpty is true, and (3) the block is NOT beyond the confidence head. When the network head is unknown the check fails open — empties are treated as retryable.

Syncing state. A self-hosted node must return eth_syncing: false four consecutive times (FullySyncedThreshold = 4 at architecture/evm/evm_state_poller.go:L21 (opens in a new tab)) before eRPC stops syncing-check requests. One syncing=true resets the counter to 1.

Block-head rollback tolerance. DefaultToleratedBlockHeadRollback = 1024 at architecture/evm/evm_state_poller.go:L26 (opens in a new tab) — the maximum block-head rollback the poller tolerates before treating it as a large rollback event (emits erpc_upstream_block_head_large_rollback gauge).

Error normalizer precedence. ExtractJsonRpcError (opens in a new tab) uses text-based matching in strict order:

1. Range-too-large → ErrEndpointRequestTooLarge
2. OP Stack sender rate limit → ErrEndpointCapacityExceeded (non-network-retryable)
3. Billing exhaustion (HTTP 402) → ErrEndpointBillingIssue
4. Rate limiting (HTTP 429 or text) → ErrEndpointCapacityExceeded
5. Block tag unsupported → ErrEndpointClientSideException
6. Missing data → ErrEndpointMissingData
7. Execution timeout → ErrEndpointServerSideException
8. EVM revert / VM exception → ErrEndpointExecutionException
9. Already-known / nonce-too-low → ErrEndpointNonceException
10. Insufficient funds → ErrEndpointExecutionException
11. Transaction rejected / out-of-gas → ErrEndpointExecutionException
12. Not-found method → ErrEndpointUnsupported; not-found block/state → ErrEndpointMissingData
13. Unsupported (HTTP 415/405, codes -32004/-32001) → ErrEndpointUnsupported
14. Malformed transaction (RLP errors) → ErrEndpointClientSideException (non-network-retryable)
15. Invalid type/map errors → ErrEndpointClientSideException (retryable)
16. Invalid args (code -32602/-32600) → ErrEndpointClientSideException
17. Unauthorized (HTTP 401/403) → ErrEndpointUnauthorized
18. Fallback → ErrEndpointServerSideException

Special: a 0x08c379a0 prefix in a successful HTTP 200 result (dt[1:11], not dt[0:10] — offset 0 is the JSON quote character) signals an EVM revert even with a 200 status.

gRPC error mapping. BDS error codes take precedence when present in gRPC status details: UNSUPPORTED_BLOCK_TAG/UNSUPPORTED_METHOD → ErrEndpointUnsupported; RANGE_OUTSIDE_AVAILABLE → ErrEndpointMissingData; RANGE_TOO_LARGE → ErrEndpointRequestTooLarge; RATE_LIMITED → ErrEndpointCapacityExceeded; TIMEOUT_ERROR/INTERNAL_ERROR → ErrEndpointServerSideException. Without BDS: Unimplemented → Unsupported; InvalidArgument → ClientSide (non-network-retryable); ResourceExhausted → CapacityExceeded; NotFound/OutOfRange → MissingData.

ExecState counter model. Per-request execution telemetry (common/exec_state.go (opens in a new tab)):

• total Attempts = UpstreamAttempts + CacheAttempts (NetworkAttempts is a rotation count, NOT summed — adding it would double-count)
• total Retries = UpstreamRetries + NetworkRetries + CacheRetries
• total Hedges = UpstreamHedges + NetworkHedges + CacheHedges
• Snapshot() loads counters independently with no global lock — under high concurrency totals may briefly drift from component sums.

UpstreamAttemptOutcome values: success, empty, transport_error, server_error, client_error, rate_limited, missing_data, exec_revert, block_unavailable, breaker_open, cancelled, timeout, skipped.

UpstreamSelectionReason values: primary, retry, hedge, consensus_slot, sweep.

Config schema

All fields under networks[*].evm.*. Populated by EvmNetworkConfig.SetDefaults() (common/defaults.go:L2060 (opens in a new tab)).

Default markEmptyAsErrorMethods (11 methods): eth_blockNumber, eth_getBlockByNumber, eth_getTransactionByHash, eth_getTransactionByBlockHashAndIndex, eth_getTransactionByBlockNumberAndIndex, eth_getUncleByBlockHashAndIndex, eth_getUncleByBlockNumberAndIndex, debug_traceTransaction, trace_transaction, trace_block, trace_get.

Explicitly excluded: eth_getBlockByHash (subgraph upstreams commonly return null by hash), eth_getTransactionReceipt and eth_getBlockReceipts (pending tx / zero-tx blocks legitimately return empty).

Non-retryable write methods (never network-retried): eth_sendTransaction, eth_createAccessList, eth_submitTransaction, eth_submitWork, eth_newFilter, eth_newBlockFilter, eth_newPendingTransactionFilter. Note: eth_sendRawTransaction is excluded because it has idempotency handling.

Observability

Prometheus metrics

Trace spans

Key log messages

Edge cases & gotchas

1. "safe" and "pending" tags are never interpolated. eRPC does not track the safe checkpoint or mempool state; these tags are passed through to the upstream unchanged.

2. Block ref "*" for transaction lookups changes connector behavior. When blockRef == "*", the cache calls ConnectorReverseIndex instead of ConnectorMainIndex. Connectors that do not implement reverse indexing will miss these entries.

3. markEmptyAsErrorMethods only fires when RetryEmpty directive is true. Even if a method is in the list, the conversion to ErrEndpointMissingData is gated on req.Directives().RetryEmpty. Set retryEmpty: true in the failsafe retry policy or as a directive default for this to activate.

4. An explicitly-set empty markEmptyAsErrorMethods: [] disables the feature entirely — including for methods in the default list. A non-nil empty slice is never replaced by the default set.

5. Custom markEmptyAsErrorMethods completely replaces the default set — there is no merge/extend. Setting markEmptyAsErrorMethods: ["custom_method"] means none of the default 11 methods will trigger empty-to-error conversion.

6. emptyResultBeyondConfidence fails open. When the network head is unknown (poller hasn't bootstrapped), it returns false — the empty result IS treated as retryable. This prevents spurious caching of null results during startup.

7. NetworkAttempts is NOT summed into the total attempts. NetworkAttempts counts upstream rotations; each rotation already generates one UpstreamAttempts increment. Summing both would double-count physical attempts.

8. ExecState.Snapshot() is eventually consistent. Each counter is loaded independently; under high concurrency Snapshot().Attempts may temporarily differ from UpstreamAttempts + CacheAttempts.

9. Error normalizer: nonce/duplicate checks must precede rejected/gas checks. Some providers use JSON-RPC code -32003 for both "already known" and "out-of-gas". The normalizer checks "already known" and "nonce too low" first to ensure idempotency detection is not masked.

10. OP Stack "sender is over rate limit" is non-network-retryable. All providers route to the same OP Stack sequencer, so retrying on a different upstream would fail identically.

11. Malformed RLP transactions are non-network-retryable. No upstream would accept an invalid-encoding transaction.

12. 200-OK revert detection uses dt[1:11] not dt[0:10]. The raw string from GetResultString() is JSON-encoded — dt[0] is the JSON double-quote character ". The 0x08c379a0 selector lives at positions 1-10. A check starting at index 0 would always fail. (architecture/evm/error_normalizer.go:L613 (opens in a new tab))

13. cache.Get fan-out has a 30s defensive backstop when the caller has no deadline. This prevents connection-pool leaks from hung connectors that lack a timeout of their own.

14. Cache compression only occurs when compressed bytes are smaller than original bytes. The compressor will not store the compressed form if compression overhead exceeds the savings.

15. deepCopyParams is called only when changes are needed. NormalizeHttpJsonRpc defers the copy until at least one mutation is confirmed; unchanged requests skip both the copy and the write-lock acquisition.

16. JavaScript Runtime is single-threaded; each goroutine needs its own instance. Sobek is not goroutine-safe. The selection policy system uses a per-goroutine pool at internal/policy/runtime_pool.go:L51 (opens in a new tab).

Source code entry points