Config
Consensus policies
AI agents: fetch https://docs.erpc.cloud/config/failsafe/consensus-policies.llms.txt for the complete machine-readable version of this page (full configuration schema, defaults, worked examples, and source links). Append `.llms.txt` to any docs URL for the same treatment.AIFor agents: /config/failsafe/consensus-policies.llms.txt

Consensus policies

A single consensus rule-set can't fit every request. A tx-hash lookup, a ten-million-block-old eth_getBalance, and a live eth_getLogs all want different quorums — but you don't want to fork your routing to get there, and you never want a looser rule-set to slip in by accident. Consensus policies let you name several ConsensusPolicyConfig blocks and pick one per request with a small JS function that runs before fan-out. Every failure — a throw, a timeout, an unknown name, a null return — falls closed to the strictest policy you designate.

What you get

  • Named consensus policies, each a full consensus config, selected per request
  • A pre-round customPolicy.evalFunction with the verified caller, the request, and live upstream health/availability in scope
  • Strategy-gated integrity grades (e.g. historical serving only for callers on a named auth strategy) — never client headers
  • Automatic availability fallback that an attacker cannot force: punishment and operator cordons pin the request to the strictest policy
  • A MissingData waiver that releases an internal quota when at least minAgreement matching participants abstained with MissingData (sibling empties/unrelated errors do not block)
  • Fail-closed by construction: eval error, timeout, or unknown name → the default policy, always
  • An X-ERPC-Consensus-Policy response header and a consensus_policy metric label so you can see which grade served each round

Quick taste

Illustrative, not a tuned production config — two named policies and a selector that upgrades authorized callers to a MissingData-waiving policy for tx-hash lookups, and fails closed to standard for everyone else:

projects[].networks[].failsafe[].consensus
erpc.yaml
consensus:  # the inline block IS the strict default policy — the round falls back to it  # whenever the selector returns null/unknown, or the eval errors or times out.  # It must be a complete, valid consensus config on its own.  maxParticipants: 3  agreementThreshold: 2  disputeBehavior: returnError  requiredParticipants:    - { tag: "type:internal", minParticipants: 1, minAgreement: 1 }    - { tag: "type:external", minParticipants: 2, minAgreement: 2 }  # each value under policies is a full consensus config  policies:    standard:                 # default / fail-closed — no waiver      maxParticipants: 3      agreementThreshold: 2      disputeBehavior: returnError      requiredParticipants:        - { tag: "type:internal", minParticipants: 1, minAgreement: 1 }        - { tag: "type:external", minParticipants: 2, minAgreement: 2 }    standard-waive-missing:   # same composition; waiver on internals only      maxParticipants: 3      agreementThreshold: 2      disputeBehavior: returnError      requiredParticipants:        - { tag: "type:internal", minParticipants: 1, minAgreement: 1, waiveAgreementOnMissingData: true }        - { tag: "type:external", minParticipants: 2, minAgreement: 2 }  customPolicy:    evalTimeout: 50ms    evalFunction: |      (ctx) => {        if (ctx.upstreams.anyPunished()) return "standard"        if (ctx.user?.auth === "consensus-waive" && ctx.request.blockNumber === null)          return "standard-waive-missing"        return "standard"      }

Agent reference

Copy one of these prompts into your AI agent session (Claude Code, Cursor, …) — each one points the agent at this page's machine-readable reference so it can do the work correctly:

Prompt Example #1: add strategy-gated historical serving
I want authorized callers (JWT role "brp:historical") to be able to read very old
blocks that our internal archive nodes have pruned, while everyone else stays on our
strict internal+external consensus. Set up eRPC named consensus policies with a
pre-round customPolicy selector, failing closed to the strictest policy for everyone
else. Work with my existing eRPC config. Read the full reference first:
https://docs.erpc.cloud/config/failsafe/consensus-policies.llms.txt
Prompt Example #2: fall back to externals when internals are down (but not when punished)
When our preferred upstreams are genuinely unavailable (unhealthy or availability-cordoned),
I want requests to fall back to a looser consensus policy — but ONLY for an
authorized role, and NEVER when a node was cordoned for misbehavior or by an operator.
Set this up with eRPC consensus policies + a customPolicy selector. Reference:
https://docs.erpc.cloud/config/failsafe/consensus-policies.llms.txt
Prompt Example #3: waive the internal quota on missing-data
Our internal archive nodes sometimes return a missing-data error for pruned state,
which currently causes a consensus composition dispute even though the external nodes
agree. Add a MissingData waiver so the internal quota is released when at least
minAgreement internals returned missing-data (aligned with that quota), kept honest by
a never-waivable external floor. Reference:
https://docs.erpc.cloud/config/failsafe/consensus-policies.llms.txt
Consensus policies — full agent referenceExpand for every option, default, and edge case — or copy this entire section into your AI assistant.

How it works

One round, one policy, chosen before fan-out. A request's consensus round runs under exactly one named ConsensusPolicyConfig. Before the round starts, a pre-round selector answers a single question — which named policy should govern this round? — from the verified caller, the request, and the candidate upstreams' live health. The selector never sees round outcomes and never grades a round; that stays in the declarative consensus config and the executor.

Request flow. (1) Auth resolves the user and stamps the authenticating strategy's name. (2) An EvalContext is built from the request, user, and candidate upstreams. (3) selector.Evaluate(ctx, evalTimeout) returns a policy name. (4) The name resolves to a ConsensusPolicyConfig — an unknown name resolves to the default policy; an empty/null return is a selector failure that also fails closed to the default. (5) The existing executor runs the round under that config, unchanged. (6) The chosen name is emitted as the X-ERPC-Consensus-Policy response header and the consensus_policy metric label.

Backward compatible. An inline consensus: block with no policies map and no customPolicy is exactly one anonymous default policy — zero migration, every existing field keeps working. customPolicy and policies are purely additive.

Fail closed, always. The default policy is the fail-closed target and must be configured as the strictest policy (the design assumes standard). Every selector failure resolves to it and is counted (and rate-limit-logged): a JS throw, a non-string return, a null/undefined return, an eval that overruns evalTimeout, or an unknown returned name. A null return is not an intentional default pick — return the string "default" to select the default deliberately. At pool capacity the selector waits for a free VM and that wait is charged to the same timeout — there is no separate pool_exhausted bypass. The selector can never fall open to a more permissive policy.

Selector engine. The engine is a standalone package that imports nothing from consensus/, health/, or auth/ — the caller builds an EvalContext from those subsystems and passes plain values in, and the vocabulary those callers populate (cordon classes, health states) is owned by the engine so the two sides cannot drift. customPolicy.evalFunction is compiled once at load; each eval borrows a pre-warmed sobek VM from a small bounded pool (8) and binds the per-request context.

Behavior changes (upgrade note)

  • Consensus config Validate() now runs at boot. A config that boots on main but fails consensus validation (broken named policy, missing waiver floor, S3-path collision, bad evalFunction) now fails startup instead of the first request.
  • Misbehavior export keys changed. The default filePattern adds {instanceId}, and S3 keys gain a -f<seq> suffix, to stop concurrent pods overwriting each other's objects. Readers of the old key shape must update.
  • Selector fail-closed reasons are explicit. A null/undefined eval return is a selector failure (counted reason="null" and logged), not an intentional default pick; return the string "default" to select the default deliberately.

Eval runs once, before fan-out. It does not extend consensus wait windows. A clean eval returns its VM to the pool, so steady-state cost is the eval itself (tens to low-hundreds of µs) — only a throw, non-string return, null return, or timeout discards and rebuilds. There is no decision cache in v1. Reuse note: because the VM is reused, an eval that closes over mutable JS state (a counter, a cached array) keeps it across requests on the same VM. Write evals as pure functions of ctx — do not depend on fresh-realm isolation.

evalTimeout is a hard wall-clock cap. On overrun the borrowed VM is interrupted and discarded (never returned to the pool, since an interrupt is the only state that poisons a VM); the request fails closed to the default. Omit the field or set 0 to get the 50ms default — negative values are rejected at load.

Caller identity is the auth strategy name. ctx.user.auth is the name field of the strategy that authenticated the caller, stamped by the auth registry onto the request's exec state. Unnamed strategies stamp "". The name is never accepted from client-controlled headers, query params, or body fields — only from the strategy config that verified the caller.

Config schema

All fields under networks[].failsafe[].consensus. (or project-level failsafe[].consensus.). Duration fields accept Go duration strings.

FieldTypeDefaultBehavior / footguns
policiesmap[string]ConsensusPolicyConfignil (inline block is the single default)Named policies. Each value is a complete consensus config — the same fields as the inline consensus: block (maxParticipants, agreementThreshold, disputeBehavior, requiredParticipants, punishMisbehavior, misbehaviorsDestination, …). A named policy must not itself define customPolicy or policies — nesting is a startup validation error. Each named policy is validated independently, so a broken one fails startup with a policies["<name>"] prefix.
cacheableResponses*boolinline/default omit → true; named policies.* omit → falseStamped onto the winning response in Consensus.Run (SetCacheable). Cache.Set combines this mark with database.*.cachingStrategy — the cache never sees policy names. Named omit→false is stamped before extends resolve, so it wins over a base's true for both extends: default and named→named inheritance (fail-safe: weaker/derived grades stay uncacheable unless that named entry opts in). Recommended posture: leave cache at defaultAllow, set cacheableResponses: false on weaker grades like fallback. When any failsafe entry on the network has customPolicy, Forward also skips Set for unmarked (nil) responses so no-consensus sibling routes cannot poison the shared store. Source: common/defaults.go, consensus/consensus.go, erpc/networks.go
policies.<name>.extendsstring"" (no inheritance)One-level inheritance only. Values: another named policy key, or default (the inline consensus: block). The child overlays the base: non-nil child fields win, including an explicit requiredParticipants: [] which clears inherited quotas. Nested extends (base itself sets extends) and unknown bases fail startup. After resolve, extends is cleared on the runtime policy. Selection surface (customPolicy / nested policies) is never inherited and is rejected on the child even when combined with extends.
customPolicy*ConsensusCustomPolicyConfignil (default policy for every request)Optional pre-round selector. When omitted, the default/inline policy applies to every request.
customPolicy.evalFunctionstringRequired when customPolicy setJS. Signature (ctx) => string | null. Compiled at load (a smoke-compile runs during validation, so a syntax error fails startup, not the first request). Returns a policy name, or null/empty for the default. A non-string return fails closed.
customPolicy.evalTimeoutDuration50msHard wall-clock cap per eval. On overrun the VM is discarded and the request fails closed to the default. Omit or 0 → default 50ms; negative values fail startup validation.
requiredParticipants[].waiveAgreementOnMissingDataboolfalseOpt-in per quota entry. Releases this entry's minAgreement quota for a round when at least minAgreement distinct upstreams matching its tag returned ErrEndpointMissingData and no matching upstream returned data. See MissingData waiver below. Put it on a permissive policy (standard-waive-missing), never on the fail-closed standard.
requiredParticipants[].waiveAgreementOnEmptyOutsideRetentionboolfalseOpt-in per quota entry (v1.1). Releases this entry's minAgreement quota when at least minAgreement distinct tag-matching upstreams returned an empty result, no matching upstream returned non-empty data, AND the block is proven outside retention — prefer each empty upstream's finite blockAvailability lower bound, else head - block > retentionBlocks. The block is the one the client's request names; for a hash lookup it is the block the answer names, once an upstream outside the agreeing set confirms it. Tag requests never waive. See Empty-outside-retention waiver below. Requires retentionBlocks > 0. Same placement rule — permissive policy only.
requiredParticipants[].retentionBlocksint640Retained depth behind the chain head for the chain-head fallback proof (outside iff head - requestedBlock > retentionBlocks). Prefer aligning with internal blockAvailability.lower.latestBlockMinus so the avail-bound proof path can fire without a network tip. Must be > 0 when waiveAgreementOnEmptyOutsideRetention is set.
blockEvidenceFieldsmap[string][]stringnilUnsafe opt-in. Per-method response paths holding the winning answer's own block number (same shape as preferHighestValueFor, plus "*.field" for array results). When no upstream outside the agreeing set confirms a hash lookup's block, the empty-outside-retention waiver trusts this one. Two colluding upstreams can then get a fabricated old tx or receipt served. See Empty-outside-retention waiver below.

Validation guardrails (startup):

  • Waiver floor. A policy that opts any entry into waiveAgreementOnMissingData or waiveAgreementOnEmptyOutsideRetention must also keep at least one never-waivable entry with minAgreement > 0. Without a floor, a unanimous abstention could waive the only quota and let a single unverified value win — the floor keeps at least one hard quorum. (A waiver-free config needs no floor.)
  • Empty-waiver needs a retention boundary. Opting any entry into waiveAgreementOnEmptyOutsideRetention requires retentionBlocks > 0 on that entry — without it, the waiver could never prove a block is outside retention and would silently never fire, so it's a startup error rather than a dropped intent.
  • Nesting. A policies["<name>"] value that defines customPolicy or policies is rejected — policies are a flat map, selected by the top-level selector only.
  • Shared S3 export path. The misbehavior rate limiter is global per upstream, but the exporter is per-policy. Two policies must not share a misbehaviorsDestination.path unless every sharing policy's filePattern contains both {timestampMs} and {policy} — otherwise two exporters can resolve the same S3 key in the same millisecond and a PutObject overwrites rather than appends. {policy} is hex-encoded (injective) so names that collide under filename sanitization (a/b vs a_b) still get distinct keys.
  • Shared punishMisbehavior. When policies is non-empty, the inline default and every named policy must carry identical punishMisbehavior (including all unset). The misbehavior limiter/sitout map is shared across policies, so differing disputeThreshold / disputeWindow / sitOutPenalty would silently first-touch-win.

Eval interface

(ctx) => string | null
// ctx.request   { method, blockNumber: number|null, network }
// ctx.user      { auth: string } | null   // auth strategy name (§4.3)
// ctx.upstreams  chainable collection of
//   { id, tags, health, blockAvailability: { lower, upper } | null }
//   health = { state: "healthy" | "unhealthy" | "cordoned",
//              cordonClasses: ("punishment"|"availability"|"operator")[],
//              cordonReason?: string }   // diagnostic only — never matched
//
// return "name"    → run policies[name]  (unknown name → fail closed to default + warn)
// return "default" → deliberately run the inline default policy
// return null      → selector failure: fails closed to the default (counted reason="null")

ctx.user is null for unauthenticated requests; guard with ctx.user?.auth === "…". blockNumber is null for methods with no numeric block (tx-hash / block-hash lookups) — the "unknown block" case. An upstream can hold several cordons at once, so cordonClasses is a set; state is "cordoned" whenever that set is non-empty, taking precedence over "unhealthy".

Sandbox. The eval runs in a bare VM that exposes only ctx and the helper API above — no env, process.env, or console. Since evalFunction is operator-writable config, it deliberately has no host-process env-var access.

Stdlib helpers (chainable, mirrors the selection-policy engine):

HelperBehavior
upstreams.withTag(t)Filter to upstreams whose tags include t (exact match).
upstreams.healthy()Keep only state == "healthy" — cordoned upstreams of any class are excluded.
upstreams.anyPunished()True if any network upstream holds a punishment cordon (consensus misbehavior sit-out), including nodes already excluded from ctx.upstreams by selection. A fail-closed gate: never downgrade while any node is punished.
upstreams.anyOperatorCordon()True if any network upstream holds an operator (admin) cordon, including nodes already excluded from ctx.upstreams by selection. Also a fail-closed gate — admin action is undifferentiated, so treat it as distrust.
upstreams.allUnavailable()True only if the set is non-empty and every member is unhealthy-but-uncordoned or holds availability cordons and nothing else. False on an empty set (a tag matching nothing is a config error, not an outage) and false if any member holds a punishment/operator/future class.
upstreams.allOperatorCordoned()True only if the set is non-empty and every member holds an operator (admin) cordon. False on an empty set. Use with withTag for a deliberate full-tag drain (e.g. upstreams.withTag("tier:primary").allOperatorCordoned() → allow a named looser policy); do not confuse with anyOperatorCordon() which pins fail-closed on a single admin cordon.
upstreams.noneSelected()True only if the set is non-empty and none of those members appear in the post-selection candidate list for this request (lag / diversity / ensureMin exclusion). False on an empty set. Use with withTag when a healthy tag side was fully excluded from the round (e.g. upstreams.withTag("tier:primary").noneSelected() → allow a named looser policy). Orthogonal to allUnavailable(), which inspects registry health rather than round membership.
upstreams.canServeBlock(n)Filter to upstreams whose configured blockAvailability range can serve block n. A non-numeric n or an upstream with no configured range is kept (attempt allowed).
user.authName of the authenticating auth strategy (plain string compare).

Worked examples

Reference selector — integrity grades + availability fallback. This is the canonical shape: fail-closed gates first (punishment / operator cordons pin the request to standard), then a strategy-gated availability fallback, then strategy-gated historical grades, then the strict default. Note the ordering — the gates come before any downgrade so an attacker who gets one node punished, or an admin who cordons a node, can never force a looser policy. For an opt-in full-tag admin drain via allOperatorCordoned(), see edge cases below (ordering matters):

projects[].networks[].failsafe[].consensus
erpc.yaml
consensus:  # the inline block IS the strict default policy the round falls back to  # (null/unknown selection, eval error, or timeout). It must be a complete,  # valid consensus config on its own — here identical to the "standard" policy.  maxParticipants: 3  agreementThreshold: 2  disputeBehavior: returnError  requiredParticipants:    - { tag: "type:internal", minParticipants: 1, minAgreement: 1 }    - { tag: "type:external", minParticipants: 2, minAgreement: 2 }  policies:    standard:                 # default / fail-closed      maxParticipants: 3      agreementThreshold: 2      disputeBehavior: returnError      requiredParticipants:        - { tag: "type:internal", minParticipants: 1, minAgreement: 1 }        - { tag: "type:external", minParticipants: 2, minAgreement: 2 }    standard-waive-missing:   # same composition; waiver for pruned/MissingData      maxParticipants: 3      agreementThreshold: 2      disputeBehavior: returnError      requiredParticipants:        - { tag: "type:internal", minParticipants: 1, minAgreement: 1, waiveAgreementOnMissingData: true }        - { tag: "type:external", minParticipants: 2, minAgreement: 2 }    historical:               # known-old block — external-only, skip internals      maxParticipants: 3      agreementThreshold: 2      requiredParticipants:        - { tag: "type:external", minParticipants: 2, minAgreement: 2 }    fallback:                 # internals genuinely down — healthy externals only      maxParticipants: 3      agreementThreshold: 2  customPolicy:    evalTimeout: 50ms    evalFunction: |      (ctx) => {        // Punishment or admin cordon anywhere → never downgrade.        if (ctx.upstreams.anyPunished()) return "standard"        if (ctx.upstreams.anyOperatorCordon()) return "standard"        const internals = ctx.upstreams.withTag("type:internal")        // Genuine outage (unhealthy or availability-cordoned only) → fallback (strategy-gated).        if (internals.allUnavailable())          return ctx.user?.auth === "consensus-fallback" ? "fallback" : "standard"        // Historical access is strategy-gated — unauthorized stay on standard (dispute on miss).        if (ctx.user?.auth === "consensus-waive") {          const n = ctx.request.blockNumber          if (n !== null && internals.canServeBlock(n).length === 0)            return "historical"              // known-old: external-only, no internal call          if (n === null)            return "standard-waive-missing"  // tx-hash etc.: try internals, waive MissingData        }        return "standard"      }

The three integrity grades this selector chooses between:

PolicyCompositionMissingData waiverWhen selected
standardinternal + externalNoDefault / fail-closed; unauthorized callers; any node punished or operator-cordoned
standard-waive-missingsame as standardYes (internals)Authorized role + unknown block (blockNumber == null, e.g. tx-hash)
historicalexternal onlyn/aAuthorized role + known-old block (internals can't serve it)

MissingData waiver. On a round running a waiving policy, an entry with waiveAgreementOnMissingData: true has its minAgreement quota released when at least minAgreement distinct upstreams matching its tag returned ErrEndpointMissingData. Sibling transport/infra errors do not block the count; a matching participant that returned data holds the quota. The waiver is safe by construction:

  • Threshold = minAgreement. Enough matching MissingData answers release the quota; fewer hold it strict → composition dispute.
  • Round-complete only. While any response can still arrive, a slower tagged upstream might return data, so the waiver holds off and the dispute stays provisional. Wait-cap cancel alone does not seal the bag — a result may already be buffered or the upstream call may ignore cancellation — so waivers stay fail-closed on a partial wait-capped collection.
  • Never-decisive-masking. The waiver only changes the outcome when releasing the entry makes the winner decisive; it can't hide a real disagreement.
  • Floor enforced. The standard-waive-missing policy keeps a never-waivable external floor (minAgreement: 2) — even with the internal quota waived, two externals must still agree.

Empty-outside-retention waiver (v1.1). Some upstreams answer with an empty result (JSON null, [], {}) rather than a missing-data error when asked for history they have legitimately pruned (e.g. post-EIP-4444 nodes drop old blocks). An entry with waiveAgreementOnEmptyOutsideRetention: true has its minAgreement quota released when at least minAgreement distinct tag-matching upstreams returned an empty result, no matching upstream returned a non-empty result, AND the block is proven outside retention. The block is the one the client's request names, or — for a hash lookup — the one the answer names once an upstream outside the agreeing set confirms it, so upstreams that agree on a fabricated block cannot supply their own proof. Sibling errors do not block the empty count; a matching non-empty result holds the quota. Proof order:

  1. Availability lower bound (preferred). Each matching empty upstream must expose a finite EvmBlockAvailabilityBounds lower bound (typically poller.latest − latestBlockMinus from blockAvailability.lower) and requestedBlock < lower for every such upstream. No network servedTip needed — this is the reliable head − retention signal when internals declare retention via availability bounds. Align latestBlockMinus with retentionBlocks.
  2. Chain-head fallback. Else head - requestedBlock > retentionBlocks, where head is the request's head Latest when evm.servedTip.quorum is "majority", otherwise the min corroborated poller latest among this round's EVM participants (≥2 positive latests; a single poller fails closed). A head under the default quorum is not trusted.

It shares the MissingData waiver's safety properties (minAgreement threshold, round-complete, never-decisive-masking, floor enforced) and adds a block proof as its anti-fabrication guard:

  • Recent nulls never waive. An empty result on a block inside retention / bounds fails the proof and holds the quota → dispute. So correlated externals returning null on recent data an internal should still serve cannot outvote that internal — the waiver only tolerates emptiness the retention model predicts.
  • Hash lookups need a confirmer outside the winners. eth_getTransactionByHash, eth_getTransactionReceipt, eth_getBlockByHash and other hash lookups name no block, so the waiver reads the block number and hash the agreeing answer names. The agreeing upstreams cannot vouch for that claim themselves: eRPC fetches eth_getBlockByNumber(<n>, false) from the round's other upstreams (abstainers first, then the rest, straight to the upstream with no retry or hedge). The first one that returns the block decides: its hash must match the answer's and the requested hash must be that block or one of its transactions. A mismatch fails closed at once; a null or an error moves on to the next. A node that pruned only its tx index still serves the block, so a geth internal can confirm its own pruned tx. When no other upstream holds the block, the quota holds → dispute. Requests that name a tag (latest) carry no proof → dispute.
  • blockEvidenceFields trusts the winners (unsafe opt-in). When every other upstream pruned the block too, nothing can confirm it. A policy that lists the method in blockEvidenceFields then judges retention against the block the agreeing answer names at those paths. The waiver no longer checks that claim: two agreeing upstreams can invent a tx or receipt in an old block and get it served. Use it only where those upstreams are already trusted to answer old data on their own. A non-winner that returns a different block still fails the round closed.
  • Fail-closed without proof. If neither avail-bound nor chain-head proof can be evaluated (no block; matching empties lack finite lower bounds and no majority tip or fewer than two participant pollers), the quota holds — the round disputes rather than waiving on faith. That case logs at info (warn for block_contradicted) and increments erpc_consensus_composition_waiver_unproven_total{reason} (no_request_block, block_unconfirmed, block_contradicted, no_avail_bound, or no_head). Mid-round deferral (responses still outstanding, collection not sealed) logs at debug only — it is not a metric.
  • Config-forced proof. retentionBlocks > 0 is mandatory when opting in (startup-enforced), so an operator cannot enable the waiver in a shape where it silently never fires.

Both waivers can coexist on one entry; when a round qualifies for both, the missing_data reason is reported. Each firing increments erpc_consensus_composition_waived_total{tag,reason} — reason is missing_data or empty_outside_retention. Alert on step changes: a waiver is a deliberate relaxation of a hard quorum.

Edge cases & gotchas

  1. The default policy must be the strictest. Everything fails closed to it — a throw, timeout, unknown name, or null return. If your default is permissive, every failure mode silently downgrades. Make it your tightest quorum.
  2. anyPunished first; partial operator cordons still pin. Put anyPunished() before any downgrade — a misbehavior sit-out must never tip a looser policy. anyOperatorCordon() still pins partial admin cordons to standard (one cordoned peer is undifferentiated distrust). Both helpers observe the network punishment / operator cordons, including upstreams already excluded from ctx.upstreams by selection (.removeCordoned()); they do not scan the candidate items list.
  3. allOperatorCordoned must run before anyOperatorCordon. A deliberate full-tag admin drain (e.g. upstreams.withTag("tier:primary").allOperatorCordoned()) is the opt-in exception that may select a looser named policy. If anyOperatorCordon() runs first, that drain branch is unreachable — when every member of the tag is operator-cordoned, anyOperatorCordon() is already true. Order: anyPunished → allOperatorCordoned (tag-scoped) → anyOperatorCordon → allUnavailable / other grades.
  4. Mixed drain fails closed. One member of the tag operator-cordoned and another only availability-cordoned / unhealthy: neither allUnavailable() (operator blocks the whitelist) nor allOperatorCordoned() (not every member has operator) is true, so the request stays on the fail-closed policy. Cordoning the remaining members (or waiting for a pure availability outage) is required to unlock the drain/fallback path.
  5. allUnavailable() is a whitelist of availability only. It is false on an empty set (a tag matching nothing is a config error, not an outage) and false if any member holds a punishment/operator/future cordon class. Only availability-cordoned or unhealthy-uncordoned members count as "unavailable".
  6. No mid-round switching. The policy is fixed before fan-out. If internals fail during a round, that round disputes under the policy it started with; the next request's eval sees the new health and can pick fallback. Recovery is automatic both ways.
  7. Never put the waiver on the fail-closed policy. waiveAgreementOnMissingData on standard defeats the point — the strict grade must dispute on missing internal data. Startup validation also requires a never-waivable minAgreement > 0 floor in any waiving policy.
  8. Strategy name never comes from headers. Only the authenticating strategy's configured name populates ctx.user.auth. Grant a grade by issuing credentials under a named strategy — consensus never parses tokens.
  9. Unknown returned name fails closed with a warn log. A typo in a returned policy name doesn't error the request — it silently resolves to the default and logs at warn. Watch consensus_policy_eval_failed_total{reason="unknown_name"}.
  10. evalTimeout discards the VM. A timeout interrupts and discards the borrowed VM (not returned to the pool); a replacement is rebuilt asynchronously. Sustained timeouts are an eval-complexity alarm — watch consensus_policy_eval_failed_total{reason="timeout"}. The pool grows on demand so capacity never bypasses eval.
  11. Two policies, one S3 path. Sharing a misbehaviorsDestination.path without both {timestampMs} and {policy} in every filePattern is a startup error — otherwise the per-policy exporters can overwrite each other's S3 objects even in the same millisecond.
  12. Named policies can't nest. A policies[<name>] value that sets customPolicy or policies is rejected at startup. Selection happens once, at the top level.
  13. default is reserved. policies["default"] is rejected at startup — that name labels the fail-closed inline default in the consensus_policy metric and the X-ERPC-Consensus-Policy header, so a named policy can't reuse it.
  14. Wait-cap does not seal for waivers. Composition disputes do not short-circuit while slots remain. When maxWaitOnResult / maxWaitOnEmpty fires, remaining slots are cancelled for latency but MissingData / empty-outside-retention waivers stay fail-closed on that partial bag — cancellation is not a round-completion barrier (buffered or cancel-ignoring results can still arrive with data that would dispute the waiver).
  15. Selector + shared cache: flush on first enable. Keys omit policy/auth and stored values carry no mark. Pre-selector warm entries remain Get-able until TTL after you add customPolicy. Flush/rotate the connector (or wait out max TTL). After enable, Forward refuses unmarked Sets on that network so sibling no-consensus routes cannot re-poison under defaultAllow. Source: erpc/networks.go.

Observability

MetricTypeLabelsWhen it fires
erpc_consensus_policy_selected_totalCounterproject, network, consensus_policyPer served round — the named policy the selector chose. Cardinality bounded by policy count (expected ≤ 10). Sustained non-default selection (e.g. fallback) is an alerting signal.
erpc_consensus_policy_eval_failed_totalCounterproject, network, reasonPer fail-closed selection where the eval ran and failed. reason: timeout, throw, unknown_name. Each falls back to the strictest default.
erpc_consensus_policy_eval_duration_secondsHistogramproject, networkPer-eval wall-clock duration. Watch the tail against evalTimeout.
erpc_consensus_composition_waived_totalCounterproject, network, tag, reasonQuota waived (missing_data / empty_outside_retention). Alert on step changes.
erpc_consensus_composition_waiver_unproven_totalCounterproject, network, reasonEmpty-outside-retention fail-closed for missing proof. reason: no_request_block (no block in the request or the answer), block_unconfirmed (no upstream outside the winners returned the answer's block), block_contradicted (one did, and it disagrees — alert on it), no_avail_bound, no_head. Mid-round deferral is debug-only, not counted.

Response header. X-ERPC-Consensus-Policy: <name> carries the chosen policy name so callers can detect a degraded grade (e.g. fallback) directly. It is emitted only when a selector is configured (consensus.customPolicy is set); plain inline-consensus setups never emit it.

Source code entry points

Related pages

  • Consensus — the rule-set each named policy configures (quorums, dispute behavior, requiredParticipants, misbehavior punishment and export). Read this first.
  • Authentication — auth strategies; set name on each strategy that should gate a consensus grade via ctx.user.auth.
  • Selection & scoring — the sibling freeform-JS engine that decides which upstreams are drawn for participant slots.