Rate Limiters

How it works

Every inbound request passes through up to four sequential rate-limit checks in this order: auth → project → network → upstream. Each check looks up the budget ID configured at that attachment point, calls TryAcquirePermit, and rejects with a layer-specific error if any matching rule is OVER_LIMIT. The layers are independent — a budget shared across layers will have its counter incremented once per layer per request, so avoid attaching the same budget ID at both project and network scope.

Rule matching uses exact string equality first, then glob (WildcardMatch). If multiple rules match a single method, all are evaluated in parallel goroutines; the first OVER_LIMIT result short-circuits the rest.

The backing store is either an in-process sharded memory cache (64 FNV-1a shards, lazy-expiry cleanup) or a Redis-backed Envoy ratelimit store. In memory mode, each eRPC process has independent counters — the effective cluster limit is maxCount × instanceCount. In Redis mode, counters are shared across all instances, but the Redis client fail-opens while the connection is being established, and again on timeout or admission pressure.

Redis fail-open paths. Three distinct conditions cause the Redis rate limiter to allow a request instead of blocking:

1. Not yet connected — getCache() returns nil; TryAcquirePermit returns (true, nil) before even reaching the Redis call. This window lasts from startup until Redis first connects.
2. Admission semaphore full — each budget has a bounded channel (max(256, connPoolSize × 32) slots). When full, the request is allowed immediately without spawning a goroutine, and erpc_rate_limiter_remote_admission_shedded_total increments. This design was added after the 2026-05-07 incident where 25k+ goroutines accumulated during a Redis contention spike.
3. Per-call timeout (getTimeout, default 1s) — doLimitWithTimeout returns after the timer fires, allows the request, and increments erpc_rate_limiter_failopen_total{reason="limit_timeout"}. The spawned goroutine continues running and completes the Redis write — the counter is updated even though the caller was not blocked.

Auto-tuner. When an upstream has rateLimitBudget set and rateLimitAutoTune is absent from config, eRPC automatically populates rateLimitAutoTune with defaults and sets enabled: true. This means adding rateLimitBudget to an upstream implicitly enables the auto-tuner unless you opt out with rateLimitAutoTune.enabled: false.

The tuner samples errorCount / totalCount over each adjustmentPeriod (default 1 minute). If error rate exceeds errorRateThreshold (default 10%), maxCount is multiplied by decreaseFactor (default 0.95). If error rate is exactly 0, maxCount is multiplied by increaseFactor (default 1.05). Partial error rates in the range (0, threshold] produce no change. The tuner adjusts rules on the budget — which may be shared with other upstreams. If two upstreams share a budget and both have auto-tuners, the tuner that fires last wins; there is no coordination.

Envoy status codes. The Envoy ratelimit library produces three response codes per descriptor. eRPC only inspects OVER_LIMIT:

Where nearLimitThreshold = floor(maxCount × nearLimitRatio). Default ratio 0.8: for maxCount=100, requests 81–100 are allowed (near-limit stats only); request 101 is blocked. eRPC never surfaces NEAR_LIMIT to callers. Source: upstream/ratelimiter_budget.go:L306

TryAcquirePermit flow. (1) If cache is nil → fail-open (true, nil). (2) GetRulesByMethod returns empty → allow (no rules match). (3) If any matching rule uses perIP/perUser/perNetwork and req == nil → return error. (4) Single matching rule: evaluate directly without spawning a goroutine. (5) Multiple rules: fan-out goroutines, atomic.Bool short-circuits on first OVER_LIMIT. Source: upstream/ratelimiter_budget.go:L153-244

Credit-weighted counting. By default each acquisition consumes 1 hit (HitsAddend: 1). When an upstream sets rateLimitCountMode: credit, the upstream layer passes the request's pre-flight vendor credit-unit estimate as the hit weight, so a heavy eth_getLogs (e.g. 60 CU on Alchemy) drains the budget 60× faster than a flat request count. A 0-CU method (e.g. eth_chainId) is admitted for free and consumes nothing. This weight rides the standard Envoy HitsAddend, so it works on both the memory and Redis stores. Only the upstream scope supports credit counting — project/network/auth scopes always count 1 per request. Source: upstream/upstream.go:L110-131

Cache key format. {cacheKeyPrefix}{domain}_{descriptor_entries_concatenated}_{time_window_bucket}. The erpc_rl_ prefix ensures isolation from other Envoy ratelimit users on the same Redis. Source: upstream/ratelimiter_registry.go:L76

Memory store eviction. Each shard prunes lazily: on every DoLimit call, if the shard's last-prune timestamp differs from the current unix second, all buckets with expiry ≤ now are deleted. This is O(#buckets) per shard — not O(#keys). Source: upstream/ratelimiter_mem_cache.go:L91-100

Admission cap derivation. remoteAdmissionCap = max(256, connPoolSize × 32). The ×32 multiplier accounts for radix's implicit pipelining window (up to 32 commands per connection). Pool of 8 → cap 256 (floor). Pool of 64 → cap 2048. Source: upstream/ratelimiter_registry.go:L281-291

Auto-tuner minimum sample floor. The tuner skips adjustment when totalCount < 10 — insufficient signal. Counters are reset at the start of maybeAdjust regardless, so a sustained pattern of exactly 9 requests per period will perpetually reset the clock. Source: upstream/ratelimiter_autotuner.go:L83-142

Auto-tuner requires a budget. An upstream without rateLimitBudget set never gets an auto-tuner instance, even if rateLimitAutoTune is explicitly configured. Source: upstream/upstream.go:L1322

Shared budgets. A budget ID is a plain string. Any number of attachment points — across projects, networks, upstreams, and auth strategies — can reference the same ID, sharing the same counters. In Redis mode this sharing extends across eRPC instances.

waitTime deprecation. The rules[].waitTime field is explicitly ignored: validation emits a warning log and discards the value. There is no queue or hold logic in the rate limiter. Remove waitTime from any existing config to suppress the warning.

Config schema

rateLimiters (top-level)

rateLimiters.budgets[]

Attachment points

User-level budget (set by auth strategy on the user object) overrides strategy-level rateLimitBudget. Source: auth/authorizer.go:L119-122

upstreams[].rateLimitAutoTune

Available only on UpstreamConfig. Implicitly created with enabled: true when rateLimitBudget != "" and rateLimitAutoTune is absent. Explicitly set rateLimitAutoTune.enabled: false to opt out.

Worked examples

All patterns below are distilled from real production fleets; comments explain the non-obvious choices.

1. Redis store with tuned pool for a multi-replica fleet. In production, each eRPC machine handles 50k+ rate-limit checks per minute; the default connPoolSize: 8 queues requests and accumulates goroutines under load. A larger pool with a generous-but-bounded getTimeout keeps the 99th-percentile check under 1ms without leaking goroutines during Redis slowdowns:

rateLimiters:  store:    driver: redis    redis:      uri: "${REDIS_URL}"      # 200ms gives the radix pipeline window (5ms) plus RTT comfortable      # headroom while still fast-failing during real outages.      # Never set below ~50ms on pipelined clients: the pipeline window      # itself takes ~5ms and a too-tight cap triggers constant fail-opens.      getTimeout: 200ms      # Default is 8 — not enough for 12+ machine fleets.      # With radix implicit pipelining (window=5ms, limit=32),      # effective in-flight = poolSize × 32: poolSize=64 supports      # ~2k concurrent checks/machine without queueing.      connPoolSize: 64    # Use distinct prefixes per environment — two deployments sharing the    # same Redis and same prefix will silently merge their counters.    cacheKeyPrefix: "erpc_prod_"  budgets:    - id: per-client      rules:        - method: "*"          maxCount: 200          period: second          perIP: true

2. Tiered per-user budgets via JWT claim. Production edge deployments serve customers across multiple subscription tiers. The JWT claim "rlm" names the budget to use per token; the auth strategy resolves it per request without any project/network attachment. Both perUser: true and perIP: true can be combined for combinatorial (user, IP) keys:

rateLimiters:  budgets:    # Free tier: 6 000 req/min total per user    - id: edge-tier-6krpm-total-unlimited-per-ip      rules:        - method: "*"          maxCount: 6000          period: minute          perUser: true    # Paid tier: 60 000 req/min total + 500 req/min per IP ceiling    - id: edge-tier-60krpm-total-500rpm-per-ip      rules:        - method: "*"          maxCount: 60000          period: minute          perUser: true        - method: "*"          maxCount: 500          period: minute          perUser: true          perIP: trueprojects:  - id: main    auth:      strategies:        - type: jwt          jwt:            # Token payload contains "rlm": "edge-tier-60krpm-total-500rpm-per-ip"            rateLimitBudgetClaimName: rlm

3. Upstream outbound budget with aggressive auto-tuner. Production uses fast adjustmentPeriod: 30s (not the 1-minute default) and an asymmetric decreaseFactor: 0.7 so the tuner cuts hard during 429 storms but restores slowly with increaseFactor: 1.1. Always set minBudget: 1 — the default is 0 which lets the auto-tuner drive maxCount to zero (always-blocked):

rateLimiters:  budgets:    - id: alchemy-global      rules:        - method: "*"          maxCount: 300          period: secondprojects:  - upstreams:      - endpoint: "https://eth-mainnet.g.alchemy.com/v2/${ALCHEMY_KEY}"        rateLimitBudget: alchemy-global        rateLimitAutoTune:          enabled: true          # Faster reaction than the 1-minute default — providers can          # 429-storm and recover inside a single minute          adjustmentPeriod: 30s          errorRateThreshold: 0.1          # Restore gently: 1.1× per period rather than default 1.05×          increaseFactor: 1.1          # Cut aggressively: 0.7× per period (not the conservative 0.95×)          # to shed load fast during a 429 storm          decreaseFactor: 0.7          # CRITICAL: 0 means no floor — auto-tuner can drive to always-blocked          minBudget: 1          maxBudget: 100000

4. Network-level admission cap on heavy methods to protect proxy capacity. After an incident where a single user's burst of eth_getBlockReceipts calls filled all Fly proxy connections for minutes, production now hard-caps just the methods empirically observed stacking up connections. Light methods like eth_blockNumber are intentionally not restricted — the cap is surgical, not global:

rateLimiters:  budgets:    - id: cronos-mainnet      rules:        # Healthy peak for receipt methods is ~5 RPS fleet-wide;        # 100 RPS gives 20× headroom while still protecting proxy capacity.        - method: "eth_getBlockReceipts|eth_getTransactionReceipt"          maxCount: 100          period: second        # Trace/debug requests are large and slow — 30 RPS matches        # their typical low volume and prevents runaway archive crawls.        - method: "debug_*|trace_*|arbtrace_*"          maxCount: 30          period: secondprojects:  - id: main    networks:      - evm:          chainId: 25        alias: cronos-mainnet        # Network-level: the (N+1)th excess request gets 429-ed in <1ms        # instead of holding a proxy slot for up to 16s on a slow upstream.        rateLimitBudget: cronos-mainnet

5. Isolated budget per vendor to prevent cross-chain auto-tune interference. When multiple chains share a single vendor budget (e.g. alchemy-global) and the auto-tuner reacts to errors on one chain, it adjusts the budget for all chains. Isolating a high-traffic chain onto its own budget prevents a polygon 429 storm from cutting Ethereum's limit:

rateLimiters:  budgets:    # Shared default for most chains    - id: alchemy-global      rules:        - method: "*"          maxCount: 300          period: second    # Isolated: polygon errors won't affect ethereum's tuned limit    - id: alchemy-polygon      rules:        - method: "*"          maxCount: 300          period: secondprojects:  - upstreams:      - id: eth-mainnet-alchemy        endpoint: "https://eth-mainnet.g.alchemy.com/v2/${ALCHEMY_KEY}"        rateLimitBudget: alchemy-global      - id: polygon-alchemy        endpoint: "https://polygon-mainnet.g.alchemy.com/v2/${ALCHEMY_KEY}"        # Isolated budget: auto-tuner reacting to polygon errors        # will not lower the limit for eth-mainnet-alchemy        rateLimitBudget: alchemy-polygon

Request/response behavior

Full error code reference:

• rateLimiters absent from config = full passthrough. When rateLimiters is omitted entirely, bootstrap() logs "no rate limiters defined which means all capacity of both local cpu/memory and remote upstreams will be used" at debug level and returns immediately — no budgets are registered and every request is allowed. Source: upstream/ratelimiter_registry.go:L48-51
• No response carries a Retry-After header from eRPC.

Best practices

• Set minBudget on auto-tuner to at least 1. Default minBudget: 0 lets the auto-tuner drive maxCount to zero (always-blocked). A floor of 10–20 preserves minimal throughput during sustained provider pressure.
• Use driver: redis with distinct cacheKeyPrefix per environment. Memory mode makes each replica's limit independent (effectiveLimit = maxCount × replicaCount). Two Redis deployments sharing a prefix merge counters across environments.
• Don't attach the same budget at both project and network scope. Each layer increments the counter independently — one request consumes two quota units.
• Prefer method: "eth_getLogs" rules over wildcard for expensive methods. Wildcard rules matched by the auto-tuner affect every method's maxCount, not just the one that triggered the 429.
• Opt out of implicit auto-tune when you manage limits manually. Adding rateLimitBudget to an upstream silently enables the auto-tuner. Add rateLimitAutoTune.enabled: false to any upstream where you want stable, manually-tuned limits.
• Never use period: "2h" or other compound durations. Only the seven canonical period names are accepted; "2h" parses successfully but is rejected at unmarshal time.
• Watch erpc_rate_limiter_failopen_total. Sustained reason="limit_timeout" spikes mean Redis is too slow; consider raising getTimeout or switching to memory mode for low-latency deployments.

Edge cases & gotchas

1. waitTime is silently ignored with a warning. The field exists for backward compatibility. Validation logs "rateLimiter.*.budget.rules.*.waitTime is deprecated and will be ignored" and discards the value. No queue or hold behaviour exists. Source: common/validation.go:L214-216
2. Auto-tuner is implicitly enabled when rateLimitBudget is set. SetDefaults creates RateLimitAutoTuneConfig{enabled: true} when rateLimitBudget != "" and rateLimitAutoTune == nil. Explicitly set rateLimitAutoTune.enabled: false to prevent auto-adjustment. Source: common/defaults.go:L1674
3. Shared budget + two auto-tuners = last-writer wins. If two upstreams share the same budget id, each has its own auto-tuner. Both independently call AdjustBudgetByFactor on the same rule objects; no locking between tuners. Source: upstream/upstream.go:L1321-L1339 (opens in a new tab)
4. Same budget at project + network layers double-counts. Each check increments the counter independently. One request consumes two quota units when the same budget id is attached at both layers. Source: erpc/projects.go:L270 (opens in a new tab), erpc/networks.go:L1112 (opens in a new tab)
5. ErrUpstreamRateLimitRuleExceeded does NOT return HTTP 429 on the wire. The HTTP status switch at erpc/http_server.go:1484-1490 lists auth/project/network errors but omits the upstream error code. An upstream-level block wraps into ErrNoUpstreamsAvailable and returns HTTP 200 with a JSON-RPC error body. Source: erpc/http_server.go:L1484-L1490 (opens in a new tab)
6. ErrRateLimitBudgetNotFound is a runtime error, not a startup error. A typo in rateLimitBudget is not caught at config validation; the error fires on the first request to hit that code path. Source: upstream/ratelimiter_registry.go:L224 (opens in a new tab)
7. maxCount: 0 blocks everything. Envoy sets overLimitThreshold=0; any increment produces OVER_LIMIT immediately. To disable a budget, remove the rateLimitBudget reference rather than setting maxCount: 0. Source: upstream/ratelimiter_budget.go:L109-L119 (opens in a new tab)
8. perIP / perUser / perNetwork fall back to a global counter when the value is empty or "n/a". IP-less callers (e.g. non-HTTP transports) all share one counter, potentially allowing far more traffic than intended for that scope. Source: upstream/ratelimiter_budget.go:L256-L264 (opens in a new tab)
9. Memory store is per-process. In a multi-replica deployment, each instance has independent counters; effective cluster limit = maxCount × instanceCount. Use driver: redis for shared counting. Source: upstream/ratelimiter_mem_cache.go:L1-L30 (opens in a new tab)
10. Redis cacheKeyPrefix collision. Two eRPC deployments sharing a Redis instance with the same prefix will merge counters across deployments. Use distinct prefixes per environment. Source: upstream/ratelimiter_registry.go:L75-L76 (opens in a new tab)
11. store block is auto-created with driver="memory" when absent. Declaring rateLimiters: without store.driver: redis silently uses memory even if Redis fields are present. No warning is emitted. Source: common/defaults.go:L2763-2769
12. Wildcard rules affect auto-tuner scope broadly. A rule method: "eth_*" matched by eth_call will have its maxCount adjusted whenever any eth_* method hits the error threshold — not only the method that triggered the change. Source: upstream/ratelimiter_autotuner.go:L113 (opens in a new tab)
13. Period "2h" is rejected. time.ParseDuration("2h") succeeds but 2*time.Hour does not equal any canonical period. Only exact matches to second, minute, hour, day, week, month, year (and their aliases) are accepted. Source: common/config.go:L1924-1944
14. Redis DoLimit goroutines outlive their callers. After getTimeout fires, the goroutine continues running and completes the Redis write. The counter is updated even though the caller was allowed through. This prevents counter drift but means Redis receives more writes than the caller observes. Source: upstream/ratelimiter_budget.go:L349 (opens in a new tab)
15. Auto-tuner adjustmentPeriod clock resets on insufficient samples. lastAdjustments[method] is updated at the start of maybeAdjust regardless of whether an actual adjustment occurs. A sustained pattern of exactly 9 requests per period (below the 10-sample floor) will perpetually reset the clock and prevent any evaluation. Source: upstream/ratelimiter_autotuner.go:L94 (opens in a new tab)
16. Auto-tuner only increases when errorRate == 0.0 exactly. Partial improvement in (0, threshold] produces no adjustment — budget will not grow back until error rate reaches exactly zero. Source: upstream/ratelimiter_autotuner.go:L107-L112 (opens in a new tab)
17. minBudget: 0 is no floor. Default minBudget=0 means no clamping floor — the auto-tuner can drive maxCount to 0, making the budget always-blocked. Set to at least 1. Source: common/config.go:L1057
18. nearLimitRatio has no effect on blocking decisions. eRPC only inspects OVER_LIMIT; NEAR_LIMIT is never surfaced. The field only affects internal Envoy stats counters, which are not exported to Prometheus. Source: upstream/ratelimiter_budget.go:L306 (opens in a new tab)
19. No enforcement on nil request when all rules are non-scoped. If req == nil is passed and all matching rules have perIP=false, perUser=false, perNetwork=false, validation does not reject the nil request — it proceeds with the method-only descriptor. Source: upstream/ratelimiter_budget.go:L188-191
20. ErrRateLimitRuleNotFound has no production call sites. The error type is defined in common/errors.go but no production code calls NewErrRateLimitRuleNotFound — it appears reserved for future use. Source: common/errors.go:L1723
21. Admission cap floor of 256 applies regardless of pool size. Even with connPoolSize: 1, the admission cap is 256. Small pools on low-traffic instances still allow 256 concurrent Redis calls per budget before shedding. Source: upstream/ratelimiter_registry.go:L281-291
22. IAM connections for the rate-limiter store recycle at a flat 11h. Unlike the main Redis connector (which adds ±30m jitter), the rate-limiter IAM pool uses a flat PoolMaxLifetime(11h) — radix lacks a jitter knob. All connections created at startup will expire in the same ~11h window. For a small pool this is acceptable; connections dialed under load spread naturally. Source: data/redis_ratelimiter_iam.go:L70-76

Observability

Trace spans: RateLimiter.TryAcquirePermit (attrs: budget, method) and RateLimiter.DoLimit (attrs: budget, method, scope, result, one per rule evaluated). Source: upstream/ratelimiter_budget.go:L161, upstream/ratelimiter_budget.go:L279

Log messages.

Source code entry points

• upstream/ratelimiter_registry.go:L1-L300 (opens in a new tab) — Registry creation, Redis connect (background retry), budget initialization, GetBudget, admission cap: remoteAdmissionCap
• upstream/ratelimiter_budget.go:L1-L460 (opens in a new tab) — RateLimiterBudget: TryAcquirePermit, parallel rule evaluation, evaluateRule, doLimitWithTimeout, AdjustBudgetByFactor
• upstream/ratelimiter_autotuner.go:L1-L145 (opens in a new tab) — RateLimitAutoTuner: RecordSuccess, RecordError, maybeAdjust with error-rate logic
• upstream/ratelimiter_mem_cache.go:L1-L130 (opens in a new tab) — In-process sharded bucketed counter cache; 64 FNV-1a shards, lazy O(#buckets) cleanup
• common/config.go:L1806-L1950 (opens in a new tab) — RateLimiterConfig, RateLimitBudgetConfig, RateLimitRuleConfig, RateLimitPeriod enum + marshal/unmarshal
• common/defaults.go:L2763-L2820 (opens in a new tab) — SetDefaults for all rate limit types; auto-tune defaults; implicit auto-tune population
• common/validation.go:L199-L216 (opens in a new tab) — Budget/rule validation; waitTime deprecation warning
• upstream/ratelimiter_leak_test.go (opens in a new tab) — Goroutine leak tests reproducing the 2026-05-07 Redis contention incident and admission panic safety

Related pages

• Auth — authentication strategies that set per-user rateLimitBudget via JWT claims or per-secret overrides.
• Projects — project-level rateLimitBudget attachment.
• Hedge — hedge bursts add request volume; combine with upstream rate limiters to cap the cost on expensive vendors.
• Retry — retries also add upstream request volume; upstream budgets apply per-attempt.
• Survive provider outages — how rate limiting and failsafe cooperate under provider pressure.