Kubernetes

How it works

kube/erpc.yml defines five Kubernetes resources: a erpc Namespace, a Deployment (single replica by default, safe to scale), a ConfigMap erpc-config with an embedded example erpc.yaml, a ClusterIP Service (port 80 → 4000), and a PodMonitor for Prometheus Operator. kube/postgres.yml adds a PersistentVolumeClaim (500 Gi), Deployment, Service, and Secret for a companion PostgreSQL cache store.

Config delivery. The Deployment mounts the ConfigMap as a volume at /erpc.yaml. To change config, update the ConfigMap and run kubectl rollout restart deployment/erpc — ConfigMap updates alone do not trigger a rolling restart.

Graceful drain. eRPC handles SIGTERM via signal.NotifyContext. After receiving SIGTERM:

1. Healthcheck starts returning 503 — readiness probe fails, pod is removed from Service endpoints.
2. server.waitBeforeShutdown elapses — in-flight requests drain.
3. HTTP server calls Shutdown.
4. server.waitAfterShutdown elapses — lets kube-proxy/Envoy close lingering TCP connections.
5. Process exits.

Set terminationGracePeriodSeconds >= waitBeforeShutdown + waitAfterShutdown + server.maxTimeout. 180s is a safe default for most workloads.

Prometheus scraping. The pod template carries annotations and a PodMonitor (monitoring.coreos.com/v1) is created in the erpc namespace:

# Pod annotations (kube/erpc.yml:L20-L23)
prometheus.io/scrape: "true"
prometheus.io/port: "4001"
prometheus.io/path: "/metrics"

The PodMonitor is labeled release: monitoring, targets the port named http, scrapes every 10s with a 5s timeout, and matches pods labeled app: erpc.

Probes. The /healthcheck HTTP endpoint drives all three probe types. See Healthcheck for the full evaluation strategy.

startupProbe:
  httpGet:
    path: /healthcheck
    port: 4000
  initialDelaySeconds: 10
  periodSeconds: 10
  failureThreshold: 6       # 60s window to absorb slow upstream init
 
readinessProbe:
  httpGet:
    path: /healthcheck
    port: 4000
  periodSeconds: 5
  failureThreshold: 2       # removed from endpoints after ~10s
 
livenessProbe:
  tcpSocket:
    port: 4000
  initialDelaySeconds: 30
  periodSeconds: 10
  failureThreshold: 3

After SIGTERM, the readiness probe fails immediately by design, signalling the orchestrator to drain traffic before the in-flight drain completes.

Config schema

No erpc.yaml fields are specific to the Kubernetes deployment layer. The following env vars are relevant in the pod spec:

CLI subcommands available in the container binary (cmd/erpc/main.go (opens in a new tab)):

Reference resource specs from the kube manifest (the only code-grounded recommendations):

Worked examples

1. Minimal production Deployment patch. Add GOMEMLIMIT, remove the CPU limit to avoid CFS throttling, and wire the Downward API for POD_NAME:

env:
  - name: GOMEMLIMIT
    value: "2700MiB"
  - name: GOGC
    value: "40"
  - name: POD_NAME
    valueFrom:
      fieldRef:
        fieldPath: metadata.name
resources:
  requests:
    memory: "3Gi"
    cpu: "2"
  limits:
    memory: "3Gi"
    # no cpu limit — avoids CFS throttling on Go goroutines

2. Graceful drain tuning for a 30s maxTimeout. Set these in erpc.yaml and match terminationGracePeriodSeconds in the pod spec:

# erpc.yaml (inside ConfigMap)
server:
  waitBeforeShutdown: 30s
  waitAfterShutdown: 30s

# Deployment pod spec
terminationGracePeriodSeconds: 180

30s drain + 30s TCP teardown + 30s max request timeout + buffer = 180s is safe for most workloads.

3. Triggering a config reload. After updating the ConfigMap with a new erpc.yaml, restart the Deployment — ConfigMap edits alone are not detected at runtime:

kubectl apply -f kube/erpc.yml
kubectl rollout restart deployment/erpc -n erpc
kubectl rollout status deployment/erpc -n erpc

4. Horizontal scaling. eRPC has no replica-count constraint. Scale freely — no sticky sessions, no leader election needed for the proxy path:

kubectl scale deployment/erpc --replicas=5 -n erpc

Best practices

• Set GOMEMLIMIT=2700MiB (or 90% of your memory limit) as a container env var. The reference manifest omits it — without it, Go's GC may allow the heap to reach the hard 3 Gi limit and trigger an OOM kill instead of a managed GC cycle.
• Remove resources.limits.cpu. Go's work-stealing scheduler is sensitive to Linux CFS throttling; a hard CPU limit raises tail latency without reducing memory usage. Set resources.requests.cpu for scheduler placement but leave limits absent.
• Pin the image tag. The reference manifest uses ghcr.io/erpc/erpc:latest — pin to a specific tag or SHA for reproducible production rollouts.
• Pin PostgreSQL too. kube/postgres.yml uses postgres:latest — pin a specific PostgreSQL version and digest for production.
• Always kubectl rollout restart after ConfigMap changes. A ConfigMap update in place does not trigger pod restarts; traffic keeps hitting the old config until you explicitly restart.
• Size terminationGracePeriodSeconds generously. It must be ≥ waitBeforeShutdown + waitAfterShutdown + server.maxTimeout; otherwise Kubernetes sends SIGKILL before the drain completes, dropping active requests.
• Allow egress to upstream RPC endpoints. eRPC makes outbound connections on 443/TCP and 80/TCP to upstream providers, plus 5432/TCP (PostgreSQL) and 6379/TCP (Redis) for cache backends. Include these in your NetworkPolicy.

Edge cases & gotchas

1. No GOMEMLIMIT in reference manifest: With 3Gi limit, Go GC may trigger OOM. Set GOMEMLIMIT=2700MiB.
2. CPU limits cause throttling: Hard CPU limits engage Linux CFS throttling on Go goroutines, raising tail latency. Omit resources.limits.cpu.
3. terminationGracePeriodSeconds too small: If shorter than waitBeforeShutdown + waitAfterShutdown + maxTimeout, Kubernetes sends SIGKILL before drain completes, dropping active requests.
4. Startup probe initialDelaySeconds too low: If upstreams are slow to respond at startup, the probe fails and Kubernetes restarts the pod in a loop. Use a startupProbe with generous failureThreshold (6 × 10s = 60s window).
5. ConfigMap update alone does not restart pods: Run kubectl rollout restart deployment/erpc or checksum the ConfigMap in pod template annotations to force a rolling update.
6. Image tag :latest is not reproducible: Reference manifest pins :latest — operators should pin a specific tag or SHA.
7. kube/postgres.yml PVC is ReadWriteOnce: Multi-zone clusters may fail to reschedule the pod to a different availability zone. Ensure your StorageClass supports cross-zone access or use a managed database instead.
8. No official Helm chart: The repository provides raw manifests under kube/. Community-maintained charts may exist on Artifact Hub.
9. Network policy egress: eRPC makes outbound connections to upstream RPC endpoints (443/TCP, 80/TCP) and optional cache backends (5432/TCP PostgreSQL, 6379/TCP Redis). Allow ingress on 4000/TCP from app pods and 4001/TCP from the Prometheus scraper.
10. kube/postgres.yml uses postgres:latest: Pin a specific PostgreSQL version and digest for production.
11. gRPC and HTTP share port 4000 by default: server.grpcPortV4 and server.httpPortV4 both default to 4000. If you override the gRPC port to a different value, a second listener is bound and you must add a corresponding containerPort and Service port entry. The manifest only exposes 4000 and 4001.
12. pprof binary present but never invoked by default: The image ships /erpc-server-pprof (built with -tags pprof) alongside the default /erpc-server. To enable profiling on port 6060, override the container command to /erpc-server-pprof; the default CMD runs the non-pprof binary.

Observability

Scraped by Prometheus via PodMonitor at 4001/metrics every 10s. Key metrics for Kubernetes-level alerting:

The PodMonitor is labeled release: monitoring and must match your Prometheus Operator's serviceMonitorSelector / podMonitorSelector. [kube/erpc.yml:L145-160]

Source code entry points

• kube/erpc.yml (opens in a new tab) — Namespace, Deployment, ConfigMap, Service, PodMonitor
• kube/postgres.yml (opens in a new tab) — PVC (500Gi), Deployment, Service, Secret for PostgreSQL
• kube/erpc.yml:L33-L36 (opens in a new tab) — reference resource sizing: 3Gi memory, 2 CPU
• kube/erpc.yml:L145-L160 (opens in a new tab) — PodMonitor spec (10s scrape interval, 5s timeout)
• kube/erpc.yml:L54-L130 (opens in a new tab) — embedded ConfigMap with Arbitrum example config
• cmd/erpc/main.go:L72-L74 (opens in a new tab) — SIGTERM graceful shutdown via signal.NotifyContext
• cmd/erpc/main.go:L279-L294 (opens in a new tab) — config file search order (12 paths)

Related pages

• Docker — single-container run and local compose stack with monitoring.
• Railway — one-click managed deploy.
• Healthcheck — probe strategy and evaluation logic.
• Monitoring & metrics — full metrics reference for dashboards and alerting.
• Authentication — secure the ingress before exposing the Service externally.