PAR-328
Add XFF-based IP/UA denylist to economy-explorer middleware
Ability to block abusive clients (IP/CIDR + user-agent) from the economy-explorer web tier, so a repeat of the Blockantir scrape can be cut off without taking the site offline.
Background
2026-07-16 → 07-20 a scraper (Blockantir-BalanceCollector/1.0, IP 136.107.101.57) made ~3.25M unauthenticated requests (~10 req/s, ~30x baseline) paginating /accounts?sort=balance + /firms — a full balance/firm harvest. It pinned the shared DB host and put the explorer into a crash loop (~21 restarts over the 4 days). Stopped on its own after the in-game ban; there is currently no way to block the client at the network/app layer. Detection/alerting is now in place (PrometheusRule → Discord); this is the missing response control.
Why not a CiliumClusterwideNetworkPolicy (the obvious first instinct)
Investigated — a CCNP is a no-op for this topology:
- The Gateway is a hostNetwork Cilium Envoy that proxies every connection, so the app pod sees Envoy as the L3 source. The real client IP only survives in
X-Forwarded-For(that's where the access log reads it). A pod-targetedfromCIDRpolicy matches nothing. - The only CCNP that would see the client IP is a host-firewall policy, but
enable-host-firewallis off, and enabling it + selecting the host endpoint flips host ingress to default-deny — requires allowlisting SSH/kube-API/node-to-node or you lock both nodes out. Not worth it for this.
So the effective + low-risk layer is the app itself, where the XFF IP + UA already exist.
Proposed implementation
- Denylist middleware in economy-explorer (sits alongside/after the existing access-log middleware), returns
403early for matching requests. - Match on client IP/CIDR (from
X-Forwarded-For, trusting the gateway hop) and User-Agent (exact + substring). - Config-driven, not code — env var (e.g.
BLOCKED_IPS/BLOCKED_UAS, comma-separated, CIDR-aware) so updates are a values change in k8s-gitops, no rebuild. - Emit a structured log line (
kind:"blocked") per rejected request so blocks are visible in Loki and can feed a "still being probed" metric. - Seed with the known abuser: IP
136.107.101.57, UABlockantir-BalanceCollector/1.0. - Apply to both tenants (democracycraft + statecraft) via the shared image; keep values per-overlay.
Notes
- Deprioritised deliberately — parked behind the current large PR; the deterrence messaging already implies this capability exists.
- Follow-on (separate tesks if wanted): rate-limit unauth
/accounts+/firmspagination and cap page depth (the deep-OFFSET queries are what caused themax_statement_timetimeouts).
Comments
No comments yet.
Activity
- tesks created the issue