Paradaux
Issues

PAR-151

0

Transaction event feed: cursor pull endpoint + webhooks (substrate for player banks)

DoneHighUnassignedTreasury API

Goal

Give player-built finance infra (banks, funds) a way to consume new ledger transactions event-driven, via the always-on REST API tailing the shared ledger. Substrate only — not network gameplay. Implements the external-consumer half of PAR-75.

(b) Cursor pull feed — GET /api/v1/accounts/{id}/transactions?since=<txn_id>&limit=N returns settled txns after the cursor, forward-ordered, capped (default 100 / max 1000), with nextCursor + hasMore.

(a) Webhooks — self-service POST/GET/PATCH/DELETE /api/v1/webhooks scoped to the API key; a background dispatcher tails the ledger into a durable outbox and POSTs HMAC-signed events with retries.

Design (approved plan)

  • Schema (economy-schema V13): webhook_subscription, webhook_delivery (UNIQUE (subscription_id, txn_id)), webhook_cursor.
  • Single dispatcher via prod Redis (already exists) + Lettuce distributed lock; also enables the Redis-backed rate limiter in prod (fixes per-pod limit gap).
  • Correctness: surface only settled rows (settlement_time <= NOW() - INTERVAL :lag SECOND) to avoid AUTO_INCREMENT visibility skips; outbox dedups on (subscription_id, txn_id).
  • Security: SSRF guard on player URLs (https, reject loopback/private/link-local/metadata, re-check at send); HMAC-SHA256 signature (X-Treasury-Signature: sha256=…) + delivery-id; rate-limited endpoints; cap subs/key; auto-disable after K failures.
  • Reuse: tesks ActivityDispatcher (monotonic poll), tesks GithubSignature (HMAC), treasury TaxWebhookServiceImpl (HttpClient), rest-api VerifiedToken/LettuceBucketProvider.

Repos

  • economy-schema — V13 migration (additive).
  • treasury-rest-api — cursor endpoint, webhook mgmt API, dispatcher, config, HMAC util, Redis lock.
  • k8s-gitops — wire RATE_LIMIT_REDIS_* + WEBHOOK_* onto prod overlays (DC + SC). No Redis provisioning (already exists).

Rollout

V13 migrate → rest-api develop→uat → prod env wiring → deploy DC + SC. Verify pull (forward/cursor/settled), webhook single-delivery across 2 replicas, failover, retry/backoff, SSRF rejection, global rate limiting.

Plan: parallel-whistling-swing.md. Aligns with the network hands-off philosophy (substrate, not gameplay).

Resources

Comments

tesks · Jun 14, 2026, 3:23 PM

Post-release defect found + fixed: tenant-shared dispatcher lock

The webhook dispatcher had stalled in prod. DC's cursor was frozen at txn 1,776,403 since ~12:30 (≈13k txns behind) while the pods were healthy.

Root cause: WebhookRedisLock used the default fixed key treasury:webhook:dispatcher, and DC and SC rest-api both point at the same redis.production. So all four pods (DC×2 + SC×2) contended for one global lease — only one tenant's dispatcher could run, and whichever tenant lost the race had its dispatcher starved. An SC pod held the lock, so DC was dead.

Fix (gitops, config-only — no rebuild): set TREASURY_WEBHOOK_LOCK_KEY per overlay (…:dispatcher:democracycraft / …:statecraft); Spring relaxed-binds it to treasury.webhook.lock-key. Each tenant now holds its own lease. Pushed to gitops main, rolled. Verified: DC's cursor immediately caught up from 1,776,403 → current (now 1 behind latest = the settlement-lag buffer, updating every few seconds).

Still outstanding for full rollout: the feature has 0 subscriptions / 0 deliveries — never exercised end-to-end in prod. Next: create a test subscription (via /me/webhooks or the REST API) pointing at a controllable endpoint and confirm a signed delivery + the delivery-health view.

Activity

  • ParadauxIO linked a pull request — PR #19 merged — Release: webhooks documentation (player guide + admin page) (PAR-162)
  • ParadauxIO linked a pull request — PR #19 open — Release: webhooks documentation (player guide + admin page) (PAR-162)
  • ParadauxIO linked a pull request — PR #19 open — Release: admin mode + webhooks documentation (PAR-161, PAR-162)
  • ParadauxIO linked a pull request — PR #19 open — Release: webhooks documentation (player guide + admin page) (PAR-162)
  • tesks commented
  • ParadauxIO linked a pull request — PR #12 merged — Release: self-service webhook management at /me/webhooks (PAR-153)
  • ParadauxIO linked a commit — Commit c174c3b — Self-service webhook management at /me/webhooks (PAR-153)
  • ParadauxIO linked a pull request — PR #12 open — Release: self-service webhook management at /me/webhooks (PAR-153)
  • ParadauxIO linked a commit — Commit c174c3b — Self-service webhook management at /me/webhooks (PAR-153)
  • ParadauxIO linked a commit — Commit 068a565 — Merge pull request #12 from MCCitiesNetwork/develop
  • ParadauxIO linked a commit — Commit 2544b25 — Add transaction event feed: cursor pull endpoint + webhooks (PAR-151)
  • ParadauxIO changed status to Status → Done
  • tesks changed status to Status → Pending Release
  • ParadauxIO linked a commit — Commit 2544b25 — Add transaction event feed: cursor pull endpoint + webhooks (PAR-151)
  • tesks created the issue