Decebal | Rust for the AI execution layer ⚙️

Prod scenario question for senior engineers: a webhook provider retries for 24 hours. Your endpoint is not idempotent. You discover duplicates in orders/invoices and some downstream side effects already happened. What do you do to fix correctness safely? Constraints: you can’t pause retries easily, events arrive out of order, you only have provider event_id + timestamp, and customer support needs a clear story. I’d look for: add an idempotency key store (event_id unique), make the handler transactional (write idempotency record + business write in one DB txn), and design rollback/compensation for already-applied side effects (refunds, voids, reverse ledger entries). Also: how do you backfill and dedupe existing bad data, add observability (high-cardinality event_id logs, duplicate rate metric, dead letter queue), and ship it behind a feature flag without making the incident worse?