Event ingestion pipeline
Replaced a nightly batch job with a streaming pipeline handling 40M events per day. Cut end-to-end latency from 6 hours to under 2 seconds and removed the single point of failure that caused two outages the year before.
- 6h → 2slatency
- -71%infra cost
- 0data loss