HTTP Ingestion degradation caused by Cloudflare KV outage
Resolved
Jul 18 at 11:34am EDT
The issue has been resolved by releasing a hotfix bypassing KV when unavailable.
Affected services
HTTP Ingestion
Created
Jul 18 at 10:42am EDT
Cloudflare is currently experiencing issues with their KV product (https://www.cloudflarestatus.com/incidents/k5nmwlwzntcj), which impacts our ability to process requests. We have seen peak failure rates of around 1% and roughly 0.0001% ongoing failure rates. Failures are not uniformly distributed over data centers, which means depending on where your requests originate from, you may not see any error at all or an error rate above 1%.
We are working on a hotfix to bypass KV in the meantime. We'll also publish a postmortem once we've managed to bring down the error rate to 0.
Affected services
HTTP Ingestion