Noise on delta, last 7 days
How loud each level is against what it is allowed to be. Nobody reads a channel that cries wolf. ?Replayed from this cluster's finding events exactly as the level rules apply them: each event's level at its own moment, a reopen within a day at no higher level is the same incident, an escalation inside a level is not news. critical and high that land within 30 minutes of each other are one incident — one message.
critical
0.0/week
0 in 7 d · budget ≤ 1/week
high
9.0/week
9 in 7 d · budget ≤ 5/week
endpoint.4xx_spike 6
error.novel 2
pod.cannot_start 1
medium
23.0/week
23 in 7 d · budget grouped
error.novel 13
scaler.failing 3
shape.degrading 2
pod.evicted 1
pod.cannot_start 1
node.unhealthy 1
job.failing 1
pod.oom 1
low
39.0/week
39 in 7 d · budget digest
node.unhealthy 27
anomaly.neo4j 3
outlier.service 3
budget.burning 2
shape.degrading 1
n1.fanout 1
shape.churn 1
pod.unstable 1
critical + high as messages (30-minute correlation): 4.0/week · medium as hourly messages: 11 in 7 d · damped as the same incident: 36
critical + high incidents 4
| high | 1d ago | execsvc: a new error shape, 101× in the last hour, never seen in 90 days (no result from branch evaluation) |
| high | 2d ago | execsvc: a new error shape, 122× in the last hour, never seen in 90 days (failed to evaluate and raise auto query for participant question — no ) |
| high | 2d ago | schema-migrate (tilda) image pull failing for at least 65 min — 1 pod |
| high | 4d ago | onboardingsvc GET /api/myprofile/v2: permission denied — 40 of 40 calls (100 %; 401×40) in the last 15 min — 180× its usual 0.2 + 5 more in the same half hour |
What each rule sent
| Rule | messages | by status |
|---|