regsvc (tilda) image pull failing for at least 117 min — 1 pod
Pod cannot start: regsvc (tilda) image pull failing for at least 117 min — 1 pod. Facts: namespace tilda, workload regsvc, why image pull failing, pending_minutes 117, since 1794904982.
What X-ray is prepared to defend about test27 right now — each one a claim
with its numbers, the evidence to check it, and what normal looks like. Detectors last ran
never.
?A finding is opened by a detector — a statistical test over the stored rollups, or a
live check of a source — and stays ONE row while the condition persists: it updates,
escalates, and resolves itself fifteen minutes after the detector stops seeing it.
Acknowledge to say you have it; snooze to hide it for a while; nothing here is a
threshold somebody configured.
Every card is a claim X-ray can back with numbers you can click through to. Findings rank by severity first, then by an impact score (severity band + the subject's share of the cluster's traffic + how long it has been open), so the top of the page is the thing to look at first.
What this page will not tell you
Pod cannot start: regsvc (tilda) image pull failing for at least 117 min — 1 pod. Facts: namespace tilda, workload regsvc, why image pull failing, pending_minutes 117, since 1794904982.
The background job pubsub.onboarding-test27-regsvc-sub on test27 failed 19 runs recently (528 failure lines; lines within one run count once), detected via its ERROR-severity log lines. A job that logs its failures below ERROR severity — a WARNING carrying an error field — is invisible to the novel-shape detector and to the page's own failed count; this detector reads the error field itself, which is why the finding now appears. The sample failure is in the facts.
The background job studioai.unknown on test27 failed 5 runs recently (532 failure lines; lines within one run count once), detected via its ERROR-severity log lines. A job that logs its failures below ERROR severity — a WARNING carrying an error field — is invisible to the novel-shape detector and to the page's own failed count; this detector reads the error field itself, which is why the finding now appears. The sample failure is in the facts.
Pod cannot start: rag-agent (tilda) image pull failing for at least 118 min — 1 pod. Facts: namespace tilda, workload rag-agent, why image pull failing, pending_minutes 118, since 1794904947.
docsvc differs from the other 7 services on 2 measures over six hours: Neo4j queries per request 107.5 against the peers' 5.8 (13.1 MADs above); p95 latency (ms) 2.77 s against the peers' 451 ms (3.6 MADs above). Peer comparison is structural more often than it is news — a service that does more work per request will sit here for as long as it does — which is why this is informational unless it is about failures.
docproc's Neo4j queries per request is 53.3 against the 7 other services' median of 5.8 (6.1 robust MADs above, six-hour window). This compares the service with its peers, not with its own history — the seasonal detector does that.
studiosvc's Neo4j load is 4,216 q/min on test27, above a normal of 0 q/min ± 20 q/min for this hour of the week (4 of the last 4 five-minute buckets, z = 210.8, 5 weeks of history). Whether users noticed depends on which endpoints carried it — the service page shows the split.
studiosvc differs from the other 7 services on 3 measures over six hours: Neo4j queries per request 161.0 against the peers' 5.8 (19.9 MADs above); p95 latency (ms) 9.24 s against the peers' 451 ms (13.7 MADs above); p99 latency (ms) 18.30 s against the peers' 1.01 s (12.1 MADs above). Peer comparison is structural more often than it is news — a service that does more work per request will sit here for as long as it does — which is why this is informational unless it is about failures.
studiosvc on test27 averaged 270.0 foreground Neo4j queries per API request over the last hour (180,895 queries for 670 calls). Dozens of queries per request is the N+1 signature: a round-trip per row instead of one query. The Journey page shows which screens pay for it, request by request.
the whole test27 cluster's Neo4j load is 4,441 q/min on test27, above a normal of 239 q/min ± 184 q/min for this hour of the week (4 of the last 4 five-minute buckets, z = 22.8, 5 weeks of history). Whether users noticed depends on which endpoints carried it — the service page shows the split.