enrollsvc: a new error shape, 6× in the last hour, never seen in 90 days (could not process study smo org binding change — Neo4jError: Neo.Clien)
warning
Error never seen before
test26
resolved
impact 32
Owner: Suresh Boddapati (backup: Sambasiva Bandarupalli)
opened 2d ago · last observed 2d ago · observed 1× · #17431 · 6b373554ce040abc25c1
What happened template
enrollsvc logged “could not process study smo org binding change — Neo4jError: Neo.ClientError.Schema.ConstraintValidationFailed (Node(?) already exists with label `ChartReviewSt” 6 times in the last hour and never before in 90 days on test26 (first seen 2026-11-16 14:02 UTC, raised at internal/pubsub.go:116). Example trace: a57aa8f28bee673ba5c66583d14d1aca.
An error shape appearing for the first time in 90 days, five or more times in an hour.
The facts
| service | enrollsvc |
| caller | internal/pubsub.go:116 |
| shape | could not process study smo org binding change — Neo4jError: Neo.ClientError.Schema.ConstraintValidationFailed (Node(?) already exists with label `ChartReviewStudy` and properties `orgid` = '867ea61b- |
| count | 6 |
| error_class | neo4j |
| first_seen | 2026-11-16 14:02:25 UTC (2d ago) |
| sample | could not process study smo org binding change — Neo4jError: Neo.ClientError.Schema.ConstraintValidationFailed (Node(12620503) already exists with label `ChartReviewStudy` and properties `orgid` = '867ea61b-3f |
| trace | a57aa8f28bee673ba5c66583d14d1aca |
| shape_hash | 1dff992f0f61fd47 |
| count_is_floor | no |
| severity_capped | critical → warning: test26 is a test-tier cluster |
| discriminator | 1dff992f0f61fd47 |
| window | 2026-11-16 13:48:32 UTC → 2026-11-16 14:48:32 UTC |
What is failing in enrollsvc — the bad events, decomposed
“Bad events” are HTTP 5xx answers from enrollsvc. In this finding's window (2026-11-16 13:48:32 UTC → 2026-11-16 14:48:32 UTC) there were 0; every line links to the grouped errors behind it.
The error shapes behind them
| 1566× | could not add chart review study node while binding study to org — Neo4jError: Neo.ClientError.Schema.ConstraintValidati… since 2026-10-09 11:44:30 UTC |
| 1081× | could not add chart review study node while binding study to org — Neo4jError: Neo.ClientError.Schema.ConstraintValidati… since 2026-11-16 06:13:23 UTC |
| 19× | failed to create node — Neo4jError: Neo.ClientError.Schema.ConstraintValidationFailed (Node(12181421) already exists wit… since 2026-10-30 19:23:45 UTC |
| 19× | failed to process db result — Neo4jError: Neo.ClientError.Schema.ConstraintValidationFailed (Node(12181421) already exis… since 2026-10-30 19:23:45 UTC |
| 19× | could not process study smo org binding change — Neo4jError: Neo.ClientError.Schema.ConstraintValidationFailed (Node(121… since 2026-10-30 06:38:56 UTC |
The story — 1 enrollsvc change landed in the hour before this opened — adjacency, not proof. Everything below is enrollsvc only.
- 2026-11-16 14:00:00 UTClatency enrollsvc p95 peaked at 4.52 s
- 2026-11-16 14:01:04 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/enrollsvc by hand (kubectl/v1.30.2 (darwin/arm64) kubernetes/3968350,gzip(gfe))
- 2026-11-16 14:48:32 UTCfinding · warning enrollsvc: a new error shape, 8× in the last hour, never seen in 90 days (could not process study smo org binding change — Neo4jError: Neo.Clien)
- 2026-11-16 14:48:32 UTCfinding · warning enrollsvc: a new error shape, 8× in the last hour, never seen in 90 days (could not process study smo org binding change — Neo4jError: Neo.Clien)
- 2026-11-16 14:48:32 UTCfinding · warning enrollsvc: a new error shape, 8× in the last hour, never seen in 90 days (could not process study smo org binding change — Neo4jError: Neo.Clien)
- 2026-11-16 14:48:32 UTCfinding · warning enrollsvc: a new error shape, 8× in the last hour, never seen in 90 days (could not process study smo org binding change — Neo4jError: Neo.Clien)
- 2026-11-16 14:48:32 UTCfinding · warning enrollsvc: a new error shape, 7× in the last hour, never seen in 90 days (could not process study smo org binding change — Neo4jError: Neo.Clien)
- 2026-11-16 14:48:32 UTCfinding · warning enrollsvc: a new error shape, 7× in the last hour, never seen in 90 days (could not process study smo org binding change — Neo4jError: Neo.Clien)
- 2026-11-16 14:48:32 UTCfinding · warning enrollsvc: a new error shape, 8× in the last hour, never seen in 90 days (could not process study smo org binding change — Neo4jError: Neo.Clien)
- 2026-11-16 14:48:32 UTCfinding · warning enrollsvc: a new error shape, 8× in the last hour, never seen in 90 days (could not process study smo org binding change — Neo4jError: Neo.Clien)
- 2026-11-16 14:48:32 UTCthis finding enrollsvc: a new error shape, 6× in the last hour, never seen in 90 days (could not process study smo org binding change — Neo4jError: Neo.Clien)
History
- 2026-11-16 14:48:32 UTCopened x-ray · severity=warning title=enrollsvc: a new error shape, 6× in the last hour, never seen in 90 days (could not process study smo org binding change — Neo4jError: Neo.Clien) finding={'id': 17431, 'fingerprint': '6b373554ce040abc25c1', 'kind': 'error.novel', 'severity': 'warning', 'cluster': 'test26', 'subject': 'enrollsvc', 'title': 'enrollsvc: a new error shape, 6× in the last hour, never seen in 90 days (could not process study smo org binding change — Neo4jError: Neo.Clien)', 'facts': {'service': 'enrollsvc', 'caller': 'internal/pubsub.go:116', 'shape': "could not process study smo org binding change — Neo4jError: Neo.ClientError.Schema.ConstraintValidationFailed (Node(?) already exists with label `ChartReviewStudy` and properties `orgid` = '867ea61b-", 'count': 6, 'error_class': 'neo4j', 'first_seen': 1794837745, 'sample': "could not process study smo org binding change — Neo4jError: Neo.ClientError.Schema.ConstraintValidationFailed (Node(12620503) already exists with label `ChartReviewStudy` and properties `orgid` = '867ea61b-3f", 'trace': 'a57aa8f28bee673ba5c66583d14d1aca', 'shape_hash': '1dff992f0f61fd47', 'window': [1794836912, 1794840512], 'count_is_floor': False, 'severity_capped': 'critical → warning: test26 is a test-tier cluster', 'discriminator': '1dff992f0f61fd47'}, 'evidence': [{'label': 'Errors, grouped', 'href': '/errors?cluster=test26&from=1794836912&to=1794840512&service=enrollsvc'}, {'label': 'Logs', 'href': '/logs?cluster=test26&from=1794836912&to=1794840512&service=enrollsvc&severity=ERROR'}, {'label': 'Example trace', 'href': '/trace/a57aa8f28bee673ba5c66583d14d1aca?cluster=test26'}], 'provenance': {'source': 'live', 'collected_at': None, 'age_seconds': None, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': 'Cloud Logging, last hour, grouped by (service, caller, shape)'}, 'impact': 32.5, 'status': 'open', 'first_seen': 1794840512, 'last_seen': 1794840512, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 1, 'narrative': None, 'narrative_kind': None, 'discriminator': '1dff992f0f61fd47', 'level': 'medium', 'told_level': None, 'episode_at': 1794840512, 'chat_thread': None, 'kind_label': 'Error never seen before', 'group': 'Errors', 'age_s': 82, 'stale_s': 82}
- 2026-11-16 15:08:36 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 17431, 'fingerprint': '6b373554ce040abc25c1', 'kind': 'error.novel', 'severity': 'warning', 'cluster': 'test26', 'subject': 'enrollsvc', 'title': 'enrollsvc: a new error shape, 6× in the last hour, never seen in 90 days (could not process study smo org binding change — Neo4jError: Neo.Clien)', 'facts': {'service': 'enrollsvc', 'caller': 'internal/pubsub.go:116', 'shape': "could not process study smo org binding change — Neo4jError: Neo.ClientError.Schema.ConstraintValidationFailed (Node(?) already exists with label `ChartReviewStudy` and properties `orgid` = '867ea61b-", 'count': 6, 'error_class': 'neo4j', 'first_seen': 1794837745, 'sample': "could not process study smo org binding change — Neo4jError: Neo.ClientError.Schema.ConstraintValidationFailed (Node(12620503) already exists with label `ChartReviewStudy` and properties `orgid` = '867ea61b-3f", 'trace': 'a57aa8f28bee673ba5c66583d14d1aca', 'shape_hash': '1dff992f0f61fd47', 'window': [1794836912, 1794840512], 'count_is_floor': False, 'severity_capped': 'critical → warning: test26 is a test-tier cluster', 'discriminator': '1dff992f0f61fd47'}, 'evidence': [{'label': 'Errors, grouped', 'href': '/errors?cluster=test26&from=1794836912&to=1794840512&service=enrollsvc'}, {'label': 'Logs', 'href': '/logs?cluster=test26&from=1794836912&to=1794840512&service=enrollsvc&severity=ERROR'}, {'label': 'Example trace', 'href': '/trace/a57aa8f28bee673ba5c66583d14d1aca?cluster=test26'}], 'provenance': {'source': 'live', 'collected_at': None, 'age_seconds': None, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': 'Cloud Logging, last hour, grouped by (service, caller, shape)'}, 'impact': 32.5, 'status': 'resolved', 'first_seen': 1794840512, 'last_seen': 1794840512, 'resolved_at': 1794841716, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 1, 'narrative': 'enrollsvc logged “could not process study smo org binding change — Neo4jError: Neo.ClientError.Schema.ConstraintValidationFailed (Node(?) already exists with label `ChartReviewSt” 6 times in the last hour and never before in 90 days on test26 (first seen 2026-11-16 14:02 UTC, raised at internal/pubsub.go:116). Example trace: a57aa8f28bee673ba5c66583d14d1aca.', 'narrative_kind': 'template', 'discriminator': '1dff992f0f61fd47', 'level': 'medium', 'told_level': None, 'episode_at': 1794840512, 'chat_thread': None, 'kind_label': 'Error never seen before', 'group': 'Errors', 'age_s': 1243, 'stale_s': 1243}
Check it yourself
Every link carries the finding's own time window, so it shows tomorrow what it shows now.
Actions
Sign in (IAP) to acknowledge, snooze or resolve.
Where the numbers come from
| source | live |
| collected_at | None |
| age_seconds | None |
| stale | False |
| shown | None |
| total | None |
| truncated | False |
| coverage_from | None |
| retain_days | None |
| note | Cloud Logging, last hour, grouped by (service, caller, shape) |
| detector | errors_novel |