What Sentry's Seer Accuracy Number Actually Measures
The 94% is root cause accuracy, not fix accuracy. Sentry's own wording draws that line itself: Seer "can correctly identify the root cause over 94% of the time, write code to fix it, and open a pull request for you." Read that sentence carefully and the number attaches to the first clause, finding the cause, not the second one, writing the code. Sentry's changelog announcement and its product blog post both repeat the same phrase, "over 94% accuracy" modifying "root cause," and neither publishes a number for how often the resulting pull request is correct as opened.
That gap isn't a gotcha. Diagnosis and fix are different problems with different failure modes, and Sentry's own docs treat them as separate steps for a reason. What's worth understanding is why the two numbers can't be the same statistic, and what that means for how much review a Seer-generated PR actually needs before it ships.
What Seer does, step by step
Sentry's Autofix documentation lays out three stages, and they matter because each one carries a different kind of risk:
- Root cause analysis. Seer reads the issue, the stack trace, and the linked code, and traces back to the most likely culprit with supporting evidence. This is the step the 94% describes.
- Solution. Seer proposes an approach to fix the identified cause, shown as a live plan you can edit. The docs describe removing steps you disagree with or adding instructions like "request additional unit tests" before it writes any code.
- Code changes. Seer generates the patch and lets you draft a PR, check the diff out locally, or hand the root cause and solution off to an external coding agent (Claude Code, a Cursor Cloud Agent, or a GitHub Copilot Cloud Agent) to implement.
Notice that step 2 exists at all. If root cause identification were the whole job, there'd be nothing to edit between diagnosis and code. The solution step is Sentry building in a checkpoint precisely because "found the right cause" and "wrote the right fix for it" are separable outcomes, and it's the step most teams skip when they're moving fast.
Why a correct diagnosis doesn't guarantee a correct fix
Root cause analysis is a search problem. Given a stack trace, logs, and a codebase, the job is to find the function, the null check, the race condition responsible. Sentry's telemetry (traces, profiles, linked source) gives Seer unusually rich evidence for that search, which is plausibly why the accuracy is high. Writing the fix is a different problem. It has to handle the actual production distribution of inputs, not regress adjacent behavior, and match how the rest of the codebase does the same thing elsewhere. A model can point at exactly the right five lines and still patch them in a way that breaks a caller it didn't see, doesn't handle the edge case that triggered the bug in the first place, or fixes the symptom in this code path while an identical copy sits three files over.
None of this makes Seer's fixes untrustworthy as a starting point. It means the honest reading of "94% accuracy" is "you can trust the diagnosis a lot more than you can trust the diff," and the two need different amounts of scrutiny before you merge.
What this looked like for one team
At Loopline, a workflow-automation startup that connects to Sentry's Autofix on every new issue with more than five events in an hour, a payments-sync job started throwing intermittent KeyError exceptions. Nadia Solheim, the backend engineer who picked up the alert, watched Seer's root cause step land fast and correctly. A webhook payload from one partner occasionally omitted an amount field the code assumed would always be present.
Seer: Root cause identified.
sync_payment()assumespayload["amount"]exists. Partner webhooks from Meridian Pay omit this field when a transaction is refunded same-day. Proposed fix: default topayload.get("amount", 0)and skip zero-amount syncs.
Nadia agreed with the diagnosis immediately; it was exactly right, and would have taken her twenty minutes of log-diving to reach on her own. Her read on the proposed fix, from the team's Slack thread:
Nadia: Diagnosis is spot on. But defaulting to 0 and skipping the sync means we silently drop same-day refunds from the ledger instead of reconciling them. That's worse than the crash. I want it to flag these for manual review, not skip them.
She used Seer's solution-step editing to add that instruction, regenerated the fix, and merged the second version. The root cause never changed and didn't need to. The fix took a domain-specific correction that no accuracy statistic about diagnosis could have told her she'd need. That's what the gap looks like in practice. Not "Seer was wrong," but "Seer was right about the cause, and the first fix still needed a human who knew what same-day refunds mean for the ledger."
“we need SSO before rollout”
“any update on single sign-on?”
“SSO came up twice on this call”
Linear issue, quotes attached
insight report in NotionThe ceiling on reading every diff yourself
At low issue volume, Nadia's workflow holds up fine. An engineer reads the diagnosis, reads the diff, and decides. The trouble starts as volume grows, because the review bottleneck isn't the diagnosis, which Seer gets right most of the time and fast. It's the fix, which needs someone who knows the domain to catch cases like Loopline's refund logic, and that judgment doesn't parallelize the way pattern-matching does.
Teams handle this two ways, and both have a ceiling. Merging Autofix PRs on green CI without a human read trades review time for risk, since tests that pass don't catch a fix that's semantically wrong in a way Loopline's refund case was. Reviewing every single one preserves safety but doesn't scale past a handful of engineers watching a growing issue stream, and it treats a crash affecting one internal test account the same as one hitting forty paying customers, because Sentry's issue view doesn't carry that distinction on its own.
The second failure is the more expensive one, because review time is the scarce resource and it's being spent uniformly regardless of stakes. What breaks first is prioritization, the ability to tell which Autofix suggestions deserve five minutes of careful reading from the ones that are safe to wave through.
What closes the prioritization gap
That prioritization gap is a customer-context problem, not a debugging problem, which is why it sits outside what Seer or any AI debugger is built to solve. Modem connects to Sentry alongside Slack, support tools, and your issue tracker, and its Sentry integration lets the agent search issues, read stack traces and sample events, and pull up the customer feedback that arrived about the same error, next to everything else those customers have said elsewhere. That turns "Seer proposed a fix for issue #4021" into "Seer proposed a fix for the bug three enterprise accounts already flagged in Slack this week," which is the fact that should decide whether Nadia reads the diff carefully or waits for the next sweep. Modem can also resolve, assign, or reprioritize the Sentry issue itself, always with your approval on each action. Modem is the company writing this guide, so factor that into how you read the recommendation; best tools to connect Sentry errors to customer feedback lines up the alternatives for this specific job if you want a comparison.
If your fix-review bottleneck is specifically about handing generated fixes to a coding agent for further work, triggering a Cursor agent directly from a Sentry error covers that handoff in more detail; the same diagnosis-versus-fix distinction applies there too.
The takeaway
The 94% is Sentry's own stated accuracy for finding the right root cause, backed by the telemetry Seer has access to, and that's the only thing it's fair to quote it for. Don't extend it to the PR that shows up in your repo. Read the diagnosis with reasonable confidence, then read the diff the way you'd read one from a fast, literal-minded new hire who's never seen your data before.
This week, before you let another Autofix PR merge on green CI alone, pick the one signal you already have for customer weight, whether that's account tier, event volume, or a Slack mention, and use it to decide which diffs get the slow read first. That's a five-minute habit, not a new tool, and it's the fix for the failure mode in this guide.
