How to turn a Sentry webhook into a reviewed fix PR

Quick answer

Create a webhook automation in Nori, then add its URL as the action of a Sentry alert rule. When the alert fires, Nori starts an agent on a fresh cloud machine with your repositories and the alert payload. The agent finds the failing code, writes a test that reproduces the error, fixes it, opens a pull request, and posts in a Slack thread asking a person to review. Nothing merges without that review.

What you get

Most new errors follow the same path. Someone sees the alert, opens the issue, reads the stack trace, finds the commit, and then decides whether it is a five-minute fix or a real problem. The first twenty minutes are the same every time, and they are the part an agent can do before anyone looks.

With this setup, the alert arrives in Slack together with an investigation and, when the cause is clear, a pull request. The on-call engineer starts at the review step.

• The agent investigates first. It reads the stack trace, checks the suspect commits, and reproduces the error in a test.
• The fix is a normal pull request. Feature branch, tests, description, your branch protection and CI.
• A person decides. The agent asks for review in Slack. It never merges and never deploys.
• The thread stays live. Reply in the thread to ask for a different fix, and the same agent continues.

What you need before you start

• A Nori organization with the GitHub app connected and the affected repositories in the workspace.
• The Nori Slack app installed, and the bot invited to the channel where alerts should land, for example #eng-alerts.
• Permission in Sentry to create an internal integration and an alert rule. Sentry documents the payload format in its webhook documentation.
• Optional: the Sentry MCP server connected in Nori, so the agent can read the full issue, breadcrumbs and event history instead of only the payload.

Step by step: from alert to pull request

1. Create the automation in Nori. In the dashboard, open Automations and create a new one with the webhook type. Or ask in Slack: @nori create a webhook automation that investigates Sentry alerts and delivers to #eng-alerts. Choose Slack thread delivery and the #eng-alerts channel.
2. Write the prompt. Describe the task only: investigate, reproduce, fix if the cause is clear, open a pull request, ask for review. The full prompt is in the worked example below. Nori adds the alert payload to the prompt for each run and marks it as untrusted data.
3. Copy the webhook URL and secret. Nori shows a URL and a secret for the automation. Treat both as credentials.
4. Create a Sentry internal integration. In Sentry, go to Settings, then Custom Integrations, and create an internal integration. Paste the Nori webhook URL as its webhook URL and enable it as an alert rule action.
5. Create the alert rule. In Sentry Alerts, create an issue alert for the project. A good first rule: "A new issue is created" in the production environment, with the action set to send a notification through your new integration. Start narrow, with one project and new issues only.
6. Test it. Use Run now on the automation in Nori, or send a test notification from the Sentry alert rule. Check that a thread appears in #eng-alerts and that the agent's reply names the right repository.
7. Review the first real run together. When the first real alert produces a pull request, read the agent's investigation as a team. Tighten the prompt where it guessed, and save anything general as a shared memory, for example "errors in payments/ always need a second reviewer".

After that the loop runs unattended: alert, session, investigation, pull request, review request.

Worked example: a TypeError in checkout

This example follows one alert from payload to merged fix. The service, file names and identifiers are illustrative.

The webhook payload (excerpt)

Sentry sends a JSON body when the alert rule fires. The agent receives it with the prompt. The fields that matter most are the title, the culprit, the release and the link back to the issue.

{
  "action": "triggered",
  "data": {
    "triggered_rule": "New issue in production",
    "event": {
      "title": "TypeError: Cannot read properties of undefined (reading 'currency')",
      "culprit": "formatLineItem(src/checkout/summary.ts)",
      "level": "error",
      "environment": "production",
      "release": "web-app@2026.11.10-3",
      "issue_id": "5120934417",
      "web_url": "https://sentry.io/organizations/example/issues/5120934417/",
      "tags": [["transaction", "/checkout"], ["browser", "Chrome 131"]]
    }
  }
}

The automation prompt

A Sentry alert fired. The payload is attached.

1. Identify the repository from the release name and culprit path.
2. Read the stack trace. If the Sentry integration is connected,
   fetch the full issue and the latest event.
3. Find the commit that introduced the error. Check commits between
   the previous release and the release in the payload first.
4. Write a failing test that reproduces the error.
5. If the cause is clear and the fix is small, fix it, run the test
   suite, and open a PR from a feature branch. Put the Sentry issue
   link in the PR description. Never push to main. Never merge.
6. If the cause is not clear, or the fix touches payments/ or auth/,
   do not open a PR. Report what you found instead.

Reply with: one-line cause, suspect commit, PR link or the reason
there is none, and what the reviewer should check. Keep it under
10 lines.

The Slack message

Eleven minutes after the alert, this appears in #eng-alerts. Nori opens the thread with a short header, and the agent's final reply follows.

#eng-alerts

Nori  14:03
Automation: Sentry alert to fix PR (webhook)

  Nori  14:14
  Cause: formatLineItem() reads item.price.currency, but gift-card
  line items have no price object since commit 3f9a2c1
  ("Add gift cards to cart", merged yesterday, release 2026.11.10-3).
  PR: web-app #1287 "Handle line items without a price in checkout
  summary"
  - New test: summary.test.ts "formats a gift-card line item"
    (fails on main, passes on the branch)
  - Falls back to the order currency when item.price is missing
  - 1,412 tests pass
  Reviewer should check: is the order currency the right fallback
  for gift cards bought in another currency?
  Sentry: https://sentry.io/organizations/example/issues/5120934417/

  Marta  14:21
  @nori gift cards are always in the order currency, so that is
  fine. But show "Gift card" as the label, not the SKU.

  Nori  14:27
  Pushed a follow-up commit to #1287: the label is "Gift card" for
  line items of type gift_card. Test updated. Ready for review.

  Marta  14:35
  Approved and merged. !done

What happened in those eleven minutes

Review steps and guardrails

An agent that reacts to production alerts needs clear limits. These are the ones to set on day one.

• Review is required. Keep branch protection on. The agent opens pull requests; a person approves and merges. Put the rule in the prompt and in your organization skillset.
• Sensitive paths report only. List directories where the agent must not propose a fix without a person, such as payments/, auth/ and database migrations.
• Payloads are data, not instructions. Error messages can contain text that users typed. Nori passes the payload to the agent as untrusted data, so text inside an error message is not treated as a request.
• Limit the alert rule, not the agent. Control volume in Sentry: new issues only, one environment, a frequency limit on the rule. Each alert that fires starts one session on one machine.
• Keep the secret private. Store the webhook URL and secret only in Sentry's integration settings.
• Check the run history. Every run is recorded on the automation with its result and transcript, so you can see what the agent looked at when a fix was wrong or when it chose not to open a pull request.

Changes to the automation itself are also controlled. When a member who is not an admin edits the prompt, the change waits for admin approval and the previous version keeps running.

Variations on the same pattern

Once the first webhook works, the same automation shape covers other sources.

• Grafana or Alertmanager. Send the alert to a second webhook automation whose prompt starts from dashboards and recent deploys instead of a stack trace.
• CI failures on main. Have CI call a webhook when main turns red, so an agent bisects and opens a revert or a fix.
• A nightly sweep. Add a cron automation that reviews unresolved Sentry issues from the last day and starts the fix automation for the ones that look tractable.
• Ticket first. If your process needs a ticket for every production error, have the agent create it through your tracker's MCP server and link it in the pull request.

The agent that runs can be Jiro, Claude Code, Codex or any other harness Nori supports, on your own subscription or keys. For how the thread side works, see the AI coding agent for Slack.

Frequently asked questions