feat: Flue-based triage agent for docs issues#17811
Open
sergical wants to merge 16 commits into
Open
Conversation
Adds a Flue agent that classifies incoming GitHub issues using existing label taxonomy (Platform, Product Area, Team, Impact, Effort) and generates structured triage reports. Runs on issue open events and via manual workflow_dispatch trigger. - .flue/agents/triage-issue.ts: Agent entry point using Sonnet - .agents/skills/classify-docs-issue.md: Classification skill with full label mapping from issue templates - .flue/AGENTS.md: Project context for the agent - .github/workflows/flue-triage-issue.yml: GitHub Actions workflow - DRY_RUN=true by default, set DRY_RUN=false for live runs Co-Authored-By: Claude <noreply@anthropic.com>
- Remove issues.opened trigger — manual dispatch only until prompt injection detection is added - Remove LINEAR_API_KEY from agent sandbox — Linear ticket creation will be a separate post-agent step - Downgrade permissions to issues: read (no write needed yet) - Fix concurrency group to match dispatch-only trigger Co-Authored-By: Claude <noreply@anthropic.com>
GitHub API calls are now wrapped as custom ToolDefs (fetch_issue, search_issues) that run in the Node process, not the sandbox. The agent never sees GH_TOKEN — it calls the tools by name and gets structured results back. Also fixes import path to @flue/runtime/client (what the CLI bundles internally, vs @flue/sdk/client which requires separate install). Co-Authored-By: Claude <noreply@anthropic.com>
Issue content is now fetched and validated in the TypeScript handler before the LLM ever sees it. If injection patterns are detected, the agent returns a flagged report and skips AI triage entirely. The skill now receives pre-parsed fields (title, body, labels, author) as arguments instead of fetching the issue itself, so the LLM never processes raw untrusted API responses. Co-Authored-By: Claude <noreply@anthropic.com>
- Import from @flue/runtime (client entrypoint was folded in) - Use local() factory instead of 'local' string - Move skill to .agents/skills/<name>/SKILL.md convention Verified: dry run against issue #17799 produces correct triage. Co-Authored-By: Claude <noreply@anthropic.com>
- Use Linear priority scale (urgent/high/medium/low) instead of impact (small/medium/large) to match the Docs team's workflow - Update existing DOCS-XXXX Linear ticket instead of creating duplicates — finds the auto-synced ticket and sets priority + labels - Fix get_linked_prs tool to return JSON strings (Flue tools must return strings) - Add PR-first check: skip deep RCA when a linked PR already exists - Handle unstructured issues (no template labels) by classifying from content alone Co-Authored-By: Claude <noreply@anthropic.com>
When DRY_RUN=false, the agent now posts the full triage report as a comment on the existing DOCS-XXXX ticket in addition to setting priority and labels. This gives Shannon/Alex the classification, related docs, suggested labels, and recommended action directly in Linear. Co-Authored-By: Claude <noreply@anthropic.com>
- Sort imports per simple-import-sort - Name the default export function (import/no-anonymous-default-export) - Apply Prettier formatting Co-Authored-By: Claude <noreply@anthropic.com>
- Use Docs team label IDs instead of DevEx team IDs - Separate priority, label, and comment into independent API calls so one failure doesn't block the others - Add error logging for failed Linear mutations Tested live: priority updates correctly, triage comments post to existing DOCS tickets. Co-Authored-By: Claude <noreply@anthropic.com>
Fetch the issue's current labels in the search query and skip issueAddLabel when the target label is already present. This avoids Linear's label-group exclusivity errors (e.g., trying to add 'Docs' when it's already on the issue from GitHub sync). Co-Authored-By: Claude <noreply@anthropic.com>
Major architecture change for security, idempotency, and debuggability: - Agent is now purely read-only — returns JSON, no secrets, no writes - New apply-triage.sh handles all writes (GitHub labels, Linear update, GitHub comment fallback) with idempotency checks - Uses <!-- flue-triage --> marker to prevent duplicate comments - Checks Linear for existing triage before posting - Falls back to GitHub comment if Linear ticket not found yet - Re-enables issues.opened trigger (agent has no write access) - Simplified triage report format — no more redundant fields - Removed suggestedLabels and dryRun (agent is always read-only) Co-Authored-By: Claude <noreply@anthropic.com>
Consolidate all orchestration (triage + writes) in the TypeScript handler instead of a separate bash script. This follows Flue's recommended pattern where the handler is the orchestration layer. Keeps all improvements from the bash approach: - Idempotency via triage marker on GitHub comments - Linear comment dedup check - Label conflict handling (skip if already present) - GitHub fallback when Linear ticket not found - Error logging without crashing Removes: - .flue/scripts/apply-triage.sh - JSON extraction logic in workflow - Multi-step workflow (now single step) Co-Authored-By: Claude <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![sentry-warden[bot]](https://avatars.githubusercontent.com/in/2755838?s=60&v=4){kind=small}
Add author_association check so the agent only runs for issues opened by MEMBER, COLLABORATOR, or OWNER. External users' issues are still triaged via manual workflow_dispatch. This prevents unbounded Anthropic API costs from spam issue creation. Addresses Warden finding D7Y-HH3. Co-Authored-By: Claude <noreply@anthropic.com>
Match sentry-javascript's pattern: only run triage when the issue has template-applied labels (Docs, Docs Platform, or Bug). Issues without templates (spam, bots) skip auto-triage but can still be triaged via workflow_dispatch. Co-Authored-By: Claude <noreply@anthropic.com>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
Fixes: - Team field now uses v.picklist with allowlist matching labels.yml instead of v.optional(v.string()) — prevents injection of arbitrary GitHub labels (Warden VK2-ZRW) - Linear fallback checks commentCreate success before setting linearOk — failed mutations now correctly trigger GitHub fallback (Cursor) - get_linked_prs fetches full PR details via /pulls/:number API to get accurate merged status (Cursor) - Idempotency fix: GitHub fallback comment uses TRIAGE_MARKER but Linear success does not, so re-runs can retry Linear when ticket appears later (Sentry bot) - Removed overly broad injection patterns (act as, curl|sh, echo, eval, base64) that would false-positive on legitimate issue content (Sentry bot) - Fixed SKILL.md step numbering (duplicate Step 2, missing Step 8) and added explicit summary instruction (Sentry bot) - linearQuery now catches fetch errors instead of crashing Not a bug: "missing filesystem tools" — sandbox: local() provides bash/grep/find via Flue's built-in tools, custom tools are additional. Co-Authored-By: Claude <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit a6e6c1b. Configure here.
- Global concurrency group (flue-triage) so only one triage runs at a time — queues instead of parallelizing - AbortSignal.timeout(120s) on the skill call — hard cap on LLM processing time per issue Combined with label gate and 10-min workflow timeout, worst case for 100 spam issues is now ~$5-10 processed sequentially (~2 min each) instead of in parallel. The ultimate backstop is setting a spend limit on the Anthropic API key itself. Co-Authored-By: Claude <noreply@anthropic.com>
Comment on lines
+68
to
+78
| | Kotlin Multiplatform SDK | `Platform: KMP` | | ||
| | Native SDK | `Platform: Native` | | ||
| | .NET SDK | `Platform: .NET` | | ||
| | PHP SDK | `Platform: PHP` | | ||
| | Python SDK | `Platform: Python` | | ||
| | React Native SDK | `Platform: React-Native` | | ||
| | Ruby SDK | `Platform: Ruby` | | ||
| | Rust SDK | `Platform: Rust` | | ||
| | Unity SDK | `Platform: Unity` | | ||
| | Unreal Engine SDK | `Platform: Unreal` | | ||
| | Sentry CLI | `Platform: CLI` | |
Contributor
There was a problem hiding this comment.
Bug: The SDK mapping table in SKILL.md is missing several options from the issue template, causing incorrect issue classification and team assignment by the triage agent.
Severity: MEDIUM
Suggested Fix
Update the mapping table in step 3 of .agents/skills/classify-docs-issue/SKILL.md to include all SDK options available in the .github/ISSUE_TEMPLATE/issue-content-01-sdks.yml dropdown. Ensure each new entry maps to a valid GitHub label.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: .agents/skills/classify-docs-issue/SKILL.md#L56-L78
Potential issue: The mapping table in `.agents/skills/classify-docs-issue/SKILL.md` is
missing several SDK options present in the
`.github/ISSUE_TEMPLATE/issue-content-01-sdks.yml` issue template, such as `PowerShell
SDK`, `All JavaScript SDKs`, and `Other`. When a user selects one of these missing
options, the AI agent fails to find a corresponding platform label. This results in
incorrect team assignment, often defaulting to the general `Team: Docs` instead of the
appropriate SDK-specific team, reducing the effectiveness of the automated triage
process.
Comment on lines
+267
to
+270
| // No TRIAGE_MARKER so re-runs can retry Linear when ticket exists | ||
| if (!linearOk) { | ||
| const commentsRes = await fetch( | ||
| `https://api.github.com/repos/${REPO}/issues/${issue.number}/comments?per_page=100`, |
Contributor
There was a problem hiding this comment.
Bug: The get_linked_prs tool hardcodes the repository name, causing it to fail or fetch incorrect data for pull requests from other repositories.
Severity: MEDIUM
Suggested Fix
Modify the get_linked_prs tool to extract the repository owner and name from the cross-referenced event payload instead of using a hardcoded value. Additionally, add a check for res.ok to handle failed API requests gracefully before attempting to parse the JSON response.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: .flue/agents/triage-issue.ts#L267-L270
Potential issue: The `get_linked_prs` tool in `.flue/agents/triage-issue.ts` incorrectly
assumes that all linked pull requests originate from the same repository
(`getsentry/sentry-docs`). It hardcodes this repository when making API calls to fetch
PR details. If a PR from another repository (e.g., `getsentry/sentry`) references a docs
issue, the API call will either fail with a 404 or fetch an unrelated PR. Because the
code does not check if the fetch was successful before parsing the JSON, the agent
receives incorrect data (e.g., `state: undefined`), leading to flawed analysis of the
issue's status.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DESCRIBE YOUR PR
Adds an AI-powered triage agent using Flue that automatically classifies incoming GitHub issues and enriches the corresponding Linear ticket.
What it does:
Architecture:
.flue/agents/triage-issue.ts— agent handler (read-only AI triage + write orchestration).agents/skills/classify-docs-issue/SKILL.md— classification skill with label mapping.flue/AGENTS.md— project context for the agent.github/workflows/flue-triage-issue.yml— triggers onissues.opened+ manual dispatchSecurity:
Tested against: #17799, #17412, #17743, #17707, #17716, #17568, #17438, #17601 — all classified correctly.
Requires secrets:
ANTHROPIC_API_KEY,LINEAR_API_KEY(optional, falls back to GitHub)IS YOUR CHANGE URGENT?
PRE-MERGE CHECKLIST