feat: support org-level issue templates from .github repo#440
Conversation
When assessing a repository that belongs to a GitHub organization, the issue_pr_templates check now falls back to the organization's .github repo for issue and PR templates when no local templates exist. Requires GITHUB_TOKEN to be set. Local templates take precedence over org-level inheritance and the fallback is only triggered when no local templates are found.
📝 WalkthroughWalkthroughThis PR adds organization-level GitHub template fallback to the issue/PR templates assessor. The new ChangesOrg-level GitHub template support
Possibly related issues
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
✨ Simplify code
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
📈 Test Coverage Report
Coverage calculated from unit tests only |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/agentready/assessors/structure.py`:
- Around line 835-847: The assessor's assess() currently lets
GitHubTemplatesFetcher.fetch_pr_templates and fetch_issue_templates raise
GitHubTemplatesError and crash the run; wrap calls to
GitHubTemplatesFetcher.extract_owner + fetch_pr_templates and
fetch_issue_templates in try/except catching GitHubTemplatesError, and on
exception degrade gracefully by leaving score unchanged, setting
pr_template_found/issue_template_found false (or whatever local flag is used),
appending evidence like "Skipped: GitHub API error fetching templates" (and
optionally processLogger.warn the error), and continue; apply the same guard
around both fetch_pr_templates and fetch_issue_templates invocations so
transient GitHub failures do not propagate.
In `@tests/unit/test_github_templates.py`:
- Around line 86-94: The tests test_no_repo_returns_empty and the similar one at
lines 146-153 incorrectly simulate a 404 because they set
mock_get.return_value.status_code = 404 but do not make raise_for_status()
raise, so the code path in GitHubTemplatesFetcher._fetch_contents() that catches
requests.HTTPError for 404 is not exercised; update those tests to have
mock_get.return_value.raise_for_status.side_effect =
requests.exceptions.HTTPError(response=mock_get.return_value) (or otherwise
raise an HTTPError) so that calling fetch_pr_templates on the
GitHubTemplatesFetcher instance triggers the exception branch and returns an
empty list.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 29fb1509-7a55-459a-a01d-4839b2544773
📒 Files selected for processing (4)
src/agentready/assessors/structure.pysrc/agentready/utils/github_templates.pytests/unit/test_github_templates.pytests/unit/test_issue_template_assessor.py
| fetcher = GitHubTemplatesFetcher() | ||
| owner = fetcher.extract_owner(repository.url) | ||
| if owner: | ||
| org_pr_templates = fetcher.fetch_pr_templates(owner) | ||
| if org_pr_templates: | ||
| score += 50 | ||
| pr_template_found = True | ||
| evidence.append( | ||
| "PR template found (inherited from org-level .github repo)" | ||
| ) | ||
| else: | ||
| evidence.append("No PR template found") | ||
| else: |
There was a problem hiding this comment.
Handle GitHub API failures in the assessor instead of propagating exceptions.
fetch_pr_templates() / fetch_issue_templates() can raise GitHubTemplatesError (e.g., 5xx/network), and assess() currently does not catch it. A transient GitHub outage can therefore fail this assessor run instead of degrading gracefully.
Suggested guard pattern
from ..utils.github_templates import GitHubTemplatesFetcher
+from ..utils.github_templates import GitHubTemplatesError
...
- if owner:
- org_pr_templates = fetcher.fetch_pr_templates(owner)
+ if owner:
+ try:
+ org_pr_templates = fetcher.fetch_pr_templates(owner)
+ except GitHubTemplatesError:
+ org_pr_templates = []
+ evidence.append("Org-level PR template check unavailable (GitHub API error)")
if org_pr_templates:
...
- if owner:
- org_issue_templates = fetcher.fetch_issue_templates(owner)
+ if owner:
+ try:
+ org_issue_templates = fetcher.fetch_issue_templates(owner)
+ except GitHubTemplatesError:
+ org_issue_templates = []
+ evidence.append("Org-level issue template check unavailable (GitHub API error)")
if len(org_issue_templates) >= 2:As per coding guidelines, "Check for proper error handling (return skipped/error Finding, don't crash)."
Also applies to: 878-896
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/agentready/assessors/structure.py` around lines 835 - 847, The assessor's
assess() currently lets GitHubTemplatesFetcher.fetch_pr_templates and
fetch_issue_templates raise GitHubTemplatesError and crash the run; wrap calls
to GitHubTemplatesFetcher.extract_owner + fetch_pr_templates and
fetch_issue_templates in try/except catching GitHubTemplatesError, and on
exception degrade gracefully by leaving score unchanged, setting
pr_template_found/issue_template_found false (or whatever local flag is used),
appending evidence like "Skipped: GitHub API error fetching templates" (and
optionally processLogger.warn the error), and continue; apply the same guard
around both fetch_pr_templates and fetch_issue_templates invocations so
transient GitHub failures do not propagate.
| def test_no_repo_returns_empty(self, mock_get): | ||
| """Test that non-existent .github repo returns empty.""" | ||
| mock_get.return_value.status_code = 404 | ||
| mock_get.return_value.raise_for_status.side_effect = None | ||
|
|
||
| fetcher = GitHubTemplatesFetcher(token="ghp_test123456789012345678901234567890") | ||
| result = fetcher.fetch_pr_templates("kagenti") | ||
| assert result == [] | ||
|
|
There was a problem hiding this comment.
404 tests are not exercising the HTTPError branch.
Both “no repo” tests set status_code = 404 but keep raise_for_status() non-raising, so _fetch_contents() never enters the 404 exception path these tests intend to validate.
Suggested test fix
+import requests
...
`@patch`("agentready.utils.github_templates.requests.get")
def test_no_repo_returns_empty(self, mock_get):
"""Test that non-existent .github repo returns empty."""
- mock_get.return_value.status_code = 404
- mock_get.return_value.raise_for_status.side_effect = None
+ response = mock_get.return_value
+ response.status_code = 404
+ response.raise_for_status.side_effect = requests.HTTPError(response=response)
...
`@patch`("agentready.utils.github_templates.requests.get")
def test_no_repo_returns_empty(self, mock_get):
"""Test that non-existent .github repo returns empty."""
- mock_get.return_value.status_code = 404
- mock_get.return_value.raise_for_status.side_effect = None
+ response = mock_get.return_value
+ response.status_code = 404
+ response.raise_for_status.side_effect = requests.HTTPError(response=response)Also applies to: 146-153
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/unit/test_github_templates.py` around lines 86 - 94, The tests
test_no_repo_returns_empty and the similar one at lines 146-153 incorrectly
simulate a 404 because they set mock_get.return_value.status_code = 404 but do
not make raise_for_status() raise, so the code path in
GitHubTemplatesFetcher._fetch_contents() that catches requests.HTTPError for 404
is not exercised; update those tests to have
mock_get.return_value.raise_for_status.side_effect =
requests.exceptions.HTTPError(response=mock_get.return_value) (or otherwise
raise an HTTPError) so that calling fetch_pr_templates on the
GitHubTemplatesFetcher instance triggers the exception branch and returns an
empty list.
|
Tested this against With Since most org Also noting the two CodeRabbit findings are valid: the uncaught This comment is from Bill Murdock with assistance from Claude Code. |
|
This was fixed in #441 instead. |
1 participant
Summary
issue_pr_templatescheck now falls back to the organization's.githubrepo for issue and PR templates when no local templates are foundGitHubTemplatesFetcherutility class to query the GitHub API for org-level templatesIssuePRTemplatesAssessorupdated to check local.github/first, then use org-level templates as fallback (requiresGITHUB_TEMPLATE_TOKEN)Problem
Repos like
kagenti/kagentihave no.github/ISSUE_TEMPLATE/directory locally, but the organizationkagentidefines templates atkagenti/.github. agentready was scoring these as completely missing templates.Implementation
New:
src/agentready/utils/github_templates.pyGitHubTemplatesFetcher— fetches issue/PR templates from an org's.githubrepo via the GitHub APIextract_owner(url)— parses the owner/org from HTTPS (https://github.com/owner/repo) and SSH (git@github.com:owner/repo.git) URLsfetch_pr_templates(owner)— returns list of PR template file pathsfetch_issue_templates(owner)— returns list of issue template file pathsGITHUB_TOKENenv var; gracefully skips when no token is availableModified:
src/agentready/assessors/structure.pyIssuePRTemplatesAssessor.assess()checks local.github/first, then falls back to org-level templates when no local templates exist andGITHUB_TOKENis availableNew:
tests/unit/test_github_templates.py(18 tests)Tests for owner extraction, token handling, PR/issue fetching, edge cases (None/empty URLs, non-GitHub URLs, missing repos, auth errors).
New:
tests/unit/test_issue_template_assessor.py(8 tests)Tests for basic assessor behavior plus org-level fallback covering scenarios:
.github)Tests
All linters pass:
black,isort,ruff.Summary by CodeRabbit
Release Notes
New Features
Tests