Trt 1989 migration indexes

[neisw]

Builds on #3532 and adds the indexes (after back-fill has completed)

Summary by CodeRabbit

• Documentation

  • Added comprehensive migration planning documents detailing database partitioning preparation and composite index implementation strategies for performance optimization.

• Performance

  • Enhanced database schema with denormalized fields to support optimized query performance through table partitioning and improved indexing.

• Tests

  • Updated test fixtures to reflect schema changes.

[openshift-merge-bot]

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: cc899706-8998-4183-a634-3e3d5879f380

📥 Commits

Reviewing files that changed from the base of the PR and between 134d39a and b252569.

📒 Files selected for processing (8)

• cmd/sippy/seed_data.go
• docs/plans/trt-1989-partitioning-prep.md
• docs/plans/trt-1989-phase2-indexes.md
• pkg/api/job_runs_test.go
• pkg/dataloader/prowloader/prow.go
• pkg/db/db.go
• pkg/db/models/prow.go
• pkg/sippyserver/pr_new_tests_worker_test.go

---

Walkthrough

This PR prepares the Sippy database schema and data loading layer for PostgreSQL table partitioning by denormalizing release and timestamp fields onto related tables. Schema models gain indexed denormalized columns, an explicit join table is configured, data loading logic is refactored to populate these fields during import, and test fixtures are updated to match the new shape.

Changes

PostgreSQL Partitioning Preparation (TRT-1989)

Layer / File(s)	Summary
Schema denormalization and migration planning pkg/db/models/prow.go, docs/plans/trt-1989-partitioning-prep.md, docs/plans/trt-1989-phase2-indexes.md	GORM models add indexed ProwJobRelease and ProwJobRunTimestamp fields to ProwJobRun, ProwJobRunTest, ProwJobRunAnnotation, ProwJobRunTestOutput, and a new explicit ProwJobRunProwPullRequest join table; comprehensive migration plan documents phased approach through schema prep, backfill, index creation, query updates, and partitioning execution.
GORM join table configuration pkg/db/db.go	Database schema setup explicitly configures ProwJobRunProwPullRequest join table with SetupJoinTable and adds it to AutoMigrate to ensure the custom model with denormalized fields is properly created and maintained.
Data loading with denormalized field population pkg/dataloader/prowloader/prow.go	GCS data loading refactored to thread prow-job ID and release context through test extraction functions, populate denormalized fields on tests/annotations/outputs, and persist ProwJobRun and join-table rows explicitly via transaction instead of relying on association hooks; function signatures updated to accept and propagate prow-job context.
Test fixture updates pkg/api/job_runs_test.go, pkg/sippyserver/pr_new_tests_worker_test.go, cmd/sippy/seed_data.go	Test fixtures in job_runs_test and pr_new_tests_worker_test updated to populate new denormalized fields on ProwJobRun and ProwJobRunTest; seed_data struct literals reformatted for consistency.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Suggested labels

approved, lgtm, jira/valid-reference

Suggested reviewers

• petr-muller
• deepsm007

🚥 Pre-merge checks | ✅ 15 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.
Single Responsibility And Clear Naming	⚠️ Warning	Struct field count violates single responsibility: ProwJobRun 18 fields, ProwJobRunTest 15 fields (recommendation ~7). Method signatures grew significantly. Ambiguous naming and missing documentation.	Use composite partition-key type to reduce struct bloat. Add GoDoc comments. Refactor large structs into focused sub-types.

✅ Passed checks (15 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The PR title 'Trt 1989 migration indexes' is directly related to the primary content of the changeset, which focuses on adding database indexes to support the TRT-1989 migration for PostgreSQL partitioning.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Go Error Handling	✅ Passed	All Go error handling patterns in new code follow best practices: no ignored errors, proper fmt.Errorf wrapping with %w, no new panics, and proper nil checks before dereferencing.
Sql Injection Prevention	✅ Passed	All database operations use GORM parameterized queries or named parameters for BigQuery. Date values in partition SQL are internally generated from time.Time, not user input.
Excessive Css In React Should Use Styles	✅ Passed	Check is not applicable to this PR. All modified files are Go backend code (6 files) and Markdown documentation (2 files). No React components, JSX, or CSS styling present.
Test Coverage For New Features	✅ Passed	PR adds denormalized fields for database partitioning. Modified functions thread new parameters without new logic. Existing tests appropriately updated to populate new model fields.
Stable And Deterministic Test Names	✅ Passed	PR modifies two test files using standard Go testing, not Ginkgo. Check not applicable.
Test Structure And Quality	✅ Passed	Custom check for Ginkgo test code is not applicable. Repository uses standard Go testing (testing.T) with testify, not Ginkgo BDD framework.
Microshift Test Compatibility	✅ Passed	PR does not add any Ginkgo e2e tests. The changes are to database schema, models, data loading code, and unit tests (using standard testing.T, not Ginkgo). The custom check is not applicable.
Single Node Openshift (Sno) Test Compatibility	✅ Passed	No new Ginkgo e2e tests added. PR only modifies database schemas, models, and data loaders for TRT-1989 partitioning. SNO compatibility check not applicable.
Topology-Aware Scheduling Compatibility	✅ Passed	PR contains only database schema and backend code changes. No deployment manifests, operator code, controllers, or scheduling constraints are introduced. Check not applicable.
Ote Binary Stdout Contract	✅ Passed	OTE Binary Stdout Contract check is not applicable. This PR modifies the Sippy project (a Prow job analysis tool), not an OTE binary. No process-level code writes to stdout.
Ipv6 And Disconnected Network Test Compatibility	✅ Passed	This PR does not add Ginkgo e2e tests. It only updates database models and Go unit tests using the standard testing framework. Check not applicable.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: neisw

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [neisw]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-merge-bot]

Scheduling required tests:
/test e2e

[openshift-ci]

@neisw: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[neisw]

/hold
#3532 first, then back-fill data and manually add indexes