Skip to content

Finalized Automation scripts for x86-64 RT kernel Upgrade #345

Open
jatinjb444 wants to merge 3 commits into
ni:nilrt/master/scarthgapfrom
jatinjb444:dev
Open

Finalized Automation scripts for x86-64 RT kernel Upgrade #345
jatinjb444 wants to merge 3 commits into
ni:nilrt/master/scarthgapfrom
jatinjb444:dev

Conversation

@jatinjb444
Copy link
Copy Markdown

@jatinjb444 jatinjb444 commented Apr 9, 2026

Changes

This PR introduces automation to fetch and merge the latest stable‑RT kernel tags into the target NI Linux Real‑Time branch.
Key changes:

Automatically clone the NI Linux kernel repository in a clean workspace.
Fetch upstream stable‑RT tags from the RT kernel repository.
Identify and merge the latest applicable RT tag into the configured target branch.
Ensure a deterministic and clean git state before merging to support re‑runs and CI usage.
Send notification emails on successful merge or on merge failure with relevant details.

This automation is focused only on the fetch and merge workflow.
Kernel build and deployment steps will be added in follow‑up changes.

Update: April 14th , 2026
Get the Cross Compile toolchain from the nirvana path and set it
Build and install the kernel on the NI's target
Copied the modules to the target
Check the network configurations working fine after reboot

Update: May 5th , 2026
Automates out‑of‑tree driver rebuild after kernel installation.
Mounts kernel source on the target via SSHFS (no local copy).
Fixes build and source symlinks under /lib/modules/.
Prepares kernel headers and invokes dkms auto install.
Integrated into the existing kernel build and test flow

Update: May 7th , 2026
Implements US‑4 and US‑5 automation:
Regenerates nati_x86_64_defconfig when required
Adds gated kernel PR creation after successful RT merge, build, install, and driver rebuild
Failure paths send immediate notification; success sends a single final summary email

Kernel PR creation is guarded behind --skip-push-and-pr and is intended to be enabled only after review.

Testing

The changes were validated by:

Running the script in a fresh workspace to verify repository cloning and RT tag discovery.
Verifying that the latest stable‑RT tag is correctly detected and merged.

After the target reboot , was able to check the latest kernel version using - { uname -r }
Verified the network modules working fine after reboot.

Confirmed SSHFS mount, header preparation, and DKMS rebuild.
Validated rebuilt drivers using dkms status.

Process

Suggested Reviewers:
@rajendra-desai-ni
@Shreejit-03

@Shreejit-03
Copy link
Copy Markdown
Contributor

Please run flake8 ( python3 -m flake8 ) on all the python files and fix all the issues we can. Some of them could be fixed by running black (python3 -m black ).

@jatinjb444 jatinjb444 force-pushed the dev branch 6 times, most recently from dbeefde to e0fcb85 Compare May 5, 2026 05:45
@jatinjb444 jatinjb444 changed the title Finalized fetch and merge script for rt-tags Finalized Automation scripts for x86-64 RT kernel Upgrade May 7, 2026
@jatinjb444 jatinjb444 force-pushed the dev branch 3 times, most recently from 5328398 to 4709c45 Compare May 8, 2026 05:58
- Install kernel on target with reboot and SSH readiness checks.
- Rebuild out-of-tree drivers automatically using DKMS.
- Kernel source is mounted via SSHFS to avoid local copy.
- Fix build/source symlinks and prepare kernel headers.

Verified on real NI RT target with DKMS-managed drivers.
Signed-off-by: Jatin Bharti <jatin.bharti@emerson.com>
Comment thread scripts/dev/upstream_merge/json_config.py
Comment thread scripts/dev/upstream_merge/json_config.py
Comment thread scripts/dev/upstream_merge/json_config.py Outdated
return 1


def run_cmd(cmd):
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is run_cmd any different from run_command in its usage and implementation? It would be better to use run_command .

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes—run_cmd differs intentionally. run_command relies on shlex.split, which breaks SSH commands that include shell operators (||, &&, pipes, quoted remote commands). The rebuild/install steps depend on shell semantics, so run_cmd executes via the shell to preserve existing behavior.

Comment thread scripts/dev/upstream_merge/kernel_build_and_test.py Outdated
Comment thread scripts/dev/upstream_merge/kernel_build_and_test.py Outdated
Comment thread scripts/dev/upstream_merge/kernel_build_and_test.py Outdated
@jatinjb444 jatinjb444 marked this pull request as ready for review May 21, 2026 09:27
@jatinjb444 jatinjb444 requested a review from a team as a code owner May 21, 2026 09:27
- Automated defconfig regeneration process
- Integrated gated kernel PR creation workflow
- Pulled latest upstream changes for consistency
- Improved SSH connection handling logic
- Cleaned and formatted json.config structure
- Enhanced overall script reliability and clarity

Signed-off-by: Jatin Bharti <jatin.bharti@emerson.com>
- Moved build functions into build.py module
- Centralized reusable build utilities
- Removed redundant code across scripts
- Cleaned up unused and inconsistent files
- Improved modularity and maintainability

Signed-off-by: Jatin Bharti <jatin.bharti@emerson.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants