OCPBUGS-84961: add kubernetes/conformance umbrella suite

[not-stbenjam]

Summary

Restores the kubernetes/conformance suite, which was previously removed. This is an umbrella suite that aggregates kubernetes/conformance/parallel and kubernetes/conformance/serial.

Uses AddGlobalSuite so the umbrella inherits qualifiers exclusively from its children via mergeParentQualifiers (openshift/origin#31261), rather than picking up a default source == "openshift:payload:hyperkube" qualifier that would match all hyperkube tests.

Companion PR

openshift/origin#31261

Test plan

• openshift-tests run kubernetes/conformance --dry-run lists the expected tests
• Test count matches kubernetes/conformance/parallel + kubernetes/conformance/serial
• Excluded tests ([Disabled:], [Disruptive], [Slow]) are not included

🤖 Generated with Claude Code

Summary by CodeRabbit

• Chores
  • Updated conformance test suite registration to ensure proper test hierarchy and organization.

[openshift-ci-robot]

@not-stbenjam: This pull request references Jira Issue OCPBUGS-84961, which is valid.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (5.0.0) matches configured target version for branch (5.0.0)
• bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Summary

• Add a kubernetes/conformance umbrella suite that aggregates tests from kubernetes/conformance/parallel and kubernetes/conformance/serial
• Enables openshift-tests run kubernetes/conformance to run all kubernetes conformance tests in a single invocation
• Uses AddGlobalSuite (not AddSuite) so the umbrella starts with zero qualifiers and inherits exclusively from its children via mergeParentQualifiers

Why AddGlobalSuite?

AddSuite prepends a source == "openshift:payload:hyperkube" qualifier to every suite. For suites with existing qualifiers, this scopes them to the correct extension — but for an empty-qualifier umbrella suite, it becomes a catch-all that matches all hyperkube tests (including [Disabled:], [Disruptive], [Slow], [Flaky]). Since qualifiers are OR'd together in the filter, this catch-all would bypass the exclusion filters in the child qualifiers.

AddGlobalSuite registers the suite with zero qualifiers. After mergeParentQualifiers runs in origin, the umbrella inherits the properly-scoped qualifiers from its children.

Companion PR

openshift/origin#31261 — adds the Parents field and mergeParentQualifiers mechanism that makes this work.

Test plan

• Verify openshift-tests run kubernetes/conformance --dry-run lists the expected tests
• Verify test count matches kubernetes/conformance/parallel + kubernetes/conformance/serial
• Verify excluded tests ([Disabled:], [Disruptive], [Slow]) are not included

🤖 Generated with Claude Code

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@not-stbenjam: the contents of this pull request could not be automatically validated.

The following commits could not be validated and must be approved by a top-level approver:

• 8697994|OCPBUGS-84961: add kubernetes/conformance umbrella suite: does not specify an upstream backport in the commit message

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: not-stbenjam
Once this PR has been reviewed and has the lgtm label, please assign jerpeter1 for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• DOWNSTREAM_OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

Hi @not-stbenjam. Thanks for your PR.

I'm waiting for a openshift member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Tip

We noticed you've done this a few times! Consider joining the org to skip this step and gain /lgtm and other bot rights. We recommend asking approvers on your previous PRs to sponsor you.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 8c2f44d9-3728-4ada-8274-0993aaddd64e

📥 Commits

Reviewing files that changed from the base of the PR and between 6bd0373 and 59c831f.

📒 Files selected for processing (1)

• openshift-hack/cmd/k8s-tests-ext/k8s-tests.go

🚧 Files skipped from review as they are similar to previous changes (1)

• openshift-hack/cmd/k8s-tests-ext/k8s-tests.go

---

Walkthrough

A new global test suite kubernetes/conformance is registered and added as a parent to the existing kubernetes/conformance/parallel and kubernetes/conformance/serial suites; all changes are in the test suite registry logic.

Changes

Test Suite Hierarchy

Layer / File(s)	Summary
kubernetes/conformance suite hierarchy openshift-hack/cmd/k8s-tests-ext/k8s-tests.go	Register a global kubernetes/conformance suite, then update kubernetes/conformance/parallel and kubernetes/conformance/serial to include kubernetes/conformance in their Parents lists.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 14 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (14 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and specifically identifies the main change: adding a kubernetes/conformance umbrella suite, which matches the core modification in the changeset.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names	✅ Passed	The PR adds test suite registration logic (not Ginkgo test definitions). The modified file contains no It(), Describe(), Context(), When() calls that would define test titles with dynamic names.
Test Structure And Quality	✅ Passed	The PR does not contain Ginkgo test code (It blocks, Describe, BeforeEach, etc.). It only modifies test suite registration configuration code in k8s-tests.go, making the custom check inapplicable.
Microshift Test Compatibility	✅ Passed	PR adds only configuration code (k8s-tests.go) that registers test suites. No new Ginkgo test definitions (It/Describe/Context/When) are present.
Single Node Openshift (Sno) Test Compatibility	✅ Passed	PR does not add new Ginkgo e2e tests (It/Describe/Context/When). Only adds test suite registry config file with AddGlobalSuite/Parents declarations.
Topology-Aware Scheduling Compatibility	✅ Passed	Changes are test suite registry code only, not deployment/operator/controller code. No scheduling constraints, affinity, nodeSelectors, or topology-dependent logic introduced.
Ote Binary Stdout Contract	✅ Passed	PR changes only add suite registration and parent references (data structure operations) with no stdout-writing code—no fmt.Print*, klog.*, or logging calls present.
Ipv6 And Disconnected Network Test Compatibility	✅ Passed	No new Ginkgo e2e tests are added; PR only registers a suite aggregator and updates suite parent relationships in k8s-tests.go.
No-Weak-Crypto	✅ Passed	PR modifies test suite configuration only (AddGlobalSuite for kubernetes/conformance, updating Parents fields). No cryptographic operations, weak crypto algorithms, or secret comparisons are present.
Container-Privileges	✅ Passed	No container/K8s manifests with privileged settings found. PR only modifies Go test suite registration in k8s-tests.go.
No-Sensitive-Data-In-Logs	✅ Passed	The PR only adds test suite registration code with no logging statements that could expose sensitive data. All fmt.Sprintf calls build test filter expressions, not logged output.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[openshift-ci-robot]

@not-stbenjam: This pull request references Jira Issue OCPBUGS-84961, which is valid.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (5.0.0) matches configured target version for branch (5.0.0)
• bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Details

In response to this:

Summary

• Add a kubernetes/conformance umbrella suite that aggregates tests from kubernetes/conformance/parallel and kubernetes/conformance/serial
• Enables openshift-tests run kubernetes/conformance to run all kubernetes conformance tests in a single invocation
• Uses AddGlobalSuite (not AddSuite) so the umbrella starts with zero qualifiers and inherits exclusively from its children via mergeParentQualifiers

Why AddGlobalSuite?

AddSuite prepends a source == "openshift:payload:hyperkube" qualifier to every suite. For suites with existing qualifiers, this scopes them to the correct extension — but for an empty-qualifier umbrella suite, it becomes a catch-all that matches all hyperkube tests (including [Disabled:], [Disruptive], [Slow], [Flaky]). Since qualifiers are OR'd together in the filter, this catch-all would bypass the exclusion filters in the child qualifiers.

AddGlobalSuite registers the suite with zero qualifiers. After mergeParentQualifiers runs in origin, the umbrella inherits the properly-scoped qualifiers from its children.

Companion PR

openshift/origin#31261 — adds the Parents field and mergeParentQualifiers mechanism that makes this work.

Test plan

• Verify openshift-tests run kubernetes/conformance --dry-run lists the expected tests
• Verify test count matches kubernetes/conformance/parallel + kubernetes/conformance/serial
• Verify excluded tests ([Disabled:], [Disruptive], [Slow]) are not included

🤖 Generated with Claude Code

Summary by CodeRabbit

• Tests
• Enhanced Kubernetes conformance test suite registration with improved organizational hierarchy and parent-child relationships for better test structuring and execution flow.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@not-stbenjam: the contents of this pull request could not be automatically validated.

The following commits could not be validated and must be approved by a top-level approver:

• df7eb4f|OCPBUGS-84961: add kubernetes/conformance umbrella suite: does not specify an upstream backport in the commit message

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

[openshift-ci-robot]

@not-stbenjam: This pull request references Jira Issue OCPBUGS-84961, which is valid.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (5.0.0) matches configured target version for branch (5.0.0)
• bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Details

In response to this:

Summary

• Add a kubernetes/conformance umbrella suite that aggregates tests from kubernetes/conformance/parallel and kubernetes/conformance/serial
• Enables openshift-tests run kubernetes/conformance to run all kubernetes conformance tests in a single invocation
• Uses AddGlobalSuite (not AddSuite) so the umbrella starts with zero qualifiers and inherits exclusively from its children via mergeParentQualifiers

Why AddGlobalSuite?

AddSuite prepends a source == "openshift:payload:hyperkube" qualifier to every suite. For suites with existing qualifiers, this scopes them to the correct extension — but for an empty-qualifier umbrella suite, it becomes a catch-all that matches all hyperkube tests (including [Disabled:], [Disruptive], [Slow], [Flaky]). Since qualifiers are OR'd together in the filter, this catch-all would bypass the exclusion filters in the child qualifiers.

AddGlobalSuite registers the suite with zero qualifiers. After mergeParentQualifiers runs in origin, the umbrella inherits the properly-scoped qualifiers from its children.

Companion PR

openshift/origin#31261 — adds the Parents field and mergeParentQualifiers mechanism that makes this work.

Test plan

• Verify openshift-tests run kubernetes/conformance --dry-run lists the expected tests
• Verify test count matches kubernetes/conformance/parallel + kubernetes/conformance/serial
• Verify excluded tests ([Disabled:], [Disruptive], [Slow]) are not included

🤖 Generated with Claude Code

Summary by CodeRabbit

• Tests
• Added a top-level "kubernetes/conformance" test suite.
• Updated conformance parallel and serial suites to be nested under the new top-level suite for clearer hierarchy and execution ordering.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@not-stbenjam: the contents of this pull request could not be automatically validated.

The following commits could not be validated and must be approved by a top-level approver:

• 2d10768|UPSTREAM: : OCPBUGS-84961: add kubernetes/conformance umbrella suite: does not specify an upstream backport in the commit message

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

[openshift-ci-robot]

@not-stbenjam: the contents of this pull request could not be automatically validated.

The following commits could not be validated and must be approved by a top-level approver:

• 6bd0373|UPSTREAM: : add kubernetes/conformance umbrella suite: does not specify an upstream backport in the commit message

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

[openshift-ci-robot]

@not-stbenjam: This pull request references Jira Issue OCPBUGS-84961, which is valid.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (5.0.0) matches configured target version for branch (5.0.0)
• bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Details

In response to this:

Summary

• Add a kubernetes/conformance umbrella suite that aggregates tests from kubernetes/conformance/parallel and kubernetes/conformance/serial
• Enables openshift-tests run kubernetes/conformance to run all kubernetes conformance tests in a single invocation
• Uses AddGlobalSuite (not AddSuite) so the umbrella starts with zero qualifiers and inherits exclusively from its children via mergeParentQualifiers

Why AddGlobalSuite?

AddSuite prepends a source == "openshift:payload:hyperkube" qualifier to every suite. For suites with existing qualifiers, this scopes them to the correct extension — but for an empty-qualifier umbrella suite, it becomes a catch-all that matches all hyperkube tests (including [Disabled:], [Disruptive], [Slow], [Flaky]). Since qualifiers are OR'd together in the filter, this catch-all would bypass the exclusion filters in the child qualifiers.

AddGlobalSuite registers the suite with zero qualifiers. After mergeParentQualifiers runs in origin, the umbrella inherits the properly-scoped qualifiers from its children.

Companion PR

openshift/origin#31261 — adds the Parents field and mergeParentQualifiers mechanism that makes this work.

Test plan

• Verify openshift-tests run kubernetes/conformance --dry-run lists the expected tests
• Verify test count matches kubernetes/conformance/parallel + kubernetes/conformance/serial
• Verify excluded tests ([Disabled:], [Disruptive], [Slow]) are not included

🤖 Generated with Claude Code

Summary by CodeRabbit

• Tests
• Introduced a top-level "kubernetes/conformance" test suite to serve as a common parent.
• Updated the "kubernetes/conformance/parallel" and "kubernetes/conformance/serial" suites to be nested under the new top-level suite for clearer hierarchy and execution ordering.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@not-stbenjam: the contents of this pull request could not be automatically validated.

The following commits could not be validated and must be approved by a top-level approver:

• 59c831f|UPSTREAM: : add kubernetes/conformance umbrella suite: does not specify an upstream backport in the commit message

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

[not-stbenjam]

/ok-to-test

[openshift-ci]

@not-stbenjam: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

Details

In response to this:

/ok-to-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[openshift-ci-robot]

@not-stbenjam: This pull request references Jira Issue OCPBUGS-84961, which is valid.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (5.0.0) matches configured target version for branch (5.0.0)
• bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Details

In response to this:

Summary

Restores the kubernetes/conformance suite, which was previously removed. This is an umbrella suite that aggregates kubernetes/conformance/parallel and kubernetes/conformance/serial.

Uses AddGlobalSuite so the umbrella inherits qualifiers exclusively from its children via mergeParentQualifiers (openshift/origin#31261), rather than picking up a default source == "openshift:payload:hyperkube" qualifier that would match all hyperkube tests.

Companion PR

openshift/origin#31261

Test plan

• openshift-tests run kubernetes/conformance --dry-run lists the expected tests
• Test count matches kubernetes/conformance/parallel + kubernetes/conformance/serial
• Excluded tests ([Disabled:], [Disruptive], [Slow]) are not included

🤖 Generated with Claude Code

Summary by CodeRabbit

• Chores
• Updated conformance test suite registration to ensure proper test hierarchy and organization.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[stbenjam]

/ok-to-test

[openshift-ci]

@not-stbenjam: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-aws-ovn-techpreview	59c831f	link	false	/test e2e-aws-ovn-techpreview

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.