In-Cluster Checks

A generic framework for running health validation rules directly on OpenShift cluster nodes using oc debug.

Key Advantages

• Direct node access - Rules run directly on cluster nodes via oc debug
• Fast execution - Parallel rule execution across multiple nodes
• Relevant rules execution - Only relevant rules run based on prerequisite checks
• Easy debugging - Full visibility into commands executed for each rule

Originally developed as part of Red Hat's Pendrive project, this framework has been extracted as open-source to benefit the broader OpenShift community.

Rules are organized by topic into domains (hardware, network, linux, storage).

Installation

Prerequisites:

• Python >= 3.12
• pip (Python package installer)

Connected Environment

Install the framework:

pip install in-cluster-checks

Or if pip is not found, use:

python3 -m pip install in-cluster-checks

Disconnected Environment

For environments without internet access:

1. Download the package on a connected machine:

   pip download in-cluster-checks --dest ./packages
   # Or: python3 -m pip download in-cluster-checks --dest ./packages

2. Transfer the packages to the disconnected environment

   Copy the entire ./packages directory to the disconnected machine. You can place it anywhere, but note the location for the next step.

3. Install from local packages:

   # If packages are in the current directory:
   pip install --no-index --find-links=./packages in-cluster-checks
   
   # Or specify the full path to where you copied them:
   pip install --no-index --find-links=/path/to/packages in-cluster-checks
   
   # Using python3 -m pip:
   # python3 -m pip install --no-index --find-links=./packages in-cluster-checks

Running in-cluster-checks

Cluster Login

Ensure you're logged into your OpenShift cluster.

You can login by one of the following options:

Login with Username and Password:

Use the cluster API URL and your credentials.

oc login https://api.your-cluster.com:6443

Login Using a Kubeconfig File:

If you already have a kubeconfig file with credentials:

export KUBECONFIG=/path/to/kubeconfig

Usage Examples

You can run in-cluster-checks with the following options:

# Run all checks. Use --output to save run results to ./cluster-checks.json
in-cluster-checks --output ./cluster-checks.json

# Output as JUnit XML (for CI dashboard integration)
in-cluster-checks --format junit --output ./cluster-checks.xml

# Run a specific rule (disables secret filtering)
in-cluster-checks --debug-rule "check_disk_usage"

# Run with debug logging
in-cluster-checks --log-level DEBUG

# Specify namespace for debug pods (default: default)
in-cluster-checks --namespace openshift-debug

To see all available options, run:

in-cluster-checks --help

Notes:

• To control execution performance, use --max-workers to set the maximum number of parallel workers (default: 50).
• When using --namespace, ensure you have permissions to create debug pods in the specified namespace.

Contributing

Contributions are welcome! Please see our CONTRIBUTING.md for detailed guidelines on how to:

• Set up your development environment
• Add new rules and domains
• Write tests
• Submit pull requests

Related Projects

• Pendrive: Red Hat's on-premise Insights validation tool (internal)
• OpenShift: Container orchestration platform

Acknowledgments

This framework was extracted from Red Hat's Pendrive project. The core validation infrastructure is generic and contains no confidential logic, making it suitable for open-source release to benefit the wider OpenShift community.

License

The 3-Clause BSD License

See LICENSE for full text.