NFD worker does not handle SIGTERM gracefully #2546

What happened:

nfd-worker exits with code 1 or 2 when receiving SIGTERM during pod termination (e.g., DaemonSet rolling updates). The process terminates abruptly without executing its graceful shutdown path. This triggers alerts in monitoring systems that track non-zero exit codes.

In a typical rollout, we observe:

• Exit code 1 (most common): nfd-master is terminated first during the rollout. The worker's next gRPC call to master gets a connection error, Run() returns that error, and main()
  hits os.Exit(1).
• Exit code 2 (less common): SIGTERM arrives while the process is idle (not mid-gRPC call). With no signal handler registered, the Go runtime terminates the process immediately.

Looking at the code in cmd/nfd-worker/main.go, the main() function creates an NfdWorker instance and calls instance.Run(), but never registers an OS signal handler to invoke instance.Stop() on SIGTERM/SIGINT:

https://github.com/kubernetes-sigs/node-feature-discovery/blob/v0.18.3/cmd/nfd-worker/main.go#L38-L73

The worker internally has a clean shutdown path — Stop() closes a channel, and Run() receives it, logs "shutting down nfd-worker", and returns nil (exit 0):

https://github.com/kubernetes-sigs/node-feature-discovery/blob/v0.18.3/pkg/nfd-worker/nfd-worker.go#L329-L346

However, since no signal handler bridges OS signals to Stop(), SIGTERM either kills the process directly (exit 2) or interrupts an in-flight gRPC call which propagates an error to Run() → os.Exit(1).

What you expected to happen:

nfd-worker should catch SIGTERM/SIGINT via signal.Notify, call Stop(), and exit cleanly with code 0. This is the standard pattern for long-running Go processes in Kubernetes.

How to reproduce it (as minimally and precisely as possible):

1. Deploy nfd-worker as a DaemonSet (e.g., via the NVIDIA gpu-operator helm chart which bundles NFD as a sub-chart)
2. Trigger a rolling update of the DaemonSet (e.g., update an annotation or image tag)
3. Observe that terminated pods report exit code 1 or 2 rather than 0
4. Alternatively: kubectl exec into a running nfd-worker pod and run kill -TERM 1 — the process exits non-zero instead of shutting down gracefully

Anything else we need to know?:

The fix is straightforward — add signal handling in cmd/nfd-worker/main.go:

ctx, cancel := signal.NotifyContext(context.Background(), syscall.SIGTERM, syscall.SIGINT)
defer cancel()

go func() {
    <-ctx.Done()
    instance.Stop()
}()

This same gap exists in all long-running NFD daemon entrypoints:

https://github.com/kubernetes-sigs/node-feature-discovery/blob/v0.18.3/cmd/nfd-worker/main.go
https://github.com/kubernetes-sigs/node-feature-discovery/blob/v0.18.3/cmd/nfd-master/main.go
https://github.com/kubernetes-sigs/node-feature-discovery/blob/v0.18.3/cmd/nfd-gc/main.go
https://github.com/kubernetes-sigs/node-feature-discovery/blob/v0.18.3/cmd/nfd-topology-updater/main.go

The worker's Run() already returns nil when Stop() is called, so wiring signals to Stop() is the only missing piece for a clean exit.

Environment:

• Kubernetes version: v1.32
• Cloud provider or hardware configuration: bare-metal (on-prem)
• OS: Ubuntu 24.04
• Kernel: N/A (not kernel-specific)
• Install tools: Helm (via NVIDIA gpu-operator chart, which includes NFD v0.15.4 as a sub-chart)
• Network plugin and version: N/A (not network-related)
• Others: Verified the signal handling gap exists in v0.15.4, v0.16.0, v0.18.3, and current master — the signal handling has never been added to the main entrypoints