Investigate whether self-check tasks should use config-level lowest_temperature

[coderabbitai]

Summary

Track the use of config.lowest_temperature in self-check actions and evaluate whether temperature selection should remain a global config-level setting or become task-/rail-specific.

Background

In PR #1874, the self-check implementation calls run_self_check_task(...) with lowest_temperature=config.lowest_temperature from the config object. This raised a review concern because self-check tasks can now be parameterized and chained, but temperature control is still sourced from a global config field.

Relevant discussion:

• PR: feat: Allow for multiple self-check rails #1874
• Review comment: feat: Allow for multiple self-check rails #1874 (comment)

Why this needs follow-up

Using a single config-level lowest_temperature may be too coarse-grained for:

• multiple sequential self-check tasks with different prompt characteristics
• future task-specific tuning needs
• clearer ownership of temperature behavior inside self-check execution paths

Affected areas

• nemoguardrails/library/self_check/input_check/actions.py
• nemoguardrails/library/self_check/output_check/actions.py
• nemoguardrails/library/self_check/utils.py
• any config/schema/docs describing lowest_temperature behavior for self-check rails

Proposed investigation

1. Identify all current call sites that pass config.lowest_temperature into self-check execution.
2. Clarify the intended contract of lowest_temperature:
   • global lower bound for all LLM tasks, or
   • behavior specific to self-check prompts, or
   • candidate for per-task/per-rail override
3. Decide whether self-check tasks should support an explicit temperature override independent of the global config.
4. Update tests and docs to reflect the chosen behavior.

Acceptance criteria

• The intended semantics of lowest_temperature are documented.
• Either:
  • the current config-level behavior is explicitly validated and documented, or
  • self-check task execution is refactored to use a more appropriate task-/rail-level configuration.
• Tests cover the selected behavior.
• The implementation for input and output self-checks is consistent.

Requested by: @Pouyanpi

[rehan243]

oh interesting, what’s your config schema look like for self-check rails right now? we hit this sort of thing in our RAG pipeline — turns out global vs per-task temperature gets messy fast, especially when you chain LLM actions with wildly different prompt types. honestly, setting lowest_temperature at the config level started biting us as soon as we added multi-step validation (one step needed 0.1, another was way too brittle below 0.3). the thing is, if you keep it global, you’ll end up leaking weird prompt artifacts downstream and can't really tune for edge cases.

not gonna lie, took us a few tries to get sane defaults. we ended up passing temperature explicitly per task, and fallback to config only if not set — so tasks like input_check and output_check could behave independently. if you go this route, you'll need to update docs and schemas though, otherwise people get confused which temperature's actually in play.

fwiw, I’d scan all call sites in those actions.py/utils.py files and see if anyone’s already doing per-task overrides. we found a couple edge cases where someone slipped in a hardcoded value and it broke the chain.