feat: Implement enterprise data protection features for log sanitization

README.md

@@ -40,6 +40,7 @@ Whether it’s a compilation error, test failure, or deployment hiccup, this plu
 * **One-click error analysis** on any console output
 * **Pipeline-ready** with a simple `explainError()` step
 * **Workspace Context** *(opt-in)* — include selected workspace files for more accurate explanations
+* **Enterprise data protection** — sanitize logs before LLM calls, preview console payloads before sending, audit sent payloads, and enforce allow/deny regex policies
 * **AI auto-fix** *(experimental)* — automatically opens a pull request on GitHub, GitLab, or Bitbucket with AI-generated code changes when a build fails
 * **AI-powered explanations** via Anthropic Claude, AWS Bedrock, Azure OpenAI, DeepSeek, Google Gemini, Microsoft Foundry, Ollama, OpenAI GPT models, Qwen, or generic Okta-authenticated company AI gateways
 * **Folder-level configuration** so teams can use project-specific settings
@@ -81,9 +82,16 @@ Whether it’s a compilation error, test failure, or deployment hiccup, this plu
 | **API URL** | AI service endpoint | **Leave empty** for official APIs where supported. **Required for Custom Okta AI and Ollama providers.** Optional Bedrock Runtime endpoint override for private VPC endpoints. |
 | **AI Model** | Model to use for analysis | *Required*.  Specify the model name offered by your selected AI provider |
 | **Custom Context** | Additional instructions or context for the AI (e.g., KB article links, organization-specific troubleshooting steps) | *Optional*. Can be overridden at the job level. |
+| **Enable Log Sanitization** | Redact secrets, internal URLs, private repo URLs, local paths, and customer email addresses before provider calls | ✅ Enabled |
+| **Preview Payload Before Sending** | Require console action users to review the sanitized payload before it is sent | Disabled |
+| **Audit Sent Payload** | Store the sanitized log payload with the build explanation action | ✅ Enabled |
+| **Payload Allow Regex** | Optional allow policy; only matching payload lines are sent | Empty |
+| **Payload Deny Regex** | Optional deny policy; matching text is replaced with `[REDACTED_SECRET]` | Empty |
 
 `Custom Okta AI` adds provider-specific fields for `Okta Token URL`, `Client ID`, `Client Secret`, and optional `Scope`, `API Version`, `App Key`, and custom access-token header settings. This is intended for generic company AI gateways that require an OAuth client-credentials exchange before the chat call.
 
+See [Enterprise Data Protection](docs/data-protection.md) for sanitizer, preview, audit, and regex policy details.
+
 4. Click **"Test Configuration"** to verify your setup
 5. Save the configuration
 
@@ -534,8 +542,9 @@ Enable debug logs:
 
 1. Use `explainError()` in `post { failure { ... } }` blocks
 2. Apply `logPattern` to focus on relevant errors
-3. Monitor usage metrics and quota outcomes to control costs (see [AI Provider Call Quotas](docs/usage-quota.md))
-4. Keep plugin updated regularly
+3. Keep log sanitization enabled and review sent payload audits for sensitive jobs
+4. Monitor usage metrics and quota outcomes to control costs (see [AI Provider Call Quotas](docs/usage-quota.md))
+5. Keep plugin updated regularly
 
 ## Support & Community
 

docs/data-protection.md

@@ -0,0 +1,40 @@
+# Enterprise Data Protection
+
+Explain Error sanitizes data before sending build failure context to an AI provider.
+
+## What is protected
+
+Log sanitization is enabled by default and replaces matched sensitive values with:
+
+```text
+[REDACTED_SECRET]
+```
+
+The default rules redact common credential assignments, bearer/basic authorization headers, GitHub and AWS-style tokens, private key blocks, internal URLs, private repository URLs, local artifact/workspace paths, and email addresses.
+
+## Preview before sending
+
+Enable **Preview Payload Before Sending** in global configuration to require console action users to review the sanitized payload before the provider call starts. The preview shows line count, redaction count, dropped line count, and the sanitized payload excerpt.
+
+Pipeline steps are non-interactive, so they sanitize and audit automatically instead of prompting.
+
+## Audit what was sent
+
+When **Audit Sent Payload** is enabled, the build's **AI Error Explanation** action stores the sanitized log payload that was sent to the provider, along with redaction and dropped-line counts. The original raw logs are not persisted by this audit field.
+
+Disable this option if your organization does not want sanitized payloads stored on build records.
+
+## Allow and deny regex policies
+
+Use **Payload Allow Regex** to send only matching payload lines. Non-matching lines are dropped before redaction.
+
+Use **Payload Deny Regex** to redact organization-specific sensitive text that is not covered by the default rules.
+
+Examples:
+
+```text
+Payload Allow Regex: (?i)(error|failed|exception|caused by)
+Payload Deny Regex: (?i)(customer-[0-9]+|tenant-[a-z0-9-]+)
+```
+
+Invalid regular expressions are rejected on the configuration page.

src/main/java/io/jenkins/plugins/explain_error/ConsoleExplainErrorAction.java

@@ -92,11 +92,7 @@
             }
 
             // Optionally allow maxLines as a parameter, default to 200
-            int maxLines = 200;
-            String maxLinesParam = req.getParameter("maxLines");
-            if (maxLinesParam != null) {
-                try { maxLines = Integer.parseInt(maxLinesParam); } catch (NumberFormatException ignore) {}
-            }
+            int maxLines = parseMaxLines(req);
 
             // Fetch the last N lines of the log
             PipelineLogExtractor logExtractor = new PipelineLogExtractor(run, maxLines, Jenkins.getAuthentication2(),
@@ -120,6 +116,40 @@
         }
     }
 
+    /**
+     * AJAX endpoint to preview the sanitized payload before sending it to the AI provider.
+     */
+    @RequirePOST
+    public void doPreviewConsolePayload(StaplerRequest2 req, StaplerResponse2 rsp) throws IOException {
+        try {
+            run.checkPermission(hudson.model.Item.READ);
+
+            int maxLines = parseMaxLines(req);
+            PipelineLogExtractor logExtractor = new PipelineLogExtractor(run, maxLines, Jenkins.getAuthentication2(),
+                    false, null);
+            List<String> logLines = logExtractor.getFailedStepLog();
+            this.urlString = logExtractor.getUrl();
+
+            ErrorExplainer explainer = new ErrorExplainer();
+            LogSanitizer.SanitizedPayload sanitizedPayload = explainer.previewPayload(String.join("\n", logLines));
+
+            rsp.setContentType("application/json");
+            rsp.setCharacterEncoding("UTF-8");
+            PrintWriter writer = rsp.getWriter();
+            JSONObject json = new JSONObject();
+            json.put("status", "success");
+            json.put("message", sanitizedPayload.text());
+            json.put("redactionCount", sanitizedPayload.redactionCount());
+            json.put("droppedLineCount", sanitizedPayload.droppedLineCount());
+            json.put("sentLineCount", sanitizedPayload.sentLineCount());
+            writer.write(json.toString());
+            writer.flush();
+        } catch (Exception e) {
+            LOGGER.severe("Error previewing console payload: " + e.getMessage());
+            writeJsonResponse(rsp, "error", "Unknown", "Error: " + e.getMessage());
+        }
+    }
+
     /**
      * AJAX endpoint to check build status.
      * Returns JSON with buildingStatus to determine if button should be shown. 0 - SUCCESS, 1 - RUNNING, 2 - FINISHED and FAILURE
@@ -167,6 +197,19 @@
         writer.flush();
     }
 
+    private int parseMaxLines(StaplerRequest2 req) {
+        int maxLines = 200;
+        String maxLinesParam = req.getParameter("maxLines");
+        if (maxLinesParam != null) {
+            try {
+                maxLines = Integer.parseInt(maxLinesParam);
+            } catch (NumberFormatException ignore) {
+                // Keep the default when the optional parameter is invalid.
+            }
+        }
+        return maxLines;
+    }
+
     /**
      * Create a response indicating this is a cached result.
      * @param explanation The cached explanation

src/main/java/io/jenkins/plugins/explain_error/ConsolePageDecorator.java

@@ -38,6 +38,10 @@ public String getProviderName() {
         return GlobalConfigurationImpl.get().getAiProvider().getProviderName();
     }
 
+    public final boolean isPayloadPreviewEnabled() {
+        return GlobalConfigurationImpl.get().isEnablePayloadPreview();
+    }
+
     /**
      * Helper method used by jelly to checked if we're on a console url.
      */

src/main/java/io/jenkins/plugins/explain_error/ErrorExplainer.java

@@ -15,6 +15,7 @@
 import java.util.logging.Level;
 import java.util.logging.Logger;
 import java.util.regex.Pattern;
+import java.util.regex.PatternSyntaxException;
 import jenkins.model.Jenkins;
 import org.apache.commons.lang3.StringUtils;
 import org.springframework.security.core.Authentication;
@@ -31,6 +32,7 @@
     private String urlString;
     private String lastErrorLogs;
     private final UsageRecorder usageRecorder;
+    private final LogSanitizer logSanitizer = new LogSanitizer();
 
     private static final Logger LOGGER = Logger.getLogger(ErrorExplainer.class.getName());
 
@@ -143,18 +145,21 @@
             PipelineLogExtractor.ExtractionResult extractionResult = extractErrorLogs(run, maxLines,
                     collectDownstreamLogs, downstreamJobPattern, authentication);
             String errorLogs = filterErrorLogs(extractionResult.logLines(), logPattern);
-            this.lastErrorLogs = errorLogs;
-            inputLogLineCount = countLines(errorLogs);
             logExtractionSummary(listener, extractionResult, maxLines);
 
             // Use step-level customContext if provided, otherwise fallback to global
             String effectiveCustomContext = StringUtils.isNotBlank(customContext) ? customContext : GlobalConfigurationImpl.get().getCustomContext();
             logToConsole(listener, "Custom context source: " + resolveCustomContextSource(customContext) + ".");
+            PreparedPayload preparedPayload = preparePayload(errorLogs, effectiveCustomContext);
+            this.lastErrorLogs = preparedPayload.errorLogs().text();
+            inputLogLineCount = preparedPayload.errorLogs().sentLineCount();
+            logSanitizationSummary(listener, preparedPayload);
 
             // Get AI explanation
             try {
                 logToConsole(listener, "Sending AI request.");
-                String explanation = provider.explainError(errorLogs, listener, language, effectiveCustomContext,
+                String explanation = provider.explainError(preparedPayload.errorLogs().text(), listener, language,
+                        preparedPayload.customContext().text(),
                         run != null ? run.getParent() : null, null);
                 LOGGER.fine(jobInfo + " AI error explanation succeeded.");
                 logToConsole(listener, "AI request completed successfully.");
@@ -167,8 +172,8 @@
                 }
 
                 // Store explanation in build action
-                ErrorExplanationAction action = new ErrorExplanationAction(explanation, urlString, errorLogs,
-                        provider.getProviderName(), provider.getModel(), inputLogLineCount);
+                ErrorExplanationAction action = createExplanationAction(explanation, errorLogs, preparedPayload,
+                        provider);
                 run.addOrReplaceAction(action);
                 logToConsole(listener, buildSavedExplanationMessage(run, action));
                 recordUsage(entryPoint, UsageEvent.Result.SUCCESS, provider, startTimeNanos, inputLogLineCount,
@@ -188,6 +193,10 @@
             LOGGER.severe(jobInfo + " Failed to explain error: " + e.getMessage());
             logToConsole(listener, "Failed to explain error: " + e.getMessage());
             return null;
+        } catch (PatternSyntaxException e) {
+            LOGGER.severe(jobInfo + " Invalid payload protection regex: " + e.getMessage());
+            logToConsole(listener, "Invalid payload protection regex: " + e.getDescription());
+            return null;
         }
     }
 
@@ -249,6 +258,7 @@
                                             UsageEvent.EntryPoint entryPoint)
             throws IOException, ExplanationException {
         String jobInfo ="[" + run.getParent().getFullName() + " #" + run.getNumber() + "]";
+        this.urlString = url;
         long startTimeNanos = System.nanoTime();
         int inputLogLineCount = countLines(errorText);
         ProviderResolution providerResolution = resolveProvider(run);
@@ -283,12 +293,14 @@
 
         try {
             // Get AI explanation with global custom context
-            String explanation = provider.explainError(errorText, new LogTaskListener(LOGGER, Level.FINE), null,
-                    GlobalConfigurationImpl.get().getCustomContext(), run.getParent(), null);
+            PreparedPayload preparedPayload = preparePayload(errorText, GlobalConfigurationImpl.get().getCustomContext());
+            inputLogLineCount = preparedPayload.errorLogs().sentLineCount();
+            String explanation = provider.explainError(preparedPayload.errorLogs().text(),
+                    new LogTaskListener(LOGGER, Level.FINE), null, preparedPayload.customContext().text(),
+                    run.getParent(), null);
             LOGGER.fine(jobInfo + " AI error explanation succeeded.");
             LOGGER.fine("Explanation length: " + explanation.length());
-            ErrorExplanationAction action = new ErrorExplanationAction(explanation, url, errorText,
-                    provider.getProviderName(), provider.getModel(), inputLogLineCount);
+            ErrorExplanationAction action = createExplanationAction(explanation, errorText, preparedPayload, provider);
             run.addOrReplaceAction(action);
             run.save();
             recordUsage(entryPoint, UsageEvent.Result.SUCCESS, provider, startTimeNanos, inputLogLineCount, false);
@@ -301,6 +313,60 @@
         }
     }
 
+    public final LogSanitizer.SanitizedPayload previewPayload(String errorText) {
+        PreparedPayload preparedPayload = preparePayload(errorText, GlobalConfigurationImpl.get().getCustomContext());
+        return new LogSanitizer.SanitizedPayload(buildAuditedPayload(preparedPayload),
+                totalRedactions(preparedPayload), totalDroppedLines(preparedPayload),
+                preparedPayload.errorLogs().sentLineCount());
+    }
+
+    final LogSanitizer.SanitizedPayload sanitizeForProvider(String payload) {
+        return logSanitizer.sanitize(payload, GlobalConfigurationImpl.get().getLogSanitizerPolicy());
+    }
+
+    private PreparedPayload preparePayload(String errorLogs, String customContext) {
+        LogSanitizer.Policy policy = GlobalConfigurationImpl.get().getLogSanitizerPolicy();
+        return new PreparedPayload(logSanitizer.sanitize(errorLogs, policy),
+                logSanitizer.sanitize(customContext, policy));
+    }
+
+    private ErrorExplanationAction createExplanationAction(String explanation, String originalErrorLogs,
+                                                           PreparedPayload preparedPayload,
+                                                           BaseAIProvider provider) {
+        String auditedPayload = GlobalConfigurationImpl.get().isAuditSentPayload()
+                ? buildAuditedPayload(preparedPayload) : null;
+        return new ErrorExplanationAction(explanation, urlString, originalErrorLogs, auditedPayload,
+                provider.getProviderName(), provider.getModel(), preparedPayload.errorLogs().sentLineCount(),
+                totalRedactions(preparedPayload), totalDroppedLines(preparedPayload));
+    }
+
+    private String buildAuditedPayload(PreparedPayload preparedPayload) {
+        String payload = preparedPayload.errorLogs().text();
+        if (StringUtils.isBlank(preparedPayload.customContext().text())) {
+            return payload;
+        }
+        return payload + "\n\n--- CUSTOM CONTEXT SENT TO AI ---\n" + preparedPayload.customContext().text();
+    }
+
+    private int totalRedactions(PreparedPayload preparedPayload) {
+        return preparedPayload.errorLogs().redactionCount() + preparedPayload.customContext().redactionCount();
+    }
+
+    private int totalDroppedLines(PreparedPayload preparedPayload) {
+        return preparedPayload.errorLogs().droppedLineCount() + preparedPayload.customContext().droppedLineCount();
+    }
+
+    private void logSanitizationSummary(TaskListener listener, PreparedPayload preparedPayload) {
+        if (!GlobalConfigurationImpl.get().isEnableLogSanitization()) {
+            logToConsole(listener, "Log sanitization is disabled.");
+            return;
+        }
+
+        logToConsole(listener, "Sanitized AI payload; redacted " + totalRedactions(preparedPayload)
+                + " sensitive value(s), dropped " + totalDroppedLines(preparedPayload)
+                + " line(s) by allow policy.");
+    }
+
     /**
      * Resolve the AI provider to use for error explanation.
      * Resolution order:
@@ -514,6 +580,10 @@
         return lineCount;
     }
 
+    private record PreparedPayload(LogSanitizer.SanitizedPayload errorLogs,
+                                   LogSanitizer.SanitizedPayload customContext) {
+    }
+
     private record ProviderResolution(@CheckForNull BaseAIProvider provider, String sourceLabel) {
     }
 }