Structured logging

The log record contract owns the JSON schema, event names, and correlation semantics. Preserve that contract when changing a log producer or its encoder. Observability principles says why each change ships its signal and what a signal must not carry. Logging is an operational interface: useful severity, trustworthy context, and bounded disclosure matter more than a console with no warnings.

Choose the level

Level	Use
ERROR	An operation failed and needs operator action. Log the failure at the boundary that owns its outcome.
WARN	An operation is degraded, or a guard refused unexpected state. Include its effect and a safe reason.
INFO	A lifecycle transition, a configuration decision, or a batch summary that an operator needs during normal operation.
DEBUG	Per-item progress, successful polls, and capture details. Keep these safe even when an operator enables them.
TRACE	Fine-grained diagnostics with bounded, safe values. Never use this level to expose private data.

A normal return from a degraded state is INFO. Do not log each item at INFO when a lifecycle event or batch summary already records the outcome. Keep an actionable failure visible at WARN or ERROR.

Retries

A failure that the work retries is not yet the outcome of the operation.

• Log each retry at WARN, with the attempt count.
• Log the verdict at ERROR, once per operation: the retry budget is exhausted, or the failure is not retryable.
• A transient outage that retries without a limit stays at WARN. Its metric and alert show that it lasts.
• Log in the retry framework's hook: Reactor's doBeforeRetry and onRetryExhaustedThrow, a Resilience4j onRetry event, a Spring RetryListener, or the scheduler's ErrorHandler. Do not count attempts by hand.
• TransientFailures decides what is transient. Do not write a second list.

The provider-event inbox follows this rule: each failed delivery is a WARN, and the dead letter is the one ERROR. A scheduled job's next run is its retry, so a transient failure of one run is a WARN. Any other failure of a run is an ERROR.

Use the existing framework facilities

Use SLF4J's fluent API for events operators need to query. Keep a stable event.name, a short human-readable message, and typed fields rather than embedding key=value fragments in the message. Use parameterized logging for ordinary diagnostic text. Do not build a second event abstraction or serialize a JSON string inside a log message.

Use {} placeholders for values. Do not concatenate values into a message or call String.format for its text. PMD checks concatenation in SLF4J messages, including fluent messages. Compile-time constant text can span several literals. The architecture tier rejects access to System.out and System.err.

log.atWarn()
        .addKeyValue("event.name", "integration.request.failed")
        .addKeyValue("http.request.method", exception.getMethod().name())
        .addKeyValue("server.address", exception.getUri().getHost())
        .addKeyValue("error.type", exception.getMostSpecificCause().getClass().getName())
        .log("External service request failed");

Hephaestus writes Spring Boot's ECS format through Logback. Boot supports both MDC and the SLF4J key-value API. Two customizers in observability/ complete the format:

• LogContextCustomizer writes the MDC and the event's key-value pairs as one nested context object.
• CredentialMaskingCustomizer masks credential formats in every value.

Put a scope identifier in MDC. Use the key names in the log record contract and StructuredLogKeys where the module boundary permits it. Set context at the boundary that knows the scope: an HTTP request, an inbox handler, a claimed run, or an outbox attempt. Keep the agent run ID and its lease epoch distinct from the attempt count.

Put a fact of one event in its key-value pairs: event.name, an outcome, a duration. An event can name a scope identifier again, for example because it also runs outside the scope. Then the record carries the event's value once. Boot's own context pairs refuse such a name and the appender drops the whole record, so do not turn LogContextCustomizer off.

Keep private data out at the source

Follow OWASP's logging guidance: never log credentials, full request/response bodies, signed URLs, private reviewed work, or raw SQL/row values. Exception messages can contain those same values. At a shared integration boundary, prefer the failure class and safe request metadata to the full exception. Preserve useful failure severity and correlation. Do not replace an operational failure with silence.

Log an exception once, at the boundary that decides the operation's outcome. A helper that rethrows adds safe context to the exception. It does not log the same failure again.

Use a class-only stack trace when frames help diagnosis. The shipped encoder removes exception messages. Never log a person's login, name, email address, or work text. Use an ID.

PMD checks known personal-data accessors and variable names on typed SLF4J logger and builder calls. The rule does not follow messages through local variables or check String.format. The test log scan checks the records that a test emits. These checks do not trace arbitrary text through the application. Review each logged value at its source.

Use fixed field names. Request-controlled keys must not become top-level log keys. The configured encoder escapes string values and keeps MDC in a separate object. This prevents JSON record injection and context collisions. It does not make the underlying string public data. CWE-117 explains why an untrusted value must not change the structure or apparent origin of a log record.

Masking is a last defense. It masks a known sensitive field by name and a known credential format in every value. Patterns must work independently of their order. No regex can reliably recognize every secret or personal datum in arbitrary prose. Never enable request-body debugging as a substitute for safe structured diagnostics.

Scope and restore correlation

Use Boot/Micrometer tracing and the existing task-context propagation. Do not generate an unrelated request ID or invent parent spans. Spring's tracing documentation defines MDC correlation and requires auto-configured HTTP-client builders for propagation. Spring's context-propagation guidance and Logback's MDC manual explain why thread-local context needs explicit lifecycle management when work moves between threads.

Test both directions: context reaches the asynchronous task, and completion or failure restores what was present before the task. Reuse the same worker thread in the regression so a workspace identifier left behind cannot pass unnoticed. Keep high-cardinality identifiers in logs/traces, not metric labels. The OpenTelemetry log data model distinguishes event attributes, severity, resource identity, and trace/span correlation. It does not make log text or an identifier proof of successful export.

Browser and agent logs

The webapp's no-console rule keeps routine browser work out of console logs. Report browser faults through the existing fault boundary, with its closed fields and no page content. Do not add a second browser log exporter. The agent runner uses the diagnostic events and typed fields in pi-diagnostics.json. Do not add prompts, model replies, tool output, credentials, or arbitrary exception text to those records. The server admits only records that match that contract.

Verify the emitted record

RedactionContractTest loads the production encoder configuration and parses its actual bytes. ShippedLogEncoder gives other tests the same encoder. Use it to prove that an event of your own reaches the record with its fields, as ReviewJobTelemetryTest does for the review lifecycle events. Its contract includes typed values, correlation fields, severity isolation, nested exception redaction, opaque sensitive fields, and exactly one JSON record despite control characters.

GlobalControllerAdviceLoggingTest supplies hostile synthetic request/database values and verifies the actual emitted event and unchanged HTTP error contract. GitLab webhook and Outline client regressions exercise their own adapters with private response/transport payloads. The GitLab failure still reports its status. Outline retains its retryability contract. MdcPropagationTest proves propagation and restoration on a reused worker thread, including a failed task.

Run focused regressions first, then the unit baseline and affected server tiers. Also inspect current CI's raw reports: a green test count is not proof that every log is appropriate. Known upstream scanner, buildpack or compiler diagnostics need an explicit disposition, not a blanket logging-level change. Assert test failures in expected-negative fixtures narrowly. Keep unexpected diagnostics visible.