Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

util/log: new config field format-options, custom timezones in log output #104265

Merged
merged 11 commits into from
Jun 14, 2023
357 changes: 94 additions & 263 deletions docs/generated/logformats.md

Large diffs are not rendered by default.

6 changes: 5 additions & 1 deletion docs/generated/logsinks.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,7 @@ Configuration options shared across all sink types:
|--|--|
| `filter` | specifies the default minimum severity for log events to be emitted to this sink, when not otherwise specified by the 'channels' sink attribute. |
| `format` | the entry format to use. |
| `format-options` | additional options for the format. |
| `redact` | whether to strip sensitive information before log events are emitted to this sink. |
| `redactable` | whether to keep redaction markers in the sink's output. The presence of redaction markers makes it possible to strip sensitive data reliably. |
| `exit-on-error` | whether the logging system should terminate the process if an error is encountered while writing to this sink. |
Expand Down Expand Up @@ -163,6 +164,7 @@ Configuration options shared across all sink types:
|--|--|
| `filter` | specifies the default minimum severity for log events to be emitted to this sink, when not otherwise specified by the 'channels' sink attribute. |
| `format` | the entry format to use. |
| `format-options` | additional options for the format. |
| `redact` | whether to strip sensitive information before log events are emitted to this sink. |
| `redactable` | whether to keep redaction markers in the sink's output. The presence of redaction markers makes it possible to strip sensitive data reliably. |
| `exit-on-error` | whether the logging system should terminate the process if an error is encountered while writing to this sink. |
Expand Down Expand Up @@ -228,6 +230,7 @@ Configuration options shared across all sink types:
|--|--|
| `filter` | specifies the default minimum severity for log events to be emitted to this sink, when not otherwise specified by the 'channels' sink attribute. |
| `format` | the entry format to use. |
| `format-options` | additional options for the format. |
| `redact` | whether to strip sensitive information before log events are emitted to this sink. |
| `redactable` | whether to keep redaction markers in the sink's output. The presence of redaction markers makes it possible to strip sensitive data reliably. |
| `exit-on-error` | whether the logging system should terminate the process if an error is encountered while writing to this sink. |
Expand Down Expand Up @@ -277,7 +280,7 @@ Type-specific configuration options:
| Field | Description |
|--|--|
| `channels` | the list of logging channels that use this sink. See the [channel selection configuration](#channel-format) section for details. |
| `no-color` | forces the omission of VT color codes in the output even when stderr is a terminal. |
| `no-color` | forces the omission of VT color codes in the output even when stderr is a terminal. This option is deprecated; its effects are equivalent to 'format-options: {colors: none}'. |


Configuration options shared across all sink types:
Expand All @@ -286,6 +289,7 @@ Configuration options shared across all sink types:
|--|--|
| `filter` | specifies the default minimum severity for log events to be emitted to this sink, when not otherwise specified by the 'channels' sink attribute. |
| `format` | the entry format to use. |
| `format-options` | additional options for the format. |
| `redact` | whether to strip sensitive information before log events are emitted to this sink. |
| `redactable` | whether to keep redaction markers in the sink's output. The presence of redaction markers makes it possible to strip sensitive data reliably. |
| `exit-on-error` | whether the logging system should terminate the process if an error is encountered while writing to this sink. |
Expand Down
1 change: 1 addition & 0 deletions pkg/util/flagutil/flagutil.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ import (
// as described in time.Parse.
var TimeFormats = []string{
log.MessageTimeFormat,
log.MessageTimeFormatWithTZ,
log.FileTimeFormat,
time.RFC3339Nano,
time.RFC3339,
Expand Down
25 changes: 22 additions & 3 deletions pkg/util/log/flags.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ import (
"fmt"
"io/fs"
"math"
"strings"

"github.com/cockroachdb/cockroach/pkg/util/envutil"
"github.com/cockroachdb/cockroach/pkg/util/log/channel"
Expand All @@ -24,6 +25,7 @@ import (
"github.com/cockroachdb/cockroach/pkg/util/log/severity"
"github.com/cockroachdb/cockroach/pkg/util/syncutil"
"github.com/cockroachdb/errors"
"github.com/cockroachdb/redact"
)

type config struct {
Expand Down Expand Up @@ -132,7 +134,7 @@ func ApplyConfig(config logconfig.Config) (logShutdownFn func(), err error) {

// Call the final value of logShutdownFn immediately if returning with error.
defer func() {
if err != nil {
if err != nil && logShutdownFn != nil {
logShutdownFn()
}
}()
Expand Down Expand Up @@ -248,6 +250,17 @@ func ApplyConfig(config logconfig.Config) (logShutdownFn func(), err error) {
if err := logging.stderrSinkInfoTemplate.applyConfig(config.Sinks.Stderr.CommonSinkConfig); err != nil {
return nil, err
}
if config.Sinks.Stderr.NoColor {
// This branch exists for backward compatibility with CockroachDB
// v23.1 and previous versions. The same effect can be obtained
// using 'format-options: {colors: none}'.
switch t := logging.stderrSinkInfoTemplate.formatter.(type) {
case *formatCrdbV1:
t.colorProfile = nil
case *formatCrdbV2:
t.colorProfile = nil
}
}
logging.stderrSinkInfoTemplate.applyFilters(config.Sinks.Stderr.Channels)

// Create the per-channel loggers.
Expand Down Expand Up @@ -434,9 +447,15 @@ func (l *sinkInfo) applyConfig(c logconfig.CommonSinkConfig) error {
l.criticality = *c.Criticality
f, ok := formatters[*c.Format]
if !ok {
return errors.Newf("unknown format: %q", *c.Format)
return errors.WithHintf(errors.Newf("unknown format: %q", *c.Format),
"Supported formats: %s.", redact.Safe(strings.Join(formatNames, ", ")))
}
l.formatter = f()
for k, v := range c.FormatOptions {
if err := l.formatter.setOption(k, v); err != nil {
return err
}
}
l.formatter = f
return nil
}

Expand Down
6 changes: 5 additions & 1 deletion pkg/util/log/format_crdb.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,10 @@ const severityChar = "IWEF"
// as used in time.Parse and time.Format.
const MessageTimeFormat = "060102 15:04:05.999999"

// MessageTimeFormatWithTZ is like MessageTimeFormat but with a numeric
// time zone included.
const MessageTimeFormatWithTZ = "060102 15:04:05.999999-070000"

// FormatLegacyEntry writes the contents of the legacy log entry struct to the specified writer.
func FormatLegacyEntry(e logpb.Entry, w io.Writer) error {
return FormatLegacyEntryWithOptionalColors(e, w, nil /* cp */)
Expand All @@ -37,7 +41,7 @@ func FormatLegacyEntry(e logpb.Entry, w io.Writer) error {
// FormatLegacyEntryWithOptionalColors is like FormatLegacyEntry but the caller can specify
// a color profile.
func FormatLegacyEntryWithOptionalColors(e logpb.Entry, w io.Writer, cp ttycolor.Profile) error {
buf := formatLogEntryInternalV1(e, false /* isHeader */, true /* showCounter */, cp)
buf := formatLogEntryInternalV1(e, false /* isHeader */, true /* showCounter */, cp, nil /* loc */)
defer putBuffer(buf)
_, err := w.Write(buf.Bytes())
return err
Expand Down
Loading