Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pkg/ottl] Added support for timezone in Time converter #32479

Merged
merged 23 commits into from
May 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
ddbf8a9
Added support for timezone
michalpristas Apr 17, 2024
4625f7a
more tests
michalpristas Apr 17, 2024
ff8b0ec
dot
michalpristas Apr 17, 2024
a46eb2a
changelog
michalpristas Apr 17, 2024
8c64310
more clear docs
michalpristas Apr 17, 2024
7072f82
do not import
michalpristas Apr 22, 2024
d53abe2
Merge branch 'main' of github.com:open-telemetry/opentelemetry-collec…
michalpristas Apr 22, 2024
55c269b
readme notice about tzdata
michalpristas Apr 22, 2024
1a982a9
Merge branch 'main' into ottl/time-timezone
michalpristas Apr 23, 2024
dd1d207
Merge branch 'main' into ottl/time-timezone
michalpristas Apr 24, 2024
d247407
Merge branch 'main' into ottl/time-timezone
michalpristas Apr 25, 2024
07c1dd4
Update pkg/ottl/ottlfuncs/README.md
michalpristas Apr 30, 2024
e8782e6
Update pkg/ottl/ottlfuncs/README.md
michalpristas Apr 30, 2024
97ab09a
Update pkg/ottl/ottlfuncs/README.md
michalpristas Apr 30, 2024
180f804
enable getting location from attribute
michalpristas Apr 30, 2024
55304e5
Merge branch 'main' of github.com:open-telemetry/opentelemetry-collec…
michalpristas Apr 30, 2024
b48bc9e
Merge branch 'main' into ottl/time-timezone
michalpristas May 2, 2024
4a084c9
Merge branch 'main' into ottl/time-timezone
michalpristas May 6, 2024
77725b9
Merge branch 'main' of github.com:open-telemetry/opentelemetry-collec…
michalpristas May 7, 2024
48d2a49
make location string again
michalpristas May 9, 2024
9486e66
Merge branch 'ottl/time-timezone' of github.com:michalpristas/opentel…
michalpristas May 9, 2024
6755997
Apply suggestions from code review
michalpristas May 9, 2024
dce1647
Merge branch 'main' into ottl/time-timezone
michalpristas May 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions .chloggen/ottl-time-timezone.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Use this changelog template to create an entry for release notes.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: enhancement

# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
component: pkg/ottl

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Added support for timezone in Time converter

# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
issues: [32140]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:

# If your change doesn't affect end users or the exported elements of any package,
# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
# Optional: The change log or logs in which this entry should be included.
# e.g. '[user]' or '[user, api]'
# Include 'user' if the change is relevant to end users.
# Include 'api' if there is a change to a library API.
# Default: '[user]'
change_logs: [user]
15 changes: 13 additions & 2 deletions pkg/ottl/ottlfuncs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1131,11 +1131,11 @@ Examples:

### Time

`Time(target, format)`
`Time(target, format, Optional[location])`

The `Time` Converter takes a string representation of a time and converts it to a Golang `time.Time`.

`target` is a string. `format` is a string.
`target` is a string. `format` is a string, `location` is an optional string.

If either `target` or `format` are nil, an error is returned. The parser used is the parser at [internal/coreinternal/parser](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/internal/coreinternal/timeutils). If the `target` and `format` do not follow the parsing rules used by this parser, an error is returned.

Expand Down Expand Up @@ -1176,13 +1176,24 @@ If either `target` or `format` are nil, an error is returned. The parser used is
|`%%` | A % sign | |
|`%c` | Date and time representation | Mon Jan 02 15:04:05 2006 |

`location` specifies a default time zone canonical ID to be used for date parsing in case it is not part of `format`.

When loading `location`, this function will look for the IANA Time Zone database in the following locations in order:
- a directory or uncompressed zip file named by the ZONEINFO environment variable
- on a Unix system, the system standard installation location
- $GOROOT/lib/time/zoneinfo.zip
- the `time/tzdata` package, if it was imported.

When building a Collector binary, importing `time/tzdata` in any Go source file will bundle the database into the binary, which guarantees the lookups will work regardless of the setup on the host setup. Note this will add roughly 500kB to binary size.

Examples:

- `Time("02/04/2023", "%m/%d/%Y")`
- `Time("Feb 15, 2023", "%b %d, %Y")`
- `Time("2023-05-26 12:34:56 HST", "%Y-%m-%d %H:%M:%S %Z")`
- `Time("1986-10-01T00:17:33 MST", "%Y-%m-%dT%H:%M:%S %Z")`
- `Time("2012-11-01T22:08:41+0000 EST", "%Y-%m-%dT%H:%M:%S%z %Z")`
- `Time("2023-05-26 12:34:56", "%Y-%m-%d %H:%M:%S", "America/New_York")`

### TraceID

Expand Down
17 changes: 12 additions & 5 deletions pkg/ottl/ottlfuncs/func_time.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,9 @@ import (
)

type TimeArguments[K any] struct {
Time ottl.StringGetter[K]
Format string
Time ottl.StringGetter[K]
Format string
Location ottl.Optional[string]
evan-bradley marked this conversation as resolved.
Show resolved Hide resolved
}

func NewTimeFactory[K any]() ottl.Factory[K] {
Expand All @@ -26,14 +27,20 @@ func createTimeFunction[K any](_ ottl.FunctionContext, oArgs ottl.Arguments) (ot
return nil, fmt.Errorf("TimeFactory args must be of type *TimeArguments[K]")
}

return Time(args.Time, args.Format)
return Time(args.Time, args.Format, args.Location)
}

func Time[K any](inputTime ottl.StringGetter[K], format string) (ottl.ExprFunc[K], error) {
func Time[K any](inputTime ottl.StringGetter[K], format string, location ottl.Optional[string]) (ottl.ExprFunc[K], error) {
if format == "" {
return nil, fmt.Errorf("format cannot be nil")
}
loc, err := timeutils.GetLocation(nil, &format)
var defaultLocation *string
if !location.IsEmpty() {
l := location.Get()
defaultLocation = &l
}

loc, err := timeutils.GetLocation(defaultLocation, &format)
if err != nil {
return nil, err
}
Expand Down
64 changes: 61 additions & 3 deletions pkg/ottl/ottlfuncs/func_time_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,15 @@ import (
)

func Test_Time(t *testing.T) {
locationAmericaNewYork, _ := time.LoadLocation("America/New_York")
locationAsiaShanghai, _ := time.LoadLocation("Asia/Shanghai")

tests := []struct {
name string
time ottl.StringGetter[any]
format string
expected time.Time
location string
}{
{
name: "simple short form",
Expand Down Expand Up @@ -151,10 +155,47 @@ func Test_Time(t *testing.T) {
format: "%Y/%m/%d",
expected: time.Date(2022, 01, 01, 0, 0, 0, 0, time.Local),
},
{
name: "with location - America",
time: &ottl.StandardStringGetter[any]{
Getter: func(_ context.Context, _ any) (any, error) {
return "2023-05-26 12:34:56", nil
},
},
format: "%Y-%m-%d %H:%M:%S",
location: "America/New_York",
expected: time.Date(2023, 5, 26, 12, 34, 56, 0, locationAmericaNewYork),
},
{
name: "with location - Asia",
time: &ottl.StandardStringGetter[any]{
Getter: func(_ context.Context, _ any) (any, error) {
return "2023-05-26 12:34:56", nil
},
},
format: "%Y-%m-%d %H:%M:%S",
location: "Asia/Shanghai",
expected: time.Date(2023, 5, 26, 12, 34, 56, 0, locationAsiaShanghai),
},
{
name: "RFC 3339 in custom format before 2000, ignore default location",
time: &ottl.StandardStringGetter[any]{
Getter: func(_ context.Context, _ any) (any, error) {
return "1986-10-01T00:17:33 MST", nil
},
},
location: "Asia/Shanghai",
format: "%Y-%m-%dT%H:%M:%S %Z",
expected: time.Date(1986, 10, 01, 00, 17, 33, 00, time.FixedZone("MST", -7*60*60)),
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
exprFunc, err := Time(tt.time, tt.format)
var locOptional ottl.Optional[string]
if tt.location != "" {
locOptional = ottl.NewTestingOptional(tt.location)
}
exprFunc, err := Time(tt.time, tt.format, locOptional)
assert.NoError(t, err)
result, err := exprFunc(nil, nil)
assert.NoError(t, err)
Expand Down Expand Up @@ -193,7 +234,8 @@ func Test_TimeError(t *testing.T) {
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
exprFunc, err := Time[any](tt.time, tt.format)
var locOptional ottl.Optional[string]
exprFunc, err := Time[any](tt.time, tt.format, locOptional)
require.NoError(t, err)
_, err = exprFunc(context.Background(), nil)
assert.ErrorContains(t, err, tt.expectedError)
Expand All @@ -207,6 +249,7 @@ func Test_TimeFormatError(t *testing.T) {
time ottl.StringGetter[any]
format string
expectedError string
location string
}{
{
name: "invalid short with no format",
Expand All @@ -218,10 +261,25 @@ func Test_TimeFormatError(t *testing.T) {
format: "",
expectedError: "format cannot be nil",
},
{
name: "with unknown location",
time: &ottl.StandardStringGetter[any]{
Getter: func(_ context.Context, _ any) (any, error) {
return "2023-05-26 12:34:56", nil
},
},
format: "%Y-%m-%d %H:%M:%S",
location: "Jupiter/Ganymede",
expectedError: "unknown time zone Jupiter/Ganymede",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
_, err := Time[any](tt.time, tt.format)
var locOptional ottl.Optional[string]
if tt.location != "" {
locOptional = ottl.NewTestingOptional(tt.location)
}
_, err := Time[any](tt.time, tt.format, locOptional)
assert.ErrorContains(t, err, tt.expectedError)
})
}
Expand Down