Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improvement(coercer transform): add a second chrono format option to convert from datetime to string #868

Conversation

xcaptain
Copy link

This is an attempt to fix #867 feel free to close if it's inappropriate

@binarylogic binarylogic requested a review from bruceg September 16, 2019 14:06
@binarylogic
Copy link
Contributor

binarylogic commented Sep 16, 2019

Thanks @xcaptain. I think the format is a little confusing given that the user is producing a string, not a timestamp. If I understand correctly, you basically want to re-format a string timestamp by converting it to a timestamp and the converting it back to a string in a different format?

@binarylogic binarylogic requested review from lukesteensen and removed request for bruceg September 16, 2019 14:08
@binarylogic binarylogic changed the title add a second chrono format option to convert from datetime to string improvement(coercer transform): add a second chrono format option to convert from datetime to string Sep 16, 2019
@xcaptain
Copy link
Author

Yes, the first format convert a string to datetime and the second format convert datetime to another string. I will give my case as an example:

I'm using vector to collect nginx access log and then store to clickhouse. nginx uses $time_iso8601 which produces datetime like 2019-09-16T18:17:54+08:00 but by default clickhouse only accept datetime like 2019-09-16 18:17:54, to accept iso 8601 datetime, I have to modify clickhouse settings https://clickhouse.yandex/docs/en/operations/settings/settings/#settings-date_time_input_format I think it's better to let vector do this datetime conversion

@binarylogic
Copy link
Contributor

binarylogic commented Sep 17, 2019

That makes sense. If you wouldn't mind, give us this week to discuss. I think there is a clearer more user-friendly approach to this. The fact that timestamp is used as the explicit type, but then produces a string is confusing. The alternative I'm considering is improving how users can transform data. Ex: coercing to a timestamp and then coercing back to a string. This is being discussed in #750.

@binarylogic binarylogic self-assigned this Sep 23, 2019
@xcaptain xcaptain force-pushed the feature/convert-timestamp-to-string branch from 6735e4e to a683eed Compare September 24, 2019 10:06
@binarylogic binarylogic force-pushed the master branch 2 times, most recently from d1acf03 to 3450767 Compare September 24, 2019 15:04
@xcaptain
Copy link
Author

@binarylogic Is there any methods to convert a timestamp from 2019-09-16T18:17:54+08:00 to 2019-09-16 18:17:54 currently?

@xcaptain xcaptain closed this Sep 26, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add another chrono format to convert timestamp to string
2 participants