[BUG] Exported Timestamp field is shown as empty brackets with report generation to CSV

[pantinor]

When generating a report from a saved search query that includes the @timestamp field, the results in the CSV are empty brackets instead of the formatted data time field as seen in the UI Discovery section of the same query.

Is it as known issue?

Using 2.13.0 but was not present in earlier build.
Noting that we tested with 2.9.0 and it works fine.

[pantinor]

@kavilla Can you help triaging this timestamp issue? I saw some backport PRs related to timestamp function but not sure because it was only referencing tests.

[jpelletier412]

Was wondering if there has been any investigation on this issue. We are experiencing the same thing. We originally noticed this in 2.13.0 and just recently upgraded to 2.14.0 and the issue is still present. We are using the download feature for monitoring trace messages and without timestamp present, the CSV download is useless in our case. Any update on this would be appreciated!

[dblock]

Doesn't look like anyone has had a chance to look at it. Your help would be greatly appreciated.

Catch All Triage - 1 2 3 4 5 6

[pantinor]

@dblock We can try to investigate, can someone point us to the relevant code source that might be good starting point? We are not very familiar with the source layout here.

[rayangou]

Issue:
The getOpenSearchData method in the dataReportHelpers.ts file doesn't handle cases where the time field is in epoch format (type=number).

Method: getOpenSearchData
File:
https://github.com/opensearch-project/dashboards-reporting/blob/main/server/routes/utils/dataReportHelpers.ts#L144
https://github.com/opensearch-project/dashboards-reporting/blob/main/server/routes/utils/dataReportHelpers.ts#L166

Proposed Solution:
Modify the condition to check for both string and number types:

if (typeof dateValue === 'string' || typeof dateValue === 'number') {
  // Existing code for handling the date value
}

[jpelletier412]

We were able to implement a workaround and I wanted to post here to help any others who needs a workaround until issue is fixed
We use Logstash to ingest our data, so during the pipeline ingestion, we create a new field and then base that value off the timestamp value. We then use Ruby to convert the date type into a string representation of the timestamp. We then define our report download to use that new field and everything works. Example of the implementation in our pipeline:

filter {
     mutate {
            add_field => {
                "timestamp_string" => "%{[timestamp]}"
            }
        }
        ruby {
            code => '
                t = Time.at(event.get("@timestamp").to_f)
                event.set("timestamp_string", t.strftime("%b %d,%Y @ %H:%M:%S.%L"))
            '
        }
}