bigquery: Timestamps no longer returned in UTC by default

[bombsimon]

Hi!

Breaking change in v1.59.0 not listed as breaking/intentional.

I'm not sure if this was intentional or if you follow semver for >= v1.0.0 but after bumping a minor version of bigquery our code stopped working as expected because the client no longer returns data the same way. This change is not noted as breaking in the release log.

With the code noted belove, this is my result:

› go get cloud.google.com/go/[email protected]
go: downgraded cloud.google.com/go/bigquery v1.59.0 => v1.58.0

› go run .
1991-01-01 01:01:01 +0000 UTC

› go get cloud.google.com/go/[email protected]
go: upgraded cloud.google.com/go/bigquery v1.58.0 => v1.59.0

› go run .
1991-01-01 02:01:01 +0100 CET

I'm aware of the DATETIME function that allows me to specify timezone, however not all queries is easy enough to manipulate by replacing the selection set to use a function.

Was this intentional? Should I invest time in figuring out a way to update my queries or is there other ways to use UTC by default? Should I convert my timestamps to UTC after fetching them as Go types?

Client

e.g. BigQuery

Environment

Local installation / Any

Go Environment

› go version
go version go1.22.0 darwin/arm64

› go env
GO111MODULE='auto'
GOARCH='arm64'
GOBIN=''
GOCACHE='/Users/simon/Library/Caches/go-build'
GOENV='/Users/simon/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/Users/simon/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='/Users/simon/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/go/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.22.0'
GCCGO='gccgo'
AR='ar'
CC='/usr/bin/clang'
CXX='/usr/bin/clang++'
CGO_ENABLED='1'
GOMOD='/Users/simon/tmp/test-go/bq/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/yx/19hjwx4n2m9f5453rs5tz1cm0000gn/T/go-build3786632168=/tmp/go-build -gno-record-gcc-switches -fno-common'

Code

package main

import (
	"context"
	"errors"
	"fmt"

	"cloud.google.com/go/bigquery"
	"google.golang.org/api/iterator"
)

func main() {
	ctx := context.Background()
	project := "some-project"

	client, err := bigquery.NewClient(ctx, project)
	if err != nil {
		panic(err)
	}

	datasetName := "some-ds"
	tableName := "some-table"
	columnName := "ts"

	// Create the table and insert data if needed, however this might take up to several minutes
	// to be visible to the client.
	/*
		table := client.Dataset(datasetName).Table(tableName)
		testSchema := []*bigquery.FieldSchema{
			{Name: columnName, Type: bigquery.TimestampFieldType},
		}

		if err := table.Create(ctx, &bigquery.TableMetadata{
			Name:   tableName,
			Schema: testSchema,
		}); err != nil {
			panic(err)
		}

		if err := table.Inserter().Put(ctx, []*bigquery.ValuesSaver{
			{
				Schema:   testSchema,
				InsertID: uuid.New().String(),
				Row: []bigquery.Value{
					time.Date(1991, 1, 1, 1, 1, 1, 0, time.UTC),
				},
			},
		}); err != nil {
			panic(err)
		}
	*/

	query := client.Query(fmt.Sprintf("SELECT %s FROM %s.%s LIMIT 1", columnName, datasetName, tableName))

	it, err := query.Read(ctx)
	if err != nil {
		panic(err)
	}

	var bqRow map[string]bigquery.Value
	if err := it.Next(&bqRow); errors.Is(err, iterator.Done) {
		return
	} else if err != nil {
		panic(err)
	}

	fmt.Println(bqRow[columnName])
}

Expected behavior

The data returned by the client to be the same in v1.58.0 and v1.59.0

Actual behavior

The data is no longer the same, it now holds a local timezone instead of UTC.

Screenshots

n/a

Additional context

Started after upgrading to v1.59.0. Most likely in #9368 and this line.

[shollyman]

Thanks for pointing this out. I hadn't considered users having a specific timezone expectation when making this change, but it's a straightforward fix. Let me send a quick PR to force alignment to UTC.

[shollyman]

This was released as 1.59.1 - #9412

[bombsimon]

Awesome! Thanks for the super quick turnaround!