Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

datastore: TestLoadCivilTimeInNonUTCZone failed #3402

Closed
flaky-bot bot opened this issue Dec 7, 2020 · 8 comments · Fixed by #3470
Closed

datastore: TestLoadCivilTimeInNonUTCZone failed #3402

flaky-bot bot opened this issue Dec 7, 2020 · 8 comments · Fixed by #3470
Assignees
@tritone @IlyaFaer
Labels
api: datastore Issues related to the Datastore API. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. 🚨 This issue needs some love. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@flaky-bot
Copy link

flaky-bot bot commented Dec 7, 2020

This test failed!

To configure my behavior, see the Build Cop Bot documentation.

If I'm commenting on this issue too often, add the buildcop: quiet label and
I will stop commenting.

commit: 45131b6
buildURL: Build Status, Sponge
status: failed

Test output
==================
WARNING: DATA RACE
Write at 0x00000153a5d8 by goroutine 70:
  cloud.google.com/go/datastore.TestLoadCivilTimeInNonUTCZone()
      /tmpfs/src/google-cloud-go/datastore/load_test.go:462 +0x434
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:1127 +0x202

Previous read at 0x00000153a5d8 by goroutine 41:

time.Now()

/usr/local/go/src/time/time.go:1073 +0xf6

time.sendTime()

/usr/local/go/src/time/sleep.go:136 +0x49


Goroutine 70 (running) created at:

testing.(*T).Run()

/usr/local/go/src/testing/testing.go:1178 +0x796

testing.runTests.func1()

/usr/local/go/src/testing/testing.go:1449 +0xa6

testing.tRunner()

/usr/local/go/src/testing/testing.go:1127 +0x202

testing.runTests()

/usr/local/go/src/testing/testing.go:1447 +0x5aa

testing.(*M).Run()

/usr/local/go/src/testing/testing.go:1357 +0x4eb

cloud.google.com/go/datastore.testMain()

/tmpfs/src/google-cloud-go/datastore/integration_test.go:106 +0x364

cloud.google.com/go/datastore.TestMain()

/tmpfs/src/google-cloud-go/datastore/integration_test.go:68 +0x38

main.main()

_testmain.go:191 +0x271


Goroutine 41 (running) created at:

[failed to restore the stack]


testing.go:1042: race detected during execution of test</pre></details>
@flaky-bot flaky-bot bot added buildcop: issue priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. labels Dec 7, 2020
@product-auto-label product-auto-label bot added the api: datastore Issues related to the Datastore API. label Dec 7, 2020
@flaky-bot
Copy link
Author

flaky-bot bot commented Dec 7, 2020

commit: b28e0b8
buildURL: Build Status, Sponge
status: failed

Test output
==================
WARNING: DATA RACE
Write at 0x00000153a5d8 by goroutine 71:
  cloud.google.com/go/datastore.TestLoadCivilTimeInNonUTCZone()
      /tmpfs/src/google-cloud-go/datastore/load_test.go:462 +0x434
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:1127 +0x202

Previous read at 0x00000153a5d8 by goroutine 43:

time.Now()

/usr/local/go/src/time/time.go:1073 +0xf6

time.sendTime()

/usr/local/go/src/time/sleep.go:136 +0x49


Goroutine 71 (running) created at:

testing.(*T).Run()

/usr/local/go/src/testing/testing.go:1178 +0x796

testing.runTests.func1()

/usr/local/go/src/testing/testing.go:1449 +0xa6

testing.tRunner()

/usr/local/go/src/testing/testing.go:1127 +0x202

testing.runTests()

/usr/local/go/src/testing/testing.go:1447 +0x5aa

testing.(*M).Run()

/usr/local/go/src/testing/testing.go:1357 +0x4eb

cloud.google.com/go/datastore.testMain()

/tmpfs/src/google-cloud-go/datastore/integration_test.go:106 +0x364

cloud.google.com/go/datastore.TestMain()

/tmpfs/src/google-cloud-go/datastore/integration_test.go:68 +0x38

main.main()

_testmain.go:191 +0x271


Goroutine 43 (running) created at:

[failed to restore the stack]


testing.go:1042: race detected during execution of test</pre></details>

@flaky-bot
Copy link
Author

flaky-bot bot commented Dec 7, 2020

commit: 26f51c5
buildURL: Build Status, Sponge
status: failed

Test output
==================
WARNING: DATA RACE
Write at 0x0000015cfc00 by goroutine 41:
  cloud.google.com/go/datastore.TestLoadCivilTimeInNonUTCZone()
      /tmpfs/src/google-cloud-go/datastore/load_test.go:462 +0x3b6
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162

Previous read at 0x0000015cfc00 by goroutine 18:

time.Now()

/usr/local/go/src/time/time.go:1060 +0xcf

time.sendTime()

/usr/local/go/src/time/sleep.go:141 +0x44


Goroutine 41 (running) created at:

testing.(*T).Run()

/usr/local/go/src/testing/testing.go:878 +0x659

testing.runTests.func1()

/usr/local/go/src/testing/testing.go:1119 +0xa8

testing.tRunner()

/usr/local/go/src/testing/testing.go:827 +0x162

testing.runTests()

/usr/local/go/src/testing/testing.go:1117 +0x4ee

testing.(*M).Run()

/usr/local/go/src/testing/testing.go:1034 +0x2ee

cloud.google.com/go/datastore.testMain()

/tmpfs/src/google-cloud-go/datastore/integration_test.go:106 +0x265

cloud.google.com/go/datastore.TestMain()

/tmpfs/src/google-cloud-go/datastore/integration_test.go:68 +0x38

main.main()

_testmain.go:186 +0x221


Goroutine 18 (running) created at:

runtime.(*timersBucket).addtimerLocked()

/usr/local/go/src/runtime/time.go:170 +0x113

context.WithDeadline()

/usr/local/go/src/context/context.go:402 +0x4db

google.golang.org/grpc.(*addrConn).createTransport()

/go/pkg/mod/google.golang.org/[email protected]/clientconn.go:1291 +0x46d

google.golang.org/grpc.(*addrConn).tryAllAddrs()

/go/pkg/mod/google.golang.org/[email protected]/clientconn.go:1227 +0x583

google.golang.org/grpc.(*addrConn).resetTransport()

/go/pkg/mod/google.golang.org/[email protected]/clientconn.go:1142 +0x32d


testing.go:771: race detected during execution of test

@flaky-bot
Copy link
Author

flaky-bot bot commented Dec 7, 2020

Looks like this issue is flaky. 😟

I'm going to leave this open and stop commenting.

A human should fix and close this.

When run at the same commit (26f51c5), this test passed in one build (Build Status, Sponge) and failed in another build (Build Status, Sponge).

@tritone
Copy link
Contributor

tritone commented Dec 7, 2020

@IlyaFaer looks like your new load test has a race condition, specifically when time.Local is modified here: https://github.com/googleapis/google-cloud-go/blob/master/datastore/load_test.go#L462

Can you come up with a way to test this that does not involve modifying time.Local?

@IlyaFaer
Copy link

IlyaFaer commented Dec 8, 2020

@tritone, yeah, it can be done, I think. Will try shortly 👌

@codyoss codyoss mentioned this issue Dec 10, 2020
Merged
gcf-merge-on-green bot pushed a commit that referenced this issue Dec 10, 2020
@codyoss
chore(datastore): skip failing test TestLoadCivilTimeInNonUTCZone (#3446
c97dcd7 
)

Tracking with #3402.
@yoshi-automation yoshi-automation added 🚨 This issue needs some love. and removed 🚨 This issue needs some love. labels Dec 13, 2020
@tritone
Copy link
Contributor

tritone commented Dec 14, 2020

Hi @IlyaFaer , any updates on this?

@IlyaFaer
Copy link

@tritone, no, I didn't find a way to do it so far. There is a couple of variants, but both ugly and requires a lot of passing args from one function to another.

@yoshi-automation yoshi-automation added the 🚨 This issue needs some love. label Dec 14, 2020
@tritone
Copy link
Contributor

tritone commented Dec 15, 2020

@tbpg and I will figure out a way to handle it.

tritone added a commit to tritone/google-cloud-go that referenced this issue Dec 15, 2020
@tritone
test(datastore): replace non-UTC time test
440d44f 
This test caused race conditions by manipulating time.Local and
there is not a good way to test the behavior while avoiding this.
Instead I replace it by passing in timezone explicitly in
setVal and writing a test for that.

Fixes googleapis#3402
@tritone tritone mentioned this issue Dec 15, 2020
Closed
tritone added a commit to tritone/google-cloud-go that referenced this issue Dec 16, 2020
@tritone
test(datastore): remove flaky timezone test
a506dd4 
This test causes a race condition and there's no simple way
to get it to work. I'll file another issue to figure out a way
to test this behavior.

Fixes googleapis#3402
@tritone tritone mentioned this issue Dec 16, 2020
Merged
tbpg pushed a commit that referenced this issue Dec 16, 2020
@tritone
test(datastore): remove flaky timezone test (#3470)
98331bd 
This test causes a race condition and there's no simple way
to get it to work. I'll file another issue to figure out a way
to test this behavior.

Fixes #3402
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tritone tritone

@IlyaFaer IlyaFaer

Labels
api: datastore Issues related to the Datastore API. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. 🚨 This issue needs some love. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Milestone
No milestone
3 participants
@tritone @IlyaFaer @yoshi-automation