roachtest: hibernate failed

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/0f5a289d49105cd317a2713b71546d84c25547f6

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stressrace instead of stress and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
make stress TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-stderr=false -maxtime 20m -timeout 10m'

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=937459&tab=buildLog

The test failed on master:
	test.go:505,hibernate.go:199,hibernate.go:214: 
		0 tests failed unexpectedly
		2 tests passed unexpectedly
		0 tests were not run that were expected to fail
		See artifacts/hibernate/log/logs/report for more details

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/f78cc21bb4d7bd3a79e44b4608c04684edf28179

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stressrace instead of stress and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
make stress TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-stderr=false -maxtime 20m -timeout 10m'

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=939960&tab=buildLog

The test failed on master:
	test.go:505,hibernate.go:199,hibernate.go:214: 
		0 tests failed unexpectedly
		2 tests passed unexpectedly
		0 tests were not run that were expected to fail
		See artifacts/hibernate/log/logs/report for more details

[BramGruneir]

Looks like new tests are passing. I'll investigate them today.

…

On Wed, Oct 3, 2018, 07:56 cockroach-teamcity ***@***.***> wrote: SHA: https://github.com/cockroachdb/cockroach/commits/f78cc21bb4d7bd3a79e44b4608c04684edf28179 Parameters: To repro, try: # Don't forget to check out a clean suitable branch and experiment with the # stress invocation until the desired results present themselves. For example, # using stressrace instead of stress and passing the '-p' stressflag which # controls concurrency. ./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh cd ~/go/src/github.com/cockroachdb/cockroach && \ make stress TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-stderr=false -maxtime 20m -timeout 10m' Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=939960&tab=buildLog The test failed on master: test.go:505,hibernate.go:199,hibernate.go:214: 0 tests failed unexpectedly 2 tests passed unexpectedly 0 tests were not run that were expected to fail See artifacts/hibernate/log/logs/report for more details — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#30877 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABihuRH5wXhWJWIfxrKwv9w1UOLxRjBWks5uhKYCgaJpZM4XEASO> .

[BramGruneir]

Here are two new passing tests:

--- PASS: org.hibernate.test.legacy.FooBarTest.testLimit (unexpected)
--- PASS: org.hibernate.test.legacy.FooBarTest.testPSCache (unexpected)

I'll see if they pass in current master vs one from a week ago.

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/6a236533824c797948eaddbd6bedf5e12eee1a13

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stressrace instead of stress and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
make stress TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-stderr=false -maxtime 20m -timeout 10m'

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=942507&tab=buildLog

The test failed on master:
	test.go:505,hibernate.go:199,hibernate.go:214: 
		0 tests failed unexpectedly
		2 tests passed unexpectedly
		0 tests were not run that were expected to fail
		See artifacts/hibernate/log/logs/report for more details

[BramGruneir]

It looks like this commit fixed 7ba018c both tests. Well done @jordanlewis!

I'll send an update to the hiberante test with the new tests expected to be passing.

[petermattis]

This makes me wonder how much analysis we've done of the hibernate test failures. Are there other test failures in there that are pointing to real bugs and not just lack of features?

[BramGruneir]

@jordanlewis and I were just discussing that actually. And probably a number of them are.

The analysis I preformed before was on a sampling of the failures, I looked at 150 of them and categorized them. Most were from features we don't have. Large object storage, cursors, etc. But it is worth the time in the 2.2 timeframe to go through all of the remaining failures now that we have a larger team and this work can be split up.

Until this roachtest was created, we didn't really have a list of what was newly failing and what was newly passing, so without those deltas, it was not possible perform any real analysis on it. Now we have the data and can proceed accordingly.

[petermattis]

@BramGruneir you might want to file an issue so that this idea doesn't get lost.

[BramGruneir]

Done, see #30975