Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

roachtest: acceptance/version-upgrade failed #104178

Closed
cockroach-teamcity opened this issue Jun 1, 2023 · 9 comments · Fixed by #105840
Closed

roachtest: acceptance/version-upgrade failed #104178

cockroach-teamcity opened this issue Jun 1, 2023 · 9 comments · Fixed by #105840
Assignees
@fqazi
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)
Milestone
23.1

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Jun 1, 2023 •
edited by herkolategan
Loading

roachtest.acceptance/version-upgrade failed with artifacts on master @ e2b704df25b012baf7d5fbe26e034f520a70f25c:

test artifacts and logs in: /artifacts/acceptance/version-upgrade/run_1
(mixedversion.go:410).Run: mixed-version test failure while running step 24 (run "test schema change step"): output in run_053215.140742790_n1_workload-run-schemac: ./workload run schemachange {pgurl:1-4} --concurrency 2 --max-ops 10 --verbose 1 returned: COMMAND_PROBLEM: exit status 1

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=4 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

/cc @cockroachdb/test-eng

This test on roachdash | Improve this report!

Jira issue: CRDB-28390
@cockroach-teamcity cockroach-teamcity added branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. labels Jun 1, 2023
@cockroach-teamcity cockroach-teamcity added this to the 23.1 milestone Jun 1, 2023
@blathers-crl blathers-crl bot added the T-testeng TestEng Team label Jun 1, 2023
@herkolategan
Copy link
Collaborator

We are still seeing issues with the schemachange workload on this test:

{
 "workerId": 0,
 "clientTimestamp": "05:32:17.070325",
 "ops": [
  "BEGIN",
  {
   "sql": "CREATE TABLE public.table8 (\"col8 _9\" INT8 NOT NULL, col8_10 TSQUERY, col8_11 STRING COLLATE de, col8_12 INT8 NOT NULL, \"'col8%8c_13\" JSONB NOT NULL, col8_14 TSQUERY NOT NULL, col8_15 TIME NOT NULL, 😖co̟l8̎_16 FLOAT4 NOT NULL, col8_17 TIMESTAMPTZ NULL, \"😓col8_!18\" STRING AS (lower(CAST(col8_11 AS STRING))) STORED, INDEX (\"'col8%8c_13\") STORING (col8_10, col8_11, col8_12, 😖co̟l8̎_16))",
   "potentialExecErr": "0A000,42601"
  }
 ],
 "expectedExecErrors": "",
 "expectedCommitErrors": "",
 "message": "***UNEXPECTED ERROR; Received an unexpected execution error.: ERROR: index element 'col8%8c_13 of type jsonb is not indexable in a non-inverted index (SQLSTATE 42P16)",
 "errorState": {
  "potentialErrors": [
   "0A000",
   "42601"
  ],

@fqazi Do you think this could be related to #103452? If so do you mind taking a look?

@fqazi
Copy link
Collaborator

fqazi commented Jun 8, 2023

Yeah, it's related, I wonder what was missed here. Let me pick this up

@fqazi fqazi self-assigned this Jun 8, 2023
@blathers-crl blathers-crl bot added the T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) label Jun 8, 2023
@renatolabs
Copy link
Contributor

renatolabs commented Jun 14, 2023 •
edited
Loading

Friendly ping, as this still flakes intermittently.

@fqazi
Copy link
Collaborator

fqazi commented Jun 14, 2023

@renatolabs Getting back to this today, sorry about that

@cockroach-teamcity cockroach-teamcity mentioned this issue Jun 16, 2023
Closed
@srosenberg
Copy link
Member

Another friendly ping. This flaked on a PR which had no related changes [1],

 "ops": [
  "BEGIN",
  {
   "sql": "CREATE TABLE public.table7 (col7_8 PG_LSN NOT NULL, \"col%%q7_9\" BIT(43) NOT NULL, UNIQUE (col7_8 DESC), INDEX (col7_8), UNIQUE (col7_8, \"col%%q7_9\" ASC), INDEX (col7_8 DESC, \"col%%q7_9\" ASC) PARTITION BY LIST (col7_8, \"col%%q7_9\") (PARTITION \"table7_\u000cpart0\" VALUES IN (('80000000/0':::PG_LSN, B'0111111111111111111111111111111111111111111'), ('FA0FAB81/699BCCEF':::PG_LSN, B'0010111011010101100001011010000110011110011'), ('5AC4D98F/6B0FA4E3':::PG_LSN, B'0100011001001010011100101101100101000110111'), ('ED189BC7/F0EEF283':::PG_LSN, B'1111100000110001000110100110010100111001111'), ('7DC35A4C/758E964B':::PG_LSN, B'1110010100011011010001001110100000010010000'), ('9D68C460/10B11F00':::PG_LSN, B'0000000100001001000000100010010010011001111'), ('2C32B2C1/710DFEF6':::PG_LSN, B'0010010100011001111001101101101011101110111'), ('8C33B6D4/AB70168B':::PG_LSN, B'0101011010101111111011010111011101111101010'), ('FC5A70C7/F55646BB':::PG_LSN, B'0111011000111100011010011000001000100010101')), PARTITION table7_part1 VALUES IN (('D6A29FD8/2590ED03':::PG_LSN, B'0110101111000000000100110011111000010100101'), ('5947D15B/2B2136F8':::PG_LSN, B'1000101110111100111011100100100101110100000'), ('7FFFFFFF/FFFFFFFF':::PG_LSN, B'1101111101111001110011000110000111001101011'), ('598A12B8/D79C58B8':::PG_LSN, B'0001001110000010101110000111010000111001100'), ('D591D661/506E6222':::PG_LSN, B'1100100010110110000010110111001000101100000'), ('8266EF71/468C440F':::PG_LSN, B'0000000000000000000000000000000000000000000'), ('48A5C82/ABD5ADC6':::PG_LSN, B'0001101000010001000111011100110000000001000'), ('147A9E59/E6637551':::PG_LSN, B'1000011110100001011001101101110100111110110'), ('7FFFFFFF/FFFFFFFF':::PG_LSN, B'0000000000000000000000000000000000000000000')), PARTITION \"table7_ part2\" VALUES IN (('0/0':::PG_LSN, B'1100001100111111110101000101011001100001010'), ('B77CD0B8/BFFBFF67':::PG_LSN, B'1110011101000101001011110011001101001000111'), ('DE3FEC39/F3CC9FFF':::PG_LSN, B'0010110011111001000101010101001001010111111'), ('654FA3E7/A1D4CF25':::PG_LSN, B'1011100111001101011010000110100111010011010'), ('E87F2535/143B96B1':::PG_LSN, B'0000000000000000000000000000000000000000000'), ('4E7D1DF2/902C045D':::PG_LSN, B'1000101011010010001011000000001001100101100'), ('477875B6/5E69DABF':::PG_LSN, B'0110000110100010001101000101111011110000110'), ('CFB73755/858B14B4':::PG_LSN, B'1000010111001100010000101100010001010110000'), ('475D87A4/558AC84':::PG_LSN, B'0000000000000000000000000000000000000000000'))), UNIQUE (col7_8, \"col%%q7_9\"), INDEX (col7_8 ASC) STORING (\"col%%q7_9\"), INDEX (\"col%%q7_9\", col7_8 DESC), UNIQUE (lower(CAST(col7_8 AS STRING)) ASC), INDEX (\"col%%q7_9\" ASC, col7_8 DESC), FAMILY (\"col%%q7_9\", col7_8))"
  }
 ],
 "expectedExecErrors": "",
 "expectedCommitErrors": "",
 "message": "***UNEXPECTED ERROR; Received an unexpected execution error.: ERROR: at or near \"not\": syntax error: unimplemented: this syntax (SQLSTATE 0A000)",

[1] https://teamcity.cockroachdb.com/repository/download/Cockroach_Ci_Tests_LocalRoachtest/10655844:id/acceptance/version-upgrade/run_1/artifacts.zip!/run_214737.937697938_n1_workload-run-schemac.log

@renatolabs
Copy link
Contributor

FWIW, the issue above is due to the use of PG_LSN and lack of version gates (some work is being done in #105391, I think there's more necessary for stop the flakes).

The previous failures in this thread are related to jsonb indexes; not sure if those are fixed.

@fqazi
Copy link
Collaborator

fqazi commented Jun 26, 2023

@renatolabs Some of the version checking in the workload is off, let me get that fixed. I have a patch for it somewhere from before vacation

@exalate-issue-sync exalate-issue-sync bot removed the T-testeng TestEng Team label Jun 26, 2023
@renatolabs
Copy link
Contributor

Another jsonb-related failure in a recent bors build:

Error: ***UNEXPECTED ERROR; Received an unexpected execution error.: ERROR: error executing StatementPhase stage 1 of 1 with 204 MutationType ops: *scop.MakePublicSecondaryIndexWriteOnly: &{{{}} 194 2}: column col%q7_14 of type jsonb is only allowed as the last column in an inverted index (SQLSTATE 0A000)

https://teamcity.cockroachdb.com/viewLog.html?buildId=10715248&buildTypeId=Cockroach_BazelEssentialCi

@fqazi
Copy link
Collaborator

fqazi commented Jun 28, 2023

@renatolabs This is a code bug where we are not correctly generating user-facing errors in the declarative schema changer. Let me try and get to this later today

@fqazi fqazi mentioned this issue Jun 29, 2023
Merged
craig bot pushed a commit that referenced this issue Jul 6, 2023
@fqazi @chengxiong-ruan
Merge #105840 #106197
818aec8 
105840: sql: version gate JSON columns on inverted indexes r=fqazi a=fqazi

Previously, our version gate for forward JSON indexes did not correctly handle the case when they are used in inverted indexes. When these columns are not the last column in an inverted index they need to be forward indexable, requiring a similar version gate. This patch fixes version gate logic for this scenario.

Fixes: #104178, Fixes #104707
Release note: None

106197: sql: delete TestGCJobWaitsForProtectedTimestamps r=chengxiong-ruan a=chengxiong-ruan

Informs #85876

In the legacy gc path, we explictly check protected timestamps. The new gc path does not need this since the mvcc gc queue handles protected timestamps, and we can remove the resonposibility of testing the PTS logic since kv is handling it.

Release note: None

Co-authored-by: Faizan Qazi <[email protected]>
Co-authored-by: Chengxiong Ruan <[email protected]>
@craig craig bot closed this as completed in 31b790e Jul 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@fqazi fqazi

Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)
Projects
None yet
Milestone
23.1
Development

Successfully merging a pull request may close this issue.

sql: version gate JSON columns on inverted indexes fqazi/cockroach
5 participants
@herkolategan @srosenberg @fqazi @cockroach-teamcity @renatolabs