Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql: implement a test generator similar to sqllite's sqllogictest for TestLogic #7734

Closed
knz opened this issue Jul 10, 2016 · 5 comments
Closed
Assignees
Labels
A-testing Testing tools and infrastructure C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)

Comments

@knz
Copy link
Contributor

knz commented Jul 10, 2016

See #7733: there are too many divergences between cockroachdb and sqllite to keep using sqllogictests as-is.

We need a test generator that exercises mixes of:

  • expressions
  • complex joins
  • complex group by
  • complex aggregates
@knz
Copy link
Contributor Author

knz commented Jul 10, 2016

Alternatively, implement a query analyzer that filters out those sqllogictests for which we have different rules than pg/sqllite. That might be slightly more tractable. Namely:

  • filter out queries using comparisons string vs. non-string
  • filter out queries using division
  • filter out queries with an error "expected type X, found type Y" with both X and Y numeric types

@knz
Copy link
Contributor Author

knz commented Aug 29, 2016

#7970 has made a great step forward already. This comprehensively covers expression computations and a few data queries. Leaving this issue open for follow-up work.

@RaduBerinde
Copy link
Member

The work around #7970 only checks for panics. What we are missing is some way of generating the correct (expected) results. This is not an easy problem. One idea would be to run the same queries against PG; we would need to either restrict the grammar to what is supported by both CRDB and PG, or somehow convert arbitrary CRDB queries into PG queries.

@petermattis petermattis added this to the Later milestone Feb 22, 2017
@jordanlewis jordanlewis added the A-testing Testing tools and infrastructure label Apr 26, 2018
@knz knz added the C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) label Apr 28, 2018
@petermattis petermattis removed this from the Later milestone Oct 5, 2018
@knz
Copy link
Contributor Author

knz commented Aug 9, 2019

This has been achieved with sqlsmith somehow, hasn't it?

@maddyblue
Copy link
Contributor

Yes I think we can close this now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-testing Testing tools and infrastructure C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)
Projects
None yet
Development

No branches or pull requests

5 participants