Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ensure concat ops are used for contains / startswith / endswith independently of datatype #8253

Closed
palisadoes opened this issue Jul 13, 2022 · 3 comments
Labels
datatypes things to do with database types, like VARCHAR and others use case not really a feature or a bug; can be support for new DB features or user use cases not anticipated
Milestone

Comments

@palisadoes
Copy link

Describe the use case

The existing documentation for these methods only has examples using strings. The examples are not applicable to VARBINARY columns where the queries fail.

  1. ColumnOperators.contains()
  2. ColumnOperators.endswith()
  3. ColumnOperators.like()
  4. ColumnOperators.startswith()

Databases / Backends / Drivers targeted

N/A

Example Use

The respective sections of documentation should be updated with edited text equivalent to the suggestions below:

ColumnOperators.contains()

  • When byte strings are stored in the database columns (eg. VARBINARY), you should use standard SQL wildcards with the like() method to represent .contains():
stmt = select(sometable).where(
    sometable.like(func.concat(func.concat('%', 'foobar'.encode()), '%'))
)    

Using .contains() with the same concatenations will produce an equivalent result.

ColumnOperators.startswith()

  • When byte strings are stored in the database columns (eg. VARBINARY), you should use standard SQL wildcards with the like() method to represent .startswith():
stmt = select(sometable).where(
    sometable.like(func.concat(func.concat('foobar'.encode()), '%'))
)    

Using .startswith() with the same concatenations will produce an equivalent result.

ColumnOperators.endswith()

  • When byte strings are stored in the database columns (eg. VARBINARY), you should use standard SQL wildcards with the like() method to represent .endswith():
stmt = select(sometable).where(
    sometable.like(func.concat(func.concat('%', 'foobar'.encode())))
)    

Using .endswith() with the same concatenations will produce an equivalent result.

Additional context

N/A

@palisadoes palisadoes added requires triage New issue that requires categorization use case not really a feature or a bug; can be support for new DB features or user use cases not anticipated labels Jul 13, 2022
@zzzeek
Copy link
Member

zzzeek commented Jul 13, 2022

So I dont think a document here is sufficient, because people tend not to find docs like these.

What I would prefer is:

  1. we figure out what these operators should do when passed a bytestring and do that
  2. we have it raise an error

1 is likely preferable

@zzzeek zzzeek changed the title Docs: VARBINARY column support for .contains(), .like(), etc. VARBINARY column support for .contains(), .like(), etc. Jul 13, 2022
@zzzeek zzzeek changed the title VARBINARY column support for .contains(), .like(), etc. _Binary column support for string operations - should implement, or raise Jul 13, 2022
@zzzeek zzzeek added datatypes things to do with database types, like VARCHAR and others and removed requires triage New issue that requires categorization labels Jul 13, 2022
@zzzeek zzzeek added this to the 2.x.x milestone Jul 13, 2022
@zzzeek zzzeek added the help wanted Extra attention is needed label Jul 13, 2022
@sqla-tester
Copy link
Collaborator

Mike Bayer has proposed a fix for this issue in the main branch:

use concat() directly for contains, startswith, endswith https://gerrit.sqlalchemy.org/c/sqlalchemy/sqlalchemy/+/3996

@sqla-tester
Copy link
Collaborator

Mike Bayer has proposed a fix for this issue in the rel_1_4 branch:

use concat() directly for contains, startswith, endswith https://gerrit.sqlalchemy.org/c/sqlalchemy/sqlalchemy/+/3997

@zzzeek zzzeek modified the milestones: 2.x.x, 1.4.x Jul 17, 2022
@zzzeek zzzeek removed the help wanted Extra attention is needed label Jul 17, 2022
@zzzeek zzzeek changed the title _Binary column support for string operations - should implement, or raise ensure concat ops are used for contains / startswith / endswith independently of datatype Jul 17, 2022
sqlalchemy-bot pushed a commit that referenced this issue Jul 18, 2022
Adjusted the SQL compilation for string containment functions
``.contains()``, ``.startswith()``, ``.endswith()`` to force the use of the
string concatenation operator, rather than relying upon the overload of the
addition operator, so that non-standard use of these operators with for
example bytestrings still produces string concatenation operators.

To accommodate this, needed to add a new _rconcat operator function,
which is private, as well as a fallback in concat_op() that works
similarly to Python builtin ops.

Fixes: #8253
Change-Id: I2b7f56492f765742d88cb2a7834ded6a2892bd7e
(cherry picked from commit 85a88df)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datatypes things to do with database types, like VARCHAR and others use case not really a feature or a bug; can be support for new DB features or user use cases not anticipated
Projects
None yet
Development

No branches or pull requests

3 participants