Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LabeledEnum grouped values using lists instead of sets #149

Closed
jace opened this issue Oct 30, 2017 · 1 comment
Closed

LabeledEnum grouped values using lists instead of sets #149

jace opened this issue Oct 30, 2017 · 1 comment

Comments

@jace
Copy link
Member

jace commented Oct 30, 2017

LabeledEnum introduced supported for grouped values using the set syntax in #148 (via b03b28a). The rationale was that sets are the appropriate data model for the in operator. However, it appears SQLAlchemy doesn't like sets for SQL IN operations and would prefer a list instead.

LabeledEnum reserves tuples for its special syntax. Sets were introduced for groups, but lists are unused. Our options:

  1. Replace sets with lists for the grouped values syntax. The only existing use is in Settlement report boxoffice#184.
  2. Support both sets and lists (thereby precluding future uses for one of them).
  3. Investigate SQLAlchemy's troubles with sets and determine if it can be solved elsewhere.
@jace
Copy link
Member Author

jace commented Oct 30, 2017

Query:

    return db.session.query(JobPost).options(*JobPost._defercols).from_statement(db.text(
        '''SELECT jobpost.id, jobpost.hashid, jobpost.datetime, jobpost.headline, jobpost.headlineb,
            jobpost.location, jobpost.company_name, jobpost.type_id, jobpost.category_id, jobpost.status,
            jobpost.pay_type, jobpost.pay_currency, jobpost.pay_cash_min, jobpost.pay_cash_max,
            jobpost.pay_equity_min, jobpost.pay_equity_max, jobpost.email_domain
            FROM (
                SELECT jobpost_tag.jobpost_id AS jobpost_id, COUNT(*) AS count FROM jobpost_tag, jobpost
                WHERE jobpost.id = jobpost_tag.jobpost_id AND tag_id IN (
                    SELECT tag_id FROM jobpost_tag WHERE jobpost_id=:id)
                AND jobpost_id != :id AND jobpost.status IN :listed
                AND jobpost.datetime >= NOW() AT TIME ZONE 'UTC' - INTERVAL '30 days'
                AND jobpost_tag.status IN :tag_present
                GROUP BY jobpost_tag.jobpost_id ORDER BY count DESC LIMIT :limit) AS matches, jobpost
            WHERE jobpost.id = matches.jobpost_id;'''
        )).params(id=self.id, listed=POSTSTATUS.LISTED, limit=limit, tag_present=tuple(TAG_TYPE.TAG_PRESENT))

SQLAlchemy error traceback:

ProgrammingError: (psycopg2.ProgrammingError) can't adapt type 'set'
[SQL: "SELECT jobpost.id, jobpost.hashid, jobpost.datetime, jobpost.headline, jobpost.headlineb,
            jobpost.location, jobpost.company_name, jobpost.type_id, jobpost.category_id, jobpost.status,
            jobpost.pay_type, jobpost.pay_currency, jobpost.pay_cash_min, jobpost.pay_cash_max,
            jobpost.pay_equity_min, jobpost.pay_equity_max, jobpost.email_domain
            FROM (
                SELECT jobpost_tag.jobpost_id AS jobpost_id, COUNT(*) AS count FROM jobpost_tag, jobpost
                WHERE jobpost.id = jobpost_tag.jobpost_id AND tag_id IN (
                    SELECT tag_id FROM jobpost_tag WHERE jobpost_id=%(id)s)
                AND jobpost_id != %(id)s AND jobpost.status IN %(listed)s
                AND jobpost.datetime >= NOW() AT TIME ZONE 'UTC' - INTERVAL '30 days'
                AND jobpost_tag.status IN %(tag_present)s
                GROUP BY jobpost_tag.jobpost_id ORDER BY count DESC LIMIT %(limit)s) AS matches, jobpost
            WHERE jobpost.id = matches.jobpost_id;"]
[parameters: {'listed': set([9, 2, 3]), 'limit': 12, 'id': 51729, 'tag_present': (0, 1, 2, 3)}]

Points of note:

  1. TAG_TYPE.TAG_PRESENT is a list, from before the support for grouping. It's explicitly cast into a tuple in this query.
  2. This is not SQLAlchemy's in_ method. It's understandable that SQLAlchemy doesn't like both sets and lists here (as guessed from the original use of a tuple in hasgeek/hasjob@feb4a50), and since LabeledEnum already reserves use of a tuple, we can't provide tuples for groups.
  3. Users of text queries should be responsible for casting data types appropriately. This is neither LabeledEnum's nor SQLAlchemy's concern.

@jace jace closed this as completed Oct 30, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant