fix DISTINCT for Oracle databases #2935
Conversation
|
Presumably this fix means we'll just have silently broken behavior, by not including the required |
|
For more background on this. From docs:
|
|
@tomchristie DRF 2.4 works fine without |
|
@jpadilla QuerySet |
I believe that the (Someone would need to track down the issue, I don't remember exactly, but git blame would probably point in the right direction) |
|
|
Right - so I'd assume merging this fix would also mean that we'd be silently breaking #2535 for users of Oracle? I assume there's no good solution here? |
… for oracle users
|
That's right, i broked #2535. Here updated fix. |
| queryset = queryset.distinct() | ||
| else: | ||
| pk_list = queryset.values_list('pk', flat=True) | ||
| queryset = view.model.objects.filter(pk__in=set(pk_list)) |
There was a problem hiding this comment.
Using view.model.objects here feels a little bit brittle. I think we can avoid needing that. Does something like the following make sense?...
if settings.DATABASES[queryset.db]["ENGINE"] != "django.db.backends.oracle":
return queryset.filter(reduce(operator.or_, or_queries)).distinct()
else:
pk_list = queryset.filter(reduce(operator.or_, or_queries)).values_list('pk', flat=True)
return queryset.filter(pk_in=set(pk_list))
There was a problem hiding this comment.
Side note here: Since when is the model attribute back on views? I thought that was removed.
|
Avoided using |
| @@ -100,13 +101,17 @@ def filter_queryset(self, request, queryset, view): | |||
|
|
|||
| orm_lookups = [self.construct_search(six.text_type(search_field)) | |||
| for search_field in search_fields] | |||
|
|
|||
| and_queries = [] | |||
There was a problem hiding this comment.
Not sure I understand the motivation here - feels more obscure after this change rather than more clear.
There was a problem hiding this comment.
I don't sure, if calling queryset.filter(pk_in=set(pk_list)) will make duplications go away, as they pk still will be in pk_list. I want to filter original queryset by pk_list.
When we call filter with multiple kwargs or several times on one queryset it is equal to logical and.
|
@tomchristie updated |
| and_queries.append(reduce(operator.or_, or_queries)) | ||
|
|
||
| if and_queries: | ||
| if settings.DATABASES[queryset.db]["ENGINE"] == "django.db.backends.oracle": |
There was a problem hiding this comment.
We'll want a brief comment here regarding needing this behavior for oracle and distinct.
There was a problem hiding this comment.
@tomchristie According to Oracle DB limits there is no capability to make a DISTINT on *LOB. There is a need to use SearchFilter with TextField (for example). That's why i pull this workaround for Oracle.
There was a problem hiding this comment.
Clarification: I was meaning that we should have a code comment for this.
|
@xordoquy thinking we can also milestone this for next release as it seems to be 99% completed. |
|
I won't have the time to review it. |
|
@tomchristie updated |
| for search_term in self.get_search_terms(request): | ||
| or_queries = [models.Q(**{orm_lookup: search_term}) | ||
| for orm_lookup in orm_lookups] | ||
| queryset = queryset.filter(reduce(operator.or_, or_queries)).distinct() |
There was a problem hiding this comment.
I'd prefer it if we didn't introduce the and_queries. We should keep each pull request as absolutely minimal as possible. Seems like there's two different changes here - the Oracle support as one, and the change in the filtering style as another. Let's just adopt the first of those two.
Apologies for the slow progress on this, but thanks for getting it nearly there! :)
|
@tomchristie i'm not sure about failed test - is it my fault? |
| @@ -152,7 +163,7 @@ def remove_invalid_fields(self, queryset, fields, view): | |||
| field.source or field_name | |||
| for field_name, field in serializer_class().fields.items() | |||
| if not getattr(field, 'write_only', False) | |||
| ] | |||
| ] | |||
There was a problem hiding this comment.
This doesn't need to be indented another level.
This is what is causing Flake8 to fail by the way.
|
Superseeded by #3103. |
fix error "DatabaseError: ORA-00932: inconsistent datatypes: expected - got NCLOB" for oracle db