Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Avoid duplicated requestable fields #1621

Merged
merged 2 commits into from
Jul 12, 2023

Conversation

islamaliev
Copy link
Contributor

@islamaliev islamaliev commented Jul 11, 2023

Relevant issue(s)

Resolves #1615

Description

Makes sure the requestable fields aren't duplicated

Tasks

  • I made sure the code is well commented, particularly hard-to-understand areas.
  • I made sure the repository-held documentation is changed accordingly.
  • I made sure the pull request title adheres to the conventional commit style (the subset used in the project can be found in tools/configs/chglog/config.yml).
  • I made sure to discuss its limitations such as threats to validity, vulnerability to mistake and misuse, robustness to invalidation of assumptions, resource requirements, ...

How has this been tested?

Intergration tests

Specify the platform(s) on which this was tested:

  • MacOS

@islamaliev islamaliev added bug Something isn't working area/mapper Related to the mapper components labels Jul 11, 2023
@islamaliev islamaliev self-assigned this Jul 11, 2023
Copy link
Contributor

@AndrewSisley AndrewSisley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this impact publicly visible behaviour? If so please add tests documenting this change.

@islamaliev
Copy link
Contributor Author

Does this impact publicly visible behaviour? If so please add tests documenting this change.

So far I didn't see any public impact. I'm not sure why. The collected fields are used internally and I assume that so far the code didn't rely on uniqueness of the fields in the list.

@AndrewSisley
Copy link
Contributor

Does this impact publicly visible behaviour? If so please add tests documenting this change.

So far I didn't see any public impact. I'm not sure why. The collected fields are used internally and I assume that so far the code didn't rely on uniqueness of the fields in the list.

Then why make the change?

@islamaliev
Copy link
Contributor Author

Does this impact publicly visible behaviour? If so please add tests documenting this change.

So far I didn't see any public impact. I'm not sure why. The collected fields are used internally and I assume that so far the code didn't rely on uniqueness of the fields in the list.

Then why make the change?

I could have added this code to the part related to secondary indexes, but I though that it would be beneficial for us to have a generic solution. Or you are asking why a separate PR? I though it was quite related to my stuff so that the score is limited. I can include this fix in secondary indexes task. Not a problem.

Copy link
Contributor

@AndrewSisley AndrewSisley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could have added this code to the part related to secondary indexes, but I though that it would be beneficial for us to have a generic solution. Or you are asking why a separate PR? I though it was quite related to my stuff so that the score is limited. I can include this fix in secondary indexes task. Not a problem.

I was asking why you are making the change at all, what it does, and what you have done to check that it does what you want it to do :) In the absence of tests it is quite hard to tell.

Approving with a minor suggestion, and assuming that you have checked it has the result you think it has (going by the linked ticket). I would prefer that there were tests for this, but going by your previous answers it looks like secondary indexes is not working because of this, and integration tests for secondary indexes will flag any regressions here?

isDuplicate := false
for _, field := range existingFields {
if field.GetIndex() == keyIndex {
isDuplicate = true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: You can drop the isDuplicate var by naming the outer loop and continuing here:

sourceLoop:
    for key := range source {
         ...
         if field.GetIndex() == keyIndex {
             continue sourceLoop
        }

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, I though about it but the label would be way up there... Anyway I will switch to use the label

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is quite a distance for sure :) No pressure if you prefer the bool

@@ -844,6 +845,11 @@ func resolveInnerFilterDependencies(
// If the key index is outside the bounds of the child mapping array, then
// this is not a relation/join and we can add it to the fields and
// continue (no child props to process)
for _, field := range existingFields {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: You are removing duplicate fields that didn't impact query results at the expense of an extra for loop for every field. Could you explain why you had to do this? Or guide me towards the explanation if it exists already.

Copy link
Contributor

@AndrewSisley AndrewSisley Jul 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The linked issue does cover this (#1615), but it does require some imagination in order to connect the dots. Was why I was quizing him quite hard RE testing.

Copy link
Collaborator

@fredcarle fredcarle Jul 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't. It shows state of variables. I understand that there was duplicates and doing this removes the duplicates. I'd like to know if it's just for cleanliness or if it has a direct impact on something else which could help support the extra for loop.

Copy link
Contributor Author

@islamaliev islamaliev Jul 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I already replied to Andy on this matter. Here is my quote:

So far I didn't see any public impact. I'm not sure why. The collected fields are used internally and I assume that so far the code didn't rely on uniqueness of the fields in the list.

I noticed this by observing weird behaviour while working on indexes.

@islamaliev islamaliev force-pushed the fix/islam/avoid-duplicated-requestable-fields branch from 510eb37 to 779e6a1 Compare July 12, 2023 08:18
@codecov
Copy link

codecov bot commented Jul 12, 2023

Codecov Report

Patch coverage: 100.00% and project coverage change: +0.10 🎉

Comparison is base (d4de22e) 76.03% compared to head (779e6a1) 76.13%.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #1621      +/-   ##
===========================================
+ Coverage    76.03%   76.13%   +0.10%     
===========================================
  Files          192      192              
  Lines        19852    19856       +4     
===========================================
+ Hits         15094    15116      +22     
+ Misses        3723     3711      -12     
+ Partials      1035     1029       -6     
Flag Coverage Δ
all-tests 76.13% <100.00%> (+0.10%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
planner/mapper/mapper.go 87.15% <100.00%> (+0.67%) ⬆️

... and 6 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d4de22e...779e6a1. Read the comment docs.

@islamaliev islamaliev merged commit ae16a95 into develop Jul 12, 2023
@islamaliev islamaliev deleted the fix/islam/avoid-duplicated-requestable-fields branch July 12, 2023 13:57
shahzadlone pushed a commit to shahzadlone/defradb that referenced this pull request Feb 23, 2024
## Relevant issue(s)

Resolves sourcenetwork#1615

## Description

Makes sure the requestable fields aren't duplicated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/mapper Related to the mapper components bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

resolveFilterDependencies() in mapper.go produces duplicated fields
3 participants