Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose join capabilities api #82

Merged
merged 5 commits into from
Feb 23, 2018

Conversation

rafaeljesus
Copy link
Contributor

@rafaeljesus rafaeljesus commented Feb 22, 2018

  • Expose more fields for generating proper join between relationships
  • Add docs

The current implementation does not allow filtering complex relationships, this PR add capabilities for the klepto users to proper filter table with relationships. Let's take this scenario:

I want to dump only the active users with their orders and it's order_items

And need to anonymise the email and firstName fields respectively

Given the klepto config

[[Tables]]
  Name = "users"
  [Tables.Anonymise]
    email = "EmailAddress"
    firstName = "FirstName"
  [Tables.Filter]
    Match = "users.active = TRUE"

[[Tables]]
  Name = "orders"
  PrimaryKey = "id"
  [[Tables.Relationships]]
    Table = "orders"
    ForeignKey = "user_id"
    ReferencedTable = "users"
    ReferencedKey = "id"
  [Tables.Filter]
    Match = "users.active = TRUE"

[[Tables]]
  Name = "order_items"
  PrimaryKey = "id"
  [[Tables.Relationships]]
    Table = "order_items"
    ForeignKey = "order_id"
    ReferencedTable = "orders"
    ReferencedKey = "id"
  [[Tables.Relationships]]
    Table = "orders"
    ForeignKey = "user_id"
    ReferencedTable = "users"
    ReferencedKey = "id"
  [Tables.Filter]
    Match = "users.active = TRUE"

Tables.Relationships fields are necessary to generate joins between tables behind the scenes e.g

generated query for dumping users:

select * from users where users.active = true

generated query for dumping orders from the users:

select * from orders
join users where users.id = orders.user_id
where users.active = true

generated query for dumping order_items from the user orders:

select * from order_items
join orders where orders.id = order_items.order_id
join users where users.id = orders.user_id
where users.active = true

FYI dumping large databases specially with the mysql-drive is pretty painful, a lot of bad conn errors are raised after some time dumping, this is where the filter comes in

@codecov
Copy link

codecov bot commented Feb 22, 2018

Codecov Report

Merging #82 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master      #82   +/-   ##
=======================================
  Coverage   32.39%   32.39%           
=======================================
  Files           5        5           
  Lines         213      213           
=======================================
  Hits           69       69           
  Misses        140      140           
  Partials        4        4

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b1f51bc...8a52460. Read the comment docs.

@rafaeljesus rafaeljesus force-pushed the features/expose-join-capabilities-api branch from adfefd3 to d165e3a Compare February 23, 2018 07:21
@rafaeljesus rafaeljesus merged commit 4868471 into master Feb 23, 2018
@rafaeljesus rafaeljesus deleted the features/expose-join-capabilities-api branch February 23, 2018 15:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants