Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add crud relation methods #1403

Merged
merged 1 commit into from
Jun 26, 2018
Merged

feat: add crud relation methods #1403

merged 1 commit into from
Jun 26, 2018

Conversation

jannyHou
Copy link
Contributor

@jannyHou jannyHou commented Jun 8, 2018

connect to #1376

Add two more relation methods:

  • delete
  • patch

* @param entity
* @param options
*/
update(entity: DataObject<T>, options?: Options): Promise<boolean>;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not a fan of the name update, because it's not clear whether it's performing a patch (update of a subset of properties provided in entity) or a full replace (setting the target model instance exactly to the property values provided in entity).

Let's pick one of the more descriptive names (depending on the actual implementation in juggler): patch or replace.

Copy link
Contributor Author

@jannyHou jannyHou Jun 13, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bajtos good point, I was following the naming convention in legacy-juggler-bridge.
IMO:

  • It makes sense that the crud methods in legacy-juggler-bridge.ts align with the names in lb3 juggler
  • Given a new opportunity to build lb4 relation from scratch, relation cruds can follow a new naming conversion, which reflects the http verb better and also distinguish partial update(patch) and full update(replace) in a more clear way.
  • It's a little weird that the Entity crud methods and relation crud methods use different naming conversion...
// entity crud method
customer.update()
// relation crud method
customer.order.patch()

I am going to sacrifice point 3 in this PR, name the function as patch, because given enough bandwidth IMO we should correct some old naming in juggler, not the other way around.
Let's discuss a better strategy to make the naming conversion more consistent across different repositories out of this story. Also we have a story to split the repository interface: #1356.

* @param entity
* @param options
*/
save(entity: DataObject<T>, options?: Options): Promise<T | null>;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the difference between update and save? I am proposing to omit the save method.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bajtos save means patchOrCreate

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While...I guess you would say that's not a good/clear name again lol, at least that's my thought, went into the code to see the difference.

*
* @param dataObjects
*/
createAll(dataObjects: DataObject<T>[]): Promise<T[]>;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK, juggler does not provide a reliable batch create where all create requests are send in a single request or transaction and where either all or no entities are created. Without that, this tool is just a syntactic sugar for dataObjects.map(dto => repo.create(dto)), which adds too little value IMO.

I am proposing to remove createAll from LB4 API.

* @param where
* @param options
*/
updateAll(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this going to perform a partial update (patch) or a full replace? I believe our connectors are performing a partial update, let's call this method patchAll.

I am also not sure if the suffix All is descriptive enough, can we come up with a better name please? E.g. patchMany or patchByQuery.

return await this.targetRepository.find(
constrainFilter(filter, this.constraint),
options,
);
}

async update(entity: DataObject<T>, options?: Options): Promise<boolean> {
return await this.targetRepository.update(entity, options);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you forgot to apply constrainFilter?

It would be great if you could work on this in a test-first/test-driven way - start with a failing unit or integration test and only then write an implementation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙈 Sorry I should have marked this PR as not ready for review

t would be great if you could work on this in a test-first/test-driven way - start with a failing unit or integration test and only then write an implementation.

True 👍 The missing function is constrainDataObject IIUC

@jannyHou jannyHou mentioned this pull request Jun 13, 2018
6 tasks
@jannyHou jannyHou force-pushed the relation/crud-methods branch from 5b81005 to ae98552 Compare June 13, 2018 22:50
@jannyHou jannyHou changed the title [WIP, tests TBD]feat: add crud relation methods [WIP, tsdocs to be add]feat: add crud relation methods Jun 13, 2018
@jannyHou jannyHou changed the title [WIP, tsdocs to be add]feat: add crud relation methods [tsdocs to be add]feat: add crud relation methods Jun 13, 2018
@jannyHou jannyHou force-pushed the relation/crud-methods branch from ae98552 to 8a3d476 Compare June 14, 2018 00:09
@jannyHou jannyHou changed the title [tsdocs to be add]feat: add crud relation methods feat: add crud relation methods Jun 14, 2018
Copy link
Member

@bajtos bajtos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test cases to add:

  • patch, replace and patchAll cannot modify the foreign key (reference) in the target model

* Create multiple target model instances
* @param dataObjects
*/
createAll(dataObjects: DataObject<T>[]): Promise<T[]>;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we remove this API please?

See #1403 (comment)

AFAIK, juggler does not provide a reliable batch create where all create requests are send in a single request or transaction and where either all or no entities are created. Without that, this tool is just a syntactic sugar for dataObjects.map(dto => repo.create(dto)), which adds too little value IMO.

I am proposing to remove createAll from LB4 API.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am ok to remove the batch create api. Any objections? @strongloop/sq-lb-apex @raymondfeng ?

* @param where Instances within the where scope are deleted
* @param options
*/
deleteAll(where?: Where, options?: Options): Promise<number>;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am proposing to rename this method to deleteMany or deleteByQuery. I find the name deleteAll misleading, because were are not going to delete all records when a query was provided.

Copy link
Contributor Author

@jannyHou jannyHou Jun 14, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds good to me, deleteMany sounds better.

* @param where Instances within the where scope are patched
* @param options
*/
patchAll(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similarly here, maybe patchMany or patchByQuery would be a better name?

);
}

async delete(entity: DataObject<T>, options?: Options): Promise<boolean> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think delete should accept the entity id only?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bajtos delete and patch are sugar functions for deleteById and patchById

options?: Options,
): Promise<number> {
return await this.targetRepository.updateAll(
dataObject,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you need to call constrainDataObject?

Test case:

  • Category hasMany products, using product.categoryId as the FK.
  • create releation repository to access products of a category with id CATID
  • call patchAll({categoryId: 123}) - this moves the products from CATID to category 123, I think it's not allowed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bajtos Good catch! I added the test case in unit test.

isDelivered: true,
};
const newData = Object.assign({id: order.id}, fieldToPatch);
const isPatched = await customerOrders.patch(toOrderEntity(newData));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const newData = Object.assign({id: order.id}, fieldToPatch);
const isPatched = await customerOrders.patch(toOrderEntity(newData));

I find this API cumbersome to use. Let's use patchById please.

const isPatched = await customerOrders.patchById(order.id, fieldsToPatch);

const order = await customerOrders.create(originalData);
delete order.customerId;

const isDeleted = await customerOrders.delete(toOrderEntity(order));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similarly here - why do we need to provide the full entity to delete it?

Here is what I would like to see instead:

await customerOrders.deleteById(order.id);

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

true. fixed after rebasing

const expectedResult = {
id: order.id,
description: 'new order',
// @jannyhou: investigating why it's undefined, pretty sure it's not caused by `constrainData`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC, your newData does not provide any value for isDelivered, therefore the model instance returned by subsequent queries return isDelivered: undefined.

Copy link
Contributor Author

@jannyHou jannyHou Jun 14, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bajtos My expectation is the field doesn't show up at all, like this:

{
id: 1
description: 'new order',
customerId: 1
}

thought?

id: order.id,
description: 'new order',
};
const newEntity = toOrderEntity(newData);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really understand the benefits of toOrderEntity, what's wrong with the following code that does not this helper?

const newData = new Order(
  id: order.id,
  description: 'new order',
});

@jannyHou
Copy link
Contributor Author

@bajtos I am following the list in acceptance criteria #1376 to implement those 6 relation crud methods, honestly I didn't think too much about why pick those 6...since as a team we agreed and groomed like that.

According to our discussion, I agree that

  • patchById is better patch,
  • deleteById is better delete,
  • rename deleteAll to deleteMany
  • rename patchAll to patchMany
  • remove createAll

So the updated list to implement would be

  • patchById
  • deleteById
  • replaceById
  • deleteMany
  • patchMany

note: we don't have replaceMany in juggler IIRC, so skip it in this story

@bajtos @strongloop/sq-lb-apex @raymondfeng thought?

@bajtos
Copy link
Member

bajtos commented Jun 14, 2018

I am following the list in acceptance criteria #1376 to implement those 6 relation crud methods, honestly I didn't think too much about why pick those 6...since as a team we agreed and groomed like that.

No worries, you did the right thing. It's just as I was reviewing the code it occurred to me the proposed API is not very easy to follow from user's perspective and thus I proposed to take this opportunity to improve it.

So the updated list to implement would be

  • patchById
  • deleteById
  • replaceById
  • deleteMany
  • patchMany

Your list looks perfect to me 👍

@jannyHou
Copy link
Contributor Author

@bajtos 😎 cool, if there is no objection from team that will be our new list. Thanks!

@b-admike
Copy link
Contributor

I agree with your proposed list as well @jannyHou 👍

@jannyHou
Copy link
Contributor Author

jannyHou commented Jun 14, 2018

@bajtos I had a talk with @b-admike and know why we the original list doesn't contain *ById. And also, the DefaultCrudRepository api I called in this PR is not correct.

For patch(or patchById), we cannot call targetRepository's update or updateById method, since the where clause is hard-coded, there is no way to apply fk constraint, see:

// DefaultCrudRepository
  updateById(id: ID, data: Partial<T>, options?: Options): Promise<boolean> {
    const idProp = this.modelClass.definition.idName();
    const where = {} as Where;
    where[idProp] = id;
    return this.updateAll(data, where, options).then(count => count > 0);
  }

By calling it, a customer can delete an order that doesn't belong to it!

Therefore patch or patchById or patchMany can only call updateAll from DefaultCrudRepository:

// DefaultCrudRepository
  updateAll(
    data: Partial<T>,
    where?: Where,
    options?: Options,
  ): Promise<number> {
    return ensurePromise(this.modelClass.updateAll(where, data, options)).then(
      result => result.count,
    );
  }

So patchMany is good in my PR. If I add patchById, it will be a sugar function of patchMany:

patchById(id: ID, data: Partial<T>, options?: Options): Promise<T> {
  const idProp = hmmmm....;
  await this.patchMany(data, constrainWhere({idProp: id}), options);
}

Currently TargetRepository is extending EntityCrudRepository, which is an interface that doesn't provide the function to get id name, to fix it, TargetRepository would extend DefaultCrudRepository instead. Then we can do const idProp = this.targetRepository.modelClass.definition.idName() in the code above^.

Give it a quick try, it also needs lots of change to constrain* functions.

Considering PR #1383 is landing soon, I will get back to this PR after 1383 merges, otherwise our change to constrain util functions are overlapping.

@bajtos
Copy link
Member

bajtos commented Jun 15, 2018

By calling it, a customer can delete an order that doesn't belong to it!

Good catch! Obviously, enforcing the relation constrain is a must have for all relation methods.

To be honest, I was never sure what's the benefit of using endpoints like PATCH /customers/{id1}/orders/{id2} - once we have the id of the related model, we can always call the non-relation API directly: PATCH /orders/{id2}.

One can argue that PATCH /customers/{id1}/orders/{id2} allows us to not expose /orders API and require all clients to access orders via /customers namespace. It would make sense to me if the order key was composed from customer id and order id, similarly to how GitHub identifies issues (org/user name + project name + project-local issue number). When using a primary key that's decoupled from relations, e.g. MongoDB's GUID or even a numeric Order id that's unique for all orders, I don't see much benefits.

Maybe it's time to revisit the relation API design and ask ourselves what use cases/database designs we would like to support? I feel that's out of scope of this story and DP3 though.

For DP3, I am proposing to simplify the relations API to the following methods:

  • find, create (already exists)
  • patchMany
  • deleteMany

Currently TargetRepository is extending EntityCrudRepository, which is an interface that doesn't provide the function to get id name, to fix it, TargetRepository would extend DefaultCrudRepository instead. Then we can do const idProp = this.targetRepository.modelClass.definition.idName() in the code above^.

I don't think it's a good idea to couple relation repositories with our default implementation of EntityCrudRepository. Have you tried to access the definition from the target model/entity directly, bypassing the target repository instance? Look how DefaultCrudRepository is obtaining definion.idName() from the model/entity constructor, I think we should be able to do the same.If not, then the relation repository should receive idName in a constructor argument.

@jannyHou
Copy link
Contributor Author

One can argue that PATCH /customers/{id1}/orders/{id2} allows us to not expose /orders API and require all clients to access orders via /customers namespace. It would make sense to me if the order key was composed from customer id and order id, similarly to how GitHub identifies issues (org/user name + project name + project-local issue number). When using a primary key that's decoupled from relations, e.g. MongoDB's GUID or even a numeric Order id that's unique for all orders, I don't see much benefits.

I agree with ^

My only concern is, from UX's perspective, say we expose an API called customer.orders.patchById when Order has a composed unique id(id + customerId), but the api doesn't work when Order uses a single unique id(id), it will confuse users.
Actually as long as we call an api that takes a where filter, the implementation for single Id or composed Id would be same, and the connector(or the api being called) will decide how to generate the Id.

@jannyHou
Copy link
Contributor Author

jannyHou commented Jun 15, 2018

I don't think it's a good idea to couple relation repositories with our default implementation of EntityCrudRepository

Agree 👍 I will try another approach.

  • find, create (already exists)
  • patchMany
  • deleteMany

Based on this list, I think we can also support replaceById, since the api under the hood takes in a where filter 🥇 .
Then as the first stage's relation crud api, we have the apis map to GET PATCH PUT DELETE POST.

@bajtos
Copy link
Member

bajtos commented Jun 18, 2018

My only concern is, from UX's perspective, say we expose an API called customer.orders.patchById when Order has a composed unique id(id + customerId), but the api doesn't work when Order uses a single unique id(id), it will confuse users.
Actually as long as we call an api that takes a where filter, the implementation for single Id or composed Id would be same, and the connector(or the api being called) will decide how to generate the Id.

I share your concerns. Unfortunately I don't have any suggestions how to approach this :(

Based on this list, I think we can also support replaceById, since the api under the hood takes in a where filter 🥇 .

That would be great 👍

Then as the first stage's relation crud api, we have the apis map to GET PATCH PUT DELETE POST.

Awesome 👌

@jannyHou jannyHou force-pushed the relation/crud-methods branch 3 times, most recently from 473a212 to 5a1589a Compare June 25, 2018 05:01
@jannyHou
Copy link
Contributor Author

@bajtos On a second thought, I changed the function names from deleteMany, patchMany to simply delete, patch:

  • find also represents an operation that finds many instances and takes in a query, so I believe a simple name without word like *many or *withQuery would be able to imply those information.
  • other functions like patchById and deleteById will be shortcuts of them, so keep the main function's name concise.

While I don't have very strong opinion on the naming convention, will respect team's suggestion.

Copy link
Member

@bajtos bajtos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

Please improve the tests per my comments below.

Additionally, please modify the tests to create also an order linked to a different customer and verify that the relation methods are not touching orders of that different customer!

@@ -41,7 +64,7 @@ export class DefaultHasManyEntityCrudRepository<
*/
constructor(
public targetRepository: TargetRepository,
public constraint: AnyObject,
public constraint: AnyObject, // public idName: string
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Forgotten comment?

await createSamples();
await testPatch();

async function testPatch() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the purpose of nesting the test code inside testPatch? We usually inline the test code in the test function itself.

it('can patch many instances', async () => {
  await createSamples();
  const patchObject = {description: 'new order'};
  // etc.
});

expect(arePatched).to.equal(2);
const patchedItems = await customerOrders.find();
expect(patchedItems).to.have.length(2);
patchedItems.forEach(order => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Executing the same assertion multiple times in a loop usually produces unhelpful test failure messages. My proposal:

const actualData = _.pick(patchedItems, ['customerId', 'description']);
expect(actualData).to.eql([
  {customerId: existingCustomerId, description: 'new order'},
  {customerId: existingCustomerId, description: 'new order'},
]);

Also please verify the following aspects of the patch operation:

  • The operation touched only the properties specified in the data (i.e. description) and no other properties were changed (e.g. isDelivered).
  • What happens when the data is trying to change the relation key (customerId) - do we silently ignore that value or reject the request with an error? I personally prefer the error to tell the clients they are doing something wrong.

}
});

async function createSamples() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test helper makes tests difficult to understand because it's not clear what kind of sample data was created. At minimum, please rename it to createTwoSamples().

To be honest, I think this helper function is a wrong abstraction. A better solution is to create a test-data builder as described in our docs and then call this builder multiple times in each test.

it('can patch many instances', async () => {
  await givenCustomerOrder({description: 'order 1'});
  await givenCustomerOrder({description: 'order 2'});
  // run the test
});

@jannyHou jannyHou force-pushed the relation/crud-methods branch from 73eec5c to f8db64b Compare June 25, 2018 21:54
Copy link
Member

@bajtos bajtos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost there! 👍

@@ -55,6 +55,7 @@ export function constrainDataObject<T extends Entity>(
): DataObject<T> {
const constrainedData = cloneDeep(originalData);
for (const c in constraint) {
if (constrainedData[c]) throw new Error(`Field "${c}" cannot be changed!`);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we use "property" rather than "field", would you mind rewording this message for consistency?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, this check is not going to catch the case where constrainedData[c] is set to a falsey value like 0 or null. Please fix, e.g. by using the following condition: if (c in constrainedData).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch!

expect(deletedOrders).to.equal(2);
const relatedOrders = await customerOrders.find();
// tslint:disable-next-line: no-unused-expression
expect(relatedOrders).to.be.empty;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please note that empty is a function, see http://shouldjs.github.io/#assertion-empty

Please remove no-unused-expression override too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! Probably because I didn't call empty as a function, tslint was complaining unused expression. Removed those comments.

await givenCustomerOrder({description: 'order 1'});
await givenCustomerOrder({description: 'order 2'});
const deletedOrders = await customerOrders.delete();
// tslint:disable-next-line: no-unused-expression
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this override necessary? Can you remove it please?

await givenCustomerOrder({description: 'order 2', isDelivered: false});
const patchObject = {description: 'new order'};
const arePatched = await customerOrders.patch(patchObject);
// tslint:disable-next-line: no-unused-expression
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this override necessary? Can you remove it please?

@jannyHou jannyHou force-pushed the relation/crud-methods branch from ac573e9 to 1a4927a Compare June 26, 2018 13:52
Copy link
Contributor

@b-admike b-admike left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! LGTM.

it('can patch many instances', async () => {
await givenCustomerOrder({description: 'order 1', isDelivered: false});
await givenCustomerOrder({description: 'order 2', isDelivered: false});
const patchObject = {description: 'new order'};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm being picky here, but can we patch the isDelivered property to be true and keep the descriptions as it? Feel free to ignore.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah that's a more practical use case 👍

@jannyHou jannyHou force-pushed the relation/crud-methods branch from 1a4927a to 5956e63 Compare June 26, 2018 15:35
Copy link
Contributor

@shimks shimks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. :shipit: after seeing the comment down below

const deletedOrders = await customerOrders.delete();
expect(deletedOrders).to.equal(2);
const relatedOrders = await customerOrders.find();
expect(relatedOrders).to.be.empty();

This comment was marked as resolved.

@@ -94,6 +94,18 @@ describe('HasMany relation', () => {
expect(relatedOrders).to.be.empty();
});

it('does not delte instance not belongs to model from', async () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does not delete instances that don't belong to the constrained instance

@jannyHou jannyHou force-pushed the relation/crud-methods branch from c20a174 to 6c10d74 Compare June 26, 2018 16:37
@jannyHou jannyHou merged commit 1fdae63 into master Jun 26, 2018
@raymondfeng raymondfeng deleted the relation/crud-methods branch June 27, 2018 21:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants