Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUGFIX] keep local state after a deletion #5058

Merged
merged 16 commits into from
Jul 14, 2017

Conversation

runspired
Copy link
Contributor

@runspired runspired commented Jul 10, 2017

If this approach is deemed good, this should be backported to beta and release.

This addresses an issue uncovered by the work to improve relationships: deleting a relationship results in a call to flushCanonical which erases any other local changes made. We can prevent local edits being removed for the common case (delete returning no additional data).

cc @igorT @hjdivad @stefanpenner for review

@runspired runspired changed the title [WIP] keep local state after a deletion [BUGFIX] keep local state after a deletion Jul 10, 2017
@@ -885,17 +885,28 @@ export default class InternalModel {
@method clearRelationships
@private
*/
clearRelationships() {
clearRelationships(resetAll = false) {
Copy link
Member

@stefanpenner stefanpenner Jul 11, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer two methods then a single method filled with if statements. Maybe clearRelationships and unloadRelationship or maybe, I think the following are more clear, rollbackRelationships and unloadRelationships?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, will refactor it tomorrow

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we may not need to split the methods, depending on the outcome of our discussion -> #5058 (comment)

assert.equal(get(pets, 'length'), 3, 'precond2 - relationship now has three pets');

run(() => {
shen.destroyRecord({})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Return

const person = store.peekRecord('person', '1');
const petsProxy = person.get('pets');

petsProxy.then((pets) => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Return

@stefanpenner
Copy link
Member

I'll take some time tonight or tomorrow to review.

I'm not sure the current semantics for unload purging inverse makes sense in the first place, as it seems at odds with resurrection...

I'll compare your tests to another approach I was playing with (one that I think corrects the above resurrection vs unload issue) and see what I missed.

@runspired
Copy link
Contributor Author

runspired commented Jul 11, 2017

So the naming here could be better but the idea is that the delete has already occurred and been committed and ergo the method uses the term "unload" because the relationship itself will be in the correct state afterwards, no additional signaling needed.

Overall it turns out that clear is called well before unloadRecord (is called during the delete), and our issue with the clearing is/was not due to the dematerialization change itself but other relationship bug fixes.

We need not fear resurrection here because these methods are called to remove a persisted deletion from the relationship.

Copy link
Member

@stefanpenner stefanpenner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some notes as I reviewed and tested locally.

@@ -715,7 +715,7 @@ export default class InternalModel {
heimdall.increment(send);
let currentState = this.currentState;

if (!currentState[name]) {
if (typeof currentState[name] !== 'function') {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems unrelated, can this be a different PR?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can be, just something I noticed as I was stepping through.

currentState.splice(idx, 1);

manyArray.set('length', currentState.length);
manyArray.notifyPropertyChange('length');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is the above set not sufficient to notify?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for async relationships an issue was surfaced in which set was not enough to notify the PromiseProxy that length had changed. This particular notify was a shot in the dark at fixing it that did not work and should be removed. See comment in model.js for more on the issue surfaced, you may have an idea as to what isn't wired correctly.

Copy link
Member

@stefanpenner stefanpenner Jul 11, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removing this notifyPropertyChange appears to have no affect on the tests. it seems safe to remove. Or we need another test case surfacing this issue

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

manyArray.set('length', currentState.length);
manyArray.notifyPropertyChange('length');

this.internalModel.notifyHasManyRemoved(this.key, internalModel, idx);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the proxy is just wrapping the hasMany, why wouldn't the above setpropagate?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unknown, see comments left in model.js

Copy link
Member

@stefanpenner stefanpenner Jul 11, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

emberjs/ember.js#15487 may fix, and if so the following would polyfill it.
[edit: removed code example, comment bellow is a less hacky fix]

Copy link
Member

@stefanpenner stefanpenner Jul 11, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

scratch that.

The issue is, ManyArray emulates Enumerable, and thus must provide it's own arrayContentWillChange and arrayContentDidChange events, these events are how ArrayProxies work, and length (amongst other things) update based on that.

. It's internal methods do this correctly, but your manual and external currentState.splice + manyArray.set('length does not. Rather then reaching into the internal state, you may want to use manyArray.internalReplace manyArray._removeInternalModels or add your own manyArray.splice.

splice(idx, count) {
  this.arrayContentWillChange(idx, count, 0);
  this.currentState.splice(idx, count);

  this.set('length', this.currentState.length);
  this.arrayContentDidChange(idx, count, 0);
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doh, that makes some sense of why this seemed so odd.

I'm still a bit confused why this (1) didn't seem to be an issue unless there was the extra proxy from the async wrapper involved and (2) the Enumerable docs suggest just set is fine (https://www.emberjs.com/api/classes/Ember.Enumerable.html) and (3) just for learning why is this.arrayContentWillChange absolutely necessary to trigger change events here?

Copy link
Member

@stefanpenner stefanpenner Jul 11, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the Enumerable docs suggest just ...

Enumerable docs appear in-complete arrayContent{Will,Did}Change stuff seems to be missing. Should b

I'm still a bit confused why this (1) didn't seem to be an issue unless there was the extra proxy from the async wrapper

As soon as there is an ArrayProxy (in this case: PromiseManyArray) at-play, this issue would arise.


if (resetAll === true) {
rel.clear();
rel.removeInverseRelationships();
Copy link
Member

@stefanpenner stefanpenner Jul 11, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

curious, do we actually want to remove the entire inverse here, or just remove our own internalModel form the inverse? If its just remove ourselves, then the clear flag feels less bad and the above recommendation to two methods can be disregarded.

Specifically:

if (resetAll === true) {
  rel.clear();
}

rel.unloadInverseInternalModel(this);

Also, would the following name change improve clarity?

rel.unloadInverseInternalModel(this); -> rel.unloadInternalModelFromInverse(this);

--

This would seem to tidy up how we deal with inverses, spuriously destroying them must lead to other issues...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We still end up with the name spaghetti with unloadInternalModelFromInverseFromOwn and unloadInternalModelFromInverseFromInverses

Copy link
Member

@stefanpenner stefanpenner Jul 11, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would seem to tidy up how we deal with inverses, spuriously destroying them must lead to other issues...

Yes this seems to cause data-loss in more scenarios (like #5052)

But, good news (after testing locally) I believe always doing unloadInverseInternalModel(this) will nearly fix #5052 the remaining assertion failures will be based on what we decide resurrection + unload + retained by inverse relationship actually means. The code-base is currently slightly confused... specifically:

the following assertion https://github.com/emberjs/data/pull/5052/files#diff-f5099af3b5cad3fc343f3f4b73eb3c97R208 will be true if resurrection does apply in this case (reviving 2 + 6) OR resurrection does not apply and the contacts list will become ['3','4','5','7']

I think @hjdivad will need to chime in re: the resurrection details. If we can decide on those, we can likely fix both issues in one swoop.


if (resetAll === true) {
rel.clear();
rel.removeInverseRelationships();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

^ same

@@ -1115,6 +1115,14 @@ const Model = Ember.Object.extend(Ember.Evented, {
this.notifyPropertyChange(key);
},

notifyHasManyRemoved(key) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

currentState.splice(idx, 1);

manyArray.set('length', currentState.length);
manyArray.notifyPropertyChange('length');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const manyArray = this._manyArray;

if (manyArray) {
const currentState = manyArray.currentState;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this block can likely be replaced with: manyArray.removeInternalModels([internalModel])

let allMembers =
// we actually want a union of members and canonicalMembers
// they should be disjoint but currently are not due to a bug
this.members.toArray().concat(this.canonicalMembers.toArray());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rather then making 3 arrays only to iterate them, maybe we should do something like:

const unload = inverseInternalModel => {
   let relationship = inverseInternalModel._relationships.get(this.inverseKey);
   relationship.unloadInverseInternalModelFromOwn(internalModel);
}

this.members.forEach(unload);
this.canonicalMembers(unload);

@@ -885,14 +885,14 @@ export default class InternalModel {
let rel = this._relationships.get(name);

rel.clear();
// rel.removeInverseRelationships();
rel.removeInverseRelationships();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hjdivad commenting these out didn't make any tests fail o.O

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm yeah it looks like the unload tests assert against the id map and the relationship payloads, but not against relationships (either our side or inverse).

/*
@method unloadDeletedRecordFromRelationships
*/
unloadDeletedRecordFromRelationships() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

per our chat out of band, let's rename this to not conflate this case with unloadRecord

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also let's add some comments here as breadcrumbs to our future selves who will clean this up.

This does fix an important bug, but currently there are two codepaths whose different semantics are not obvious within the same level of abstraction, ie that rel.clear() causes a flushCanonical but unloadDeletedInverseInternalModel does not is not clear, and that this is the entire point is also not clear.

@@ -266,6 +269,33 @@ export default class Relationship {
this.flushCanonicalLater();
}

unloadDeletedInverseInternalModel(internalModel) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the case where internalModel != this.internalModel?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per our discussion out of band, let's fix the inconsistency in naming here. unloadDeletedInverseInternalModel and unloadDeletedInverseInternalModelFromOwn both mention an "inverse internal model" but are referring to different sides of the relationship.

const rebel = store.peekRecord('pet', '3');

assert.equal(get(shen, 'name'), 'Shenanigans', 'precond - relationships work');
assert.equal(get(pets, 'length'), 1, 'precond - relationship has only one pet to start');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's change these assertions to deepEqual an array of ids rather than just checking the length

pets.pushObjects([rambo, rebel]);
});

assert.equal(get(pets, 'length'), 3, 'precond2 - relationship now has three pets');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's change these assertions to deepEqual an array of ids rather than just checking the length

});
});

assert.equal(get(pets, 'length'), 2, 'relationship now has two pets');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's change these assertions to deepEqual an array of ids rather than just checking the length

shen.unloadRecord();

assert.equal(get(pets, 'length'), 2, 'relationship now has two pets');
assert.equal(get(petsProxy, 'length'), 2, 'proxy now reflects two pets');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's change these assertions to deepEqual an array of ids rather than just checking the length

@@ -63,6 +63,25 @@ export default class BelongsToRelationship extends Relationship {
this.notifyBelongsToChanged();
}

unloadDeletedInverseInternalModelFromOwn(internalModel) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

foo.set(belongsTo, new);
new.destroyRecord();

foo.get(belongsTo) === null;
``

We should not restore canonical when deleting new; we should let belongsto be null.  That's consistent with [current behaviour](https://ember-twiddle.com/7f94a0e81ea38d9022a5007ef8be15c8?openFiles=adapters.application.js%2C) and in any case, restoring here is a bit wat

});
});

test('deleting an item that is the current state of a belongsTo restores canonicalState', function(assert) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

  1. this is a change in semantics (see this twiddle)
  2. this is surprising behaviour

the parent record's hasMany is a situation in which this limitation will be encountered should other
local changes to the relationship still exist.
*/
test('returning new hasMany relationship info from a delete supplements local state', function(assert) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/supplements/supersedes

Supplements is exactly what it does not do ^_^

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wrote the test for what we want to happen, and it should fail once what we want does happen, should fix that test description to account for this

const rebel = store.peekRecord('pet', '3');

assert.equal(get(shen, 'name'), 'Shenanigans', 'precond - relationships work');
assert.equal(get(pets, 'length'), 2, 'precond - relationship has only one pet to start');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as before, let's deepEqual ids instead of just checking length in this test

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

@method removeDeletedInternalModelFromInverse
@private
*/
removeDeletedInternalModelFromInverse(internalModel) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the case where internalModel != this.internalModel?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there isn't one afaik, just seemed better to pass this in with the original implementation. using this.internalModel is probably a bit safer

Copy link
Member

@hjdivad hjdivad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit: pending rebase &c.

Although As discussed out of band we should clean up the semantics around this, but that cleanup should be a 2.15 effort and here we should focus on the bugfix

@method unloadDeletedRecordFromRelationships
@private
*/
unloadDeletedRecordFromRelationships() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use unload as the term for unloading records, maybe we should call this method something else, like remove?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Absolutely.

@runspired I guess we missed an instance of the renaming

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

confirm, will rename

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about: removeFromInverseRelationships, example:

internalModel.removeFromInverseRelationships();

The method would now describes concisely what is does not when to use it. The when and why should be provided via docs, or derived contextually from where it is used.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@@ -63,6 +63,19 @@ export default class BelongsToRelationship extends Relationship {
this.notifyBelongsToChanged();
}

removeDeletedInternalModelFromOwn(internalModel) {
super.removeDeletedInternalModelFromOwn(internalModel);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought we moved away from calling super in relationships?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We never landed that

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:(

@private
*/
clearRelationships() {
_resetAllRelationshipsToEmpty() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the difference in behavior between the isNew and the deletion case?

this.canonicalMembers.forEach(unload);
}

removeDeletedInternalModelFromOwn(internalModel) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe Deleted should be removed from this record name. In general a method name should describe what it does, not why or when it should be invoked.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will follow guidance on your other comment for renaming

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@runspired this function is quite similar to removeInternalModelFromOwn what is the divergence?

Copy link
Member

@stefanpenner stefanpenner Jul 14, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The divergence (but feel free to confirm):

  • removeInternalModelFromOwn: is removing from current state (awaiting future canonical flush)
  • removeDeletedInternalModelFromOwn: is removing from both current state (members) and canonical state (canonicalMembers)

The naming of these methods sorta blows, but naming is hard. So at the very least we may need to document them somewhat.

Some ideas:

Removes a given internal model from the relationships currentState, but not from that relationships canonicalState. This results in an optimistic delete, and subsequent confirmation (typically server side) will update the canonicalState.

@method removeInternalModelFromOwn
@param internalModel
Removes a given internal model from both a relationships currentState and canonical state. This typically occurs when an internalRecord's deletion is persisted.

@method removeDeletedInternalModelFromOwn
@param internalModel

removeDeletedInternalModelFromOwn(internalModel) {
this.canonicalMembers.delete(internalModel);
this.members.delete(internalModel);
this.notifyRecordRelationshipRemoved(internalModel);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this method do anything? I only see two implementations and they appear to be noops.

https://github.com/emberjs/data/search?utf8=%E2%9C%93&q=notifyRecordRelationshipRemoved&

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this was put here for something with dematerialization that was never completed maybe?

@method unloadDeletedRecordFromRelationships
@private
*/
unloadDeletedRecordFromRelationships() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about: removeFromInverseRelationships, example:

internalModel.removeFromInverseRelationships();

The method would now describes concisely what is does not when to use it. The when and why should be provided via docs, or derived contextually from where it is used.

@@ -671,7 +671,7 @@ const RootState = {
isDirty: false,

setup(internalModel) {
internalModel.clearRelationships();
internalModel.unloadDeletedRecordFromRelationships();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

internalModel.removeFromInverseRelationships()

if (this._relationships.has(name)) {
let rel = this._relationships.get(name);

rel.removeDeletedInternalModelFromInverse();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should destroyRelationships also get this treatment?

@@ -111,6 +111,26 @@ export default class ManyRelationship extends Relationship {
super.removeCanonicalInternalModelFromOwn(internalModel, idx);
}

removeDeletedInternalModelFromOwn(internalModel) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems related to removeInternalModelFromOwn, why diverge?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removeInternalModelFromOwn removes only current state while removeDeletedInternalModelFromOwn removes from current and canonical state.

@hjdivad
Copy link
Member

hjdivad commented Jul 13, 2017

Summary of issues

  1. lose local changes in hasMany on a delete of a canonical record and the server returns no new canonical data.
  2. lose local changes in hasMany more generally during flushCanonical as flushCanonical does not diff
  3. unload clears inverse relationships eagerly, preventing intra-loop rematerialization
  4. unload perf issue possibly overwalking during gc

This pr addresses 1.

cc @runspired @igorT @stefanpenner

this.canonicalMembers.forEach(unload);
}

removeDeletedInternalModelFromOwn(internalModel) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@runspired this function is quite similar to removeInternalModelFromOwn what is the divergence?

@emberjs emberjs deleted a comment from runspired Jul 14, 2017
this.canonicalMembers.forEach(unload);
}

removeDeletedInternalModelFromOwn(internalModel) {
Copy link
Member

@stefanpenner stefanpenner Jul 14, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The divergence (but feel free to confirm):

  • removeInternalModelFromOwn: is removing from current state (awaiting future canonical flush)
  • removeDeletedInternalModelFromOwn: is removing from both current state (members) and canonical state (canonicalMembers)

The naming of these methods sorta blows, but naming is hard. So at the very least we may need to document them somewhat.

Some ideas:

Removes a given internal model from the relationships currentState, but not from that relationships canonicalState. This results in an optimistic delete, and subsequent confirmation (typically server side) will update the canonicalState.

@method removeInternalModelFromOwn
@param internalModel
Removes a given internal model from both a relationships currentState and canonical state. This typically occurs when an internalRecord's deletion is persisted.

@method removeDeletedInternalModelFromOwn
@param internalModel

@emberjs emberjs deleted a comment from runspired Jul 14, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants