Data: Support adding and updating entities

[youknowriad]

As an example: This PR refactors the HierarchicalTaxonomySelector component to use the data module instead of apiFetch.

This PR is very interesting in several aspects because it shows where we fall shart currently in Async Flow in the Data module and hopefully will allow us to solve this issue.

Notes:

1- First thing to note is that it's not fully functional yet because we need to solve a fundamental problem first: The list of categories is not being refreshed once we add a new category. That's because the queried-data state doest't invalidate the query resolvers/selectors when we add new records.

Options I can think of:

• Invalidate all the resolvers performing queries. The difficulty is that it's not possible to gather automatically all these resolvers. Not sure there's a technical solution with this approach.

• Rewrite the queried-data selectors to avoid relying on the IDs returned by the API request but perform the filtering client-side: Doesn't seem great neither because it's not always possible to recreate the filters client side especially since this is extensible by plugins server-side.

Which is to say, I'm not certain how to move forward with this, I need some insights/help @aduth

2- This PR also shows that to create an action composed of two actions:

• Save category
• Add the saved category to the categories of the post

we need to know when the first composed action finishes and we need to know its return value (the ID created).

This is solved in this PR by using the return value of the generators. For me it seems an ok compromise and it works well. but I know this can be considered a "bad" practice

[jorgefilipecosta]

We will be generating a new reference inside buildTermsTree which will probably trigger lost of unnecessary rerenders. As possible solutions, we may create a memoized selector that returns the tree already build. Or create a local memoized one instance of build terms tree, I think something like this may work:

export default compose( [
    (() => {
        const memoizedBuildtermsTree = memize( buildTermsTree, { maxSize: 1 } );
        return withSelect( ( select, { slug } ) => {
            ....
        } );
    })()

[jorgefilipecosta]

Invalidate all the resolvers performing queries. The difficulty is that it's not possible to gather automatically all these resolvers. Not sure there's a technical solution with this approach.

By invaliding, does it mean all resolvers would need to query the server again to retrieve the query?

[youknowriad]

By invaliding, does it mean all resolvers would need to query the server again to retrieve the query?

yes, that's the idea, on the next "selector" call of these resolvers, they will fetch again. So, only the components that are still mounted.

[aduth]

Rewrite the queried-data selectors to avoid relying on the IDs returned by the API request but perform the filtering client-side: Doesn't seem great neither because it's not always possible to recreate the filters client side especially since this is extensible by plugins server-side.

See also Query Manager, where this was its second goal to the first we mimicked already by queried-data (normalizing pagination fields). The need to respect plugin extensions is more pronounced than what we were dealing with in Calypso, so it might not be the best option to explore. Will need to think on this more...

[jorgefilipecosta]

Would add a parameter to getEntityRecords that allows the query to be updated be an option?
E.g:

getEntityRecords( 'taxonomy', slug, DEFAULT_QUERY, onCategory( previousQueryResult, newCategory ) {
	add new category id to previousQueryResult and sort by count after adding
}

We may then provide helpers, e.g: a dumb update that just adds new elements, a sorted helper that adds sorted by a given property etc...

[jorgefilipecosta]

Existing problem but we should have a constant EMPTY_TERMS = [] and use it instead of [] so we avoid generating a new reference.

[youknowriad]

@jorgefilipecosta I'm thinking ideally, this is not handled on the caller's side but on the data module side.

A rought proposal:

const resolver = {
  fulfill: () => {},
   shouldInvalidate: ( action, ...args ) => {
      return action.type === 'ADD_ENTITY' && args.name === action.name;
   }
}

Ideally, these invalidation rules can be "composed". If a resolvers ( getEntityRecords ) uses a given action ( receiveItems ) and receiveItems defines that it can be invalidated by addItems then getEntityRecords is also automatically invalidated by addItems

[jorgefilipecosta]

I would prefer if we did not have to make a fetch request right after adding new items. I guess a dumb adder, a sorter and a filter by property would cover 99% of the cases and maybe we can generate the updater automatically based on the query that is already being, for complex cases we would invalid and request again.

[aduth]

I would prefer if we did not have to make a fetch request right after adding new items.

Considering that there is nothing about resolvers that necessitates that their implementation incur a network request, I don't know that this is strictly problematic. While an additional implementation could naively destroy the local cache and trigger a follow-up refetch, I could envision some advanced resolver which has its own reconciling cache to more efficiently resolve data after an invalidation.

[aduth]

A rought proposal:

Thought: Would the inverse work / be better? Where an action define (by names?) the selectors it would invalidate?

[youknowriad]

Thought: Would the inverse work / be better? Where an action define (by names?) the selectors it would invalidate?

how do you refer to partially applied selectors? say you want to invalidate all calls to a selector with a given first argument.

[aduth]

With that, I guess you could not be granular at all. Still possible, but would require designing selectors to not be so generic. Leaning towards your original proposal. Did you have any worries about it?

[aduth]

I'm more okay with a solution which is "wasteful" in invalidating more than it needs to than one which requires recreating querying in the client, which I feel would be impossible to have be 100% accurate (and thus lead to inconsistencies).

[youknowriad]

I added cache invalidation using the mechanism suggested above #10089 (comment)

This is lacking some unit tests but should be considered for merge soon.

[youknowriad]

@aduth .delete didn't work, probably a bug upstream in EquivalentKeyMap

[aduth]

Minor: the data reducer does not hold a cache of data per-se, it only tracks whether resolution has occurred. Therefore, maybe a better name is INVALIDATE_RESOLUTION or INVALIDATE_IS_RESOLVED

[youknowriad]

Thanks @aduth for the updates, I also added some tests and I think this should be ready for a final review.

[TimothyBJacobs]

@youknowriad sorry. To clarify, could returning the action value in the withDispatch hoc happen before 5.0? Or is that not happening at all? It’d be extremely helpful for being able to show in the UI when long running actions are occurring.

[youknowriad]

@TimothyBJacobs The workaround is to trigger an action setting a flag at the beginning of your async action and unsetting the flag and the end and use a selector to retrieve this flag.

That said, I agree we need something more consistent across actions, whether it's the promise returned by withDispatch (which has drawbacks: don't forget to cancel the promise when the component unmounts) or automatically identify and track the actions in core/data (like resolvers). I'm not certain yet what's the best approach.

I don't see it as crucial before 5.0 because there are workarounds but it's nice to have and if we get something in, that would be great.

[TimothyBJacobs]

Gotcha. Thank you for the alternative! Agreed not necessary for 5.0, but would be nice. If something made it into the data module after API freeze, would it still be included in 5.0? Or is the deadline right around the corner.

[youknowriad]

I think enhancements to the API are fine. Breaking changes not that welcome. But don't take my word for it :)