Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hooks: add support for async filters and actions #64204

Merged
merged 2 commits into from
Sep 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions packages/hooks/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@

## Unreleased

### New Features

- added new `doActionAsync` and `applyFiltersAsync` functions to run hooks in async mode ([#64204](https://github.com/WordPress/gutenberg/pull/64204)).

## 4.8.0 (2024-09-19)

## 4.7.0 (2024-09-05)
Expand Down
2 changes: 2 additions & 0 deletions packages/hooks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,9 @@ One notable difference between the JS and PHP hooks API is that in the JS versio
- `removeAllActions( 'hookName' )`
- `removeAllFilters( 'hookName' )`
- `doAction( 'hookName', arg1, arg2, moreArgs, finalArg )`
- `doActionAsync( 'hookName', arg1, arg2, moreArgs, finalArg )`
- `applyFilters( 'hookName', content, arg1, arg2, moreArgs, finalArg )`
- `applyFiltersAsync( 'hookName', content, arg1, arg2, moreArgs, finalArg )`
- `doingAction( 'hookName' )`
- `doingFilter( 'hookName' )`
- `didAction( 'hookName' )`
Expand Down
7 changes: 2 additions & 5 deletions packages/hooks/src/createCurrentHook.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,8 @@
function createCurrentHook( hooks, storeKey ) {
return function currentHook() {
const hooksStore = hooks[ storeKey ];

return (
hooksStore.__current[ hooksStore.__current.length - 1 ]?.name ??
null
);
const currentArray = Array.from( hooksStore.__current );
return currentArray.at( -1 )?.name ?? null;
};
}

Expand Down
10 changes: 5 additions & 5 deletions packages/hooks/src/createDoingHook.js
Original file line number Diff line number Diff line change
Expand Up @@ -24,13 +24,13 @@ function createDoingHook( hooks, storeKey ) {

// If the hookName was not passed, check for any current hook.
if ( 'undefined' === typeof hookName ) {
return 'undefined' !== typeof hooksStore.__current[ 0 ];
return hooksStore.__current.size > 0;
}

// Return the __current hook.
return hooksStore.__current[ 0 ]
? hookName === hooksStore.__current[ 0 ].name
: false;
// Find if the `hookName` hook is in `__current`.
return Array.from( hooksStore.__current ).some(
( hook ) => hook.name === hookName
);
};
}

Expand Down
10 changes: 6 additions & 4 deletions packages/hooks/src/createHooks.js
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,11 @@ export class _Hooks {
constructor() {
/** @type {import('.').Store} actions */
this.actions = Object.create( null );
this.actions.__current = [];
this.actions.__current = new Set();

/** @type {import('.').Store} filters */
this.filters = Object.create( null );
this.filters.__current = [];
this.filters.__current = new Set();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had some concerns about switching the storage type, worried that someone could be using the internals (using the internals for the PHP version of the hooks is not uncommon), but I didn't spot any custom usage, so should be fine.

Copy link
Member

@adamsilverstein adamsilverstein Aug 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious about why this is changed - is there a benefit to Set over [] or any risk this could break something? (eg is add not allowing duplicates a concern?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A hook run must be able to reliable add and remove itself to __current when it starts and ends. With async hooks, multiple hooks can run at the same time, independently and in parallel, and array .push and .pop is no longer reilable. You could be .popping not your own record, but someone else's. That gets fixed with a Set (of currently running hooks).


this.addAction = createAddHook( this, 'actions' );
this.addFilter = createAddHook( this, 'filters' );
Expand All @@ -34,8 +34,10 @@ export class _Hooks {
this.hasFilter = createHasHook( this, 'filters' );
this.removeAllActions = createRemoveHook( this, 'actions', true );
this.removeAllFilters = createRemoveHook( this, 'filters', true );
this.doAction = createRunHook( this, 'actions' );
this.applyFilters = createRunHook( this, 'filters', true );
this.doAction = createRunHook( this, 'actions', false, false );
this.doActionAsync = createRunHook( this, 'actions', false, true );
this.applyFilters = createRunHook( this, 'filters', true, false );
this.applyFiltersAsync = createRunHook( this, 'filters', true, true );
this.currentAction = createCurrentHook( this, 'actions' );
this.currentFilter = createCurrentHook( this, 'filters' );
this.doingAction = createDoingHook( this, 'actions' );
Expand Down
57 changes: 37 additions & 20 deletions packages/hooks/src/createRunHook.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,15 @@
* registered to a hook of the specified type, optionally returning the final
* value of the call chain.
*
* @param {import('.').Hooks} hooks Hooks instance.
* @param {import('.').Hooks} hooks Hooks instance.
* @param {import('.').StoreKey} storeKey
* @param {boolean} [returnFirstArg=false] Whether each hook callback is expected to
* return its first argument.
* @param {boolean} returnFirstArg Whether each hook callback is expected to return its first argument.
* @param {boolean} async Whether the hook callback should be run asynchronously
*
* @return {(hookName:string, ...args: unknown[]) => undefined|unknown} Function that runs hook callbacks.
*/
function createRunHook( hooks, storeKey, returnFirstArg = false ) {
return function runHooks( hookName, ...args ) {
function createRunHook( hooks, storeKey, returnFirstArg, async ) {
return function runHook( hookName, ...args ) {
const hooksStore = hooks[ storeKey ];

if ( ! hooksStore[ hookName ] ) {
Expand Down Expand Up @@ -42,26 +42,43 @@ function createRunHook( hooks, storeKey, returnFirstArg = false ) {
currentIndex: 0,
};

hooksStore.__current.push( hookInfo );

while ( hookInfo.currentIndex < handlers.length ) {
const handler = handlers[ hookInfo.currentIndex ];

const result = handler.callback.apply( null, args );
if ( returnFirstArg ) {
args[ 0 ] = result;
async function asyncRunner() {
try {
hooksStore.__current.add( hookInfo );
let result = returnFirstArg ? args[ 0 ] : undefined;
while ( hookInfo.currentIndex < handlers.length ) {
const handler = handlers[ hookInfo.currentIndex ];
result = await handler.callback.apply( null, args );
if ( returnFirstArg ) {
args[ 0 ] = result;
}
hookInfo.currentIndex++;
}
return returnFirstArg ? result : undefined;
} finally {
hooksStore.__current.delete( hookInfo );
}

hookInfo.currentIndex++;
}

hooksStore.__current.pop();

if ( returnFirstArg ) {
return args[ 0 ];
function syncRunner() {
try {
hooksStore.__current.add( hookInfo );
let result = returnFirstArg ? args[ 0 ] : undefined;
while ( hookInfo.currentIndex < handlers.length ) {
const handler = handlers[ hookInfo.currentIndex ];
result = handler.callback.apply( null, args );
if ( returnFirstArg ) {
args[ 0 ] = result;
}
hookInfo.currentIndex++;
}
return returnFirstArg ? result : undefined;
} finally {
hooksStore.__current.delete( hookInfo );
}
}

return undefined;
return ( async ? asyncRunner : syncRunner )();
};
}

Expand Down
6 changes: 5 additions & 1 deletion packages/hooks/src/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ import createHooks from './createHooks';
*/

/**
* @typedef {Record<string, Hook> & {__current: Current[]}} Store
* @typedef {Record<string, Hook> & {__current: Set<Current>}} Store
*/

/**
Expand All @@ -48,7 +48,9 @@ const {
removeAllActions,
removeAllFilters,
doAction,
doActionAsync,
applyFilters,
applyFiltersAsync,
currentAction,
currentFilter,
doingAction,
Expand All @@ -70,7 +72,9 @@ export {
removeAllActions,
removeAllFilters,
doAction,
doActionAsync,
applyFilters,
applyFiltersAsync,
currentAction,
currentFilter,
doingAction,
Expand Down
150 changes: 150 additions & 0 deletions packages/hooks/src/test/index.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,9 @@ import {
removeAllActions,
removeAllFilters,
doAction,
doActionAsync,
applyFilters,
applyFiltersAsync,
currentAction,
currentFilter,
doingAction,
Expand Down Expand Up @@ -943,3 +945,151 @@ test( 'checking hasFilter with named callbacks and removeAllActions', () => {
expect( hasFilter( 'test.filter', 'my_callback' ) ).toBe( false );
expect( hasFilter( 'test.filter', 'my_second_callback' ) ).toBe( false );
} );

describe( 'async filter', () => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also add tests that verify the functions that work with the current hook work well with async filters/actions? doingAction comes to mind.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added tests that verify that doingAction/doingFilter return correct results when multiple hooks are running at once, in parallel.

test( 'runs all registered handlers', async () => {
addFilter( 'test.async.filter', 'callback_plus1', ( value ) => {
return new Promise( ( r ) =>
setTimeout( () => r( value + 1 ), 10 )
);
} );
addFilter( 'test.async.filter', 'callback_times2', ( value ) => {
return new Promise( ( r ) =>
setTimeout( () => r( value * 2 ), 10 )
);
} );

expect( await applyFiltersAsync( 'test.async.filter', 2 ) ).toBe( 6 );
} );

test( 'aborts when handler throws an error', async () => {
const sqrt = jest.fn( async ( value ) => {
if ( value < 0 ) {
throw new Error( 'cannot pass negative value to sqrt' );
}
return Math.sqrt( value );
} );

const plus1 = jest.fn( async ( value ) => {
return value + 1;
} );

addFilter( 'test.async.filter', 'callback_sqrt', sqrt );
addFilter( 'test.async.filter', 'callback_plus1', plus1 );

await expect(
applyFiltersAsync( 'test.async.filter', -1 )
).rejects.toThrow( 'cannot pass negative value to sqrt' );
expect( sqrt ).toHaveBeenCalledTimes( 1 );
expect( plus1 ).not.toHaveBeenCalled();
} );

test( 'is correctly tracked by doingFilter and didFilter', async () => {
addFilter( 'test.async.filter', 'callback_doing', async ( value ) => {
await new Promise( ( r ) => setTimeout( () => r(), 10 ) );
expect( doingFilter( 'test.async.filter' ) ).toBe( true );
return value;
} );

expect( doingFilter( 'test.async.filter' ) ).toBe( false );
expect( didFilter( 'test.async.filter' ) ).toBe( 0 );
await applyFiltersAsync( 'test.async.filter', 0 );
expect( doingFilter( 'test.async.filter' ) ).toBe( false );
expect( didFilter( 'test.async.filter' ) ).toBe( 1 );
} );

test( 'is correctly tracked when multiple filters run at once', async () => {
addFilter( 'test.async.filter1', 'callback_doing', async ( value ) => {
await new Promise( ( r ) => setTimeout( () => r(), 10 ) );
expect( doingFilter( 'test.async.filter1' ) ).toBe( true );
await new Promise( ( r ) => setTimeout( () => r(), 10 ) );
return value;
} );
addFilter( 'test.async.filter2', 'callback_doing', async ( value ) => {
await new Promise( ( r ) => setTimeout( () => r(), 10 ) );
expect( doingFilter( 'test.async.filter2' ) ).toBe( true );
await new Promise( ( r ) => setTimeout( () => r(), 10 ) );
return value;
} );

await Promise.all( [
applyFiltersAsync( 'test.async.filter1', 0 ),
applyFiltersAsync( 'test.async.filter2', 0 ),
] );
} );
} );

describe( 'async action', () => {
test( 'runs all registered handlers sequentially', async () => {
const outputs = [];
const action1 = async () => {
outputs.push( 1 );
await new Promise( ( r ) => setTimeout( () => r(), 10 ) );
outputs.push( 2 );
};

const action2 = async () => {
outputs.push( 3 );
await new Promise( ( r ) => setTimeout( () => r(), 10 ) );
outputs.push( 4 );
};

addAction( 'test.async.action', 'action1', action1 );
addAction( 'test.async.action', 'action2', action2 );

await doActionAsync( 'test.async.action' );
expect( outputs ).toEqual( [ 1, 2, 3, 4 ] );
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I wonder if this should be the right order. I feel like the correct order should be 1, 3, 2, 4 (in other words, internally we'd use Promise.all to allow for parallel actions).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like the order doesn't matter in actions (as opposed to filters)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addAction also has priority as argument, both in JS and PHP versions. So the order matters and can be influenced.

The only difference between actions and filters is that filters process a "value" that is passed from one handler to another.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes true but I feel that for most actions, we don't really care about the order. I wonder if it should be an option. For instance, this has the waterfall effect if multiple plugins trigger REST APIs for instance in an async action...

Anyway, I can leave with both approaches.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now I understand, the registered actions could also run in parallel, and there would be one Promise.all or Promise.allSettled that waits and only then doAsyncAction returns.

Why not, but that means entering a new territory where there is not precedent with existing PHP and JS actions. These all run sequentially.

Do you have a use case for async actions? Filters are easy, they always must be sequential, but actions can run in different modes.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Theoretically, order should matter for both actions and filters, specifically if we want to mirror the hook behavior on the PHP side. Order does matter there, and is predictable. It's important for consistent behavior when you have multiple functions hooked to the same action.

Imagine you're hooking 10 functions from separate plugins, each of them registering tabs in a tabbed layout. If you care about the tab order (and you likely do), then you'll likely care about the consistency of the action order.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

think about the "post save" action we have or the "pre save" action. How do you see plugins using this one?

The known use cases for preSavePost are:

Validations can run in parallel, because the result is a boolean success/fail. One failed validation is sufficient to fail them all.

Modifying edits must run sequentially, the output of one handler is fed to the next one.

For savePost (after the entity was saved to REST) the known use case is saving metaboxes. After the post itself is saved, the metabox forms are saved to _wpMetaBoxUrl. These handlers could run in parallel.

After some consideration, I believe that async actions are inherently parallel, because the input of any action doesn't depend on the output of any other. While async filters are inherently sequential.

I'm not sure about handling errors. Until now the sync hook didn't really support throwing errors. Stale data would be left inside hooksStore.__current if any handler threw an exception, there was no try/catch block.

In the current state of this PR, throwing an error immediately aborts the hook run. Further handlers are called only when all previous handlers succeed. It's not possible, for example, that a later handler recovers from an error thrown by an earlier one. Like you can do with promises: promise.then().catch().then(). I'm not sure if this lack of flexibility is a problem or not.

In PHP, a handler can return a wp_error and the next handler has complete freedom what to do with that error.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Webpack has a very sophisticated system for various kinds of sync and async hooks, published separately as the tapable package. These are the kinds of async hooks that they provide:

  • AsyncSeriesHook: all handlers run one at at time, in a sequence. If any handler throws an error, further handlers are not called and the hook run finishes with that error. All handlers receive the same parameters (passed to the "run" function) and don't return anything. The return type of handlers and the "run" function is Promise<void>.
  • AsyncSeriesBailHook: a slight variation of the previous hook where a handler can abort the process not only by throwing an error, but also by returning a non-undefined value. If a handler has "something to say", it will return that, and it will become the result of the hook run. The same logic as with errors, but the semantic is not an error. The hook can have both a regular result and an error. The return type is Promise<T | undefined>.
  • AsyncSeriesWaterfallHook: here each handler's first argument is the return value of the previous handler. The remaining argument are constant. Very similar to a WordPress filter.
  • AsyncParallelHook: all handlers are running in parallel, started at the same time. If any of them throws an error, the hook run finishes with that error. The other handlers still run (we can't abort them) but their return values will be ignored. Very similar to Promise.all.
  • AsyncParallelBailHook: all handlers also run in parallel, and if any of them bails (returns non-undefined value) or throws, that becomes the result of the hook run. Other handlers continue to run but their results are ignored.

Currently, this PR implements an equivalent of AsyncSeriesHook (doAsyncAction) and AsyncSeriesWaterfallHook (applyAsyncFilters).

We could also add AsyncParallelHook, to do an action in parallel. Could be a parameter of doAsyncAction (parallel = true | false), but there is no good way to distinguish it from other parameters. So a better option is doAsyncActionParallel method.

The "bail" logic doesn't have any prior art in WordPress and I think we don't need it. It can be always simulated by each handler doing a check:

if ( value !== undefined ) {
  return value;
}
// start doing the actual job

There are WordPress filters, like determine_current_user, that are of the "bail" type and do exactly this.

So, what do you think? How much inspiration should we take here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still hesitant about which one do we need more parallel or not but I'm thinking that we should pick one for now and see how far we can get with it.

Your call, if you think "serial/waterfall" is better as default, we can go with it for now and consider a separate function later for parallel if need be.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the base webpack codebase, they have 42 async hook, and only 6 of them are parallel. The majority, 36, are serial. That suggests that parallel actions are an exception.

So, yes, I'll go with doActionAsync and applyFiltersAsync being serial by default. We can add doActionAsyncParallel at any time later when there is a demand.

Also, doActionAsync seems to be a better name than doAsyncAction. It follows a convention where the base name is doAction and there is a series of modifiers. Core has do_action_deprecated or do_action_ref_array, so this convention is already established. I'll modify the PR accordingly.

} );

test( 'aborts when handler throws an error', async () => {
const outputs = [];
const action1 = async () => {
throw new Error( 'aborting' );
};

const action2 = async () => {
outputs.push( 3 );
await new Promise( ( r ) => setTimeout( () => r(), 10 ) );
outputs.push( 4 );
};

addAction( 'test.async.action', 'action1', action1 );
addAction( 'test.async.action', 'action2', action2 );

await expect( doActionAsync( 'test.async.action' ) ).rejects.toThrow(
'aborting'
);
expect( outputs ).toEqual( [] );
} );

test( 'is correctly tracked by doingAction and didAction', async () => {
addAction( 'test.async.action', 'callback_doing', async () => {
await new Promise( ( r ) => setTimeout( () => r(), 10 ) );
expect( doingAction( 'test.async.action' ) ).toBe( true );
} );

expect( doingAction( 'test.async.action' ) ).toBe( false );
expect( didAction( 'test.async.action' ) ).toBe( 0 );
await doActionAsync( 'test.async.action', 0 );
expect( doingAction( 'test.async.action' ) ).toBe( false );
expect( didAction( 'test.async.action' ) ).toBe( 1 );
} );

test( 'is correctly tracked when multiple actions run at once', async () => {
addAction( 'test.async.action1', 'callback_doing', async () => {
await new Promise( ( r ) => setTimeout( () => r(), 10 ) );
expect( doingAction( 'test.async.action1' ) ).toBe( true );
await new Promise( ( r ) => setTimeout( () => r(), 10 ) );
} );
addAction( 'test.async.action2', 'callback_doing', async () => {
await new Promise( ( r ) => setTimeout( () => r(), 10 ) );
expect( doingAction( 'test.async.action2' ) ).toBe( true );
await new Promise( ( r ) => setTimeout( () => r(), 10 ) );
} );

await Promise.all( [
doActionAsync( 'test.async.action1', 0 ),
doActionAsync( 'test.async.action2', 0 ),
] );
} );
} );
Loading