Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement contextSeries - alternative to waterfall #779

Closed
wants to merge 1 commit into from

Conversation

macobo
Copy link

@macobo macobo commented Jun 6, 2015

This function is mainly useful as an alternative to watefall in versions of javascript, which support destructing.

Main problem with waterfall is that it becomes hard to debug/refactor after initial write due to needing to passaround arguments in exactly the correct order. This function when used with destructing gets around that problem by allowing to name steps in the series and then each function can grab results it cares about.

Example (in coffeescript):

# Adapted from one of my work scripts

getCurrentShards = (r, callback) -> # ...
cleanupOldShardMoves = ({shardInfo}, callback) -> # ...
createTask = ({shardInfo}, callback) -> # ...
setMoveStatus = (newStatus, {taskId, state}, callback) -> # ...
lockShards = ({taskId, shardInfo}, callback) -> # ...
# etc...

tasks = [
  ['shardInfo', getCurrentShards]
  cleanupOldShardMoves
  ['taskId', createTask]
  ['state', _.partial(setMoveStatus, 'initial')]
  lockShards
  _.partial(setMoveStatus, 'moving')
  moveInBatches
  _.partial(setMoveStatus, 'unlocking')
  unlockShards
  ['timestamp', updateMetadata]
]
async.contextSeries tasks, (err, {timestamp, taskId}) ->
  # ...
  log.info "Shard moves completed successfully at #{format timestamp} for task #{taskId}"

This function can prove to be even more useful by adding an optional seedData object with values which are made available to each task in the contextSeries array. I have omitted that however due to it not really fitting in with the rest of the API.

This function is mainly useful as an alternative to watefall in versions of javascript, which support destructing.

Main problem with waterfall is that it becomes hard to debug/refactor after initial write due to needing to pass
around arguments in _exactly_ the correct order. This function when used with destructing gets around that limitation
by allowing to name steps in the series and then each function can grab results it cares about.
@macobo macobo force-pushed the karl-contextSeries branch from 9e407c6 to 478579f Compare June 6, 2015 15:33
@aearly aearly added the feature label Jun 7, 2015
@aearly
Copy link
Collaborator

aearly commented Jun 7, 2015

I don't feel like this is any more useful than auto. Sure, you have guaranteed in-order execution, but you can easily do that in auto by setting your dependencies correctly.

@aearly
Copy link
Collaborator

aearly commented Jun 7, 2015

Also, the proposed feature in #635 -- essentially autoLimit -- might also be a way to enforce sequentiality.

@macobo
Copy link
Author

macobo commented Jun 7, 2015

I would argue it is more useful than auto for sequential operations because of not needing to specify previous steps name and autos downright weird function argument ordering.

As mentioned before, we're using it as an alternative to waterfall which always is a huge pain to refactor due to having to pass arguments through in the exact correct order.

@aearly
Copy link
Collaborator

aearly commented Jun 7, 2015

I hear you about auto's weird parameter ordering. I want to swap the ordering, but need to figure out the best way to phase it in.

In the meantime, I often write a simple combinator that flips the arguments:

function flip(fn) {
  return function (results, cb) {
    return fn(cb, results);
  };
}

function normalAsyncFunction(options, callback) {...}

async.auto({
  //..
  "task": ["dependency", flip(normalAsyncFunction)]
  //..
})

@aearly aearly closed this Jun 7, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants