Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Integrate TaskManager into NodeGraph and Discovery #445

Merged
merged 40 commits into from
Sep 21, 2022
Merged

Integrate TaskManager into NodeGraph and Discovery #445

merged 40 commits into from
Sep 21, 2022

Commits on Sep 21, 2022

  1. feat: PolykeyAgent.ts using TaskManager

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    a69f513 View commit details
    Browse the repository at this point in the history

  2. feat: updated NodeManager to use TaskManager

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    66ee630 View commit details
    Browse the repository at this point in the history

  3. feat: updated NodeConnectionManager to use TaskManager

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    41d3a00 View commit details
    Browse the repository at this point in the history

  4. feat: adding cancellability to NodeManager handlers

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    eb1b557 View commit details
    Browse the repository at this point in the history

  5. fix: small bug with pingNode

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    6e4322b View commit details
    Browse the repository at this point in the history

  6. fix: bugs with nodeConnectionManager.syncNodeGraph

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    27316f9 View commit details
    Browse the repository at this point in the history

  7. tests: cleaning up dependencies in tests

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    f1ab40b View commit details
    Browse the repository at this point in the history

  8. fix: getRemoteNodeClosestNodes shouldn't throw connection errors 

    `getRemoteNodeClosestNodes` was throwing an connection error in certain conditions. If it failed to connect to a node it should've just skipped that node.

#418
    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    a4470ad View commit details
    Browse the repository at this point in the history

  9. fix: excessive connections from refreshBuckets 

    Removing excessive logging for using connections. We don't need a 3 log messages for each time we use an existing connection.

Adding 'jitter' or spacing to the `refreshBuckets` delays so that they don't run all at once. This is implemented with a `refreshBucketDelaySpread` paramater that specifies the multiple of the delay to spread across. defaults to 0.5 for 50%

Adding a 'heuristic' to `refreshBucket` to prevent it from contacting the same nodes repeatably. Currently this is just a check in `getClosestGlobalNodes` where if we find less than `nodeBucketLimit` nodes we just reset the timer on all `refreshBucket` tasks.

Adding tests for checking the spread of `refreshBucket` delays. Another test for resetting the timer on `refreshBucket` tasks if a `findNode` finds less than 20 nodes.

#415
    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    ca5e675 View commit details
    Browse the repository at this point in the history

  10. feat: nodeConnectionManager.getClosestGlobalNodes can optionally sk… 

    …ip recently offline nodes

This is done with an in-memory map of `nodeIdstring` to some data tracking the backoff period. it defaults to 5 min and doubles each failure.

#413
    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    206dceb View commit details
    Browse the repository at this point in the history

  11. feat: added handler to detect promise deadlocks to ExitHandlers.ts 

    #307
    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    c65e758 View commit details
    Browse the repository at this point in the history

  12. fix: handerIds are derived from class and handler function names

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    f9b1dbe View commit details
    Browse the repository at this point in the history

  13. fix: fixing up setNode garbage collection.

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    20ea395 View commit details
    Browse the repository at this point in the history

  14. fix: refactored nodeManager.setNode garbage collection

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    26695b5 View commit details
    Browse the repository at this point in the history

  15. tests: proper stopping of taskManager in tests

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    ca9af18 View commit details
    Browse the repository at this point in the history

  16. fix: updating task paths 

    `TaskPaths` should take the form of `[basePath, handlerId, ...extra]`. basePath is the `this.constructor.name` for the domain the handler is registered for.
    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    3be12d7 View commit details
    Browse the repository at this point in the history

  17. feat: task handlers are now timedCancellable

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    19bce20 View commit details
    Browse the repository at this point in the history

  18. fix: depending on return from updateTask to check existence

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    ad7c751 View commit details
    Browse the repository at this point in the history

  19. fix: replacing private with protected

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    81dbac6 View commit details
    Browse the repository at this point in the history

  20. fix: removing Queue.ts

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    66181b4 View commit details
    Browse the repository at this point in the history

  21. fix: removing @ready from TaskManager.cancelTask

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    cd47c74 View commit details
    Browse the repository at this point in the history

  22. docs: adding description to isConnectionError

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    34c658e View commit details
    Browse the repository at this point in the history

  23. fix: added cancellability and blocking to nodeManager.setNode

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    b9d248b View commit details
    Browse the repository at this point in the history

  24. fix: extracted refreshBucketsDelayJitter into nodes utils

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    fe817a2 View commit details
    Browse the repository at this point in the history

  25. fix: fixes to errors and adding un-recoverable error handlers 

    Un-recoverable errors include `ErrorBinUncaughtException`, `ErrorBinUnhandledRejection` and `ErrorBinAsynchronousDeadlock`.

#414
    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    ca2c966 View commit details
    Browse the repository at this point in the history

  26. fix: moved 'syncNodeGraph from NodeConnectionManager to NodeManager`

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    97ce1d8 View commit details
    Browse the repository at this point in the history

  27. fix: pingNodes inside of garbageCollectBucket now have timeouts s… 

    …eparate from the overall timer

other fixes have been applied.
    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    1229695 View commit details
    Browse the repository at this point in the history

  28. fix: cleaning up ephemeral tasks when stopping NodeManager

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    48298f8 View commit details
    Browse the repository at this point in the history

  29. fix: cleaning up errors

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    9f3d2c0 View commit details
    Browse the repository at this point in the history

  30. fix: small fix to updateRefreshBucketDelay

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    45360d4 View commit details
    Browse the repository at this point in the history

  31. fix: rollback of proxy changes, was out of scope for this PR

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    81e5532 View commit details
    Browse the repository at this point in the history

  32. fix: small fix to garbageCollectBucket concurrent pinging

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    a33ea26 View commit details
    Browse the repository at this point in the history

  33. fix: updated default timeout for NodeConnectionManager.pingNode

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    f11197c View commit details
    Browse the repository at this point in the history

  34. fix: using Symbols for cancelling tasks

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    0f729b1 View commit details
    Browse the repository at this point in the history

  35. fix: TaskManager should extend the CreateDestroyStartStop interface

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    607095b View commit details
    Browse the repository at this point in the history

  36. fix: test wasn't overriding key-pair generation

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    43027e7 View commit details
    Browse the repository at this point in the history

  37. fix: TaskManager's stopProcessing and stopTasks are now properl… 

    …y idempotent

the @ready decorator caused them to throw if ran while `taskManager` was not running. They needed to be called during incomplete startup, so I removed the decorator.
    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    0267d3b View commit details
    Browse the repository at this point in the history

  38. tests: slightly increasing timeouts for two tests

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    ccc61a8 View commit details
    Browse the repository at this point in the history

  39. tests: general fixes for tests failing in CI

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    b29ba9e View commit details
    Browse the repository at this point in the history

  40. syntax: formatting change for ExitHandlers.ts

    @tegefaulkes
    tegefaulkes committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    23f470b View commit details
    Browse the repository at this point in the history