Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DYN-4228 Convert StrongConnectComponent to use List vs Dictionary input type #10355

Merged

Conversation

saintentropy
Copy link
Contributor

@saintentropy saintentropy commented Feb 4, 2020

Purpose

https://jira.autodesk.com/browse/DYN-4228

The purpose of this PR is to remove a performance bottleneck that affect graphs with larger node quantities. In the graph compilation phase, cyclic dependencies in the graph logic are detected for error and warning purpose. Part of this detection process creates a numerous Dictionary collections which incurred significant time and memory allocation. This PR refactors the StrongConnectComponent method to utilize existing data found in the GraphNode object (specifically the dependencyGraphListID property) vs the creation of new Map data structure for a lookup key.

Declarations

Check these if you believe they are true

  • The codebase is in a better state after this PR
  • Is documented according to the standards
  • The level of testing this PR includes is appropriate
  • User facing strings, if any, are extracted into *.resx files
  • All tests pass using the self-service CI.
  • Snapshot of UI changes, if any.
  • Changes to the API follow Semantic Versioning and are documented in the API Changes document.

Reviewers

@mjkkirschner @aparajit-pratap

FYIs

@reddyashish

@saintentropy
Copy link
Contributor Author

Screen Shot 2020-02-04 at 3 13 10 PM

Rough perf comparison for a the compilation step for a sample graph of 320 nodes

@@ -337,18 +337,17 @@ private bool CyclicDependencyTest(ProtoCore.AssociativeGraph.GraphNode node, ref
return true;
}

var indexMap = new Dictionary<GraphNode, int>();
var lowlinkMap = new Dictionary<GraphNode, int>();
var indexList = new int[codeBlock.instrStream.dependencyGraph.GraphList.Count];
Copy link
Contributor

@aparajit-pratap aparajit-pratap Feb 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the actual place where we're getting the perf improvement; since you are pre-allocating the size of the array in this case rather than initializing an empty dictionary that needs to be expanded every other time values are added to it?

@saintentropy saintentropy changed the title WIP: Convert StrongConnectComponent to use List vs Dictionary input type Convert StrongConnectComponent to use List vs Dictionary input type Oct 18, 2021
@saintentropy saintentropy changed the title Convert StrongConnectComponent to use List vs Dictionary input type [DYN-4228] Convert StrongConnectComponent to use List vs Dictionary input type Oct 18, 2021
@saintentropy saintentropy changed the title [DYN-4228] Convert StrongConnectComponent to use List vs Dictionary input type DYN-4228 Convert StrongConnectComponent to use List vs Dictionary input type Oct 18, 2021
@saintentropy saintentropy removed the WIP label Oct 18, 2021
Copy link
Contributor

@aparajit-pratap aparajit-pratap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Waiting for PR checks and tests to pass before merging.

Copy link
Contributor

@aparajit-pratap aparajit-pratap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Waiting for PR checks and tests to pass before merging.

@aparajit-pratap
Copy link
Contributor

@aparajit-pratap aparajit-pratap merged commit 78e0559 into DynamoDS:master Apr 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants