Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make GraphProcessor trace optional #460

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

AryamanAgrawalIronclad
Copy link
Contributor

@AryamanAgrawalIronclad AryamanAgrawalIronclad commented Dec 3, 2024

  • Arguably, trace events can become add to emit events being a bottleneck when running a large number of graphs parallelly.
  • This PR makes trace emit events optional with the parameter defaulting to true.
  • includeTrace only exists as an override for a user who would like to disable traces for their graph runs.
  • Conditional wrappers have been placed in a way such that they don't hinder or change the current execution however, a review would be helpful to make sure that is the case for all changes in the file.

@AryamanAgrawalIronclad AryamanAgrawalIronclad changed the title Make traces optional Make GraphProcessor trace optional Dec 3, 2024
Copy link
Contributor

@ZhangYiJiang ZhangYiJiang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you run lint and prettier fix? It seems CI is missing setup for them, which will need to be fixed in a separate PR

@@ -75,7 +76,7 @@ export function coreCreateProcessor(project: Project, options: RunGraphOptions)
throw new Error(`Graph not found, and no main graph specified.`);
}

const processor = new GraphProcessor(project, graphId as GraphId, options.registry);
const processor = new GraphProcessor(project, graphId as GraphId, options.registry, options.includeTrace);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be preferred for optional properties to be grouped into an options object so they can be specified in any order, but this class is exported out of core which technically makes it part of the public interface. While I think most users would use it through createGraphProcessor we can't update this without a breaking change, but we can at least note this for the future

Comment on lines +877 to +882
if(this.#includeTrace){
this.#emitter.emit(
'trace',
`Node ${node.title} has required inputs nodes: ${inputNodes.map((n) => n.title).join(', ')}`,
);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also we should consider adding a small util to wrap this so future calls to emitting trace don't miss the check

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants