Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Group profiling tool "Potential Problems" #2659

Closed
revans2 opened this issue Jun 9, 2021 · 0 comments · Fixed by #2669
Closed

[FEA] Group profiling tool "Potential Problems" #2659

revans2 opened this issue Jun 9, 2021 · 0 comments · Fixed by #2669
Assignees
Labels
bug Something isn't working feature request New feature or request

Comments

@revans2
Copy link
Collaborator

revans2 commented Jun 9, 2021

I ran the profiling tool on some integration test runs as kind of a stress test, and I got some output like.

App Name,App ID,Score,Potential Problems,SQL Dataframe Duration,App Duration,Executor CPU Time Percent
rapids spark plugin integration tests (python),app-20201105103038-0001,68.54,"UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF",2119989,3093153,44.0
rapids spark plugin integration tests (python),app-20210526070813-0008,49.4,"UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF,UDF",1471194,2978171,21.01

This is rather ugly and it might be better to just have UDF and a count of how many times it showed up.

@revans2 revans2 added feature request New feature or request ? - Needs Triage Need team to review and classify labels Jun 9, 2021
@tgravescs tgravescs self-assigned this Jun 9, 2021
@tgravescs tgravescs added bug Something isn't working and removed ? - Needs Triage Need team to review and classify labels Jun 9, 2021
@tgravescs tgravescs added this to the June 7 - June 18 milestone Jun 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working feature request New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants