You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Since current job shuffle data clean up policy is to call remove_job_data for every executor which may not contain any shuffle data of the job. For a large cluster with many executors, many redundant warn logs will be printed in the scheduler side when it detects the executor fail to remove the job shuffle data.
Describe the solution you'd like
It's better for the executor to be silent when no job shuffle data exists.
Describe alternatives you've considered
Additional context
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Since current job shuffle data clean up policy is to call remove_job_data for every executor which may not contain any shuffle data of the job. For a large cluster with many executors, many redundant warn logs will be printed in the scheduler side when it detects the executor fail to remove the job shuffle data.
Describe the solution you'd like
It's better for the executor to be silent when no job shuffle data exists.
Describe alternatives you've considered
Additional context
The text was updated successfully, but these errors were encountered: