Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[<Ray component: Core|RLlib|etc...>] #47223

Open
anyscalesam opened this issue Aug 20, 2024 · 1 comment
Open

[<Ray component: Core|RLlib|etc...>] #47223

anyscalesam opened this issue Aug 20, 2024 · 1 comment
Labels
bug Something that is supposed to be working; but isn't core Issues that should be addressed in Ray Core observability Issues related to the Ray Dashboard, Logging, Metrics, Tracing, and/or Profiling P3 Issue moderate in impact or severity

Comments

@anyscalesam
Copy link
Contributor

What happened + What you expected to happen

Issue: The environment variable RAY_DEDUP_LOGS=0, which is intended to disable log deduplication in Ray, does not seem to be deterministically configurable.

Details: A user tried to set RAY_DEDUP_LOGS=0 as referenced in the Ray documentation to prevent Ray from deduplicating logs while using Ray Tune with a PyTorch model. Despite following the instructions, the deduplication still occurred, and the user couldn't find clear guidance on how to properly set this variable or which object controls it.

Request: Investigate and clarify how to correctly configure the RAY_DEDUP_LOGS environment variable to ensure deterministic behavior.

Versions / Dependencies

many

Reproduction script

see various examples in discourse thread here https://discuss.ray.io/t/how-to-set-ray-dedup-logs-0/10465

Issue Severity

None

@anyscalesam anyscalesam added bug Something that is supposed to be working; but isn't triage Needs triage (eg: priority, bug/not-bug, and owning component) core Issues that should be addressed in Ray Core observability Issues related to the Ray Dashboard, Logging, Metrics, Tracing, and/or Profiling P2 Important issue, but not time-critical and removed triage Needs triage (eg: priority, bug/not-bug, and owning component) labels Aug 20, 2024
@MengjinYan
Copy link
Collaborator

It seems that the environment variable should be added before the import ray in the python code.

And I believe it is just clarified in the PR here: #47913.

@jjyao jjyao added P3 Issue moderate in impact or severity and removed P2 Important issue, but not time-critical labels Oct 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something that is supposed to be working; but isn't core Issues that should be addressed in Ray Core observability Issues related to the Ray Dashboard, Logging, Metrics, Tracing, and/or Profiling P3 Issue moderate in impact or severity
Projects
None yet
Development

No branches or pull requests

3 participants