Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resolve buffering issues in TeeStream and capture_output #3449

Merged
merged 26 commits into from
Feb 4, 2025

Conversation

jsiirola
Copy link
Member

Fixes #3084, fixes #3031, fixes #2845.

Summary/Motivation:

The HiGHS interface exposed some issues in the logic in TeeStream and capture_output where the two could end up fighting with each other. This PR should resolve all issues with console log management that we have been seeing with HiGHS.

Changes proposed in this PR:

  • Ensure that flushing a TeeStream handle is propagated through tho the target output streams
  • Allow passing a TeeStream to capture_output to prevent ending up with nested TeeStream instances
  • TeeStream should dup all target file descriptors so that subsequent context managers (like capture_output) don't commandeer them (causing a weird circular reference loop)
  • Update how capture_output handles redirection (do not let redirect_fd update the sys.stdout / sys.stderr streams)
  • Add tests for buffered and unbuffered output through TeeStream (and capture_output)
  • Update all solver interfaces to standardize how we use capture_output / TeeStream for managing the console output.

Legal Acknowledgement

By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution:

  1. I agree my contributions are submitted under the BSD license.
  2. I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

@FBumann
Copy link

FBumann commented Jan 14, 2025

Would love to see this resolved.
Kepp the good work up ;)

@mrmundt
Copy link
Contributor

mrmundt commented Jan 16, 2025

Would love to see this resolved. Kepp the good work up ;)

Thanks! We have been having lots of issues with our testing infrastructure that we need to fix before we can get any actually functional PRs merged. Stay tuned!

@@ -50,6 +50,32 @@
logger = logging.getLogger(__name__)


class _SignalFlush(object):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(object) - grrrrr

Comment on lines +115 to +116
# NOTE: do not flush: we will test flush in the next test
# t.STDOUT.flush()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leftover?

@mrmundt mrmundt merged commit 8a50553 into Pyomo:main Feb 4, 2025
34 checks passed
@jsiirola jsiirola deleted the tee-stream-fixes branch February 5, 2025 14:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants