⚡️ Enhancements, Refactoring, and Maintenance Updates for StreamGear API #401
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
💬 When the output codec is set to "copy" (stream copy mode), certain video processing parameters like "-vf" (video filters) and "-aspect" (aspect ratio) are not supported and can lead to errors and invalid output files. ♻️ This commit refactors the internal `PreProcess` method in StreamGear API to handle the stream copy mode correctly: - 🥅 Moved the existing code for setting "-vf" and "-aspect" inside conditional block that checks if the output stream codec is not "copy". - 🔊 Added an else block to log warnings and discard "-vf" and "-aspect" in stream copy mode.
- 🧑💻 Update the StreamGear usage examples for streaming live audio from an external device - Fixed typo in source, code comments and filename in usage example code. - Updated usage example for device video source. - ✏️ Fixed minor typo in `js_hook.py.` StreamGear: - 💡 Fix minor typos and formatting issues in code comments for better clarity Helper: - 🩹Update `extract_time` helper function regex to handle milliseconds.
♻️ The `terminate()` method in StreamGear is now deprecated and will be removed in a future release. Developers should use the new `close()` method instead, which provides a more descriptive name like in WriteGear API for terminating StreamGear processes safely. - 📌 Pinned `typing_extensions` dependency to `>=4.7.1` for using the `@deprecated` decorator. See issue fastapi/fastapi#9808 - 🗑️ Deprecate the `terminate()` method in StreamGear and added backward compatibility. - ⚡️ Introduce a new `close()` method to safely terminate StreamGear processes - 📝 Minor formatting and docstring updates
…#399) - 💬 pyzmq version `24.0.1` has a bug with Cython, and it breaks the installation process. See issue [cython/cython#5238](cython/cython#5238).
💬 The updates aim to provide a better understanding of the StreamGear API's functionality, modes of operation, and usage scenarios. The documentation now includes more detailed explanations, practical examples, and best practices for working with StreamGear in various situations. - 📝 Improve the overview section's description and wording. - 🧑💻 Update usage examples for both Single-Source Mode and Real-time Frames Mode. - ♻️ Refactored sections for Live Streaming and RGB Mode usage. - ♿️ Clarify warnings, alerts, and important information. - 🎨 Fix markdown formatting and code highlighting issues. - 🗑️ Addressed deprecation of the `terminate()` method in favor of new `close()` method. - 🚸 Enhance overall clarity and readability of the documentation. WriteGear: - ⚡️ Simplified the logic for formatting output parameters. CI: - 👷 Updated Streamgear tests to use new `close()` method instead of deprecated `terminate()`.
- ⚡️ Implemented a custom `deprecated` decorator function - This decorator can be used to mark functions or parameters as deprecated - It displays a warning message when a deprecated function or parameter is used StreamGear: - ⚡️ Updated the `stream` method to use the new custom `deprecated` decorator - 🗑️ Marked the `rgb_mode` parameter as deprecated with a relevant warning message - This parameter will be removed in a future version, and only BGR format frames will be supported - 🚩 Added a new `-enable_force_termination` attribute, similar to WriteGear API. - When set to True, this parameter will force the termination of the FFmpeg process - This option can be useful in cases where the FFmpeg process needs to be terminated immediately - 🔊 Enhanced logging messages for better clarity and readability - 🧑💻 Improved parameter validation and added more descriptive warning/error messages - 🎨 Refactored some conditions and error handling for better code maintainability - 💡 Updated docstrings and comments to better reflect the current functionality - 📝 Improved code documentation for better understanding and easier maintenance - 🩹 Fixed `libx264rgb` encoder not compatible with `-profile:v` FFmpeg parameter. Setup - ⏪️ Removed the `typing_extensions` package as core dependency - This package was previously required for the `deprecated` decorator. - With the introduction of the custom `deprecated` decorator in helper.py, this dependency is no longer needed.
- ♻️ Refactored the logic for terminating the FFmpeg process in the `close` method - Instead of directly terminating the process, the code now sends a CTRL_BREAK_EVENT signal - This allows for a more graceful termination and avoids potential issues with abrupt termination - If the forced termination option is enabled, the process is directly terminated - ✏️ Fixed a typo in an error message related to stream initiation - 🎨 Refactored and simplified some logging statements using Python's ternary operator 🍻 WriteGear: Mirrored logic for terminating the FFmpeg process in the `close` method
- 🔥 Removed the obsolete usage example for deprecation RGB mode with StreamGear. - 🗑️ Added a deprecation warning admonition for the `rgb_mode` parameter in the `stream()` method. - 💡 Updated the docstring for the `stream()` method and `transcode_source()` method. - 🚩 Added documentation and usage of the new `-enable_force_termination` parameter. - 📝 Updated the documentation for the `-disable_force_termination` parameter in WriteGear API. - 💬 Added a new FAQ entry about deprecated `rgb_mode` parameter. - 🎨 Minor formatting and wording improvements in the documentation.
- 🏗️ Removed the `-aspect` parameter from the default FFmpeg pipeline - Previously, StreamGear would enforce a simplified aspect ratio using this parameter, which forces FFmpeg to use non-square pixels, leading to unwanted distortion on the output. - 🎨 Updated warning messages for better clarity.
- 🔊 Updated the StreamGear docs file to clarify the purpose and potential side effects of the `-enable_force_termination` parameter. StreamGear API: - 🧑💻 Modified the warning message to mention that forced termination can cause corrupted output in certain scenarios. - 🔊 Changed the log message in StreamGear to print a warning instead of an info message when forced termination is enabled.
…Mode only 💬 Live streaming is intended for low-latency streaming of real-time frames, where chunks contain only the most recent frames. It doesn't make sense when streaming from a video file, as the entire file can be streamed normally without the need for live streaming. - 🏗️ Disabled live streaming if `-video_source` is provided (Single-Source Mode) - 🔊 Log an error message when live streaming is attempted in Single-Source Mode, otherwise log normally in Real-time Frames Mode. Docs: - 📝 Refine description of `-streams` attribute of StreamGear API for better clarity in params. - Clarify primary stream generation and user-defined secondary streams. - Improve formatting and language for better readability. - ♻️ Replace usage of "tip" admonition with "example" for usage examples. - 🚸 Add warning for unsupported `-livestream` parameter in Single-Source Mode. - 📝 Updated respective notices for deprecate `terminate()` method and `rgb_mode` parameter. - 🔥 Remove unsupported live-streaming usage examples in Single-Source Mode.
…hance clarity and readability - 🧑💻 Simplified and clarified descriptions for `output`, `format`, `custom_ffmpeg`, and `stream_params` parameters. - 📝 Improved examples for defining output paths, filenames, and URLs. - 🚸 Enhanced warnings, tips, and information admonitions for better user guidance. - 🎨 Reformatted code examples to provide clearer usage patterns. - ✏️ Updated formatting and grammar for consistency and precision.
- 🔥 Removed unused imports.
abhiTronix
added
ENHANCEMENT ⚡
This was
linked to
issues
May 31, 2024
💬 The `close()` methods in StreamGear and WriteGear were sending the `CTRL_BREAK_EVENT` signal to terminate the underlying process, which only works on Windows systems. This commit modifies the code to send the appropriate signal based on the operating system: - For Windows, it sends the `CTRL_BREAK_EVENT` signal - For non-Windows systems, it sends the `SIGINT` signal
…ate_dash_stream` method. - 🔊 Updated logging message format in `evaluate_streams` method of for consistency. WriteGear: - 🐛 Fixed dictionary comprehension logic to strip only string values. CI: - 💚 Fixed expected duration value in parameterized test case from `8` to `8.44` since `test_extract_time` function now supports floating point values.
… Mode 🚑️ This commit addresses an bug where the `-vcodec copy` parameter was incorrectly defined when using the Real-time Frames Mode in StreamGear. Stream copy is not compatible with this mode since it requires encoding the frames before streaming. Additionally, If the Real-time Frames Mode is active and `-vcodec copy` is specified, a warning log message is printed to notify the user that the stream copy parameter is being discarded.
- 🐛 Updated logic to use `zmq.Context()` instead of `zmq.Context.instance()` to isolate contexts in order to fix `zmq.error.ZMQError: Address in use (addr='inproc://zeromq.zap.01')` bug. - ♻️ Refactored code. CI: - 💚 Fixed typos in NetGear Tests method names.
- ⏪️ Reverted Isolated contexts for Secure Modes
…4.0.1` on Windows.
- 🧑💻 Added warnings about Secure Mode issues with PyZMQ versions > 24.0.1 on Windows in NetGear API:
- 💬 Secure Mode functionality is broken on Windows with PyZMQ versions > 24.0.1 due to changes in `zmq.auth` module.
- 💬 Users should downgrade to PyZMQ version <= 24.0.1 to use Secure Mode on Windows.
- 🧑💻 Added warnings about Stream copy (`-vcodec copy`) is not compatible with Real-time Frames Mode as this mode requires re-encoding of incoming frames in StreamGear API.
- 💡 Refined warning message for stream copy compatibility for clarity.
CI:
- 👷 Added a skip condition in `test_netgear.py` for Windows platform when PyZMQ version > 24.0.1:
- Ensured relevant tests are skipped on incompatible PyZMQ versions to prevent test failures.
NetGear:
- ⏪️ Reverted Handle graceful termination of ZMQ Context.
- 👷 CI: Applied short-circuiting to simplify code in NetGear Tests.
…dio streams only. WriteGear: - ⏪️ Reverted terminating the FFmpeg process in the `close` method, handled by `terminate()`
⚡️ StreamGear: Discarded invalid audio stream.
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## testing #401 +/- ##
===========================================
+ Coverage 93.09% 93.85% +0.76%
===========================================
Files 15 15
Lines 3026 3092 +66
===========================================
+ Hits 2817 2902 +85
+ Misses 209 190 -19 ☔ View full report in Codecov by Sentry. |
…xts. - 🚑️ Fix an issue where the secure mode failed to work due to conflicting ZMQ contexts, when Server and Client run on same thread. - Moved ZMQ Authenticator and Certificates handling code together, independent of mode activated in API. - Handle Authenticator and Certification errors more gracefully, by handling the "Address in use" error, and disable secure mode if errors occur. - Logged Authenticator start/stop events. - 🥅 Handle socket session expiration more gracefully in the `recv_handler` internal method. - Fixed `msg_json` undefined when terminating context in the `recv_handler` internal method forcefully. - 🩹 Ensure proper termination of the ZMQ context and socket when closing the NetGear instance. - ⚡️ Set the `WindowsSelectorEventLoopPolicy` for Python `3.8` and above on Windows to ensure compatibility with ZMQ event loop. - ♻️ Simplify and refactor conditional statements and variable assignments with short-circuiting and formatting. - 🔊 Improve logging for various events, such as Authenticator termination, thread termination, and secure mode activation. - 🚩 Added new imports. Docs: - 🔥 Removed warning for Secure Mode issues with PyZMQ `versions > 24.0.1` on Windows. - 📝 Added Admonition for warning users about the Client's end must run before the Server's end to establish a secure connection in Secure Mode.
- 👷 Added `"127.0.0.1"` address to allow common endpoint for connection. - 👷 Added `"jpeg_compression":False` to disable frame compression, allowing frame to be same while assertion. - ⏪️ Reverted skip condition for Windows platform when PyZMQ version `> 24.0.1`. - ☂️ Improved code coverage.
📝 Docs: Fixed typos in admonitions.
abhiTronix
added
SOLVED 🏁
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Brief Description
This PR includes significant refactoring, documentation updates, and significant enhancements to the StreamGear, along with minor improvements to WriteGear, and NetGear APIs. It introduces safer process termination methods, improved parameter handling, and enhanced documentation for better user guidance. Additionally, it addresses several bugs and improves the overall maintainability and clarity of the codebase.
Key Changes
StreamGear API:
-vcodec copywhen using this mode.terminate()method, introducingclose()for safer process termination.-enable_force_terminationattribute for immediate FFmpeg process termination.-livestreamparameter to Real-time Frames Mode only.close()methods for handling gracefully signal interruptions on different systems with device audio streams.WriteGear API:
NetGear API:
recv_handler.msg_jsonundefined when terminating context in therecv_handlermethod.Helper Class:
extract_timehelper function regex to handle milliseconds.Setup
pyzmqversion to address installation issues (Fixes [Bug]: update pyzmq version #399).Documentation:
-livestreamparameter in Single-Source Mode.terminate()method in favor of the newclose()method.terminate()method andrgb_modeparameter.-enable_force_terminationparameter.rgb_modeparameter.rgb_modeparameter in thestream()method.-enable_force_terminationparameter.stream()method andtranscode_source()method.-streamsattribute of the StreamGear API.-disable_force_terminationparameter in WriteGear API.js_hook.py.CI
close()method instead of the deprecated terminate() method.kill=Trueinclose()in NetGear Tests.8to8.44sincetest_extract_timefunction now supports floating point values.test_secure_modeNetGear test:"127.0.0.1"address to allow common endpoint for connection."jpeg_compression":Falseto disable frame compression, allowing frame to be the same while assertion.Maintenance:
Requirements / Checklist
Related Issue
#385
#396
#399
Context
This Pull Request aims to introduce various improvements, refactoring efforts, and maintenance updates to the StreamGear API. The changes encompass bug fixes, documentation updates, feature enhancements, and code quality improvements, along with minor improvements to WriteGear, and NetGear APIs. It introduces safer process termination methods, improved parameter handling, and enhanced documentation for better user guidance. Additionally, it addresses several bugs and improves the overall maintainability and clarity of the codebase.
Types of changes