Make opencv optional

[Poikilos]

cv2 (formerly cv) could be removed:

I'll definitely check this out within the next few days. I'm currently working on something big: removing cv2 as a dependency.

-#38 (comment)

but maybe make it optional, since it is handy for advanced operations on the video.

Such as:

In the module:

ENABLE_CV2 = True
try:
    import cv2
except ImportError:
    ENABLE_CV2 = False
. . .
if ENABLE_CV2:
    # ... do stuff
if ENABLE_CV2:
    # ... more stuff in some other part of the code

...then split off a different requirements.txt called requirements-opencv.txt and just add opencv_python to that.

And pip install pyvidplayer2[opencv] can be defined if you do:

• setup.py:

    extras_require = {
        'opencv':  ["opencv_python"]
    }

• pyproject.toml:

[project.optional-dependencies]
opencv = ["opencv_python"]

[Poikilos]

However, I didn't get a good look at the part of the code that actually requires opencv, so if there is some way for me to use opencv on the pyvidplayer2 video efficiently without the pyvidplayer2 module using opencv, that's fine too (Maybe add an example?).

[anrayliu]

Cv2 will indeed be an optional package. It seems like ffmpeg can do almost everything that cv2 can, except ffmpeg can also accept videos in memory. My biggest motivation for removing cv2 is because that would reduce the dependencies down to just pygame and ffmpeg, which would be really simple and lightweight. Also, what kind of video operations were you thinking of doing with cv2?

[Poikilos]

Well I want to use pyvidplayer2 for some simple video editing (I know all the supposed drawbacks of not using a video editing framework like MLT. However, I am planning a unique rotoscoping program, probably not an NLE. A rationale is at the readme here: https://github.com/Hierosoft/rotocanvas). I'm likely to want to want to have:

• Scene detection: https://github.com/Breakthrough/PySceneDetect
• Object tracking: https://www.geeksforgeeks.org/getting-started-with-object-tracking-using-opencv/

[anrayliu]

What kind of interaction would you need with the video and cv2 for this? Reading video frames?

[Poikilos]

The examples I gave are python_opencv examples, but only used opencv briefly. When I save the video, I will merely use ffmpeg. pyvidplayer2 won't be involved in that step I don't think.

Therefore, I don't know enough yet. I don't know if I will need to use opencv with pyvidplayer2. I will likely just use opencv to analyze the source video apart from the GUI, save the metadata, then overlay my own widgets based on the metadata (such as boxes the user can move if the object tracking is wrong, but also to attach a sprite, blur effect, etc. to the tracked target).

The reason pyvidplayer2 is useful regardless of that is:

• pyvidplayer2 could preview ffmpeg filters (and maybe opencv filters) as part of the realtime preview. I don't even know how much control I will have over ffmpeg without subclassing pyvidplayer2 heavily. If I find a neat way to do it and put most of the code in my program, I can submit a small PR. I haven't looked at the ffmpeg part of pyvidplayer2's code much yet either, so I don't know what that would take.
• pyvidplayer2 can use the same timecodes as ffmpeg uses so I assume I can do frame-accurate editing (Though I understand I may have to re-encode portions if using mp4 and similar formats due to keyframe issues etc). In other words, if I can go to a timecode in pyvidplayer2, I can use that same timecode in ffmpeg commands. Therefore I can add markers or cut points that I can store in a project and later pass along to ffmpeg. Unedited clips will remain lossless (-c:v copy -c:a copy). Then I will splice the edited clips with the unedited clips ("smart rendering").
  • I currently don't know of any open source video editing tools with both effects and "smart rendering" :edit: See avidemux comment below

My old program rotocanvas-cs very slow as it was using the ffmpeg command & saving each preview to a file (I wasn't good at using stdout back then, but should have probably used a library anyway): https://github.com/Poikilos/rotocanvas-cs. It used the code from another repo for the timecode construction function (can be used in -ss option in ffmpeg command for cutting): https://github.com/Poikilos/RetroEngine-cs/blob/f43c73a24b01ea37fbb5c1fa1d86c03031ef01bf/RConvert.cs#L257 . This code goes to the "middle" of the frame. As in, if it is 60 fps, to go to the second frame I have to convert 1.5/60 seconds to a timecode that works reliably for -ss (1.0/60 gets the previous frame all or most of the time). Maybe I made some other mistake, but that is the only way I could get it working.

[anrayliu]

Interesting. Have you browsed the documentation for things that may help you? For example I just added (hasn't been pushed to pypi yet but is available in the repo) a seek_frame function, which should quite accurately seek to a frame. In addition, frame_data and frame_surf both contain the video's current rendered frame as a numpy array and pygame surface, respectively. Also, check out the built-in post processing (which is what I was planning to have cv2 be an optional package for). However, yes, there isn't an audio post processing feature yet. How pyvidplayer2 uses ffmpeg is inaccessible to users.

[Poikilos]

For accuracy I should say I found out today that AVIDemux supports "smart rendering" (a combination of filtered clips and lossless clips): https://www.avidemux.org/admWiki/doku.php?id=using:video_filters :edit: It appears AVIDemux doesn't have smart rendering either. Only all or none of the video can have filters, and the "copy" codec has to be turned off when using filters, according to the documentation.