Memory Leak In VideoFileClip

[mdallen22]

VideoFileClip defines a __del__ method and has a reference cycle https://github.com/Zulko/moviepy/blob/master/moviepy/video/io/VideoFileClip.py#L74. This causes the gc to ignore any VideoFileClip instances, https://docs.python.org/2/library/gc.html#gc.garbage. An immediate solution is to del video_file_clip.make_frame for all VideoFileClip instances. But it should not be my responsibility to clean up VideoFileClip.

[Zulko]

Can you provide an example of where it doesn't do what you want ? That would clarify where we are going and help me see the problems. Also can you remind me what to use to check for memory allocations in Python ?

[mdallen22]

Copy this script in a folder with an image named "test.gif".

from moviepy.editor import VideoFileClip
from guppy import hpy
import sys

def main():
    clip = VideoFileClip('test.gif')
    if len(sys.argv) > 1 and sys.argv[1] == 'del':
        del clip.make_frame

h = hpy()
h.setrelheap()
main()
print h.heap()

Run the script without a del argument

 Index  Count   %     Size   % Cumulative  % Kind (class / dict of class)
     0      4  12     1328  15      1328  15 _sre.SRE_Pattern
     1      1   3     1048  12      2376  27 dict (no owner)
     2      1   3     1048  12      3424  39 dict of codecs.CodecInfo
     3      1   3     1048  12      4472  51 dict of moviepy.video.io.VideoFileClip.VideoFileClip
     4      1   3     1048  12      5520  63 dict of
                                             moviepy.video.io.ffmpeg_reader.FFMPEG_VideoReader
     5      1   3     1048  12      6568  75 dict of subprocess.Popen
     6      1   3      448   5      7016  80 types.FrameType
     7      5  15      384   4      7400  85 tuple
     8      2   6      288   3      7688  88 file
     9      2   6      240   3      7928  91 function

Run the script with a del argument

 Index  Count   %     Size   % Cumulative  % Kind (class / dict of class)
     0      4  24     1328  28      1328  28 _sre.SRE_Pattern
     1      1   6     1048  22      2376  50 dict of codecs.CodecInfo
     2      1   6     1048  22      3424  72 dict of subprocess.Popen
     3      1   6      448   9      3872  81 types.FrameType
     4      4  24      320   7      4192  88 tuple
     5      2  12      288   6      4480  94 file
     6      1   6       96   2      4576  96 codecs.CodecInfo
     7      1   6       88   2      4664  98 __builtin__.weakref
     8      1   6       64   1      4728  99 subprocess.Popen
     9      1   6       24   1      4752 100 int

The second run is what I want to see, moviepy.video.io.VideoFileClip.VideoFileClip should no longer be in the heap after the function call.

[Zulko]

Thanks for the explanations, I think I fixed it. Can you try the new version (PyPI or github) and tell me it it works ?

[Red5d]

Still got a memory leak or something for me. Each time I write out a video file, more ffmpeg processes that don't go away are created and more memory is used. (v0.2.1.9.07 from pip on Linux)

[Zulko]

With exactly the same code as before ?

[Red5d]

Yep. Same code.

[Red5d]

I think this is resolved now. I'm not experiencing this problem anymore.

[keikoro]

Sounds like this problem disappeared a while ago. Closing.