Problems with complex mask

[RealJTG]

I'm trying to cut out some areas with mask, shift them in time, mix together and finally put on background, but there are blend artifacts — accuracy loss or something.

from moviepy.editor import *

def cycle(clip, ismask=False, transparent=False):
    d = clip.duration
    result = CompositeVideoClip([
                clip, 
                clip.set_start(d/2),
                clip.set_start(d)
             ], 
             ismask=ismask, transparent=transparent) \
             .subclip(d/2, 3*d/2)
    return result

gif = VideoFileClip("./source.gif", audio=False)
gif_mask = VideoFileClip("./source_mask.gif", audio=False)
bg = gif.copy()

mask_mf = lambda t: gif_mask.reader.get_frame(t)[:,:,0]/255.0
mask = VideoClip(ismask=True, make_frame=mask_mf).set_duration(gif.duration)
cat = gif.set_mask(mask)

composition = CompositeVideoClip([bg, cycle(cat, transparent=True)])
composition.preview()
#composition.write_gif('result.gif', fuzz=10, fps=24)

There are no artifacts with "binary" mask:
mask_mf = lambda t: gif_mask.reader.get_frame(t)[:,:,0]/255

[Zulko]

Thanks for the interesting issue. This is indeed puzzling, for the moment I can't understand if it's a bug (= MoviePy really not working as expected), a problem in your code, or maybe a problem with the gifs or how they are read. Have you used masks for other purposes yet ? Have you had any issue ? I have made complex things with masks before and never had any issue. I'll have a better look into it when I have time.

PS: is this for reddit/perfectloops or something like this ;) ?

[Zulko]

I think I can confirm that your problem is a due to the combined facts that (1) the GIFs are somehow corrupted (some frames cannot be read by MoviePy) and (2) you are making copies of clips which all refer to the same video reader.

I'll try to explain: a videofileclip in moviepy has a "reader" which is like a pipe to the data on the hard drive. When you make a copy of the clip (for instance with clip2 = clip.set_start(t0) ) the reader is not copied (because it takes lots of place in the memory), the reader will be shared by clip and its derivative clip2. Now, if clip asks the reader for a frame at some time t and the reader cannot read it (because the file is somehow corrupted like in your example), the reader will instead give the last frame read. Normally this is the frame just before t and the reading error is barely noticeable (the video staggers a little as it has two identical frames in a row). But if two clips are using the same reader, the last frame read can also be a frame anywhere else in the video.

This is what happens in your example: the reader can't read a frame and instead it gives the last frame read, which has actually been read by another clip, at a completely different time. So the clip and the masks are out of sync and strange stuff happens.

[Zulko]

I don't want to spoil the fun, but here is a version that works :D. It only needs the source, not the mask gif. See the result here.

https://gist.github.com/Zulko/9e65c119b9c9ad027608

[RealJTG]

Thanks, I'll dig around a bit