Transparency with CompositeVideoClip issue

[Sherif-GLH]

i used this function for zooming

def Zoom(clip,mode='in',position='center',speed=1):
    fps = clip.fps
    duration = clip.duration
    total_frames = int(duration*fps)
    def main(get_frame,t):
        frame = get_frame(t)
        h,w = frame.shape[:2]
        i = t*fps
        if mode == 'out':
            i = total_frames-i
        zoom = 1+(i*((0.1*speed)/total_frames))
        positions = {'center':[(w-(w*zoom))/2,(h-(h*zoom))/2],
                     'left':[0,(h-(h*zoom))/2],
                     'right':[(w-(w*zoom)),(h-(h*zoom))/2],
                     'top':[(w-(w*zoom))/2,0],
                     'topleft':[0,0],
                     'topright':[(w-(w*zoom)),0],
                     'bottom':[(w-(w*zoom))/2,(h-(h*zoom))],
                     'bottomleft':[0,(h-(h*zoom))],
                     'bottomright':[(w-(w*zoom)),(h-(h*zoom))]}
        tx,ty = positions[position]
        M = np.array([[zoom,0,tx], [0,zoom,ty]])
        frame = cv2.warpAffine(frame,M,(w,h))
        return frame
    return clip.transform(main)

with this as the main code

background_video = VideoFileClip('BG Template.mp4')
image_clip = ImageClip("final_output1.png").with_fps(30).with_duration(pause_duration)
animated_image = Zoom(image_clip,mode='in',position='center',speed=2)
animated_image = animated_image.with_effects([ vfx.CrossFadeIn(0.2)])
animated_image.write_videofile("test.mov", codec="mpeg4", fps=30 )
new = VideoFileClip("test.mov", has_mask=True)
clips = [
    background_video.subclipped(0, 15), 
    new  
]
video = CompositeVideoClip(clips)
video.write_videofile("video1.mp4", codec="libx264",fps=30)

to get an image with smooth zooming as a foreground pic
with a background video
but the image is still being cropped even after i added a transparent layer to avoid cropping
here's how i added the transparent layer

def add_transparent_layer(image_path, output_path, canvas_width=1920, canvas_height=1080):
    image = Image.open(image_path).convert("RGBA")
    canvas = Image.new("RGBA", (canvas_width, canvas_height), (0, 0, 0, 0))
    x_offset = (canvas_width - image.width) // 2
    y_offset = (canvas_height - image.height) // 2
    canvas.paste(image, (x_offset, y_offset), image)
    canvas.save(output_path, format="PNG")

the "test.mov" video is worked fine without any issues but after combining it with the background video using the composite i causes the cropping issue

[Implosiv3]

Hi, can you upload the result to be easier to understand the error, or just a link to the video?

I think maybe you are writing the animated_image to manually check the alpha layer, but you can use it directly without writing it using .with_mask() and using it as you use the new.

Also, if this helps you, a frame is just a numpy array. If it has no mask, the frame has 3 values from 0 to 255 (RGB). If there is a mask, the frame has 4 values, the same 3 first are from 0 to 255 (RGB = color) and the last one from 0 to 1 (A = alpha layer). You could modify it directly with numpy instead of PIL if you know how to.

[OsaAjani]

Ideally a video of the expected result and the actual result would be a big help.

[Sherif-GLH]

here's a detailed explanaition :

• I have a picture i need to resized over time with a background video
  the expected output need to be like this :
  https://drive.google.com/file/d/1Ag9Qunad0CMKhYpEyCJjsehG32-COAGA/view?usp=sharing
  Firstly, I make have added a transparent layer to an image with (1920x1080) size with this code :

def add_transparent_layer(image_path, output_path, canvas_width=1920, canvas_height=1080):
    image = Image.open(image_path).convert("RGBA")
    canvas = Image.new("RGBA", (canvas_width, canvas_height), (0, 0, 0, 0))
    x_offset = (canvas_width - image.width) // 2
    y_offset = (canvas_height - image.height) // 2
    canvas.paste(image, (x_offset, y_offset), image)
    canvas.save(output_path, format="PNG")

the transparent layer was added successfuly and here is the output:

then i have applied the Zoom Function :

def Zoom(clip,mode='in',position='center',speed=1):
    fps = clip.fps
    duration = clip.duration
    total_frames = int(duration*fps)
    def main(get_frame,t):
        frame = get_frame(t)
        h,w = frame.shape[:2]
        i = t*fps
        if mode == 'out':
            i = total_frames-i
        zoom = 1+(i*((0.1*speed)/total_frames))
        positions = {'center':[(w-(w*zoom))/2,(h-(h*zoom))/2],
                     'left':[0,(h-(h*zoom))/2],
                     'right':[(w-(w*zoom)),(h-(h*zoom))/2],
                     'top':[(w-(w*zoom))/2,0],
                     'topleft':[0,0],
                     'topright':[(w-(w*zoom)),0],
                     'bottom':[(w-(w*zoom))/2,(h-(h*zoom))],
                     'bottomleft':[0,(h-(h*zoom))],
                     'bottomright':[(w-(w*zoom)),(h-(h*zoom))]}
        tx,ty = positions[position]
        M = np.array([[zoom,0,tx], [0,zoom,ty]])
        frame = cv2.warpAffine(frame,M,(w,h))
        return frame
    return clip.transform(main)

• the zoom function is definetly crop the image from its edges, so we need to add the transparent layer to keep the image details and let the transparent area to be cropped
  for testing this part of code i write the video :

image_clip = ImageClip("final_output1.png").with_fps(30).with_duration(10)
animated_image = Zoom(image_clip,mode='in',position='center',speed=2)
animated_image.write_videofile("test.mov", codec="prores_ks", preset="4444", fps=30 )

and the test was successfuly done, here's the link :
https://drive.google.com/file/d/1JWbmwgMNPRjQ-HvsizBC0jVWAqxE22g4/view?usp=sharing

• acutally the problem starts from here

new = VideoFileClip("test.mov", has_mask=True)
clips = [
    background_video.subclipped(0, 15),
    new
]
video = CompositeVideoClip(clips)
video.write_videofile("video1.mp4", codec="libx264",fps=30)

after writing the video i found that the image is beign cropped and avoiding the transparent layer
here's the link of my output:
https://drive.google.com/file/d/1DOisIjf3vHGCoxqxGY4u8kb-rHDPFurr/view?usp=sharing

[Implosiv3]

Ok, thank you for your links.

First I detect is one thing. Your zoom effect is actually applying a zoom on the image by manipulating the pixels but keeping the same frame size (as you do here cv2.warpAffine(frame,M,(w,h)), the (w, h) is forcing the same original size), thats why the image you see in your output has always the same size.

I've downloaded your second link and when playing locally I can see that the image size is still the same (I rounded 2 corners) even when the image is aparently zooming. I don't know why the image is apparently zooming, I haven't work too much with ImageClips individually nor with the specific codec and preset you use for rendering. Maybe one expert can help you (us) with it.

So, when you put the image in front of the background, as the image is always having the same size because you are forcing the frame to it (as I told you in the 2nd paragraph), the content is zoomed (as your zoom code does) but the size is the same. You are not zooming a clip.

The result you want for your video is resizing the ImageClip, or resizing each frame of the clip (but not keeping the image). You can put the image as it is, make it an ImageClip, put in the center of the scene (in the foreground as you do) and resize the clip progressively.

You know the video duration (video.duration) and also the duration of each frame (1 / fps). You can try to resize it by 1.5x maybe. So the ImageClip resize factor must go from 1.0 to 1.5 to be done frame by frame, but you need to keep the ('center', 'center') position.

def resize_progressively(t, duration):
    return 1 + 0.5 * t / duration

video.resized(lambda t: resize_progressively(t, video.duration)
"""
This, that is apparently complicated, is telling the video its size
(as a clip) for each time it is rendering one of its frames. The 't'
is the time moment for the current frame. Imagine a video of
duration=1s and fps=30. For frame 0, t=0, for frame 6, t=6/30
=> t=0.2. For frame 29, t=29/30 => t=0.966...7.
1 + 0.5 * 0 / 1 = 1, but 1 + 0.5 * 0.966...7 ~ 1.5. Also, as you see,
the progression is not perfect, you have to adjust it, this is just
a quick example.
"""

I hope this clearifies something. Playing with moviepy is very funny and powerfull :) (you can see this if you don't trust me #2119 (comment))

[Sherif-GLH]

thanks for your help , it was really appreciated
I found the problem that i'm facing,

def add_transparent_layer(image_path, output_path, canvas_width=1920, canvas_height=1080):
    image = Image.open(image_path).convert("RGBA")
    canvas = Image.new("RGBA", (canvas_width, canvas_height), (0, 0, 0, 0))
    x_offset = (canvas_width - image.width) // 2
    y_offset = (canvas_height - image.height) // 2
    canvas.paste(image, (x_offset, y_offset), image)
    canvas.save(output_path, format="PNG")

in this function where i add the transparent layer
specifically in this line :
canvas.paste(image, (x_offset, y_offset), image)

it applies an alpha channel mask that is static to the initial image size

i tried to change the mask but i'm very confused about how many similiar variables are there
with_mask , is_mask , has_mask ,
i need to apply a mask for each frame in the "test.mov" video that matches the size of the image at its current state

if you could help me will be very appreciated

[Implosiv3]

Ok, let me introduce you to the incredible world of video masks, haha. A VideoClip can be normal or mask. The normal one contains frames that are 3 RGB values between [0, 255], and the mask one contains frames that are 1 Alpha value between [0, 1]. One thing is the video frame concept and another one is the image frame concept. And, as you can see, moviepy allow having VideoClips with masks. But, how is this possible if a VideoClip can only be normal or mask type? Simple. A normal VideoClip can have attached another mask VideoClip which describes its alpha layer. Also, if no mask is attached, it is like a full-opaque mask is attached (when processing, non-mask is replaced by full-opaque mask).

So, is_mask is the boolean that says if a VideoClip is mask or not, has_mask is a boolean that tells you if there is a mask VideoClip attached or not, and with_mask is a method that actually attaches a mask to the VideoClip instance that method belongs. If you don't provide an argument to this with_mask method, a default full-opaque mask is attached, but if you provide another mask VideoClip, that one will be attached.

So, you usually attach a mask to a VideoClip when you need some kind of transparency (because if not, you just let the default full-opaque-on-processing one work at the end).

Now that you know the idea, you need to build your own mask VideoClip frame by frame to put on it the mask you want to be applied on the normal VideoClip you'll attach it. And now you know that a normal frame is a numpy of 3 RGB values between [0, 255], so a mask frame is a numpy array of 1 Alpha value between [0, 1], so you know how to build the numpy mask parameter to pass to a mask VideoClip to be instantiated and, lately, attached to your normal VideoClip.

:)

[OsaAjani]

Hi guys, as this is more an in-depth dive into moviepy than an issue strictly speaking, I will move that to discussion where it will be easier to go back and forth !