OSError: [WinError 6] The handle is invalid... concatenating clips

[hackydunk]

Expected Behavior

I want to choose clips randomly from a directory and then concatenate them into a compilation. My code works fine a random number of times before inevitably failing with a WinError 6 seen below.

Actual Behavior

Traceback (most recent call last):
  File "C:/Users/Administrator/PycharmProjects/Snapchat/Compile.py", line 26, in <module>
    + random.choice(os.listdir('D:\Snapchat Project\Good Videos')))
  File "C:\Users\Administrator\PycharmProjects\Snapchat\venv\lib\site-packages\moviepy\video\io\VideoFileClip.py", line 121, in __init__
    nbytes=audio_nbytes)
  File "C:\Users\Administrator\PycharmProjects\Snapchat\venv\lib\site-packages\moviepy\audio\io\AudioFileClip.py", line 72, in __init__
    buffersize=buffersize)
  File "C:\Users\Administrator\PycharmProjects\Snapchat\venv\lib\site-packages\moviepy\audio\io\readers.py", line 50, in __init__
    infos = ffmpeg_parse_infos(filename)
  File "C:\Users\Administrator\PycharmProjects\Snapchat\venv\lib\site-packages\moviepy\video\io\ffmpeg_reader.py", line 256, in ffmpeg_parse_infos
    proc = sp.Popen(cmd, **popen_params)
  File "C:\Program Files (x86)\Python36-32\lib\subprocess.py", line 596, in __init__
    _cleanup()
  File "C:\Program Files (x86)\Python36-32\lib\subprocess.py", line 205, in _cleanup
    res = inst._internal_poll(_deadstate=sys.maxsize)
  File "C:\Program Files (x86)\Python36-32\lib\subprocess.py", line 1035, in _internal_poll
    if _WaitForSingleObject(self._handle, 0) == _WAIT_OBJECT_0:
OSError: [WinError 6] The handle is invalid

Steps to Reproduce the Problem

from moviepy.editor import *
import os
import random


directory = r'D:\Snapchat Project\Compilations\\'

for root, dirs, files in os.walk(directory):
    existing_videos = len(files)


number_of_videos = 50
duration = 0
intro = VideoFileClip('D:\Snapchat Project\intro.mp4')
clips = [intro]
final_duration = random.randint(300, 500)
number = 0

for video in range(existing_videos + 1, existing_videos + number_of_videos):

    while duration < final_duration:

        print('D:\Snapchat Project\Good Videos\\'
              + random.choice(os.listdir('D:\Snapchat Project\Good Videos')))
        clip = VideoFileClip('D:\Snapchat Project\Good Videos\\'
                             + random.choice(os.listdir('D:\Snapchat Project\Good Videos')))

        if (clip.duration < 11) and (clip.duration > 4):

            clip = clip.resize(width=360, height=640)
            clips.append(clip)
            duration = duration + clip.duration

        else:

            continue

    clips.append(intro)

    final_clip = concatenate_videoclips(clips)
    final_clip.write_videofile('D:\Snapchat Project\Compilations\Comp ' + str(video) + '.mp4', fps=25)

    duration = 0
    clips = [intro]

Specifications

• Python Version: 3.6
• Moviepy Version: 0.2.3.5
• Platform Name: Windows
• Platform Version: Server 2012

I'm using PyCharm for development

[tburrows13]

I'm guessing that what is happening is that you are trying to load the same clip twice.

Can you run this to test my hypothesis?

from moviepy.editor import *
import os
import random

file = random.choice(os.listdir('D:\Snapchat Project\Good Videos'))
vid1 = VideoFileClip('D:\Snapchat Project\Good Videos\\' + file)
vid2 = VideoFileClip('D:\Snapchat Project\Good Videos\\' + file)

I expect it will fail with the same error as you had before. After you've ran this report the results back here and I can help you avoid it.

Thanks.

[hackydunk]

I ran the test and there were no errors present.

Process finished with exit code 0

The folder I'm pulling data from has 65,000 unique clips in it. The chances of it pulling the same one are slim to none.

[tburrows13]

Oh ok. This is definitely a bug and it has come up loads with windows and moviepy. Unfortunately there isn't a definitive solution. Try calling clip.close() after adding it to clips.
Does it always complete at least one video?

[hackydunk]

Since I posted this I have added this function

def close_clip(vidya_clip):
    # noinspection PyBroadException
    try:
        vidya_clip.reader.close()
        del vidya_clip.reader
        if vidya_clip.audio is not None:
            vidya_clip.audio.reader.close_proc()
            del vidya_clip.audio
        del vidya_clip
    except Exception:
        # sys.exc_clear()
        pass

Then I call clip.close() on every clip at the end of my program.

I have a theory that running another moviepy program simultaneously is causing the issue.

[tburrows13]

Ok. In which case I'd suggest doing your movie stuff inside a process. This will delete all references when the function ends. Note that I haven't tested this code.

from moviepy.editor import *
import os
import random
from multiprocessing import Process


def create_video():
    duration = 0
    while duration < final_duration:
        clips = [intro]
        print('D:\Snapchat Project\Good Videos\\'
              + random.choice(os.listdir('D:\Snapchat Project\Good Videos')))
        clip = VideoFileClip('D:\Snapchat Project\Good Videos\\'
                             + random.choice(os.listdir('D:\Snapchat Project\Good Videos')))

        if (clip.duration < 11) and (clip.duration > 4):
            clip = clip.resize(width=360, height=640)
            clips.append(clip)
            duration = duration + clip.duration
        else:
            continue

    clips.append(intro)

    final_clip = concatenate_videoclips(clips)
    final_clip.write_videofile('D:\Snapchat Project\Compilations\Comp ' + str(video) + '.mp4', fps=25)
    duration = 0


directory = r'D:\Snapchat Project\Compilations\\'

for root, dirs, files in os.walk(directory):
    existing_videos = len(files)


number_of_videos = 50
intro = VideoFileClip('D:\Snapchat Project\intro.mp4')
final_duration = random.randint(300, 500)

for video in range(existing_videos + 1, existing_videos + number_of_videos):
    process = Process(target=create_video)
    process.start()
    process.join()

[hackydunk]

Final code below... I tried using the map.Pool from python's multiprocessing package, but I wasn't able to get it working successfully.

This doesn't eliminate the WinError 6, but when it happens the process ends and another one takes its place.

Thanks for your help! I'll mark this as closed.

from moviepy.editor import *
from multiprocessing import Process
from random import randint
import random
import uuid


def close_clip(vidya_clip):
    # noinspection PyBroadException
    try:
        vidya_clip.reader.close()
        del vidya_clip.reader
        if vidya_clip.audio is not None:
            vidya_clip.audio.reader.close_proc()
            del vidya_clip.audio
        del vidya_clip
    except Exception:
        # sys.exc_clear()
        pass


def get_video_clips():

    directory = r'D:\Snapchat Project\Good Videos'
    intro = VideoFileClip(r'D:\Snapchat Project\intro.mp4').set_start(0)
    video_clips = [intro]
    duration = intro.duration

    while duration < (randint(300, 500)):

        video_clip = VideoFileClip(directory + '\\' + random.choice(os.listdir(directory))).set_start(duration)

        if 4 < video_clip.duration < 11:

            video_clip = video_clip.resize(width=360, height=640)
            video_clips.append(video_clip)
            duration = duration + video_clip.duration

        else:
            continue

    outro = VideoFileClip(r'D:\Snapchat Project\intro.mp4').set_start(duration)
    video_clips.append(outro)
    duration = duration + outro.duration

    return video_clips,  duration


def get_text_clips(duration):

    count = 1
    text_clips = []

    for txt in ["text1",
                "text2",
                "text3",
                "text4",
                "text5",
                "text6",
                "text7",
                "text8"]:

        text_clip = TextClip(txt, color='white', fontsize=20)\
            .set_duration(10)\
            .set_pos('bottom')\
            .crossfadein(2)\
            .crossfadeout(2)\
            .set_start(duration/9*count)

        text_clips.append(text_clip)
        count = count+1

    return text_clips


def create_video():

    unique_filename = str(uuid.uuid4())

    clips, clips_duration = get_video_clips()
    clips.extend(get_text_clips(clips_duration))

    video = CompositeVideoClip(clips, size=(360, 640))
    video.write_videofile('D:\Snapchat Project\Compilations\\' + unique_filename + '.mp4', fps=30)

    for clip in clips:
        close_clip(clip)


def main():

    p1 = Process(target=create_video)
    p2 = Process(target=create_video)
    p3 = Process(target=create_video)
    p4 = Process(target=create_video)
    p5 = Process(target=create_video)

    p1.start()
    p2.start()
    p3.start()
    p4.start()
    p5.start()

    p1.join()
    p2.join()
    p3.join()
    p4.join()
    p5.join()


if __name__ == "__main__":

    number_of_videos = 50

    for i in range(number_of_videos):
        # noinspection PyBroadException
        try:
            main()
        except Exception:
            pass

[shahla69]

hi
Would you mind telling me where my problem is ?
I run a Python file, but it does not run full and ends in the middle of the run!!

C:\Users\Hatami\Desktop\test\AdaptiveTrafficSignalControl-master>python four_int
ersects.py
Traceback (most recent call last):
  File "four_intersects.py", line 403, in <module>
    main()
  File "four_intersects.py", line 395, in main
    fixed_control_run(10)
  File "four_intersects.py", line 168, in fixed_control_run
    game_env.reset()
  File "C:\Users\Hatami\AppData\Local\Programs\Python\Python36\lib\site-packages
\gym\wrappers\time_limit.py", line 44, in reset
    return self.env.reset()
  File "C:\Users\Hatami\AppData\Local\Programs\Python\Python36\lib\site-packages
\gym\envs\toy_text\four_intersects_env.py", line 121, in reset
    self.start_sumo()
  File "C:\Users\Hatami\AppData\Local\Programs\Python\Python36\lib\site-packages
\gym\envs\toy_text\four_intersects_env.py", line 108, in start_sumo
    traci.start(self.sumo_cmd)
  File "C:\Program Files (x86)\DLR\Sumo\tools\traci\__init__.py", line 82, in st
art
    sumoProcess = subprocess.Popen(cmd + ["--remote-port", str(port)])
  File "C:\Users\Hatami\AppData\Local\Programs\Python\Python36\lib\subprocess.py
", line 594, in __init__
    _cleanup()
  File "C:\Users\Hatami\AppData\Local\Programs\Python\Python36\lib\subprocess.py
", line 205, in _cleanup
    res = inst._internal_poll(_deadstate=sys.maxsize)
  File "C:\Users\Hatami\AppData\Local\Programs\Python\Python36\lib\subprocess.py
", line 1027, in _internal_poll
    if _WaitForSingleObject(self._handle, 0) == _WAIT_OBJECT_0:
OSError: [WinError 6] The handle is invalid

python3.6
windows 7
please help me

[lily1720]

I have the same question.
Please help me ,too

[kalaschnik]

@lily1720 @shahla69
I encountered the same issue on Windows (not on Linux), as this is Windows-specific thing; see https://bugs.python.org/issue37380

This issue got fixed with Python 3.8. Try if upgrading to 3.8 solves WinError 6 for you. It did for me.

[zillurbmb51]

Hello,
I am having the same issue. I am using python 3.8. Any help?

> library(reticulate)
> use_condaenv("py3.8", required = TRUE)
> reticulate::repl_python()
Python 3.8.6 (C:/Users/zillu/anaconda3/envs/py3.8/python.exe)
Reticulate 1.16 REPL -- A Python interpreter in R.
>>> from pyspark import SparkContext, SparkConf
>>> conf = pyspark.SparkConf().setAppName("Spark Tutorials")
NameError: name 'pyspark' is not defined
>>> conf = SparkConf().setAppName("Spark Tutorials")
>>> sc = SparkContext.getOrCreate(conf=conf)
OSError: [WinError 6] The handle is invalid
>>> import subprocess
>>> subprocess._cleanup = lambda: None
>>> sc = SparkContext.getOrCreate(conf=conf)
OSError: [WinError 6] The handle is invalid

[jesvijonathan]

This "win 6" error happens only with process from multiprocessing but never occurs when using thread from threading..

File "c:/Users/jesvi/Documents/GitHub/Jesvi-Bot-Telegram/scripts/main.py", line 144, in thread_test p.start() File "C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\process.py", line 121, in start self._popen = self._Popen(self) File "C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\context.py", line 224, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\context.py", line 327, in _Popen return Popen(process_obj) File "C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__ reduction.dump(process_obj, to_child) File "C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) TypeError: cannot pickle '_thread.lock' object Traceback (most recent call last): File "<string>", line 1, in <module> File "C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\spawn.py", line 107, in spawn_main new_handle = reduction.duplicate(pipe_handle, File "C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\reduction.py", line 79, in duplicate return _winapi.DuplicateHandle( OSError: [WinError 6] The handle is invalid

Spec :
Python 3.8.6
Windows 10
VS code

any alternate solution for the time being ?

[koutoftimer]

@hackydunk @tburrows13 and anyone who will read this thread in the future: the problem is that you have to close all opened files implicitly if you are using Windows.

So, if you have some code that creates VideoFileClip or AudioFileClip or any file clip, do not forget to close it, e.g.:

videos: list[VideoFileClip] = read_videos()
images: list[ImageClip]     = read_images()
audios: list[AudioFileClip] = read_audios()

do_the_suff_you_need(videos, images, audio)

for clip in videos + images + audios:
    clip.close()

There is the snippet from the source code of the Clip base class:

    def close(self):
        """ 
            Release any resources that are in use.
        """

        #    Implementation note for subclasses:
        #
        #    * Memory-based resources can be left to the garbage-collector.
        #    * However, any open files should be closed, and subprocesses
        #      should be terminated.
        #    * Be wary that shallow copies are frequently used.
        #      Closing a Clip may affect its copies.
        #    * Therefore, should NOT be called by __del__().
        pass