how to encode and output file from anime4k？

[lsl330]

I have no problem playing anime4k video via mpv, and I have no problem playing it via command line, but I want to output the video to 1080p, how should I achieve that?

command
mpv input.mkv --no-sub --vo=gpu --glsl-shaders="/shaders/Anime4K_Clamp_Highlights.glsl;/shaders/Anime4K_Restore_CNN_VL.glsl;/shaders/Anime4K_Upscale_CNN_x2_VL.glsl;/shaders/Anime4K_Restore_CNN_M.glsl;/shaders/Anime4K_AutoDownscalePre_x2.glsl;/shaders/Anime4K_AutoDownscalePre_x4.glsl;~~/shaders/Anime4K_Upscale_CNN_x2_M.glsl"
it work and play as well

but command
mpv input.mkv --no-sub --vo=gpu --glsl-shaders="/shaders/Anime4K_Clamp_Highlights.glsl;/shaders/Anime4K_Restore_CNN_VL.glsl;/shaders/Anime4K_Upscale_CNN_x2_VL.glsl;/shaders/Anime4K_Restore_CNN_M.glsl;/shaders/Anime4K_AutoDownscalePre_x2.glsl;/shaders/Anime4K_AutoDownscalePre_x4.glsl;~~/shaders/Anime4K_Upscale_CNN_x2_M.glsl" -vf scale=1440:1080 --o=output.mkv
No effect，
how can i set the Resolution to 1080p and output the anime4k video？

[hooke007]

no meaning of using vo=gpu whiling encoding....
To be clear, do not use it.
vf=gpu="w=1920:h=1080"

https://mpv.io/manual/master/#video-filters-gpu

[lsl330]

no meaning of using vo=gpu whiling encoding.... To be clear, do not use it. vf=gpu="w=1920:h=1080"

https://mpv.io/manual/master/#video-filters-gpu

thank you for reply,but when i use
vf=gpu="w=1920:h=1080"
it have an error,
Option vf: gpu doesn't exist.
Error parsing option vf (option parameter could not be parsed)
Setting commandline option --vf=gpu=w=1440:h=1080 failed.

[hooke007]

This requires that the EGL implementation supports off-screen rendering on the default display. (This is the case with Mesa.)

See manual. windows does not support now.
#9354

[lsl330]

This requires that the EGL implementation supports off-screen rendering on the default display. (This is the case with Mesa.)

See manual. windows does not support now. #9354

thank you, it seems that I can only wait until later to add support for Win

[haasn]

If your ffmpeg is built with the corresponding options, you can use the vf_libplacebo filter in ffmpeg to apply shaders, including Anime4K, while transcoding.

[lsl330]

If your ffmpeg is built with the corresponding options, you can use the vf_libplacebo filter in ffmpeg to apply shaders, including Anime4K, while transcoding.

more detial？
标语言英语（美式）
Can you please give me the full command line?

[aypro]

I compiled the latest ffmpeg commit with libplacebo filter, but i'm getting this error when trying to add it to my command
ffmpeg -y -i clip.avs -vf libplacebo=custom_shader_path=Anime4K_Thin_HQ.glsl -vcodec libx264 -an out.mp4

[avisynth @ 000001c5f71da4c0] Stream #0: not enough frames to estimate rate; consider increasing probesize
Guessed Channel Layout for Input Stream #0.1 : stereo
Input #0, avisynth, from 'clip.avs':
Duration: 00:00:05.84, start: 0.000000, bitrate: 1 kb/s
Stream #0:0: Video: rawvideo (Y3[11][10] / 0xA0B3359), yuv420p10le, 1920x1080, 23.98 fps, 23.98 tbr, 23.98 tbn
Stream #0:1: Audio: pcm_s32le, 48000 Hz, stereo, s32, 3072 kb/s
Stream mapping:
Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
Press [q] to stop, [?] for help
Impossible to convert between the formats supported by the filter 'graph 0 input from stream 0:0' and the filter 'auto_scale_0'
Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented
Error while processing the decoded data for stream #0:0
Conversion failed!

[haasn]

@lsl330 @aypro it's implemented as a vulkan filter so you have to upload the frame to vulkan first:

ffmpeg -i CLIP -init_hw_device vulkan -vf hwupload,libplacebo,hwdownload -vcodec libx264 OUT

[aypro]

Thank you @haasn , this is what ended up working for me, just had to explicitly add the format on both ends, also i don't know if this is the correct way to apply multiple shaders
ffmpeg -y -i IN.MKV -init_hw_device vulkan -vf format=yuv420p,hwupload,libplacebo=custom_shader_path=Anime4K_Clamp_Highlights.glsl,libplacebo=custom_shader_path=Anime4K_Restore_CNN_Moderate_VL.glsl,libplacebo=custom_shader_path=Anime4K_Darken_HQ.glsl,libplacebo=custom_shader_path=Anime4K_Thin_HQ.glsl,hwdownload,format=yuv420p -vcodec libx264 -an out.mp4

[haasn]

That's a bit wasteful, it would be much better to cat all shaders into the same file and load that. (Though I'll probably fix vf_libplacebo to accept multiple shaders soon-ish)

That said, to make it significantly less wasteful, you could add format=rgba (or rgba64le) to the first libplacebo filter and format=yuv420p to the last libplacebo filter. (That way you at least avoid constantly converting between rgb and yuv420p)

[xiaowu2001]

I have problem when I use bat to upscale video.

However something was wrong in cmd:
Trailing option(s) found in the command: may be ignored.
Device creation failed: -12.
Failed to set value 'vulkan' for option 'init_hw_device': Cannot allocate memory
Error parsing global options: Cannot allocate memory

[xiaowu2001]

Thank you @haasn , this is what ended up working for me, just had to explicitly add the format on both ends, also i don't know if this is the correct way to apply multiple shaders ffmpeg -y -i IN.MKV -init_hw_device vulkan -vf format=yuv420p,hwupload,libplacebo=custom_shader_path=Anime4K_Clamp_Highlights.glsl,libplacebo=custom_shader_path=Anime4K_Restore_CNN_Moderate_VL.glsl,libplacebo=custom_shader_path=Anime4K_Darken_HQ.glsl,libplacebo=custom_shader_path=Anime4K_Thin_HQ.glsl,hwdownload,format=yuv420p -vcodec libx264 -an out.mp4

Can you upscale video by this command successfully? There's something wrong when I use the command. the error in cmd:
"Trailing option(s) found in the command: may be ignored.
Device creation failed: -12.
Failed to set value 'vulkan' for option 'init_hw_device': Cannot allocate memory"
I'm not good at ffmpeg, and I'm not sure if it's the problem of command "-init_hw_device vulkan".

[hooke007]

check if your local build support vulkan ffmpeg -init_hw_device list

[xiaowu2001]

check if your local build support vulkan ffmpeg -init_hw_device list

Supported hardware device types:
cuda
dxva2
qsv
d3d11va

[hooke007]

So, obviously no.

[xiaowu2001]

Can I use "cuda" instead of "vulkan"?
Or how can I output enlarged video by other way?

So, obviously no.

[xiaowu2001]

That's a bit wasteful, it would be much better to cat all shaders into the same file and load that. (Though I'll probably fix vf_libplacebo to accept multiple shaders soon-ish)

That said, to make it significantly less wasteful, you could add format=rgba (or rgba64le) to the first libplacebo filter and format=yuv420p to the last libplacebo filter. (That way you at least avoid constantly converting between rgb and yuv420p)

My ffmpeg(v5.0) build doesn't support vulkan, how to solve the problem or if there's anyway else to achieve outputting videos from Anime4K? And I also have a question about ffmpeg -i CLIP -init_hw_device vulkan -vf hwupload,libplacebo,hwdownload -vcodec libx264 OUT, "libplacebo" is used to set custom path of filter, right?

[k4yt3x]

With a standard Linux installation of GLSL for MPV, here's the full command I used to render the output. It works like a charm. IIRC this should be using the same shaders as Mode A.

mpv input.mp4 --no-sub --glsl-shaders="~/.config/mpv/shaders/Anime4K_Clamp_Highlights.glsl:~/.config/mpv/shaders/Anime4K_Restore_CNN_VL.glsl:~/.config/mpv/shaders/Anime4K_Upscale_CNN_x2_VL.glsl:~/.config/mpv/shaders/Anime4K_AutoDownscalePre_x2.glsl:~/.config/mpv/shaders/Anime4K_AutoDownscalePre_x4.glsl:~/.config/mpv/shaders/Anime4K_Upscale_CNN_x2_M.glsl" -vf=gpu="w=3840:h=2160" --o=output.mkv

Edit: this should be easier to read.

mpv input.mp4 \
    --no-sub \
    --glsl-shaders="\
~/.config/mpv/shaders/Anime4K_Clamp_Highlights.glsl:\
~/.config/mpv/shaders/Anime4K_Restore_CNN_VL.glsl:\
~/.config/mpv/shaders/Anime4K_Upscale_CNN_x2_VL.glsl:\
~/.config/mpv/shaders/Anime4K_AutoDownscalePre_x2.glsl:\
~/.config/mpv/shaders/Anime4K_AutoDownscalePre_x4.glsl:\
~/.config/mpv/shaders/Anime4K_Upscale_CNN_x2_M.glsl" \
    -vf=gpu="w=1216:h=2160" \
    --o=output.mkv

[infinisil]

@k4yt3x There's a mistake in your last command, the width should be 3840, not 1216

[k4yt3x]

@k4yt3x There's a mistake in your last command, the width should be 3840, not 1216

That's for the specific video I'm processing. Just do the calculations and change it to whatever you need it to be.

[studentnice05]

This requires that the EGL implementation supports off-screen rendering on the default display. (This is the case with Mesa.)

See manual. windows does not support now. #9354

现在在windows上不能使用--vf=gpu的话是不是就无法保存anime4k放大的视频了？Windows在运行下面的命令时候总是会报错：mpv input.mp4 --glsl-shaders="%appdata%/mpv/shaders/Anime4K_Clamp_Highlights.glsl" --vf=gpu -vf scale="1920:1080" --o=output.mkv
报错：Option vf: gpu doesn't exist.

[Hrxn]

it's --vo=gpu

[hooke007]

报错：Option vf: gpu doesn't exist.

Because there is no implement in windows. #9589 (comment)

it's --vo=gpu

No, it's vf not vo. It's clear in the doc and previous comments.

[mikigal]

I encoded video using this command:
mpv input.mp4 --glsl-shader="~~/shaders/Anime4K_Clamp_Highlights.glsl:~~/shaders/Anime4K_Restore_CNN_VL.glsl:~~/shaders/Anime4K_Upscale_CNN_x2_VL.glsl:~~/shaders/Anime4K_AutoDownscalePre_x2.glsl:~~/shaders/Anime4K_AutoDownscalePre_x4.glsl:~~/shaders/Anime4K_Upscale_CNN_x2_M.glsl" -vf scale="2560:1440" -o output.mp4

I got output file 1440p resolution, but it looks shaders are not applied - output file looks same as input. It just scaled file.
When I use vf=gpu nothing happens.

Did anyone find way to encode video with Anime4K shaders?

Platform: Windows

[haasn]

Did anyone find way to encode video with Anime4K shaders?

FFmpeg with vf_libplacebo is the recommended way.

[mikigal]

Did anyone find way to encode video with Anime4K shaders?

FFmpeg with vf_libplacebo is the recommended way.

Already tried all ways from this thread, still getting errors.
Can you share correct parameters for ffmpeg?

[ryandash]

Yep, thanks for the quick catch. I will update the shaders
Hello i keep getting:
[libplacebo @ 0000015a5016e3c0] Missing device feature: dynamicRendering
[libplacebo @ 0000015a5016e3c0] Imported Vulkan device was not created with all required features!
[libplacebo @ 0000015a5016e3c0] Failed importing vulkan device
[libplacebo @ 0000015a5016e3c0] Failed importing vulkan device to libplacebo!
[Parsed_libplacebo_2 @ 0000015a4bad60c0] Query format failed for 'Parsed_libplacebo_2': Generic error in an external library
Error reinitializing filters!
Failed to inject frame into filter network: Generic error in an external library
Error while processing the decoded data for stream #0:0
Conversion failed!

This error.

I have gtx 1650 and it says it supports vulkan

Is that 1650 in a laptop or desktop?
What OS are you running?
Is your OS up to date?
Do you have integrated and dedicated graphics?
Are your drivers up to date?
Are you using the latest version of ffmpeg?
What command are you using with ffmpeg?
Are you even trying to use ffmpeg?
I don't mean to be the devil but all you have given us is errors and basically useless info... Can you provide more information that will actually help understand what the issue could be?

What you have said is.. I have errors and this is my model of graphics card...

If you have a laptop with integrated and dedicated graphics. This might be a solution to your problem. IDK

https://www.reddit.com/r/vulkan/comments/qsthqq/enable_vulkan_for_gtx_1650_laptop/

[Sanaan01]

I don't mean to be the devil but all you have given us is errors and basically useless info... Can you provide more information that will actually help understand what the issue could be?

Sorry for not providng more info

the script i use was:
ffmpeg -y -vsync 0 -i "ASilentVoice(KoenoKatachi).mkv" -init_hw_device vulkan -vf format=yuv420p10,hwupload,libplacebo=w=3840:h=2160:upscaler=ewa_lanczos:custom_shader_path=shaders/Anime4K_ModeA+A.glsl,hwdownload,format=yuv420p10 "A Silent Voice (Koe no Katachi)4K2.mkv"

i am using the latest version of ffmpeg
I have a desktop 1650
Tried it after updating my drivers still same error
OS is up to date running windows 11
@FarisR99 I'm going to attempt to to do what you said to do may you help me I am new to ffmpeg hos do I build it using and older version of libplacebo

[FarisR99]

@Sanaan01 If you're using a prebuilt ffmpeg binary, try looking for an older release from a month ago. Much easier than building it yourself - though I do have documentation on building ffmpeg and mpv yourself if needed.

[Sanaan01]

Okay will try that when I downloaded ffmpeg i just used the latest release @FarisR99 march 20th build is fine i presume

[Sanaan01]

Thank you both for your time it works now I assume I should use CPU encoder to give better quality right?

[Tama47]

I should use CPU encoder to give better quality right?

I don't see a noticeable difference in quality, so I just use GPU.

[Tama47]

Does anyone know how to make this work on an Apple Silicon Mac with Videotoolbox?

[Sanaan01]

Is it common for file size to be reduced drastically like a 40gb 1080p remuc of a silent voice went to 6gb when its converted to 4K

[FarisR99]

Is it common for file size to be reduced drastically like a 40gb 1080p remuc of a silent voice went to 6gb when its converted to 4K

Completely depends on the encoder and encoder settings, but this is definitely possible.
Personally I use NVENC H264 for compatibility with a variable bitrate target of 25000Kbps, and even that is probably overkill. With AV1 encoding I can compress videos to much lower bitrates with very similar VMAF scores.

NVENC on a 1650 uses the older architecture (cant remember the names) and isn't as good as 20 series+, but use whatever produces a video where you don't notice obvious artifacts
It's much faster than CPU encoding still (although file size will be larger)

[Sanaan01]

It's much faster than CPU encoding still (although file size will be larger)

Yea it went speed 0.2x took a while left it on overnight.
I used the default settings that were given it gave such huge storage savings without any drop in quality it looks even better imo. But ill stick to nvenc because the main reason i tried anime4k instead of anything else was its speed.

[ryandash]

Does anyone know how to make this work on an Apple Silicon Mac with Videotoolbox?

I don't have an apple silicon Mac to try so I can only suggest options based on documentation that I have read.

Try this for h264
ffmpeg -y -vsync 0 -i "input.mkv" -init_hw_device vulkan -vf format=yuv420p10,hwupload,libplacebo=w=3840:h=2160:upscaler=ewa_lanczos:custom_shader_path=shaders/Anime4K_ModeA.glsl,hwdownload,format=yuv420p10 -c:v h264_videotoolbox -q:v 50 "output.mkv"

Or maybe this for h265
ffmpeg -y -vsync 0 -i "input.mkv" -init_hw_device vulkan -vf format=yuv420p10,hwupload,libplacebo=w=3840:h=2160:upscaler=ewa_lanczos:custom_shader_path=shaders/Anime4K_ModeA.glsl,hwdownload,format=yuv420p10 -c:v hevc_videotoolbox -q:v 50 "output.mkv"

The 50 can be 0 - 100 where 0 is the worst and 100 is the best. Try out different numbers with small video clips and compare quality to find the best number for your use case.

[Tama47]

@ryandash thanks for the suggestions!

Unfortunately, I don't think Apple Silicon Mac supports vulkan.

This is the error I got:

-vsync is deprecated. Use -fps_mode
Passing a number to -vsync is deprecated, use a string argument as described in the manual.
Device creation failed: -12.
Failed to set value 'vulkan' for option 'init_hw_device': Cannot allocate memory
Error parsing global options: Cannot allocate memory

ffmpeg -hwaccels:

Hardware acceleration methods:
videotoolbox

Is it possible to apply the shaders without using vulkan?

Edit: I've also tried -init_hw_device videotoolbox, but that also throws me an error.

[AVFilterGraph @ 0x600000a50360] No such filter: 'libplacebo'
Error reinitializing filters!
Failed to inject frame into filter network: Filter not found
Error while processing the decoded data for stream #0:0
[libvorbis @ 0x134e11a10] 35 frames left in the queue on closing

[FarisR99]

@Tama47 If you use MPV instead of ffmpeg, OpenGL rendering is an option (provided it's compiled with the required libraries). But I don't know if it'll work on your setup.

[jonaspero]

Can anyone help me?
Im using this command, the video looks great. But there seems to be some delay in the audio, around 3-4 seconds. Tried with multiple files.

.\ffmpeg.exe -y -vsync 0 -i "a.mkv" -init_hw_device vulkan -vf format=yuv420p10,hwupload,libplacebo=w=7680:h=4320:upscaler=ewa_lanczos:custom_shader_path=shaders/ac.glsl,hwdownload,format=yuv420p10 -c:v hevc_nvenc -c:a copy -cq 24 -bf 5 -refs 5 -preset p7 "a.mkv"

[ryandash]

Can anyone help me? Im using this command, the video looks great. But there seems to be some delay in the audio, around 3-4 seconds. Tried with multiple files.

.\ffmpeg.exe -y -vsync 0 -i "a.mkv" -init_hw_device vulkan -vf format=yuv420p10,hwupload,libplacebo=w=7680:h=4320:upscaler=ewa_lanczos:custom_shader_path=shaders/ac.glsl,hwdownload,format=yuv420p10 -c:v hevc_nvenc -c:a copy -cq 24 -bf 5 -refs 5 -preset p7 "a.mkv"

Do you have the same issues with just w=3840:h=2160?

I do not know what device you are using for playback but can the device play other local 8k content?

8k can be pretty demanding especially because you are encoding your video with h.265 and not h.264.
If you are watching the video on a device where you can monitor resource usage, how is your CPU and GPU usage?

Are you using Bluetooth audio? IDK what player you are using but depending on the device and player, bluetooth audio sync could be an issue. Reconnecting the Bluetooth device may fix that.

[Tama47]

@FarisR99 What would the commands be for mpv? I would also like to give it a try.

[FarisR99]

@FarisR99 What would the commands be for mpv? I would also like to give it a try.

@Tama47 I forgot that the OpenGL implementation was only added as part of the MR adding support for vf=gpu to Windows, followed by my own commit to allow selecting the api as an argument

My fork of mpv is public which has these changes merged into master, but a few additional tweaks like respecting --vulkan-device choice when using vulkan with vf=gpu. Clone it and revert the latest 2 commits (which makes vulkan support a requirement), then build that with the required EGL libraries. Run mpv --vf=gpu=api=help and ensure egl is listed - if so you should be able to use --vf=gpu=w=3840:h=2160:api=egl.

[Sanaan01]

I got a question to ask does a higher cq value mean more compressed or less compressed and does it mean that your getting higher quality?
In this following code

ffmpeg -y -vsync 0 -i "input.mkv" -init_hw_device vulkan -vf format=yuv420p10,hwupload,libplacebo=w=3840:h=2160:upscaler=ewa_lanczos:custom_shader_path=shaders/Anime4K_ModeA.glsl,hwdownload,format=yuv420p10 -c:v hevc_nvenc -cq 24 -bf 5 -refs 5 -preset p7 "output.mkv"

And what about cpu encoding I don't see a way to change the values on that script?

[mikigal]

Hello everyone,
I created GUI application which allows to save upscaled anime to disk using Anime4K shaders, it internally uses ffmpeg
https://github.com/mikigal/Anime4K-GUI

[llyyr]

Please stop, I don't want to see random video files in the wild ruined by your meme sharpening filter

[1412367]

Hello everyone, I created GUI application which allows to save upscaled anime to disk using Anime4K shaders, it internally uses ffmpeg https://github.com/mikigal/Anime4K-GUI

Thank you, you're really a lifesaver

[marcussacana]

I'm trying with ffmpeg but I'm crashing with out of memory error, my GPU is an 1650 with 4GB of VRAM
ffmpeg.exe -y -i %%i -init_hw_device vulkan -vf format=yuv420p,hwupload,libplacebo=w=3840:h=2160:upscaler=ewa_lanczos:custom_shader_path=%SHADER%,hwdownload,format=yuv420p -cq 19 -c:v hevc_nvenc "%%~dpni_out.mkv"
any idea of what can be wrong?

[Jules-A]

I'm trying with ffmpeg but I'm crashing with out of memory error, my GPU is an 1650 with 4GB of VRAM ffmpeg.exe -y -i %%i -init_hw_device vulkan -vf format=yuv420p,hwupload,libplacebo=w=3840:h=2160:upscaler=ewa_lanczos:custom_shader_path=%SHADER%,hwdownload,format=yuv420p -cq 19 -c:v hevc_nvenc "%%~dpni_out.mkv" any idea of what can be wrong?

Ffmpeg runs out of VRAM on my 12GB 6700XT even without shaders so I think it's just an Ffmpeg/libplacebo bug, probably need 16GB VRAM or something to not crash (some files worked fine though).

[marcussacana]

I see, someone reported the problem? I'm not sure if I have the right to report memory issue with a 4GB card

[marcussacana]

I created an ticket in ffmpeg bug report site: https://trac.ffmpeg.org/ticket/10665

[Jules-A]

I created an ticket in ffmpeg bug report site: https://trac.ffmpeg.org/ticket/10665

I don't have an account there but this is what it looks like for me: https://i.imgur.com/nDQ1DQe.png Doing just 1080p->1440p:
ffmpeg -y -i "input4.mp4" -init_hw_device vulkan -c:v libx264 -preset fast -crf 28 -c:a copy -c:s mov_text -metadata:s:s:0 language=English -disposition:s:s:0 forced -vf format=yuv420p,hwupload,libplacebo=w=2560:h=1440:upscaler=ewa_lanczos,hwdownload,format=yuv420p "Cells at Work! - S1E4 - Food Poisoning - 4.mp4"

[marcussacana]

I added your print as proof, thx, should make the problem more clear since some people just say in nowdays to buy a new card when they know you has 4gb only, lol.

[ghost]

Lookin' good.