XBMC XvBA crashes when play a video H.264 Level 5.1

[Freespacer]

Hello,

XBMC XvBA (standalone) crashes when I let played a video with the format H.264 Level 5.1 (killa sample) over HDMI (HD-TV) but I can not find out the really issue. Please let me know if you need more information.

AMD Catalyst 12.6 (final release) / fglrx 8.98.2 / Radeon HD 6970
openSUSE 12.1 (64-bit)

XBMC crash log: http://sprunge.us/WYMM

Compile option from the spec file:
./configure
--prefix=%{_prefix} --bindir=%{_bindir} --includedir=%{_includedir}
--libdir=%{_libdir} --datadir=%{_datadir}
--enable-xvba
--disable-vaapi
--disable-vdpau
--disable-pulse
--disable-debug
CFLAGS="$RPM_OPT_FLAGS -fPIC -D__STDC_CONSTANT_MACROS"
CXXFLAGS="$RPM_OPT_FLAGS -fPIC -D__STDC_CONSTANT_MACROS"
LDFLAGS="-fPIC"
LIBS="-L%{_libdir}/mysql -lhdhomerun $LIBS"
ASFLAGS=-fPIC

GCC version:

gcc -v --version

Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib64/gcc/x86_64-suse-linux/4.6/lto-wrapper
gcc (SUSE Linux) 4.6.2
Copyright (C) 2011 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Target: x86_64-suse-linux
Configured with: ../configure --prefix=/usr --infodir=/usr/share/info --mandir=/usr/share/man --libdir=/usr/lib64 --libexecdir=/usr/lib64 --enable-languages=c,c++,objc,fortran,obj-c++,java,ada --enable-checking=release --with-gxx-include-dir=/usr/include/c++/4.6 --enable-ssp --disable-libssp --disable-plugin --with-bugurl=http://bugs.opensuse.org/ --with-pkgversion='SUSE Linux' --disable-libgcj --disable-libmudflap --with-slibdir=/lib64 --with-system-zlib --enable-__cxa_atexit --enable-libstdcxx-allocator=new --disable-libstdcxx-pch --enable-version-specific-runtime-libs --program-suffix=-4.6 --enable-linux-futex --without-system-libunwind --with-arch-32=i586 --with-tune=generic --build=x86_64-suse-linux
Thread model: posix
gcc version 4.6.2 (SUSE Linux)
COLLECT_GCC_OPTIONS='-v' '-v' '--version' '-mtune=generic' '-march=x86-64'
/usr/lib64/gcc/x86_64-suse-linux/4.6/cc1 -quiet -v -v help-dummy -quiet -dumpbase help-dummy -mtune=generic -march=x86-64 -auxbase help-dummy -version --version -o /tmp/ccidEMIV.s
GNU C (SUSE Linux) version 4.6.2 (x86_64-suse-linux)
compiled by GNU C version 4.6.2, GMP version 5.0.2, MPFR version 3.0.1, MPC version 0.8.2
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
COLLECT_GCC_OPTIONS='-v' '-v' '--version' '-mtune=generic' '-march=x86-64'
/usr/lib64/gcc/x86_64-suse-linux/4.6/../../../../x86_64-suse-linux/bin/as --64 --version -o /tmp/ccfp0viP.o /tmp/ccidEMIV.s
GNU assembler (GNU Binutils; openSUSE 12.1) 2.21.1
Copyright 2011 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or later.
This program has absolutely no warranty.
This assembler was configured for a target of `x86_64-suse-linux'.
COMPILER_PATH=/usr/lib64/gcc/x86_64-suse-linux/4.6/:/usr/lib64/gcc/x86_64-suse-linux/4.6/:/usr/lib64/gcc/x86_64-suse-linux/:/usr/lib64/gcc/x86_64-suse-linux/4.6/:/usr/lib64/gcc/x86_64-suse-linux/:/usr/lib64/gcc/x86_64-suse-linux/4.6/../../../../x86_64-suse-linux/bin/
LIBRARY_PATH=/usr/lib64/gcc/x86_64-suse-linux/4.6/:/usr/lib64/gcc/x86_64-suse-linux/4.6/../../../../lib64/:/lib/../lib64/:/usr/lib/../lib64/:/usr/lib64/gcc/x86_64-suse-linux/4.6/../../../../x86_64-suse-linux/lib/:/usr/lib64/gcc/x86_64-suse-linux/4.6/../../../:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-v' '-v' '--version' '-mtune=generic' '-march=x86-64'
/usr/lib64/gcc/x86_64-suse-linux/4.6/collect2 --build-id --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 --version /usr/lib64/gcc/x86_64-suse-linux/4.6/../../../../lib64/crt1.o /usr/lib64/gcc/x86_64-suse-linux/4.6/../../../../lib64/crti.o /usr/lib64/gcc/x86_64-suse-linux/4.6/crtbegin.o -L/usr/lib64/gcc/x86_64-suse-linux/4.6 -L/usr/lib64/gcc/x86_64-suse-linux/4.6/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib64/gcc/x86_64-suse-linux/4.6/../../../../x86_64-suse-linux/lib -L/usr/lib64/gcc/x86_64-suse-linux/4.6/../../.. /tmp/ccfp0viP.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib64/gcc/x86_64-suse-linux/4.6/crtend.o /usr/lib64/gcc/x86_64-suse-linux/4.6/../../../../lib64/crtn.o
collect2 version 4.6.2 (x86-64 Linux/ELF)
/usr/lib64/gcc/x86_64-suse-linux/4.6/../../../../x86_64-suse-linux/bin/ld --build-id --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 --version /usr/lib64/gcc/x86_64-suse-linux/4.6/../../../../lib64/crt1.o /usr/lib64/gcc/x86_64-suse-linux/4.6/../../../../lib64/crti.o /usr/lib64/gcc/x86_64-suse-linux/4.6/crtbegin.o -L/usr/lib64/gcc/x86_64-suse-linux/4.6 -L/usr/lib64/gcc/x86_64-suse-linux/4.6/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib64/gcc/x86_64-suse-linux/4.6/../../../../x86_64-suse-linux/lib -L/usr/lib64/gcc/x86_64-suse-linux/4.6/../../.. /tmp/ccfp0viP.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib64/gcc/x86_64-suse-linux/4.6/crtend.o /usr/lib64/gcc/x86_64-suse-linux/4.6/../../../../lib64/crtn.o
GNU ld (GNU Binutils; openSUSE 12.1) 2.21.1
Copyright 2011 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.

[Freespacer]

Additonal information:

H.264 Level 5.1 support is enabled in the driver:
aticonfig --set-pcs-u32=MCIL,HWUVD_H264Level51Support,1

This issue seems that have nothing to do with Level 5.1 setting.

[fritsch]

As we talked on IRC. Can you give a short feedback on these two questions:
a) Does it work if you disable xvba?
b) Do other videos for example Level 4.1@High work with and without xvba enabled?

[FernetMenta]

Could you configure with --enable-debug? I would like to get more information from the crashlog,

[Freespacer]

I have recompile the XBMC with --enable-debug. I have 2 difference crash log file.

First crash logfile with killa.sample video (enabled XvBA):
http://sprunge.us/SZOf

Second crash logfile with YouTube Add-on to play a youtube video (enabled XvBA):
http://sprunge.us/SbWG

[Freespacer]

I can play the killa.sample video and an YouTube vIdeo with disabled XvBA acceleration.

[FernetMenta]

Hmm, random locations for crashes. Mostly an indicator for writing out of bounds. I have checked this part of the code but could not find any problems. Interesting that it seems to happen only on your system. Do you have a chance to run with debugger and watch the variables? At least it would help to know the exact lines where the crash happened. I am wondering why those are not printed in the crashlog. Normally they show up when it was compiled with -g

[Freespacer]

Okay, the debug symbol are missing. I have installed the debuginfo for XBMC (XvBA). I hope the crash log can tell more about the ffmpeg/XvBA issue.

New crash report with killa.sample video (enabled XvBA):
http://sprunge.us/EjiS

[FernetMenta]

Something goes entirely wrong here. It should never get to this line when hw acceleration is active. Check error_resilience.c line 916. Can you place ffmpeg log messages into all functions in xvba_h264.c? Just to see if those are called. Simply use error logs in order to make sure they show up in the log. E.g.
av_log(h->s.avctx, AV_LOG_ERROR, "marker xx.\n");

[Freespacer]

I have set the log handler in all functions. But in the log file it does not print out any "marker". I am afraid that every single function in xvba_h264.c is not in used during enabled XvBA.

[Freespacer]

fritsch told me in the IRC to remove any special flags from configure parameters but it has no effect.

config.log: http://sprunge.us/RYaF

[FernetMenta]

Strange. I have no better idea than to check if avctx->hwaccel has ever been set or it's getting cleared by whatever. Can you check the value of avctx->hwaccel (pointer) at the beginning of XVBA::FFGetBuffer and at the end.

[Freespacer]

I add this line at the beginning of FFGetBuffer function:

diff -rup xbmc.orig/xbmc/cores/dvdplayer/DVDCodecs/Video/XVBA.cpp xbmc-git/xbmc/cores/dvdplayer/DVDCodecs/Video/XVBA.cpp
--- xbmc.orig/xbmc/cores/dvdplayer/DVDCodecs/Video/XVBA.cpp 2012-06-15 10:32:50.858300045 +0200
+++ xbmc-git/xbmc/cores/dvdplayer/DVDCodecs/Video/XVBA.cpp 2012-07-04 22:13:03.950666910 +0200
@@ -1049,6 +1049,7 @@ void CDecoder::FFDrawSlice(struct AVCode

int CDecoder::FFGetBuffer(AVCodecContext avctx, AVFrame *pic)
{
+CLog::Log(LOGERROR,"Pointer: %p\n", avctx->hwaccel);
CDVDVideoCodecFFmpeg ctx = (CDVDVideoCodecFFmpeg_)avctx->opaque;
CDecoder_ xvba = (CDecoder*)ctx->GetHardware();

The crash log print out:
20:28:06 T:140585679709952 ERROR: Pointer: (nil)

Full crash log:
http://sprunge.us/ENOC

[FernetMenta]

avctx->hwaccel should be set in libavcodec/h264.c, decode_slice_header (line 2816). If the codec context is not initialized, avctx->get_format is called. This is actually a callback into XBMC which opens the decoder, in our case xvba. Can you check whether hwaccel is set:

s->avctx->hwaccel = ff_find_hwaccel(s->avctx->codec->id, s->avctx->pix_fmt);

[fritsch]

We added a lot of av_logs yesterday.
hwaccel is NIL after the above method and more important, as we debugged ff_find_hwaccel, there is no single hwaccel found, the list is empty.

On the other side ffmpeg build logs shows that all accels are added ...

freespacer will provide some logs later.

I don't have an idea anymore, could be that linking horribly fails?

[fritsch]

Here is the ffmpeg log: http://sprunge.us/PVfb

Concerning h264, we get:
h264_vaapi_hwaccel=yes
h264_vaapi_hwaccel_select='vaapi h264_decoder'
h264_vda_hwaccel=yes
h264_vda_hwaccel_deps='VideoDecodeAcceleration_VDADecoder_h pthreads'
h264_vda_hwaccel_select='vda h264_decoder'
h264_vdpau_decoder=yes
h264_vdpau_decoder_select='vdpau h264_decoder'
h264_xvba_hwaccel=yes
h264_xvba_hwaccel_select=xvba

Does this fail to register xvba with the h264_decoder?

[fritsch]

Okay, spamming like hell:
https://github.com/FernetMenta/xbmc/blob/master/lib/ffmpeg/configure#L1428

should this be: h264_xvba_hwaccel_select="xvba h264_decoder"

[FernetMenta]

Freespacer, did you change anything on the configure scripts?
your ffmpeg config.log:

check_header amdxvba.h
check_cpp
BEGIN /tmp/ffconf.miBetCct.c
    1   #include <amdxvba.h>

This is mine:

check_header amd/amdxvba.h
check_cpp
BEGIN /tmp/ffconf.J5ZQAXaX.c
    1   #include <amd/amdxvba.h>

I think the build dependency is not satisfied and the xvba stuff is not linked. The amdxvba.h header is expected to be found in /usr/include/amd/amdxvba.h

@fritsch

Does this fail to register xvba with the h264_decoder?

Yep, this seems to be wrong but I think this is not the problem we are chasing here. Can you please submit a PR?

[fritsch]

@FernetMenta:
opensuse installs amdxvba.h directly to /usr/include so he has to change this.
Here is the patch suse uses: i did not find a position where he misses this to replace: http://sprunge.us/YGXb

Will submit a PR.

[FernetMenta]

He should not change the configure script, just copy the header into include/amd/amdxvba.h

Please tell me if anything has been changed on the code.

[FernetMenta]

The amdxvba header is not part of Catalyst. The reference against which we develop is here:
http://developer.amd.com/downloads/xvba-sdk-0.74-404001.tar.gz

[fritsch]

@FernetMenta
freespacer got it working. It seems the the missing bit in configure was the cause.
He kept everything he has, only changed this configure line.

[FernetMenta]

Great. Then I close this issue.