OpenMAX locked when seeking multiple times #824
Comments
|
Can you point to a file that is confirmed to hit this problem? (e.g link to a specific Big Buck Bunny or a sample you have tested). |
|
Thank you for your answer. I could reproduce with other video files, but the test I reported above was run with the "standard" big buck bunny file from the website: http://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_1080p_h264.mov. I just reproduced the condition again, took me more hours, probably around 24. In this case I ran using start_debug=1 and vcdbg log assert returned: Now I killed omxplayer and it can't start again without a reboot. Is there anything else I can do to get some more info about this condition before I reboot? |
|
Can you run "sudo vcdbg log ex" |
|
|
I am experiencing the same issue with a custom application and omxplayer. |
|
@popcornmix were you able to reproduce the issue by any chance? I forgot to say that I'm currently testing with Pi1. Not sure if this happens also in Pi2 and Pi3. |
|
I've reproduced this with the VideoCore debugger attached. |
|
We run many PI's and we had this bug since PI1 (PI B) so I believe that its independent of the PI variant. |
|
@6by9 I appreciate you took the time to have a look at this! Did you succeed in reproducing the issue again by any chance? How much did it take for you to reproduce the first time on Pi3? I guess #449 was different from this, but in that case it was much quicker to reproduce on Pi1 and harder on Pi3 or when overclock was applied. Maybe switching to a Pi1 may help fixing this as well? |
|
Yes, I think we know what is happening, although it's only a hypothesis on the cause rather than proof. I had to reproduce the failure several times to get sufficient information, which generally meant leaving it overnight so it's taken a while. I've a local fix that I left running last night. If it is still running when I get in then I'll suggest popcornmix makes a test firmware with it in for you to try out. The only niggle in my testing is that my monitor doesn't actually have speakers to tell for sure that the audio is still working! AIUI This is going to be independent of flavour of Pi - it's timing in the DMA for HDMI audio, and that is common across all of them. #449 was the IPC doing the wrong thing, so the ARM spec had an effect. |
|
|
|
Locked up in what appears to be a different way and the debugger failed. Need to kick it off again :-( |
|
Please let me know if there is anything I can try to do to assist you. |
|
Sorry for the delay - it doesn't help that this takes so long to reproduce. I have seen a different issue with the fix where omxplayer has stalled, but everything on the GPU looks fine, and it is still responding normally. Whilst that instance of omxplayer appears to have locked up (not responding to the dbus commands), killing and restarting it works fine. The current change has been merged internally, so should be in the next rpi-update. |
|
When can we expect the current the fix in rpi-update? |
popcornmix
added a commit
that referenced
this issue
Jul 11, 2017
firmware: usb: Force MSD app to use CM3 pin conf firmware: IL ISP: Fix typo in logging firmware: IL ISP: Add black level and lens shading controls firmware: isp: Correct ISP Bayer stride calcs for supported formats firmware: Remove unused duplicate versions of vc_sm_defs.h firmware: IL camera: add get_parameter for OMX_IndexConfigCustomAwbGains firmware: IL resize: Support get_parameter OMX_IndexConfigCommonInputCro firmware: MMAL/RIL: Add MMAL_PARAMETER_RESIZE_PARAMS / OMX_IndexParamResize mapping firmware: ISP IL: Add lresize output firmware: MMAL/RIL: Add mapping for OMX_IndexConfigCommon[In|Out]putCrop firmware: MMAL/RIL: Correct handling of MMAL_PARAMETER_VIDEO_SOURCE_PATTERN firmware: IL ISP: Add H & V flip support firmware: IL ISP: Implement OMX_IndexConfigCommonInputCrop firmware: imx219: Refactor exposure calculations firmware: dmalib: Stop spinning on dma_pause if END is signalled See: #824
popcornmix
added a commit
to Hexxeh/rpi-firmware
that referenced
this issue
Jul 11, 2017
firmware: usb: Force MSD app to use CM3 pin conf firmware: IL ISP: Fix typo in logging firmware: IL ISP: Add black level and lens shading controls firmware: isp: Correct ISP Bayer stride calcs for supported formats firmware: Remove unused duplicate versions of vc_sm_defs.h firmware: IL camera: add get_parameter for OMX_IndexConfigCustomAwbGains firmware: IL resize: Support get_parameter OMX_IndexConfigCommonInputCro firmware: MMAL/RIL: Add MMAL_PARAMETER_RESIZE_PARAMS / OMX_IndexParamResize mapping firmware: ISP IL: Add lresize output firmware: MMAL/RIL: Add mapping for OMX_IndexConfigCommon[In|Out]putCrop firmware: MMAL/RIL: Correct handling of MMAL_PARAMETER_VIDEO_SOURCE_PATTERN firmware: IL ISP: Add H & V flip support firmware: IL ISP: Implement OMX_IndexConfigCommonInputCrop firmware: imx219: Refactor exposure calculations firmware: dmalib: Stop spinning on dma_pause if END is signalled See: raspberrypi/firmware#824
|
rpi-update is pushed. |
|
I've just caught the stall I'd seen overnight again - it looks to be my network locking up, nothing to do with omxplayer or the GPU. New test set up to run overnight with everything local to the Pi. |
|
great news! Thank you. Also a queston, do I understand it correctly that the fix is in one of these files below? because only these change since the last commit. |
|
Yes |
|
My test setup is still misbehaving. Has anyone else tried the new firmware and got any improvement? |
|
Yes, I'm testing it and seems fine so far. More than 4 days and omxplayer is still running properly. I have other pi's running tests and so far no lock. What misbehaviour did you notice? |
|
@carlonluca Thanks for the response, and that is good news. I was running with the debugger attached but that has been known to produce odd results of itself. I'll revert my test rig to running without the debugger overnight and see what I get. |
|
Generally happened within 12/24 hours yes. The update seems very good so far. Need more time anyway for precise results. |
|
Hello @6by9, do you have any news about this topic by any chance? Unfortunately it seems a few locks still occur. I just reproduced on omxplayer (unfortunately not a debug build): Again dbus seems stuck during SetPosition: took a couple of days to reproduce. The only difference here is that I changed my seek script to something like this (unsure if this is relevant or not):
It seems however that restarting omxplayer is still possible now without rebooting first. Do you have any idea? |
|
I was able to reproduce again. This time omxplayer includes the symbols: again dbus call to SetPosition is hanging. Took me 10/15 hours to reproduce this time. |
|
I haven't been looking into this for a while as your previous response had been that you hadn't managed to reproduce a lockup. |
|
It would be awesome, thank you. |
|
For some unknown reason it seems I'm now able to reproduce in less than 24 hours again. I'm trying to determine why, but can you confirm, just to be safe, that I'm testing on a firmware which is including your latest patches? |
|
Hello, i have the same question about the firmware. How to make sure all fixes are in effect? It would be great if you could provide with checksums to important files. Thanks. |
|
The commit was tagged against this issue, so #824 (reference) showed up in this issue log. Commited to the firmware and Hexxeh/rpi-firmware repos on 11 July. |
|
@6by9 I suspect that you actually fixed an issue with your latest patch, but I suspect another issue is still there. I keep getting a similar issue, but at this point it seems that the GPU is not stuck. Killing omxplayer and starting it again is sufficient to make it run properly. No more reboot needed. This is another backtrace that I dumped when omxplayer was stuck: It takes me 2 or 3 days to get this. What I notice is that this seems to only happen when running on more than 1 core. Pi1 does not seem to be affected and also Pi3 running with only 1 CPU enabled in the kernel seems to be ok. Is it possible this is something only related to omxplayer maybe? Is there something that I can do to investigate this further? Can you advise on something that I can do? |
|
This seems to have dropped from view. Anyone have any further comments? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hello! I'm experiencing a issue very similar to #449. I can reproduce this with my application, but I can also reproduce with omxplayer.
This is what I experience: repeating a seek command for long time results in a complete lock of the OpenMAX libraries, and often of the entire GPU. If I repeat seek for around 7 hours (but may be more or less), omxplayer freezes, the seek command does not return and the GPU is stuck.
I built omxplayer in debug and this is what I get when locked (I had to build a new gdb cause the one shipped with raspbian was crashing):
It seems that once again calling the OpenMAX function locks into ilcs_execute_function_ex like in
#449, but in this case the queue is empty. In my tests it seems this typically happens in SetConfig, resuming from the buffering procedure. I have the same identical behaviour in my application, which is very similar to omxplayer.
I can reproduce in a few hours by running omxplayer and executing a script like this:
while [ true ]; do ./dbuscontrol.sh setposition 0; sleep 1; doneThis is a huge problem as not only the application freezes, but also the board needs a complete reboot.
The text was updated successfully, but these errors were encountered: