H264 bad video after passing through network; issue with packetization/depacketization apparently..

[forlayo]

I am trying to fix an issue that comes from sending the Android camera using SipSorcery, after several tests I ended up recording the stream in raw h264 just before sending it and just when receiving it at the other side. And the issue is still there so I would say that something is not going well on network transmission.

As a proof of this I've recorded the h264 stream just before calling OnVideoSourceEncodedSample on video source and recorded same on the other party on OnVideoFrameReceived.

As you an see on file recorded at OnVideoSourceEncodedSample video looks pretty fine, but at receiving side at OnVideoFrameReceived. it looks with artefacts, specially on movement.

Note: You can play these raw h264 files with ffplay, vlc or Elmedia Player.

Comparing (using h264streamanalysis tool) the stream we can see that there are errors, on received stream at NAL 98 (for example) the type is not correct, neither the size and it's followed by an invalid frame as wel..

[forlayo]

This could be related with #378 I think..

[forlayo]

I am checking with latest version of master, just wondering if the changes to rtp mtu to 1200 fixes my issue. But it's still happening..

[forlayo]

Could it be an issue related to packet loss ? Just basing my answer as the receiving side is somehow corrupt and the effect is like this one (more or less) -> http://www.mediapro.cc/rtp抗丢包传输方案/

If so.. I think I am on a big issue as this library doesn't have any mechanism for packet loss right ?

[forlayo]

I saw on a bit old Microsoft example a full implementation of FEC using XOR and also Reed Solomon, just in case this helps -> https://github.com/conferencexp/conferencexp/blob/master/MSR.LST.Net.Rtp/fec.cs#L47

And Java implementation of ULPFEC (like what's currently in use in WebRTC) https://github.com/jitsi/libjitsi/tree/master/src/main/java/org/jitsi/impl/neomedia/transform/fec

[sipsorcery]

There is no proper RTCP feedback mechanism in the sipsorcery library and minimal work has been done to wire up the video encoders to trigger key frames in the event of packet loss. I'm adding this issue to the list of enhancements. It's a BIG job to sort this out.