Retransmission, Loss, and Recovery

For interactive real-time media we generally do not have time to request the source to retransmit a packet and to receive the new copy ⇒ live without it or recover it using Forward Error Correction (FEC), i.e., send sufficient redundant data to enable recovery.

However, for non-interactive media we can use retransmission at the cost of a longer delay before starting playout.

If you do have to generate output, but do not have any samples to play:

audio

Comfort noise: play white noise or play noise like in the last samples {as humans get uncomfortable with complete silence, they think the connection is broken!} [Mattisson 1995]
if you are using highly encoded audio even a BER of 10^-5 will produce very noticeable errors

video

show the same (complete) video frame again
you can drop every 100th frame (for a BER of 10^-2), but the user will not notice! [Xiaoning 1994]

There may also be compression applied to RTP see [RFC 3545 ].

Slide Notes

[Mattisson 1995] Thomas Mattisson, ‘Integration of Computer Telephony into the Ericsson Corporate Network’, Master’s thesis, KTH, Teleinformatics, Stockholm, Sweden, February 1995 [Online]. Available: http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-98625 Links to an external site.

[Xiaoning 1994] Yang Xiaoning, ‘A New Controlled Video Frame Loss Scheme for Video Transmission Over High Speed Networks’, Master’s thesis, National University of Singapore, Dept. of Electrical Engineering, Singapore, 1994.

[RFC 3545] T. Koren, S. Casner, J. Geevarghese, B. Thompson, and P. Ruddy, ‘Enhanced Compressed RTP (CRTP) for Links with High Delay, Packet Loss and Reordering’, Internet Request for Comments, vol. RFC 3545 (Proposed Standard), Jul. 2003 [Online]. Available: http://www.rfc-editor.org/rfc/rfc3545.txt Links to an external site.

Transcript

[slide87] Now, one approach that we can take is using forward error correction. What does forward error correction do? It says that in this packet here, if I lose it, I lose the details about the audio for that period of time. But if in this packet, I put not only its data, but a copy of the data that would be there, if I lose this packet, what happens? I get the next one, and I say, oh, I didn't get the audio in time, but oh, by the way, I have this copy of what would have been there, and I play it out. And typically, this might be encoded by a different CODEC, let's say a GSM CODEC, and this might be coded by a G.711 CODEC, so this will be small. But in any case, we saw the size of this thing was only about 33 octets, so hey, even if we double it, remember, keep in mind the overhead that we had, so we could lose every other packet, and we still wouldn't lose any content. Now, it was at the cost of an increased delay, because we had to wait for the next packet to arrive, but we didn't lose anything, which is really very, very cool. So people have used forward error correction, and they have shown that even if you have very highly encoded audio, at a bit error rate of about 10 to the minus 5, you get noticeable effects of it. 10 to the minus 5, what's that correspond to? Typical twisted pair wiring. So that means we need to do some sort of coding on top of this, or we're going to end up with bad effects, so we take advantage of that. In video, what's cool is you can actually have a bit error rate of 10 to the minus 2, which seems absurd to me, but if you realize, that just means, once in 100 frames, we throw the whole frame away. You won't notice it. The trick is not putting up splotches in the middle of a frame, because we had a corrupt piece of the frame, and show something which doesn't fit with the frames before and the frames afterward. That's very noticeable to our eye. So it's about hiding it.