VoIP Checklist

[ylecollen]

Here's my notes on building a comprehensive VoIP UX (inherited from ye olde TNG days):

P0:

• Ensure "early hangup" works from both caller & callee side
• Ensure "late hangup" works from both caller & callee side (i.e. clearing the call after being answered)
• Ensure multiple consecutive calls work after the various different termination scenarios
• Ensure that termination reasons are correctly displayed in the UI (e.g. "answered elsewhere", "lost media", "lost signalling", "cancelled", "hung up", "other error")
  answered elsewhere : a toast is displayed.
• Ensure inbound calls are picked up when the app is launched from cold
• Ensure inbound calls are picked up when the app is foregrounded after being backgrounded
• Ensure that multiple devices ring simultaneously for inbound calls
• Ensure that if the call is answered on another device, we tell the user what the error was
• Ensure that if the call is rejected by another device, we tell the user that's what happened
• Ensure that devices which are busy on other calls do not cause inbound calls to be globally rejected (see Handle overlapping VoIP calls correctly with 'busy local' and 'busy everywhere'. element-web#1964)
• Ensure that we don't interact badly with rate limiting (i.e. we correctly retry VoIP signalling like m.call.candidates if we get a 429 'too many requests' response back)
• Ensure that we don't try to re-answer stale m.call.invite events (m.call.invite and m.call.hangup matching the same call_id in the same sync resp, so first m.call.invite must be discarded)
• Ensure 1:1 call is made for 1:1 joined members (invited or banned users do not count)

P1:

• Need local voice mute button
• Need local video mute button
• Need ability to flip between rear & front facing camera
• Need ability to toggle speakerphone on & off
• When on video call, default to speakerphone being on.
• When on voice call, default to speakerphone being off.
• Ensure that the right microphone is used when on speakerphone (some handsets have a dedicated microphone used when on speakerphone to avoid feedback)
• When a video call (and not a voice call) is foregrounded, wedge the backlight on.
• When a voice call (and not a video call) is foregrounded and not on speakerphone, turn the backlight off when the proximity sensor fires (i.e. the phone is brought to the ear).
• When physical headset is plugged in, switch to headset.
• Only pause background music when actually on a call
• Play 'ringtone' and vibrate when the callee receives a call.
• Ensure "glare" is handled gracefully if caller & callee try to call each other at the same time. Some tiebreaker (e.g. the user with the lexographically lower user ID) should assume the role of caller and the call should proceed.
• Ensure there is no echo when on speakerphone (as heard by the remote device from a different room)
• Ensure lipsync is correct
• Ensure bandwidth/framerate/latency recovers out of going out of reception or being on a bad network
• Ensure trickle ICE works
• Ensure TURN works over UDP
• Ensure TURN works over TCP (port 3478)
• Ensure TURN works over TCP+SSL (port 443)
• Ensure permission prompts for microphone and camera are handled well on first use and don't disrupt the call setup (especially if the call setup is happening in the background)
• Ensure if user cancels permissions for microphone and camera the call still continues as well as it can (and that the user has the ability to easily uncancel the permissions in future)

P2:

• Ensure that incoming GSM calls put the call on hold
• Ensure that ongoing calls are taken off hold when the app is subsequently resumed
• Need indicator to say whether the remote user is currently on hold (especially if this is due to them receiving a GSM call)
• Need indicator to say whether the remote user has muted voice or video (rather than the call crapping out)
• Need network quality warning, so that if the call quality is having problems the user understand it's the network's fault rather than the app's fault.
• Set default audio playback volume (or warn if it is at zero), otherwise users think there's audio and don't realise they have to turn up the volume control.
• Play 'pushing' tone when waiting for the other user to 'respond to your push' (i.e. return an m.call.ringing or whatever)
• Play 'ring back tone' when the other user's client(s) is actually ringing. (This is different to a ringtone played for inbound calls!)
• Show inbound call screen on lockscreen if at all possible.
• Ensure "back to app" works and video & audio is still captured and audio is still played back when the app is foregrounded but the call UI is backgrounded
• Ensure that both voice and video calls continue when the app is backgrounded (although the OS will typically disables video capture)
• Ensure that the phone can be locked and screen turned off (e.g. by pressing power button on iOS) whilst on a call and still have audio continue.
• Ensure that after N minutes of the phone being locked and screen turned off the call continues and isn't cancelled as a powersaving measure.

P3:

• Prompt to optionally degrade to voip if insufficient network to support video
• After N seconds of total media failure, abort call and give option to call back
• Don't need an explicit "hold" button though as we don't have an explicit line 1/line 2 thing going on (yet) other than when something happens to the call (e.g. GSM interruption) to interrupt voice and video capture.
• Ensure that other interruptions to the call (e.g. backgrounding it and capturing voice or video with another app) put the call on hold.
• Ensure that secondary noise-cancelling microphones are correctly used for both speakerphone and non-speakerphone calls. (this is probably best done by googling to see if we're using the right API)
• Support bluetooth headset
• Ensure "back to app" button works when an outbound call is being setup (i.e. the call media sets up properly despite connecting in the background)
• Ensure backgrounded calls work when the app is backgrounded during call setup
• Ensure that if the OS disables video capture due to backgrounding, the mute is propagated to the receiver
• Ensure there is no echo when switching speakerphone on and off (may require explicitly telling the VoIP stack when this happens so it knows to recalibrate the echo cancellation filter)
• Ensure ICE candidate discovery is not too aggressive

In future:

• Support group calling as per vector-web.
• Support upgrading voip to video call
• Support downgrading video to voip call
• Support upgrading voip/video call to conference
• Support downgrading voip/video call from conference
• Support receiving WebRTC screenshare screens
• Ensure calls survive switching networks
• Ensure video is played in a thumbnail when "back to app" (like youtube)
• Prompt for user feedback after calls (rate from 1-5 stars)
• Report media logs for calls (i.e. phone home with the webrtc.internals graphs, or the TNG equivalent)

[pedroGitt]

Related deffect: #284

[sbts]

I'd like to upvote P3 item " Support bluetooth headset"

This is a fairly fundamental thing to support and catches me out every time.
I then have to spend time hunting up a wired headset (which I otherwise NEVER use)

I'm happy to raise a separate issue for this if you prefer

[jfrederickson]

Show inbound call screen on lockscreen if at all possible.

Is this meant to include #554 - or if not, can that be added to this list as well? (Assuming I'm not just missing it.)

[ghost]

Where are we wih this checklist ?
I see that TURN support with SSL support is a P1 but there are still bugs opened like #5954

Would be great to see some progress here and/or communication around it.

[ara4n]

Heads up that there was a confusing checklist entry on this talking about needing to match DTLS fingerprints with E2EE keys. This was a 4 year old thinko on my side, so I've removed it. The rationale is:

• E2E protects the DTLS fingerprints in the SDP of the offer/answer, so you can be sure the SDP came from the right user.
• and WebRTC then ensures the DTLS handshake on the wire actually matches the fingerprint in the SDP.
• the really annoying thing is you can't select the keys used for DTLS. but insertable streams mitigates that.