SylkServer

h1. SylkServer WebRTC Video Conference

https://webrtc-test.sipthor.net

h2. Design

Two types of conferences are being supported: ad-hoc conferences and moderated conferences.

h3. Ad-hoc conferences

An ad-hoc conference is a conference where all participants have the same status and no one is controlling what other are participants are doing. The participants are rendered in a matrix or up to 3x3 depending of how many participants are in the room. The layout switches automatically for everybody as participants join or leave.

The conference room has a fixed total bitrate configured by the server, that can be specified per room or globally with the max_bitrate setting in webrtcgateway.ini (see below). This bitrate is shared by all participants in the room, meaning that the more participants are in the room, the less bitrate each participant will use for the video stream they send, keeping the total room usage constant to the value configured by max_bitrate. The bitrate adjustment per participant is done automatically by sylkserver as participants join or leave the room, by diving the available bitrate among the number of participants. The end result of this is that each participant will send a fraction of max_bitrate (which is determined by the number of participants in the room) and will always receive a total combined of max_bitrate from all the participants in the room, no matter how many participants are in the room. The formula to compute the bitrate per participant is shown below:

<pre>

participant_send_bitrate = max_bitrate / max(number_of_participants - 1, 1)

</pre>

Using this formula we can make sure that each participant always receives max_bitrate traffic in incoming video streams, independent of the number of participants. The traffic send/received by each party can be expressed like (considering N to be the number of participants and N>1):

<pre>

participant_sent_traffic     = max_bitrate / (N - 1)

participant_received_traffic = max_bitrate

sylkserver_sent_traffic      = max_bitrate * N            (participant_received_traffic * N)

sylkserver_received_traffic  = max_bitrate * N / (N - 1)  (participant_sent_traffic     * N)

</pre>

h3. Moderated conferences

A moderated conference is a conference where a moderator can decide the flow of the conference. The moderator is the first participant to join the conference. The moderator has the ability to see a list with all the participants, can select 1 or 2 of them to be the active speakers and also has the ability to mute other participants (audio and/or video). The moderator can also change the active speakers at any time.

The other participants will see the selected active speakers in full-sized video and the other participants as thumbnails. They will not be able to choose which other participant to watch, the conference view in their browser will be controlled by the moderator that decides who is the active speaker that everybody else sees on their screen in full-sized video.

The active speakers selected by the moderator will have their bitrate set to either max_bitrate (for 1 active speaker) or max_bitrate/2 (for 2 active speakers), while everybody else will have their bitrate set to a low value (64kb/s), just enough to have them represented in small thumbnails on other participant's screens.

h2. Features

h3. Ad-hoc conferences

Ad-hoc conferences are best suited for conversations with family/friends, since bandwidth/bitrate is managed automatically and does not involve a dedicated person to control the flow of the conference. However they can also be used for any other video conferences that imply a free-flowing type of discussion where any participant can jump into the conversation at any time.

h3. Moderated conferences

Moderate conferences are best suited for a business environment, where participants have to make some sort of presentation in front of the other participants and a moderator is assigned to control the flow of the conference and give the microphone to the appropriate participant, while the others are just watching the active speaker. They can also be used for a conference with 2 active participants that are having a public debate on a subject, while every other participant is just watching it and eventually asking questions.

h2. Configuration

Sylkserver allows the maximum bitrate and video codec to be configured, globally or per room with the following settings in webrtcgateway.ini file:

<pre>

; Maximum video bitrate allowed per sender in a room in bits/s. This value is

; applied to any room that doesn't define its own. The value is any integer

; number between 64000 and 4194304. Default value is 2016000 (~2Mb/s).

; max_bitrate = 2016000

; The video codec to be used by all participants in a room. This value is

; applied to any room that doesn't define its own.

; Possible values are: h264, vp8 and vp9. Default is vp9.

; video_codec = vp9

</pre>

h2. Client support

h2. Things that were explored

h2. Measurements

These load measurements were done on a Macbook Pro 15" with a 2.3GHz Intel Core I7 CPU, while having 7 participants in the room with each using 336Kb/s. The measurement shows the CPU usage in Firefox web browser with the aforementioned conditions, for the specified video codecs and resolutions which are used by all participants:

<pre>

 * H264/VGA - 150% CPU

 * H264/HD  - 250% CPU

 * VP9/VGA  - 220% CPU

 * VP9/HD   - 350% CPU

</pre>

h2. Remaining tasks

 * sylkserver: control interface for moderator

 * janus: patch to request full frames when a paused video is resumed

 * Rebuild mobile version

h2. Conclusions

h2. Software that was modified

In order to implement the bandwidth management and CPU load optimizations the following software was modified:

# sylkserver https://github.com/AGProjects/sylkserver

# sylk-webrtc https://github.com/AGProjects/sylk-webrtc

# sylkrtc.js https://github.com/AGProjects/sylkrtc.js

# python-application https://github.com/AGProjects/python-application

# python-sipsimple https://github.com/AGProjects/python-sipsimple