SIP SIMPLE Client SDK

[[TOC(Design*, depth=1)]]

= Video blueprint =

Design and implement Ticket [ticket:18]

== Goals ==

The goal is to implement the support for video session is the SIPSIMPLE middleware. For the video encoding/decoding VLC library shall be used, which supports H.264 codec (first codec to be supported).

The video stream should behave like the audio stream at the transport level: SRTP and ICE should be usable. It must comply with the IMediaStream interface, the same way AudioStream does.

== Integration ==

In the picture below the integration of the video support into SIPSIMPLE is shown divided into it's different components:

(IMAGE)

* PJSIP: a new transport is needed. Instead of creating a whole new transport, which would require to implement SRTP and ICE again, a transport adapter will be implemented. A transport adapter sits between a real transport and a pjmedia_stream. To the stream, this adapter will look like a media transport, and to existing media transport, this adapter will look like a stream. The benefit of this approach is we can use the same adapter for both kind of media transports, that is the UDP and ICE media transport. This is exactly the approach that was taken for SRTP. This transport adapter will be responsible for encoding/decoding the video information.

* Middleware:


* VideoTransport: the VideoTransport will use RTPTransport to carry the video data and will be responsible for building the SDP for the video stream. A new option will be added to RTPTransport so that it starts the video transport adapter instead of the regular transport when needed.


* VideoStream: implements IMediaStream interface. Will export a plugable mechanism so that the application layer can access the video data and display it in a window for example, similar to ExternalVNCViewer on MSRP streams.

* Application: the application will receive the video data from the stream and 'paint' it on a window.

== Video acquisition ==

First approach will == Roadmap ==

In order to achive the goal, the following subtasks need to be done:
* Implement a wrapper class around the VLC Python bindings. http://wiki.videolan.org/Python_bindings
* Create videostream.py implementing IMediaStream interface.
* Use the VLC wrapper class to create pjmedia_videostream object which will do encode/decode the video acquisition at low level stream.
* Bundle the VLC Python wrapper together with SIPSIMPLE so that it get's compiled and pass installed when installing SIPSIMPLE.
* Create sip_video_session.py script (or modify sip_audio_session_script) so that it to transport_video.

== Roadmap ==

This milestones should uses video capabilities.

At the time of mangling the SDPs, two approaches can be achieved taken in order to get add video working:

support:
* Modify PJSIP's PJMEDIA component to encode/decode video streams and handle them the same as audio streams from the upper layers.
* Just use PJMEDIA to do the SDP negotiation: make SIPSIMPLE able negotiation and audio part. Video would be external to generate and negotiate the PJMEDIA layer.

For this case the second approach seems to be a valid video SDP.
* Null video: SIPSIMPLE better one. When receiving an incoming call or making a call, the SDPs will generate a valid RTP payload with dummy data be mangled so that they include H.264 stream information i.e.
{{{
...
m=video 6001 RTP/AVP 98
a=rtpmap:98 H264/90000
...
}}}

So in the case of an incoming call, an VLC player instance will be exchanged after a successful SDP negotiation.
* Video reception an still image sending: add opened for playing the ability to receive stream at the IP and display port specified by the remote SDP. For an outgoing call, an VLC will stream H.264 video stream and generate taken from a valid selected video stream source (webcam, still picture, ...) from an still image.
* Video acquisition: send real the sipsimpleclient listen IP and will randomly select a port (in a given range) which doesn't interfere with the audio ports.

This way audio and video data. components are sepparate, but it's not necesary to modify the underlying media handling component: pjmedia.

== Components ==

[[Image(videostream-components.png)]]

== Tools Classes ==

libVLC will be used for encoding and decoding

[[Image(videostream-classes.png)]]

== Code mockup ==

There are 3 files attached containing a simple code mockup:
* videostream.py: Class representing the actual video data. It has ctypes based Python bindings, that should be used at stream. Contains necessary functions to initialize/stop the application stream using VLC as the engine.
* vlcw.py: Wrapper class around the VLC python bindings. Provide some higher level functions to display remote video. Acquisition will most likely be done in C.

A libVLC shared library will need used by the VideoStream class.
* sip_video_session.py: Script to be built with all necessary module statically linked: h264 encoder/decoder, core modules, etc. test videostream from a console interface.