(Last updated on Mon 15 jul 2013, version 0.11.90.1)
This HOWTO describes the basic usage of the GStreamer RTSP libraries and how you
can build simple server applications with it.
The server relies heavily on the RTSP infrastructure of GStreamer. This includes
all of the media acquisition, decoding, encoding, payloading and UDP/TCP
streaming. We use the rtpbin element for all the session management. Most of
the RTSP message parsing and construction in the server is done using the RTSP
library that comes with gst-plugins-base.
The result is that the server is rather small (a few 11000 lines of code) and easy
to understand and extend. In its current state of development, things change
fast, API and ABI are unstable. We encourage people to use it for their various
use cases and participate by suggesting changes/features.
Most of the server is built as a library containing a bunch of GObject objects
that provide reasonable default functionality but has a fair amount of hooks
to override the default behaviour.
The server currently integrates with the glib mainloop nicely. It's currently
not meant to be used in high-load scenarios and because no security audit has
been done, you should probably not put it on a public IP address.
You need to initialize GStreamer before using any of the RTSP server functions.
main (int argc, char *argv)
gst_init (&argc, &argv);
The server itself currently does not have any specific initialisation function
but that might change in the future.
* Creating the server
The first thing you want to do is create a new GstRTSPServer object. This object
will handle all the new client connections to your server once it is added to a
GMainLoop. You can create a new server object like this:
server = gst_rtsp_server_new ();
The server will by default listen on port 8554 for new connections. This can be
changed by calling gst_rtsp_server_set_service() or with the 'service' GObject
property. This makes it possible to run multiple server instances listening on
multiple ports on one machine.
We can make the server start listening on its default port by attaching it to a
mainloop. The following example shows how this is done and will start a server
on the default 8554 port. For any request we make, we will get a NOT_FOUND
error code because we need to configure more things before the server becomes
main (int argc, char *argv)
gst_init (&argc, &argv);
server = gst_rtsp_server_new ();
/* make a mainloop for the default context */
loop = g_main_loop_new (NULL, FALSE);
/* attach the server to the default maincontext */
gst_rtsp_server_attach (server, NULL);
/* start serving */
The server manages four other objects: GstRTSPSessionPool,
GstRTSPMountPoints, GstRTSPAuth and GstRTSPThreadPool.
The GstRTSPSessionPool is an object that keeps track of all the active sessions
in the server. A session will usually be kept for each client that performed a
SETUP request for a certain media stream. It contains the configuration that
the client negotiated with the server to receive the particular stream, ie. the
transport used and port pairs for UDP along with the state of the streaming.
The default implementation of the session pool is usually sufficient but
alternative implementation can be used by the server.
The GstRTSPMountPoints object is more interesting and needs more configuration
before the server object is useful. This object manages the mapping from a
request URL to a specific stream and its configuration. We explain in the next
topic how to configure this object.
GstRTSPAuth is an object that authenticates users and authorizes actions
performed by users. By default, a server does not have a GstRTSPAuth object and
thus does not try to perform any authentication or authorization.
GstRTSPThreadPool manages the threads used for client connections and media
pipelines. The server has a default implementation of a threadpool that should
be sufficient in most cases.
* Making url mount points
Next we need to define what media is attached to a particular URL. What we want
to achieve is that when the user asks our server for a specific URL, say /test,
that we create (or reuse) a GStreamer pipeline that produces one or more RTP
The object that can create such pipeline is called a GstRTSPMediaFactory object.
The default implementation of GstRTSPMediaFactory allows you to easily create
GStreamer pipelines using the gst-launch syntax. It is possible to create a
GstRTSPMediaFactory subclass that uses different methods for constructing
The default GstRTSPMediaFactory can be configured with a gst-launch line that
produces a toplevel bin (use '(' and ')' around the pipeline description to
force a toplevel GstBin instead of the default GstPipeline toplevel element).
The pipeline description should contain elements named payN, one for each
stream (ex. pay0, pay1, ...). Also, for increased compatibility each stream
should have a different payload type which can be configured on the payloader.
The following code snippet illustrates how to create a media factory that
creates an RTP feed of an H264 encoded test video signal.
factory = gst_rtsp_media_factory_new ();
"( videotestsrc ! x264enc ! rtph264pay pt=96 name=pay0 )");
Now that we have the media factory, we can attach it to a specific url. To do
this we get the default GstRTSPMountPoints from our server and add the url to
factory mount points to it like this:
...create server..create factory..
/* get the default mount points from the server */
mounts = gst_rtsp_server_get_mount_points (server);
/* attach the video test signal to the "/test" URL */
gst_rtsp_mount_points_add_factory (mounts, "/test", factory);
When starting the server now and directing an RTP client to the URL (like with
vlc, mplayer or gstreamer):
a test signal will be streamed to the client. The full example code can be
found in the examples/test-readme.c file.
Note that by default the factory will create a new pipeline for each client. If
you want to share a pipeline between clients, use
* more on GstRTSPMediaFactory
The GstRTSPMediaFactory is responsible for creating and caching GstRTSPMedia
A freshly created GstRTSPMedia object from the factory initially only contains a
GstElement containing the elements to produce the RTP streams for the media and
a GPtrArray of GstRTSPStream objects describing the payloader and its source
pad. The media is unprepared in this state.
Usually the url will determine what kind of pipeline should be created. You can
for example use query parameters to configure certain parts of the pipeline or
select encoders and payloaders based on some url pattern.
When dealing with a live stream from, for example, a webcam, it can be
interesting to share the pipeline with multiple clients. This must be done when
only one instance of the video capture element can be used at a time. In this
case, the shared property of GstRTSPMedia must be used to instruct the default
GstRTSPMediaFactory implementation to cache the media.
When all objects created from a factory can be shared, you can set the shared
property directly on the factory.
* more on GstRTSPMedia
After creating the GstRTSPMedia object from the factory, it can be prepared
with gst_rtsp_media_prepare(). This method will put those objects in a
GstPipeline and will construct and link the streaming elements and the
rtpbin session manager object.
The _prepare() method will then preroll the pipeline in order to figure out the
caps on the payloaders. After the GstRTSPMedia prerolled it will be in the
prepared state and can be used for creating SDP files or for streaming to
The prepare method will also create 2 UDP ports for each stream that can be
used for sending and receiving RTP/RTCP from clients. These port numbers will
have to be negotiated with the client in the SETUP requests.
When preparing a GstRTSPMedia, an appsink and asppsrc is also constructed
for streaming the stream over TCP when requested.
Media is prepared by the server when DESCRIBE or SETUP requests are received
from the client.
* the GstRTSPClient object
When a server detects a new client connection on its port, it will accept the
connection, check if the connection is allowed and then call the vmethod
create_client. The default implementation of this function will create
a new GstRTCPClient object, will configure the session pool, mount points,
auth and thread pool objects in it.
The server will then attach the new client to a server mainloop to let it
handle further communication with the client. In RTSP it is usual to keep
the connection open between multiple RTSP requests. The client watch will
be dispatched by the server mainloop when a new GstRTSPMessage is received,
which will then be handled and a response will be sent.
The GstRTSPClient object remains alive for as long as a client has a TCP
connection open with the server. Since is possible for a client to open and close
the TCP connection between requests, we cannot store the state related
to the configured RTSP session in the GstRTSPClient object. This server state
is instead stored in the GstRTSPSession object, identified with the session
This object contains state about a specific RTSP session identified with a
session id. This state contains the configured streams and their associated
When a GstRTSPClient performs a SETUP request, the server will allocate a new
GstRTSPSession with a unique session id from the GstRTSPSessionPool. The pool
maintains a list of all existing sessions and makes sure that no session id is
used multiple times. The session id is sent to the client so that the client
can refer to its previously configured state by sending the session id in
A client will then use the session id to configure one or more
GstRTSPSessionMedia objects, identified by their url. This SessionMedia object
contains the configuration of a GstRTSPMedia and its configured
* GstRTSPSessionMedia and GstRTSPStreamTransport
A GstRTSPSessionMedia is identified by a URL and is referenced by a
GstRTSPSession. It is created as soon as a client performs a SETUP operation on
a particular URL. It will contain a link to the GstRTSPMedia object associated
with the URL along with the state of the media and the configured transports
for each of the streams in the media.
Each SETUP request performed by the client will configure a
GstRTSPStreamTransport object linked to by the GstRTSPSessionMedia structure.
It will contain the transport information needed to send this stream to the
client. The GstRTSPStreamTransport also contains a link to the GstRTSPStream
object that generates the actual data to be streamed to the client.
Note how GstRTSPMedia and GstRTSPStream (the providers of the data to
stream) are decoupled from GstRTSPSessionMedia and GstRTSPStreamTransport (the
configuration of how to send this stream to a client) in order to be able to
send the data of one GstRTSPMedia to multiple clients.
* media control
After a client has configured the transports for a GstRTSPMedia and its
GstRTSPStreams, the client can play/pause/stop the stream.
The GstRTSPMedia object was prepared in the DESCRIBE call (or during SETUP when
the client skipped the DESCRIBE request). As seen earlier, this configures a
couple of udpsink and udpsrc elements to respectively send and receive the
media to clients.
When a client performs a PLAY request, its configured destination UDP ports are
added to the GstRTSPStream target destinations, at which point data will
be sent to the client. The corresponding GstRTSPMedia object will be set to the
PLAYING state if it was not allready in order to send the data to the
The server needs to prepare an RTP-Info header field in the PLAY response,
which consists of the sequence number and the RTP timestamp of the next RTP
packet. In order to achive this, the server queries the payloaders for this
information when it prerolled the pipeline.
When a client performs a PAUSE request, the destination UDP ports are removed
from the GstRTSPStream object and the GstRTSPMedia object is set to PAUSED
if no other destinations are configured anymore.
A seek is performed when a client sends a Range header in the PLAY request.
This only works when not dealing with shared (live) streams.
The server performs a GStreamer flushing seek on the media, waits for the
pipeline to preroll again and then responds to the client after collecting the
new RTP sequence number and timestamp from the payloaders.
* session management
The server has to react to clients that suddenly disappear because of network
problems or otherwise. It needs to make sure that it can reasonable free the
resources that are used by the various objects in use for streaming when the
client appears to be gone.
Each of the GstRTSPSession objects managed by a GstRTSPSessionPool has
therefore a last_access field that contains the timestamp of when activity from
a client was last recorded.
Various ways exist to detect activity from a client:
- RTSP keepalive requests. When a client is receiving RTP data, the RTSP TCP
connection is largely unused. It is the client's responsability to
periodically send keep-alive requests over the TCP channel.
Whenever a keep-alive request is received by the server (any request that
contains a session id, usually an OPTION or GET_PARAMETER request) the
last_access of the session is updated.
- Since it is not required for a client to keep the RTSP TCP connection open
while streaming, gst-rtsp-server also detects activity from clients by
looking at the RTCP messages it receives.
When an RTCP message is received from a client, the server looks in its list
of active ports if this message originates from a known host/port pair that
is currently active in a GstRTSPSession. If this is the case, the session is
Since the server does not know anything about the port number that will be
used by the client to send RTCP, this method does not always work. Later
RTSP RFCs will include support for negotiating this port number with the
server. Most clients however use the same port number for sending and
receiving RTCP exactly for this reason.
If there was no activity in a particular session for a long time (by default 60
seconds), the application should remove the session from the pool. For this,
the application should periodically (say every 2 seconds) check if no sessions
expired and call gst_rtsp_session_pool_cleanup() to remove them.
When a session is removed from the sessionpool and its last reference is
unreffef, all related objects and media are destroyed as if a TEARDOWN happened
from the client.
A TEARDOWN request will first locate the GstRTSPSessionMedia of the URL. It
will then remove all transports from the streams, making sure that streaming
stops to the clients. It will then remove the GstRTSPSessionMedia and
GstRTSPStreamTransport objects. Finally the GstRTSPSession is released back
into the pool.
When there are no more references to the GstRTSPMedia, the media pipeline is
shut down (with _unprepare) and destroyed. This will then also destroy the
The security of the server and the policy is implemented in a GstRTSPAuth
object. The object is reponsible for:
- authenticate the user of the server.
- check if the current user is authorized to perform an operation.
For critical operations, the server will call gst_rtsp_auth_check() with
a string describing the operation which should be validated. The installed
GstRTSPAuth object is then responsible for checking if the operation is
Implementations of GstRTSPAuth objects can use the following infrastructure
bits of the rtsp server to implement these checks:
- GstRTSPToken: a generic structure describing roles and permissions granted
to a user.
- GstRTSPPermissions: a generic list of roles and matching permissions. These
can be attached to media and facties currently.
An Auth implementation will usually authenticate a user, using method such as
Basic authentication or client certificates or perhaps simply use the IP address.
The result of the authentication of the user will be a GstRTSPToken that is made
current in the context of the ongoing request.
The auth module can then implement the various checks in the server by looking
at the current token and, if needed, compare it to the required GstRTSPPermissions
of the current object.
The security is deliberately kept generic with a default implementation of the
GstRTSPAuth object providing a usable and simple implementaion. To make more
complicated security modules, the auth object should be subclassed and new
implementations for the checks needs to be made.
- Toplevel object listening for connections and creating new
- Handle RTSP Requests from connected clients. All other objects
are called by this object.
- Helper structure contaning the current state of the request
handled by the client.
- Maps a url to a GstRTSPMediaFactory implementation. The default
implementation uses a simple hashtable to map a url to a factory.
- Creates and caches GstRTSPMedia objects. The default implementation
can create GstRTSPMedia objects based on gst-launch syntax.
- Specialized GstRTSPMediaFactory that can stream the content of any
- The object that contains the media pipeline and various GstRTSPStream
objects that produce RTP packets
- Manages the elements to stream a stream of a GstRTSPMedia to one or
- Creates and manages GstRTSPSession objects identified by an id.
- An object containing the various GstRTSPSessionMedia objects managed
by this session.
- The state of a GstRTSPMedia and the configuration of a GstRTSPStream
objects. The configuration for the GstRTSPStream is stored in
- Configuration of how a GstRTSPStream is send to a particular client. It
contains the transport that was negotiated with the client in the SETUP
- helper functions for creating SDP messages from gstRTSPMedia
- a pool of multicast and unicast addresses used in streaming
- a pool of threads used for various server tasks such as handling clients and
managin media pipelines.
- Hooks for checking authorizations, all client activity will call this
object with the GstRTSPContext structure. By default it supports
- Credentials of a user. This contrains the roles that the user is allowed
to assume and other permissions or capabilities of the user.
- A list of permissions for each role. The permissions are usually attached
to objects to describe what roles have what permissions.
- object to handle get and set parameter requests.