docs/design/part-trickmodes.txt: More documentation on trickmodes.

Original commit message from CVS:
* docs/design/part-trickmodes.txt:
More documentation on trickmodes.

1 files changed, 163 insertions, 4 deletions

diff --git a/docs/design/part-trickmodes.txt b/docs/design/part-trickmodes.txt
index 69c2e7c98e..7bda266b29 100644
--- a/docs/design/part-trickmodes.txt
+++ b/docs/design/part-trickmodes.txt
@@ -1,13 +1,172 @@
 Trickmodes
 ==========
 
+GStreamer provides API for performing various trickmode playback. This includes:
 
-forward playback 
-----------------
+ - server side trickmodes
+ - client side fast/slow forward playback
+ - client side fast/slow backwards playback
 
+Server side trickmodes mean that a source (network source) can provide a
+stream with different playback speed and direction. The client does not have to
+perform any special algorithms to decode this stream.
 
+Client side trickmodes mean that the decoding client (GStreamer) performs the
+needed algorithms to change the direction and speed of the media file.
 
+Seeking can both be done in a playback pipeline and a transcoding pipeline.
 
 
-backwards playback 
-------------------
+General seeking overview
+------------------------
+
+Consider a typical playback pipeline:
+
+                          .---------.  .------.
+              .-------.   | decoder |->| sink |  
+  .--------.  |       |-->'---------'  '------'
+  | source |->| demux |
+  '--------'  |       |-->.---------.  .------.
+              '-------'   | decoder |->| sink |
+                          '---------'  '------'
+
+The pipeline is initially configured to play back at speed 1.0 starting from 
+position 0 and stopping at the total duration of the file. 
+
+When performing a seek, the following steps have to be taken by the application:
+
+ 1) Create a seek event:
+
+    The seek event contains:
+
+    - various flags describing:
+      - where to seek to (KEY_UNIT)
+      - how accurate the seek should be (ACCURATE)
+      - how to perform the seek (FLUSH) 
+      - what to do when the stop position is reached (SEGMENT).
+    - a format to seek in, this can be time, bytes, units (frames, samples), ...
+    - a playback rate, 1.0 is normal playback speed, positive values bigger than 1.0
+      mean fast playback. negative values mean reverse playback. A playback speed of
+      0.0 is not allowed.
+    - a start position, this value has to be between 0 and the total duration of the 
+      file. It can also be relative to the previously configured start value.
+    - a stop position, this value has to be between 0 and the total duration. It can
+      also be relative to the previously configured stop value.
+
+    See also gst_event_new_seek().
+
+
+ 2) Send the seek event to the pipeline with gst_element_send_event()
+
+    By default the pipeline will send the event to all sink elements.
+    By default an element will forward the event upstream on all sinkpads.
+    Elements can modify the format of the seek event. The most common format is
+    GST_FORMAT_TIME.
+
+    One element will actually perform the seek, this is usually the demuxer or
+    source element. For more information on how to perform the different seek
+    types see part-seeking.txt.
+
+    For client side trickmode a NEW_SEGMENT event will be sent downstream with
+    the new rate and start/stop positions. All elements prepare themselves to
+    handle the rate (see below).
+
+    for server side trick mode a NEW_SEGMENT event is sent downstream with a
+    rate of 1.0 and the start/stop positions. The elements will configure themselves
+    for normal playback speed since the server will perform the rate conversions.
+
+    When the seek succeeds, the _send_event() function will return TRUE.
+
+
+Server side trickmode
+---------------------
+
+The source element operates in push mode. It can reopen a server connection requesting
+a new byte or time position and a new playback speed. The capabilities can be queried
+from the server when the connection is opened.
+
+We assume the source element is derived from the GstPushSrc base class. The base source
+should be configured with gst_base_src_set_format (src, GST_FORMAT_TIME). 
+
+The do_seek method will be called on the push src subclass with the seek information
+passed in the GstSegment argument.
+
+The rate value in the segment should be used to reopen the connection to the server
+requesting data at the new speed and possibly a new playback position.
+
+When the server connection was successfully reopened, set the rate of the segment
+to 1.0 so that the client side trickmode is not enabled. Alternatively a combination
+of client side and serverside trickmode can be used, for example if the server does
+not support certain rates, the client can perform rate conversion for the remainder.
+
+
+               source               server
+   do_seek       |                     |
+     ----------->|                     |
+                 |  reopen connection  |
+                 |-------------------->|
+                 |                     .
+                 |   success           .
+                 |<--------------------|
+       modify    |                     |
+     rate to 1.0 |                     |
+                 |                     |
+      return     |                     |
+       TRUE      |                     |
+                 |                     |
+ 
+After performing the seek, the source will inform the downstream elements of the
+new segment that is to be played back. Since the segment will have a rate of 1.0,
+no client side trick modes are enabled.
+
+
+client side forward trickmodes 
+------------------------------
+
+The seek happens as stated above. a NEW_SEGMENT event is sent downstream with a rate
+different from 1.0. Plugins receiving the NEW_SEGMENT can decide to perform the 
+rate conversion of the media data (retimestamp video frames, resample audio, ...).
+A plugin can also decide to drop frames in the case of fast playback or use a more
+efficient decoding algorithm (skip B frames, ...). If a plugin decides to resample
+or retimestamp, it should modify the NEW_SEGMENT with a rate of 1.0 so that 
+downstream elements don't resample again.
+
+The GStreamer base audio and video sinks will resample automatically if they receive
+a NEW_SEGMENT event with a rate different from 1.0.
+
+
+client side backwards trickmode 
+-------------------------------
+
+For backwards playback the following rules apply:
+
+ - the rate in the NEW_SEGMENT is less than 0.0.
+ - the NEW_SEGMENT start position is less than the stop position, playback will
+   however happen from stop to start in reverse.
+
+For plugins the following rules apply:
+   
+ - A source plugin sends data in chunks starting from the last chunk of the file. 
+   The actual bytes are not reversed.
+
+ - A demuxer accumulates the chunks. As soon as a keyframe is found, everything
+   starting from the keyframe up to the accumulated data is sent downstream.
+   Timestamps on the buffers are set starting from the stop position to start,
+   effectively going backwards.
+
+ - A decoder decodes and accumulates all frames. If a new keyframe is received, 
+   all accumulated frames are sent backwards. 
+
+ - A sink reverses (for audio) and retimestamps (audio, video) the buffers before
+   playing them back. Retimestamping occurs relative to the stop position, making
+   the timestamps increase again and suitable for synchronizing against the clock.
+
+ - for transcoding, audio and video resamplers can be used to reverse, resample and
+   retimestamp the buffers.
+
+Notes:
+
+ - The clock keeps running forward.
+ - backwards playback potentially uses a lot of memory as frames and undecoded data
+   get buffered.
+