docs/design/: Add doc about synchronisation

Original commit message from CVS:
* docs/design/Makefile.am:
* docs/design/part-synchronisation.txt:
Add doc about synchronisation
* docs/design/draft-latency.txt:
* docs/design/part-TODO.txt:
* docs/design/part-clocks.txt:
* docs/design/part-events.txt:
* docs/design/part-gstbus.txt:
* docs/design/part-gstpipeline.txt:
* docs/design/part-live-source.txt:
* docs/design/part-messages.txt:
* docs/design/part-overview.txt:
* docs/design/part-streams.txt:
* docs/design/part-trickmodes.txt:
Documentation updates.

13 files changed, 260 insertions, 127 deletions

diff --git a/docs/design/Makefile.am b/docs/design/Makefile.am
index 72c61d7b51..a2fdb3f28e 100644
--- a/docs/design/Makefile.am
+++ b/docs/design/Makefile.am
@@ -39,6 +39,7 @@ EXTRA_DIST = \
 	part-standards.txt \
 	part-states.txt \
 	part-streams.txt \
+	part-synchronisation.txt \
 	part-TODO.txt \
 	part-trickmodes.txt
 
diff --git a/docs/design/draft-latency.txt b/docs/design/draft-latency.txt
index 583e213394..82e6342131 100644
--- a/docs/design/draft-latency.txt
+++ b/docs/design/draft-latency.txt
@@ -199,26 +199,21 @@ that no sink is set to PLAYING before it is prerolled.
 
 In order to do this, the pipeline (at the GstBin level) keeps track of all
 elements that require preroll (the ones that return ASYNC from the state
-change). It keeps a custom GstBinState GST_MESSAGE_ELEMENT internally for
-those elements.
+change). These elements posted a ASYNC_START message without a matching
+ASYNC_DONE message.
 
-When the pipeline did not receive a NO_PREROLL state change return from any
-element, it can forget about the GstBinState messages because the state change
-to PLAYING will proceed when all elements commited their state when they are
-prerolled automatically.
+The pipeline will not change the state of the elements that are still doing an
+ASYNC state change.
 
-When the pipeline received a NO_PREROLL state change return from an element, it
-keeps the ELEMENT messages. The pipeline will not change the state of the
-elements that are still doing an ASYNC state change.
+When an ASYNC element prerolls, it commits its state to PAUSED and posts an
+ASYNC_DONE message. The pipeline notices this ASYNC_DONE message and matches it
+with the ASYNC_START message it cached for the corresponding element. 
 
-When an ASYNC element prerolls, it commits its state to PAUSED and posts a
-PREROLLED message. The pipeline notices this PREROLLED message and matches it
-with the GstBinState message it cached for the corresponding element. 
+When all ASYNC_START messages are matched with an ASYNC_DONE message, the
+pipeline proceeds with setting the elements to the final state again.
 
-When all GstBinState messages are matched with a PREROLLED message, the
-pipeline proceeds with setting the PREROLLED sinks to their pending state.
 The base time of the element was already set by the pipeline when it changed the
-nopreroll element to PLAYING. This operation has to be performed in the
+NO_PREROLL element to PLAYING. This operation has to be performed in the
 separate async state change thread (like the one currently used for going from
 PAUSED->PLAYING in a non-live pipeline).
 
@@ -235,7 +230,7 @@ Latency compensation
 As an extension to the revised state changes we can perform latency calculation
 and compensation before we proceed to the PLAYING state. 
 
-When the pipeline collected all PREROLLED messages it can calculate the global
+When the pipeline collected all ASYNC_DONE messages it can calculate the global
 latency as follows:
 
   - perform a latency query on all sinks.
@@ -262,11 +257,12 @@ ex2:
   MIN (50, 40) = 40 >= 33 -> latency = 33
 
 The latency is set on the pipeline by sending a LATENCY event to the sinks
-that posted the PREROLLED message. This event configures the total latency on
-the sinks. The sink forwards this LATENCY event upstream so that
-intermediate elements can configure themselves as well.
+in the pipeline. This event configures the total latency on the sinks. The
+sink forwards this LATENCY event upstream so that intermediate elements can
+configure themselves as well.
 
-After this step, the pipeline continues setting the pending state on the sinks.
+After this step, the pipeline continues setting the pending state on its
+elements.
 
 A sink adds the latency value, received in the LATENCY event, to
 the times used for synchronizing against the clock. This will effectively
@@ -289,23 +285,19 @@ A flush in a pipeline can happen in the following cases:
  - flush preformed by an element
     - after receiving a navigation event (DVD, ...)
 
-When a playing sink is flushed by a FLUSH_START event, a LOST_PREROLL message is
-posted by the element. As part of the message, the state of the element is
-included. The element also goes to a pending PAUSED state.
+When a playing sink is flushed by a FLUSH_START event, an ASYNC_START  message is
+posted by the element. As part of the message, the fact that the element got
+flushes is included. The element also goes to a pending PAUSED state and has to
+be set to the PLAYING state again later.
 
-The message is kept in a GstBinState message by the parent bin. When the element
-prerolls, it posts a PREROLLED message. 
+The ASYNC_START message is kept by the parent bin. When the element prerolls,
+it posts an ASYNC_DONE message. 
 
-When all LOST_PREROLL messages are matched with a PREROLLED message, the bin
-will capture a new base time from the clock and will bring all the prerolled
-sinks back to PLAYING (or whatever their state was when they posted the
-LOST_PREROLL message) after setting the new base time on them. It's also possible
+When all ASYNC_START messages are matched with an ASYNC_DONE message, the bin
+will capture a new base_time from the clock and will bring all the sinks back to
+PLAYING after setting the new base time on them. It's also possible
 to perform additional latency calculations and adjustments before doing this. 
 
-The difference with the NEED_PREROLL/PREROLLED and LOST_PREROLL/PREROLLED
-message pair is that the latter makes the pipeline acquire a new base time for
-the PREROLLED elements.
-
 
 Dynamically adjusting latency
 -----------------------------
diff --git a/docs/design/part-TODO.txt b/docs/design/part-TODO.txt
index 01ec91a963..e2d0b48226 100644
--- a/docs/design/part-TODO.txt
+++ b/docs/design/part-TODO.txt
@@ -22,9 +22,6 @@ API/ABI
 - use | instead of + as divider in serialization of Flags
   (gstvalue/gststructure)
 
-- with the addition the PREROLLED message, we can probably get rid of the
-  STATE_DIRTY message.
-
 
 IMPLEMENTATION
 --------------
diff --git a/docs/design/part-clocks.txt b/docs/design/part-clocks.txt
index c5e157c282..032ca3c208 100644
--- a/docs/design/part-clocks.txt
+++ b/docs/design/part-clocks.txt
@@ -7,7 +7,7 @@ implementation but time is always expessed in nanoseconds. Since the
 baseline of the clock is undefined, the clock time returned is not
 meaningfull in itself, what matters are the deltas between two clock
 times.
-The time reported by the clock is called the absolute time.
+The time reported by the clock is called the absolute_time.
 
 
 Clock Selection
@@ -30,68 +30,8 @@ CLOCK_LOST message is posted. The application must then set the pipeline to
 PAUSED and PLAYING again in order to let the pipeline select a new clock
 and distribute a new base time.
 
-
-Time in GStreamer
------------------
-
-The absolute time is used to calculate the running time. The running time
-is defined as follows:
-
- - If the pipeline is NULL/READY, the running time is undefined.
- - In PAUSED, the running time remains at the time when it was last 
-   PAUSED. When the stream is PAUSED for the first time, the running time
-   is 0. 
- - In PLAYING, the running time is the delta between the absolute time
-   and the base time. The base time is defined as the absolute time minus
-   the running time at the time when the pipeline is set to PLAYING.
- - after a flushing seek, the running time is set to 0 (see part-seeking.txt)
-
-The running time is completely managed by the GstPipeline object using the
-GstClock absolute time. It basically represents the elapsed time that the
-pipeline spend in the PLAYING state.
-
-
-Timestamps
-----------
-
-NOTE: this info is inaccurate, see part-synchronisation.txt for how the clock
-time is used to synchronize buffer timestamps.
-
-The combination of the last NEWSEGMENT event and the buffer timestamps 
-express the presentation stream time of the buffer. The stream time
-of a buffer is calculated as follows:
-
- ST = TS - DT        where: TS = buffer timestamp
-                            DT = NEWSEGMENT timestamp
-			    ST = buffer stream time
-
-The reason for not making the buffer times express the stream time directly
-is for the following reasons:
-
- - demuxers are easier if they can just copy the timestamps as encoded in
-   the file. The initial NEWSEGMENT event would contain the lowest timestamp in
-   the stream which makes the buffer stream time start from 0.
- - pipelines requiring retimestamping of buffers can efficiently adjust
-   the timestamp in the NEWSEGMENT events and have all buffers retimestamped
-   automatically.
- - resync after different kinds of seeks is easier.
-
-If an element wants to synchronize a buffer to the clock it needs to first
-calculate the buffer stream time and then bring the stream time to the
-absolute clock time.
-
-Converting a timestamp (in stream time) to absolute time is performed using
-the following formula:
-
-    AT = BT + ST               where: AT = absolute time
-                                      BT = base time
-                                      ST = stream time
-
-The pipeline base time is propagated to all the element during the PAUSED
-to PLAYING state change. All elements are therefore able to convert the 
-stream time to the absolute time. It is possible to specify an aditional
-delay to the base time to compensate for the delay it takes to perform
-the state change using the GstPipeline "delay" property.
+The clock selection is performed as part of the state change from PAUSED to
+PLAYING and is described in part-states.txt.
 
 
 Clock features
diff --git a/docs/design/part-events.txt b/docs/design/part-events.txt
index 8f2e64ff58..ad2672dc95 100644
--- a/docs/design/part-events.txt
+++ b/docs/design/part-events.txt
@@ -242,7 +242,7 @@ contains a single GstClockTime with the required latency. The latency value is
 calculated by the pipeline and distributed to all sink elements before they are
 set to PLAYING. The sinks will add the configured latency value to the
 timestamps of the buffer in order to delay their presentation.
-See also part-latency.txt.
+(See also part-latency.txt).
 
 
 DRAIN
diff --git a/docs/design/part-gstbus.txt b/docs/design/part-gstbus.txt
index 61c73069c9..e246605c8a 100644
--- a/docs/design/part-gstbus.txt
+++ b/docs/design/part-gstbus.txt
@@ -7,8 +7,9 @@ a first-in first-out way from the streaming threads to the application.
 Since the application typically only wants to deal with delivery of these
 messages from one thread, the GstBus will marshall the messages between
 different threads. This is important since the actual streaming of media
-is done in another thread than the application. It is also important to not
-block the streaming thread while the application deals with the message.
+is done in another threads (streaming threads) than the application. It is
+also important to not block the streaming threads while the application deals
+with the message.
 
 The GstBus provides support for GSource based notifications. This makes it
 possible to handle the delivery in the glib mainloop. Different GSources 
@@ -21,7 +22,7 @@ posted message.
 The bus can be polled with the gst_bus_poll() method. This methods blocks
 up to the specified timeout value until one of the specified messages types
 is posted on the bus. The application can then _pop() the messages from the
-bus to handle them.
+bus to handle them. 
 
 It is also possible to get messages from the bus without any thread 
 marshalling with the gst_bus_set_sync_handler() method. This makes it
diff --git a/docs/design/part-gstpipeline.txt b/docs/design/part-gstpipeline.txt
index 8bb5c098e6..7a0064578a 100644
--- a/docs/design/part-gstpipeline.txt
+++ b/docs/design/part-gstpipeline.txt
@@ -7,8 +7,10 @@ children with a clock.
 A GstPipeline also provides a toplevel GstBus (see part-gstbus.txt)
 
 The pipeline also calculates the running_time based on the selected
-clock (see part-clocks.txt).
+clock (see also clocks.txt and part-synchronisation.txt).
 
+The pipeline will calculate a global latency for the elements in the pipeline.
+(See also part-latency.txt).
 
 State changes
 -------------
@@ -25,6 +27,7 @@ GstBin, the pipeline performs the following actions during a state change:
  - PAUSED -> PLAYING:
     - Select and a clock.
     - calculate base_time using the running_time.
+    - calculate and distribute latency.
     - set clock and base_time on all elements before performing the
       state change.
 
@@ -35,7 +38,7 @@ GstBin, the pipeline performs the following actions during a state change:
     - set the bus to flushing (when auto-flushing is enabled)
 
 The running_time represents the total elapsed time, measured in clock units,
-that the pipeline spent in the PLAYING state.
+that the pipeline spent in the PLAYING state (see part-synchronisation.txt).
 
 
 Clock selection
@@ -43,7 +46,7 @@ Clock selection
 
 Since all of the children of a GstPipeline must use the same clock, the
 pipeline must select a clock. This clock selection happens when the pipeline
-gopes to the PLAYING state.
+goes to the PLAYING state.
 
 The default clock selection algorithm works as follows:
 
@@ -61,7 +64,7 @@ The default clock selection algorithm works as follows:
       a more upstream element.
       
  - use GstSystemClock, this only happens when no element provides a
-   clock.
+   usable clock.
 
 The application can influence this clock selection with two methods:
 gst_pipeline_use_clock() and gst_pipeline_auto_clock(). 
@@ -74,3 +77,11 @@ possible.
 The _auto_clock() method removes the fixed clock and reactivates the auto-
 matic clock selection algorithm described above.
 
+
+GstBus
+------
+
+A GstPipeline provides a GstBus to the application. The bus can be retrieved
+with gst_pipeline_get_bus() and can then be used to retrieve messages posted by
+the elements in the pipeline (see part-gstbus.txt).
+
diff --git a/docs/design/part-live-source.txt b/docs/design/part-live-source.txt
index ada16606ad..59691334fe 100644
--- a/docs/design/part-live-source.txt
+++ b/docs/design/part-live-source.txt
@@ -15,7 +15,8 @@ since such a buffer might never arrive.
 
 Live sources return NO_PREROLL when going to the PAUSED state to inform the
 bin/pipeline that this element will not be able to produce data in the
-PAUSED state.
+PAUSED state. NO_PREROLL should be returned for both READY->PAUSED and
+PLAYING->PAUSED.
 
 When performing a get_state() on a bin with a non-zero timeout value, the
 bin must be sure that there are no live sources in the pipeline because else
diff --git a/docs/design/part-messages.txt b/docs/design/part-messages.txt
index 904f4daaba..d8ebd1d325 100644
--- a/docs/design/part-messages.txt
+++ b/docs/design/part-messages.txt
@@ -10,7 +10,7 @@ requiring an API change while allowing a wide range of different types
 of messages.
 
 Messages are posted by objects in the pipeline and are passed to the 
-application using the GstBus. 
+application using the GstBus (See also part-gstbus.txt and part-gstpipeline.txt). 
 
 
 Message types
@@ -112,14 +112,14 @@ Message types
 
     An element posts this message when it has detected or updated the stream duration.
 
-  GST_MESSAGE_LOST_PREROLL:
+  GST_MESSAGE_ASYNC_START:
 
-    Posted by sinks when they lose their preroll buffer in the PAUSED or PLAYING
-    state caused by a FLUSH_START event.
+    Posted by sinks when they start an asynchronous state change.
 
-  GST_MESSAGE_PREROLLED:
+  GST_MESSAGE_ASYNC_DONE:
 
-    Posted by sinks when they receive the first data buffer.
+    Posted by sinks when they receive the first data buffer and complete the
+    asynchronous state change.
 
   GST_MESSAGE_LATENCY:
 
diff --git a/docs/design/part-overview.txt b/docs/design/part-overview.txt
index 5e700a3583..0a3cf47db8 100644
--- a/docs/design/part-overview.txt
+++ b/docs/design/part-overview.txt
@@ -166,6 +166,7 @@ Pipeline
    - Manage running_time based on the selected clock. Running_time is the elapsed
      time the pipeline spent in the PLAYING state and is used for
      synchronisation.
+   - Manage latency in the pipeline.
    - Provide means for elements to comunicate with the application by the GstBus.
    - Manage the global state of the elements such as Errors and end-of-stream.
  
@@ -365,7 +366,8 @@ Pipeline states
 
  Before going to PLAYING, the pipeline select a clock and samples the current time of 
  the clock. This is the base_time. It then distributes this time to all elements. 
- Elements can then synchronize against the clock using the buffer timestamp+base time. 
+ Elements can then synchronize against the clock using the buffer running_time +
+ base_time (See also part-synchronisation.txt). 
 
  The following chain of state changes then takes place:
 
@@ -385,7 +387,7 @@ Pipeline status
 
  The bus is distributed to all elements added to the pipeline. The elements use the bus
  to post messages on. Various message types exist such as ERRORS, WARNINGS, EOS, 
- STATE_CHANGED,  etc..
+ STATE_CHANGED, etc..
 
  The pipeline handles EOS messages received from elements in a special way. It will
  only forward the message to the application when all sink elements have posted an
@@ -412,7 +414,8 @@ Pipeline EOS
  The EOS event will eventually arrive in the sink element. The sink will then post
  an EOS message on the bus to inform the pipeline that a particular stream has
  finished. When all sinks have reported EOS, the pipeline forwards the EOS message
- to the application. 
+ to the application. The EOS message is only forwarded to the application in the
+ PLAYING state.
 
  When in EOS, the pipeline remains in the PLAYING state, it is the applications
  responsability to PAUSE or READY the pipeline. The application can also issue
diff --git a/docs/design/part-streams.txt b/docs/design/part-streams.txt
index 65048b0474..7a21e9982a 100644
--- a/docs/design/part-streams.txt
+++ b/docs/design/part-streams.txt
@@ -34,21 +34,31 @@ Typical stream
 
   1) NEW_SEGMENT, rate, start/stop, time
      - marks valid buffer timestamp range (start, stop)
-     - marks stream_time of buffers (time)
-     - marks playback rate (rate)
-     - marks applied rate (applied_rate)
+     - marks stream_time of buffers (time). This is the stream time of buffers
+       with a timestamp of NS.start.
+     - marks playback rate (rate). This is the required playback rate.
+     - marks applied rate (applied_rate). This is the already applied playback
+       rate. (See also part-trickmodes.txt)
 
   2) N buffers
-     - displayable buffers are between start/stop of the NEW_SEGMENT
+     - displayable buffers are between start/stop of the NEW_SEGMENT. Buffers
+       outside the segment range should be dropped or clipped.
 
-     - display_time: (B.timestamp - NS.start) * NS.abs_rate
-        * used to calculate stream_time and sync_time
+     - running_time: 
+     
+         if (NS.rate > 0.0)
+           running_time = (B.timestamp - NS.start) / NS.abs_rate + NS.accum
+         else
+           running_time = (NS.stop - B.timestamp) / NS.abs_rate + NS.accum
 
-     - stream_time:  display_time + NS.time
-        * current position in stream between 0 and duration
+        * used to synchronize against the clock (See also
+	  part-synchronisation.txt).
 
-     - sync_time:    display_time + NS.accum + base_time 
-        * used to synchronize against the clock.
+     - stream_time:
+
+         stream_time = (B.timestamp - NS.start) * NS.abs_applied_rate + NS.time
+
+        * current position in stream between 0 and duration.
 
   3) EOS
      - marks the end of data, nothing is to be expected after EOS
diff --git a/docs/design/part-synchronisation.txt b/docs/design/part-synchronisation.txt
new file mode 100644
index 0000000000..918d13cb2b
--- /dev/null
+++ b/docs/design/part-synchronisation.txt
@@ -0,0 +1,176 @@
+Synchronisation
+---------------
+
+This document outlines the techniques used for doing synchronised playback of
+multiple streams.
+
+Synchronisation in a GstPipeline is achieved using the following 3 components:
+
+ - a GstClock, which is global for all elements in a GstPipeline.
+ - Timestamps on a GstBuffer.
+ - the NEW_SEGMENT event preceeding the buffers.
+
+
+A GstClock
+----------
+
+This object provides a counter that represents the current time in nanoseconds.
+This value is called the absolute_time.
+
+Different sources exist for this counter:
+
+ - the systemtime (with g_get_current_time() and with microsend accuracy)
+ - an audio device (based on number of samples played)
+ - a network source based on packets received + timestamps in those packets (a
+   typical example is an RTP source)
+ - ...
+
+In GStreamer any element can provide a GstClock object that can be used in the
+pipeline. The GstPipeline object will select a clock from all the providers and
+will distribute it to all other elements (see part-gstpipeline.txt).
+
+A GstClock always counts time upwards and does not necessarily start at 0.
+
+
+Running time
+------------
+
+After a pipeline selected a clock it will maintain the running_time based on the
+selected clock. This running_time represents the total time spent in the PLAYING
+state and is calculated as follows:
+
+ - If the pipeline is NULL/READY, the running_time is undefined.
+ - In PAUSED, the running_time remains at the time when it was last
+   PAUSED. When the stream is PAUSED for the first time, the running_time
+   is 0.
+ - In PLAYING, the running_time is the delta between the absolute_time
+   and the base time. The base time is defined as the absolute_time minus
+   the running_time at the time when the pipeline is set to PLAYING.
+ - after a flushing seek, the running_time is set to 0 (see part-seeking.txt).
+   This is accomplished by redistributing a new base_time to the elements that
+   got flushed.
+
+This algorithm captures the running_time when the pipeline is set from PLAYING
+to PAUSED and restores this time based on the current absolute_time when going
+back to PLAYING. This allows for both clocks that progress when in the PAUSED
+state (systemclock) and clocks that don't (audioclock).
+
+The clock and pipeline now provides a running_time to all elements that want to
+perform synchronisation. Indeed, the running time can be observed in each
+element (during the PLAYING state) as:
+  
+  running_time = absolute_time - base_time
+
+
+Timestamps
+----------
+ 
+The GstBuffer timestamps and the preceeding NEW_SEGMENT event (See
+part-streams.txt) define a transformation of the buffers to running_time as
+follows:
+
+The following notation is used:
+
+ B: GstBuffer 
+  - B.timestamp = buffer timestamp (GST_BUFFER_TIMESTAMP)
+
+ NS:  NEWSEGMENT event preceeding the buffers.
+  - NS.start: start field in the NEWSEGMENT event
+  - NS.rate: rate field of NEWSEGMENT event
+  - NS.abs_rate: absolute value of rate field of NEWSEGMENT event
+  - NS.time: time field in the NEWSEGMENT event
+  - NS.accum: total accumulated time of all previous segments
+
+Valid buffers for synchronisation are those with B.timestamp between NS.start
+and NS.stop. All other buffers outside this range should be dropped or clipped
+to these boundaries (see also part-segments.txt).
+
+The following transformation to running_time exist:
+
+    if (NS.rate > 0.0)
+      running_time = (B.timestamp - NS.start) / NS.abs_rate + NS.accum
+    else
+      running_time = (NS.stop - B.timestamp) / NS.abs_rate + NS.accum
+
+The first displayable buffer will yield a value of 0 (since B.timestamp ==
+NS.start and NS.accum == 0).
+
+For NS.rate > 1.0, the timestamps will be scaled down to increase the playback
+rate. Likewise, a rate between 0.0 and 1.0 will slow down playback.
+
+For negative rates, timestamps are received stop NS.stop to NS.start so that the
+first buffer received will be transformed into running_time of 0 (B.timestamp ==
+NS.stop and NS.accum == 0).
+
+
+Synchronisation
+---------------
+
+As we have seen, we can get a running_time:
+
+ - using the clock and the element's base_time with:
+
+    C.running_time = absolute_time - base_time
+
+ - using the buffer timestamp and the preceeding NEWSEGMENT event as (assuming
+   positive playback rate):
+
+    B.running_time = (B.timestamp - NS.start) / NS.abs_rate + NS.accum
+
+We prefix C. and B. before the two running times to note how they were
+calculated.
+
+The task of synchronized playback is to make sure that we play be buffer with
+B.running_time at the moment when the clock reaches the same C.running_time.
+
+Thus the following must hold:
+
+   B.running_time = C.running_time
+
+expaning:
+
+   B.running_time = absolute_time - base_time
+
+or:
+
+   absolute_time = B.running_time + base_time
+
+The absolute_time when a buffer with B.running_time should be played is noted
+with B.sync_time. Thus:
+
+  B.sync_time = B.running_time + base_time
+
+One then waits for the clock to reach B.sync_time before rendering the buffer in
+the sink (See also part-clocks.txt).
+
+For multiple streams this means that buffers with the same running_time are to
+be displayed at the same time. 
+
+A demuxer must make sure that the NEWSEGMENT it emits on its output pads yield
+the same running_time for buffers that should be played synchronized. This
+usually means sending the same NEWSEGMENT on all pads and making sure that the
+synchronized buffers have the same timestamps.
+
+
+Stream time
+-----------
+
+The stream time is also known as the position in the stream and is a value
+between 0 and the total duration of the media file.
+
+It is the stream time that is used for:
+
+  - report the POSITION query in the pipeline
+  - the position used in seek queries
+  - the position used to synchronize controller values
+
+Stream time is calculated using the buffer times and the preceeding NEWSEGMENT
+event as follows:
+
+    stream_time = (B.timestamp - NS.start) * NS.abs_applied_rate + NS.time
+ 
+For negative rates, B.timestamp will go backwards from NS.stop to NS.start,
+making the stream time go backwards.
+
+Note that the stream time is never used for synchronisation against the clock.
+
diff --git a/docs/design/part-trickmodes.txt b/docs/design/part-trickmodes.txt
index 253550b610..a53e7c66ee 100644
--- a/docs/design/part-trickmodes.txt
+++ b/docs/design/part-trickmodes.txt
@@ -199,3 +199,4 @@ Notes:
  - backwards playback potentially uses a lot of memory as frames and undecoded
    data gets buffered.
 
+