SOME OF THE FIRST IDEAS, PRETTY OUTDATED ---------------------------------------- During the course of a discussion on IRC, it turned out that there are many possible ways to handle the capabilities. A capability is basically a set of properties attached to a mimetype in order to more closely describe the mimetype. Capabilities are supposed to be attached to pads so that the autoplugging algorithm has more specific information to connect compatible pads. We present 3 possible implementation for the capabilities. we need to pick one of them. 1. static capabilities ---------------------- When an element is created, it creates its pads like: mpg123->sinkpad = gst_pad_new ("sink", GST_PAD_SINK); gst_element_add_pad (GST_ELEMENT (mpg123), mpg123->sinkpad); mpg123->srcpad = gst_pad_new ("src", GST_PAD_SRC); gst_element_add_pad (GST_ELEMENT (mpg123), mpg123->srcpad); In the static capabilities case, it will attach a GstCaps* structure to the pad. The GstCaps structure in the above example might look like: static GstCapsFactory mpg123_sink_caps = { "audio/mp3", "layer", GST_CAPS_INT_RANGE (1, 3), "bitrate", GST_CAPS_INT_RANGE (8, 320), NULL }; with mpg123sinkcaps = gst_caps_register (mpg123_sink_caps); the factory can be converted into a GstCaps* structure. The GstCaps* structure is attached to the pad with: gst_pad_add_caps (mpg123->sinkpad, mpg123sinkcaps); The GstElement would then have a sinkpad with the given mimetype (audio/mp3) and with the capabilitities of accepting mpeg layer 1 to 3 and a bitrate from 8 up to 320 Kbps. Likewise, the src pad could be set up in the same way. An example capability factory could look like: static GstCapsFactory mpg123_src_caps = { "audio/raw", "format", GST_CAPS_BITFIELD (...), "depth", GST_CAPS_INT (16), "rate", GST_CAPS_INT_RANGE (4000, 96000), "channels", GST_CAPS_INT_RANGE (1, 2), NULL }; All GstElements would present their pads with the appropriate capabilities structure. The autoplugger would then proceed (once the source media type is known with a typefind function) in finding all the elements with compatible pads and connecting them into a pipeline. All elements of the complete pipeline could then be constructed with one single pass. No new elements should be added to the pipeline because we can figure out all the possibilities using the pad capabilities. We call this the static case because the capabilities of the pads are supposed to stay the same after creating the element. While the ability to completly setup the pipeline before actually starting playback is an advantage regarding performance, one obvious problem with this setup is that the static case may be too static in some cases. We can illustrate this with the following setup: ----------) (------------ mpg123 ! ! audiosink src sink ! ! ----------) (------------ The mpg123 element has its src capabilities set up as mpg123_src_caps in the above example. The audio renderer has its capabilities set up with the following factory: static GstCapsFactory audio_sink_caps = { "audio/raw", "format", GST_CAPS_BITFIELD (...), "depth", GST_CAPS_INT (16), "rate", GST_CAPS_INT_RANGE (22000, 44000), "channels", GST_CAPS_INT_RANGE (1, 2), NULL }; The static autoplugger has to be carefull when connecting the mpg123 element with the audiosink because it is theoretically possible that the mpg123 element outputs raw audio with a rate that cannot be handled by the audiosink (ex. 4000KHz). In the absence of another audiosink with more capabilities, the autoplugging of this simple pipeline will not be possible and would fail. the autoplugging algorithm would probably select another element to insert between the mpg123 element and the audiosink in order to handle the (uncommon) case of a rate conversion (audioscaler). It is clear that this static setup might even fail or work suboptimal for even the common case and should therefore be considered as too restrictive. 2. dynamic capabilities ----------------------- The idea of dynamic capabilities is that the capabilities are not set at element create time but rather while the pipeline is running. An element would still list its mime type using: gst_pad_add_type_id(mpg123->sinkpad, mp3type); The idea would then be that a rough draft of the pipeline would be built afer the media type of the stream has been detected with the typefind functions. The rough draft would consist of laying out a global plan to reach the renderer(s). this plan would basically list the set of conversions that have to be performed. (mime-type to mime-type conversion). Elements that accept the src mime-type are tried by giving it a buffer. If the element accepts the buffer, it will set its capabilities for both the sink pad and the src pad. At that time other elements can be tried and added to the src pad, until we reach the renderer. As usual one has to be carefull to add just the minimum amount of elements to reach the renderer. The global plan will help with that. Since we basically do not use the capabilities of the sink pad one has to question the need for sink pad capabilities in the first place. We might also have a hard time trying different elements until we find a compatible one that does not cause a dead end at some point. 3. combined setup ----------------- This combined setup will minimise the effort needed to try different elements encountered by option 2 while still allowing a more dynamic setup based on the actual media stream we are handling. The combined setup will list/add the sink capabilities at create time. It will only set the mime-type of its src pads. As with option2, a global plan will be built. At runtime the src pads will actually specify the capabilities they need for any element that wants to be connected to its source pads. In this case we specify the capabilities for all the sink pads of an element at create time. The capabilities of the src pads would only become available when data has been processed by the element. The autoplugger would then be able to choose an element that can handle the capability listed by the src pad. in our previous example: ----------) (------------ mpg123 ! ! audiosink src sink ! ! ----------) (------------ the audiosink element would specify its sink pad capabilities at create time, while the mpg123 elements src pad would not yet have any capabilities set. When data is handled by the mpg123 element, a capability would be added to the mpg123 src pad. This capability might be: static GstCapsFactory mpg123_src_caps = { "audio/raw", "format", GST_CAPS_INT (S16), "depth", GST_CAPS_INT (16), "rate", GST_CAPS_INT (44000), "channels", GST_CAPS_INT (2), NULL }; This capability would be compatible with the audiosinks sinkpad capabilities and the autoplugger would therefore be able to connect the two elements. While allowing a more flexible setup with option3, compared to option1, we introduce a slightly higher overhead when we need to dynamically connect elements. This overhead will not be as big as option2 because we don't have to 'try' elements. so: src caps: added at runtime to list the caps needed for an element that wants to connect to this pad. sink caps: the (static) capabilities that this sinkpad has.