The A/V Control object
Support in HbbTV
Available since: HbbTV 1.0 (ETSI TS 102 796 V1.1.1, OIPF DAE V1.1)
See clause A.2.5 of the present document.
The reference to the <object> element being defined in clause 4.8.4 of the HTML5 specification is revised to be clause 4.7.4.
An onPlaySpeedChanged event shall be generated for all calls to the play() method regardless of the value returned by the method call and whether the play speed changes or not.
In the present document, the allocationMethod property is not included but requirements for DYNAMIC_ALLOCATION shall apply.
The transitions in the state diagram of Figure 19 shall be mandatory.
Comment
Security: Trusted (applies to downloaded content)
A.2.5 Extensions to the A/V Control object
A.2.5.1 New queue method
The following method shall be added to the A/V Control embedded object.
| Boolean queue(String url) | ||
| Description | Queue the media referred to by url for playback after the current media item has finished playing. If a media item is already queued, url will not be queued for playback and this method will return false. If the item is queued successfully, this method returns true. If no media is currently playing, the queued item will be played immediately.
If url is null, any currently queued item will be removed from the queue and this method will return true. If an A/V Control object is an audio object (as defined by clause 7.14 of the OIPF DAE specification [1]) then queued media items shall only contain audio. Otherwise, if an A/V Control object is a video object | then queued media items shall always contain video and may also contain audio and other media components. When the current media item has finished playing, the A/V Control object shall transition to the finished state, update the value of the data property with the URL of the queued media item and automatically start playback of the queued media item. The A/V Control object may transition to the connecting or buffering states before entering the playing state when the queued media item is being presented. Implementations may pre-buffer data from the queued URL before the current media item has finished playing in order to reduce the delay between items. Play speed is not affected by transitioning between the current and queued media item. To avoid race conditions when queueing multiple items for playback, applications should wait for the currently queued item to begin playback before queuing subsequent items, e.g. by queueing the subsequent item when the A/V Control object transitions to the connecting, buffering or playing state for the currently queued item. |
|
| Arguments | url | The media item to be queued, or null to remove the currently-queued item. |
Calling stop(),modifying the data and/or type property or entering the error state shall cause any queued media item to be discarded.
Play control keys (OK, play, stop, pause, fast forward, fast rewind and other trick play keys) shall not be handled by the A/V Control object and no action shall be taken by the terminal for these keys when they have been requested by an application. DOM 2 events shall be generated for these keys whether the A/V Control object is focused or not.
The timing of automatic transitions from the error state to the stopped state is implementation dependent; applications should not rely on the A/V Control object remaining in the error state after an error has occurred and should listen for play state change events in order to detect errors.
If the AVControl object’s play() method returns true then at least one play state change event shall be generated.
The error property shall be available in the stopped state. After an automatic transition from the error state to the stopped state, the value of the error property shall be preserved.
The following value shall be added to the list of valid values for the error property:
- undefined – no error has occurred;
- 7 – content blocked due to parental control.
A.2.5.2 State machine and related changes
This clause describes a set of changes to the state machine for the A/V Control object defined in clause 7.14.1.1 of the OIPF DAE specification [1]:
- An A/V Control object with a CSS rule of display:none shall not be loaded and hence shall not be decoding audio or video.
A.2.5.3 Support for TTML subtitles
The following extensions shall apply to support TTML subtitles as defined by clause 7.3.1.5. The <object> element of an A/V Control object shall contain a <param> element for each subtitle component that is carried out-of-band.
The attributes of the <param> element shall have the following values.
| Attribute name | Attribute value | ||||||||
| name | “subtitles” or “captions” | ||||||||
| value | Shall contain a list of key:value tuples separated by space.
The following keys shall be supported:
|
The terminal shall support the component selection API defined in clause 7.14.4.1 of OIPF DAE [1] for in-band and out-of-band delivered TTML subtitles as defined in clause 7.3.1.5. The following mappings for the properties of the AVSubtitleComponent class as defined in clause A.2.13 shall apply.
| property | Value |
| encoding | Shall have the value “application/ttml+xml” |
| language | The value defined by the ‘srclang’ key used in the <param> element. |
| hearingImpaired | Shall be true if the value of the ‘name’ attribute is ‘captions’, false otherwise. |
| label | The value defined by the ‘label’ key used in the <param> element. |
EXAMPLE:
<object type=’video/mp4′ data=’http://mycdn.de/video.mp4′>
<param name=’subtitles ‘ value=’srclang:de src: http%3A%2F%2Fmycdn.de%2Fsubtitles_de.ttml’ />
<param name=’captions ‘ value=’srclang:en src:
http%3A%2F%2Fmycdn.de%2Fsubtitles_hearing_impaired.ttml’ />
A.2.5.4 Support for media synchronization with subtitle-only streams
The media synchronization capabilities of the terminal as defined in clause 10.2.8 allow for delivery of subtitles as broadband streams that are re-synchronized in the terminal with another stream, either broadcast or broadband.
The terminal shall support the rendering of subtitles when carried in a separate subtitle-only stream using the A/V Control object if the type of the subtitle-only stream is supported by the terminal for media synchronization. The requirements on terminals for supporting subtitle-only streams are defined in clause 10.2.8.4. In any case, the following restrictions shall apply:
- For DVB Subtitles:
- the video component is provided by the broadcast service;
- the type attribute of the A/V Control object is “video/mpeg”;
- the subtitle stream is delivered with a Content Type of “image/vnd.dvb.subtitle”; and
- the subtitle stream is a single programme transport stream.
- For TTML subtitles:
- the video component is provided by the broadcast service;
- either:
- the type attribute of the A/V Control object is “application/ttml+xml”;
- the subtitle stream is delivered with a Content Type of “application/ttml+xml”; and
- the subtitle stream is a single XML document.
- or:
- the subtitle stream is delivered using MPEG-DASH; and
- multi-stream media synchronization is used, as specified in clause 10.2.8.
The terminal shall render the subtitle stream on the subtitles plane with scaling and positioning defined by the subtitle object element and not by any video element that it is synchronized with.
A.2.5.5 Using an A/V Control object to play downloaded content
Clause 7.14.1.3 of the OIPF DAE specification [1], “Using an A/V Control object to play downloaded content” shall be modified by the addition of the text shown underlined.
If an A/V Control object is used to play content that has been downloaded and stored on the OITF (by using method setSource() as defined in clause 7.14.7 of the OIPF DAE specification [1]) then the following holds:
- if the download was triggered using registerDownloadURL() with a contentType other than “application/vnd.oipf.ContentAccessDownload+xml” or the download was triggered using a Content Access Download Descriptor with <TransferType> value “playable_download” as defined in clause E.1, then:
- if the play() method is called before sufficient data has been download to initiate playback, then the play state of the A/V Control object SHALL be set to 6 (‘error’) with a detailed error code of 5 (“content not available”).
A.2.5.6 Extension to PlayStateChange event
In clause 7.14.1 of the OIPF DAE specification [1], the description of the ‘onPlayStateChange’ property is replaced with the following:
- The function that is called when the play state of the A/V control object object changes for any reason.
- The specified function shall be called with the argument state. This argument is defined as follows:
- Number state – the new state of the A/V control object. Valid values are given in the definition of the playState property [Req. 5.7.1.f].
A.2.5.7 Other modifications to the A/V Control object object
In clause 7.14.3.1 of the OIPF DAE specification [1], the definition of the property onPlayPositionChanged( Integer position ) is changed as shown:
The function that is called when change occurs in the play position of a channel due to the use of trick play functions random access.
NOTE: Most of the text in this section was found in Annex B of previous versions of this specification. It has been moved here and re-structured for readability. No additional text has been included from CEA-2014-A.
The HTML object element (defined in section 4.8.4 of the HTML5 specification as referenced in [OIPF_DAE2_WEB]) SHALL support presentation of video and audio or just audio. This presentation SHALL be indicated by setting the type attribute to one of the “video/” or “audio/” MIME types that is listed in section 3 of [OIPF_MEDIA2] and corresponds to a combination of system layer format, video format and audio format that is supported by the OITF.
When an HTML object is presenting video or audio as defined in this section, the following requirements shall apply:
- HTML objects presenting video and/or audio SHALL be accessible through the DOM and SHALL support the properties and methods defined by the tables below in addition to those of HTMLObjectElement as defined in the HTML5 specification as referenced in [OIPF_DAE2_WEB].
| Property type and name | Property Definition |
| String data | String data [RW] – media URL. If the value of data is changed while media is playing playback is stopped (resulting in a play state change). The default value is the empty string. If the value of this attribute is changed, the related data-attribute inside the DOM tree SHOULD be changed accordingly. If the value of this attribute is set to an empty string or is changed, the resources (files, server connections, etc…) currently owned by the object SHALL be released. The value set in this property MAY include a temporal fragment interval according to section 4.2.1 of [Media Fragments URI] in which case the derived begin time and end time SHALL serve as bounds for playback. The Normal Play Time format SHALL be used. The begin time SHALL behave as start-of-media and the end time SHALL behave as end-of-media. If the value of temporal fragment interval is changed then there will be no change in the play state unless the interval is changed to period outside of the current play position. |
| readonly Number playPosition | The play position in number of milliseconds since the beginning as denoted by the server (i.e. in relation to NPT 0.0 as described in section 3.6 of [RFC2326]) of the media referenced by attribute data when data refers to a single media item. If the play position cannot be determined, the playPosition SHALL be undefined.. |
| readonly Number playTime | The estimated total duration in milliseconds of the media referenced by data when data refers to a single media item. If the duration of the media cannot be determined, the playTime SHALL be undefined. |
| readonly Number playState | Indication of the current play state as follows: 0 – stopped; user (or script) has stopped playback of the current media, or playback has not yet started. 1 – playing; the current media pointed to by data is currently playing. 2 – paused; the current media pointed to by data has been paused. 3 – connecting; connect to media server, i.e. waiting for connection to media server to be established, upon first connection or after the connection was lost. In addition, DRM rights necessary for playback of protected content are also retrieved during this state. 4 – buffering; the buffer is being filled in order to have sufficient data available to initiate or continue playback. In this state, playback is stalled due to insufficient data in the buffer to continue playback. The player waits until sufficient data has been buffered to continue playback. For video objects, whilst being in this state, the player SHOULD show the last completed video frame that was shown before entering this state. This playstate is an intermediate state to reach playState 1 (‘playing’). The OITF SHOULD buffer the content in the background whilst in playState 2 (‘paused’). However, this background buffering does not result into a state change to state 4. 5 – finished; the playback of the current media has reached the end of the media. 6 – error; an error occurred during media playback, preventing the current media to start/continue playing. |
| readonly Number error | Error details; only significant if the value of playState equals 6: 0 – A/V format not supported. 1 – cannot connect to server or connection lost. 2 – unidentified error. 3 – insufficient resources. 4 – content corrupt or invalid. 5 – content not available. 6 – content not available at given position. |
| readonly Number speed | Play speed relative to real-time as defined by 5.7.1.f.6 of [CEA-2014-A]. |
| Object onPlayStateChange | DOM-0 event handler called when the value of the playState property changes as defined by 5.7.1.f.9 of [CEA-2014-A]. |
Table 9: Properties of the A/V Control Object when the type attribute refers to video or audio
| Property type and name | Property Definition |
| String width | The width of the area used for rendering the video object. This property is only writable if property fullScreen has value false. The effect of changes to width SHALL be in accordance with requirement 5.7.1.c of [CEA-2014-A]. |
| String height | The height of the area used for rendering the video object. This property is only writable if property fullScreen has value false. The effect of changes to height SHALL be in accordance with requirement 5.7.1.c of [CEA-2014-A]. |
| readonly Boolean fullScreen | Indicates whether an object presenting video is in full screen mode or not – as defined in by 5.7.1.g.3 of [CEA-2014-A]. |
| Object onFullScreenChange | DOM-0 event handler called when the value of fullScreen changes as defined in by 5.7.1.g.4 of [CEA-2014-A]. |
| Object onfocus | DOM-0 event handler called when the object gains focus as defined by 5.7.1.g.7 of [CEA-2014-A]. |
| Object onblur | DOM-0 event handler called when the object loses focus as defined by 5.7.1.g.8 of [CEA-2014-A]. |
Table 10: Additional Properties of the A/V Control Object when the type attribute refers to video
| Method signature | Method Definition |
| Boolean play(Number speed) | Plays the media referenced by data, starting at the current play position denoted by playPosition, at the supported speed closest to the value of attribute speed. Negative speeds reverse playback. If no speed is specified, it defaults to 1. A speed of 0 will pause playback. This method SHALL always return true. If the playback reached the beginning of the media at rewind playback speed, then the play state SHALL be changed to 2 (‘paused’). A play speed event (see section 7.14.3.2 of this specification) SHALL be generated when the operation has completed, regardless of the new play speed. If the play speed is not changed, the argument of the event SHALL be set to the previous play speed. |
| Boolean stop() | Stops playback and resets the playPosition to 0 as defined by 5.7.1.f.12 of [CEA-2014-A]. |
| Boolean seek(Number pos) | If seek() is called while the player is in state 1 (“playing”), then it sets the current play position (in milliseconds) to the value of pos and MAY change play state to 4 (‘buffering’). If the player is in state 2 (‘paused’), then the seek() method seeks to the new position, but the play state and the rendered image is not changed. If the player is in states 0 (“stopped”), 5 (“finished”) or 6 (“error”), then the new play position SHALL be retained and SHALL be used (if possible) as the starting position for playing back the content item indicated by the data property when the play() method is called. NOTE: changing the content item resets the play position to the beginning of the new content item. If the player is in states 3 (“connecting”) or 4 (“buffering”) then the seek() method seeks to the new play position and MAY change play state to 3 (“connecting”). If the new playback position is valid, the value of the playPosition attribute SHALL be set to the new value before this method returns. Returns true if the method succeeded, and false otherwise. A play position event (see section 7.14.3.2 of this specification) will be generated when the operation has completed, regardless of the success of the operation. If the operation fails, the argument of the event SHALL be set to the previous play position. |
| Boolean setVolume(Number volume) | Sets the audio volume as defined by 5.7.1.f.14 of [CEA-2014-A]. |
Table 11: Methods of the A/V Control Object when the type attribute refers to video or audio
| Method signature | Method Definition |
| setFullScreen(Boolean fullscreen) | Sets the object to full screen mode or windowed mode as defined by 5.7.1.g.5 of [CEA-2014-A]. |
| focus() | Sets the input focus to this object as defined by 5.7.1.g.6 of [CEA-2014-A]. |
Table 12: Additional Methods of the A/V Control Object when the type attribute refers to video
- In addition to the properties and methods listed above, the following table lists other requirements from CEA-2014-A that SHALL also apply for the A/V Control object as defined in this section.
| CEA-2014 Requirement | Summary |
| 5.7.1.a.3 | Indication of the initial aspect ratio of the video content. |
| 5.7.1.b.3 | Access to both video and audio objects by name/id through the HTMLDocument interface. |
| 5.7.1.c | Full-screen or windowed mode for video objects. |
| 5.7.1.d | CSS properties that apply to video objects. |
| 5.7.1.e | CSS z-index, opacity and RGBA colour values. NOTE: In this specification, the value of the “<overlay>” element is never “none” and hence z-index and opacity are required to be supported. |
Table 13: Additional applicable requirements from CEA-2014
State diagram for A/V Control objects
The following state transition diagram SHOULD be used for an A/V Control object:
Figure 16: State diagram for embedded A/V Control objects (normative)
The following clarifications apply:
- A detailed description for all the states in this state diagram is given above in the definition of the playState property.
- If the value of the allocationMethod property is DYNAMIC_ALLOCATION the following SHALL apply:
- Scarce resources for playback using the A/V Control object, such as the MPEG decoder, are claimed during state 3 (‘connecting’), state 4 (‘buffering’) or during state transitions from state 3 (‘connecting’) to state 4 (‘buffering’), from state 4 (‘buffering’) to state 1 (‘playing’) or from state 0 (‘stopped’) or from state 3 (‘connecting’) to state 2 (‘paused’).
- If at any point in time during playback the scarce resources are not available anymore, due to a resource conflict, then the play state of the A/V Control object SHALL be set to 6 (‘error’) with a detailed error code of 3 (‘insufficient resources’).
- Scarce resources for playback using the A/V Control object SHALL be released when state 6 (‘error’) or 0 (‘stopped’) or 5 (‘finished’) are reached.
- In addition, if the A/V Control object gets destroyed, e.g. because another URL is loaded into the containing window, scarce resources claimed for playback using the A/V Control object SHALL be released, except in cases described for the optional persist property of A/V Control objects.
- When the data attribute and/or the type attribute of the HTMLObjectElement representing the A/V Control object is given a different value, the object SHALL go to state 0 (‘stopped’).
- For playback of DRM protected content, the rights for playback are retrieved during state 3 (‘connecting’).
- If the play position reaches the end of the available content the A/V Control object SHALL be set to state 5 (‘finished’) in addition to generating a playback speed change of zero.
- If there is an attempt to play() with a speed in the positive direction (forward or > 0) and there is no content available then the request fails.
- If the play position reaches the beginning of the available content the A/V Control object SHALL be set to state 2 (‘paused’) in addition to generating a playback speed change of zero.
- If there is an attempt to play() with a speed in the negative direction (rewind or < 0) and there is no content available then the request fails.
- If seek() is performed beyond the available content the request is rejected and the current playout is maintained.
- When a A/V Control object stops being rendered as defined in section 10.1 of the HTML5 specification as referenced by [OIPF_DAE2_WEB] an OITF MAY release scarce resources allocated for that object. Vice versa, an A/V Control object which is not visible but is still being rendered SHALL still be decoding video if it is in the playing state and any audio associated with the currently playing media will still be audible. State transitions caused by calls to methods on the A/V Control object, or due to permanent or transient errors, will occur as shown above regardless of the visibility of the object. Section 4.4.4 describes the effect on scarce resources when an A/V Control object is removed from the DOM tree.
- NOTE: as implied by the text above, rendering state and visibility are not equivalent. This implies, just to give two examples, that display:none will affect the object state while visibility:hidden will not. When an A/V Control object is destroyed (e.g. by the A/V Control object being garbage collected, or because of a page transition within the application), presentation of streamed audio or video shall be terminated.
- When not presenting video, the A/V Control object SHALL be rendered as an opaque black rectangle.
Using an A/V Control object to play streaming content
If an A/V Control object is used to play streamed content using either RTSP or HTTP the OITF then the following holds:
- play(0) is called in state 0 (‘stopped’), the A/V Control object SHALL automatically go to play state 2 (‘paused’). The necessary resources are secured and no external signalling is performed.
- play(0) is called in the connecting or buffering state, the A/V Control object SHALL automatically go to play state 2 (‘paused’)
- play() is called in the paused state with an argument other than 0, the A/V Control object SHALL transition to one of the following states as follows:
- connection to the server, the A/V Control object SHALL transition to the connecting state.
- connection to the server but insufficient content is buffered to resume playback immediately, the A/V Control object SHALL transition to the buffering state.
- connection to the server and sufficient content is buffered to resume playback immediately, the A/V Control object SHALL transition to the playing state.
Using an A/V Control object to play downloaded content
If an A/V Control object is used to play content that has been downloaded and stored on the OITF (by using method setSource() as defined in section 7.14.7) then the following holds:
- if the download was triggered using registerDownloadURL or the download was triggered using a Content Access Download Descriptor with <TransferType> value “playable_download” as defined in Annex E.1, then:
- if the play() method is called before sufficient data has been download to initiate playback, then the play state of the A/V Control object SHALL be set to 6 (‘error’) with a detailed error code of 5 (“content not available”).
- if the downloaded content was triggered using a Content Access Download Descriptor with <TransferType> value “full_download” as defined in Annex E.1, then:
- if the play() method is called whilst the content is still downloading and has not yet successfully completed, then the play state of the A/V Control object SHALL be set to 6 (‘error’) with a detailed error code of 5 (“content not available”).
Using an A/V Control object to play recorded content
If an A/V Control object is used to play content that has been recorded or is being recorded on the OITF (by using method setSource() as defined in section 7.14.7) then the following holds:
- if the play() method is called before sufficient data has been recorded to initiate playback, then the play state of the A/V Control object SHALL be set to 6 (‘error’) with a detailed error code of 5 (“content not available”).
Using the A/V Control object to play content fragments
If the OITF indicates support through the temporalClipping capability indicator (see section 9.3.22) for playing content fragments then it SHALL support the Media Fragment URI [Media Fragments URI] according to section 8.3.1.
User Input and the A/V Control object
If an A/V Control object has input focus:
- The OITF SHALL not block execution of scripts of the document from which the focus was moved to the video object, even when the video is playing full-screen and has input focus.
- The OITF SHALL NOT handle the VK_OK, VK_PLAY, VK_PAUSE, VK_PLAY_PAUSE, VK_STOP, VK_FAST_FWD, VK_REWIND, VK_NEXT or VK_PREV keys.