Summary
Currently only one extension that renders in the video playback area can be active at any one time. This proposal provides broadcasters the ability to have multiple extensions present in the video playback area. Summary:
- Create a third kind of extensions called “Component Extensions” (CEs)
- CEs occupy smaller portions of the video area
- CEs can be hidden by the viewer
Motivation
Because video overlay extensions can render to the entire playback area, there are a number of technical and UX challenges to enabling multiple such extensions in a stack. Extensions are not aware of each other, and are not required to provide a viewer-customizable or positionable user interface. Therefore it would be very easy for multiple full-player video overlays to unintentionally interfere with each other and result in a poor user experience.
However, restricting users to a single video overlay creates a significant adoption problem; smaller extensions that might increase engagement for a broadcaster’s channel will be overlooked in favor of bigger, “Swiss army knife” type extensions that broadcasters have come to rely on and are unwilling to give up.
To solve both of these problems simultaneously, we propose creating Component Extensions, which will be smaller than the full video area, can be locally hidden by the viewer, and allow more than one extension to be active at a time.
Detailed design
CEs will appear in the current extension manager alongside existing panel and overlay extensions. Future plans to separate the extension manager from the extension discovery process will include component extensions as a category that can be searched for and filtered on.
The below descriptions assume that the CEs are being viewed on a desktop web browser; additional considerations for mobile are beyond the scope of this document and will be determined when a complete mobile design is created.
Rendering style
CEs, like other extensions, are realized as an HTML iframe. Therefore they will occupy a rectangular portion of the video player, although they can be partially transparent to effectively have an arbitrary outline.
Each active CE will have an associated icon that appears at the bottom of the video player; that icon will present a popup menu allowing the user to take actions related to that specific extension (e.g., providing their true Twitch identity).
Each CE’s popup menu will contain a visibility toggle switch. When a viewer chooses to make a CE invisible, it will be hidden from view (but will continue to receive network updates as though it were visible). It can be restored by toggling the visibility switch again.
CE minification status will not be preserved across reloads, in accordance with the general principle that the broadcaster should ultimately have control over the appearance of their channel. We do not intend minification of CEs to be a way for viewers to “disable” an extension in even a semi-permanent way.
Broadcaster Configuration
Upon activation of a CE, the broadcaster will be presented with a modal dialog that shows a mock video and the current position and name of any active CEs (including the one currently being activated) as solid rectangles. Off-limits portions of the video will be clearly highlighted.
Broadcasters can drag the rectangular CE proxies around to select their initial position. Overlap between multiple CEs is not permitted.
Broadcasters will have a maximum of two active CEs at a time.
Developer Setup
Adding CEs to extensions will require some additional fields in the extension setup portion of the developer site.
Specifying Sizing
Developers will specify a required aspect ratio for their CE, as well as a sizing constraint that will control the size of the CE relative to the interactive portion of the video player. The size control is a single percentage which controls the length of the longer edge of the CE relative to the corresponding edge of the interactable portion of the video player. In the event that the CE is square, the height shall be (arbitrarily) considered the longer edge.
As a concrete example, suppose that an extension wishes to have a 3x4 aspect ratio, and wants the height (the long edge) to occupy 15% of the interactable portion of the video player. The developer would specify the following parameters (detailed below) as part of the extension setup:
| Field | Value |
|---|---|
| Aspect Width | 3 |
| Aspect Height | 4 |
| Size | .15 |
If the interactable area of the video changes size, either because the browser window is resized, or the player is changed to/from fullscreen mode, the extension will resize itself accordingly.
To provide a consistent rendering paradigm, Twitch will (by default) apply a CSS zoom style to the CE body which will allow the extension developer to treat their iframe as being 1024 pixels long along the longer edge (this default size can be overridden by the developer as well). If this behavior is undesirable, the developer can disable it by specifying a false value for the Zoom parameter.
Additional fields added to extension setup
New CE-specific fields. These fields are only required if “component” is provided as an anchor type.
| Field | Type | Description |
|---|---|---|
| Aspect Width | Integer | Specifies width, in “units” (see “specifying sizing” above), of the CE. |
| Aspect Height | Integer | Specifies height, in “units” (see “specifying sizing” above), of the CE. |
| Size | Integer | Size of the long edge of the CE, relative to the length of the corresponding interactable dimension of the video (see “specifying sizing”, above). |
| Zoom | Boolean (optional) | Specify whether CSS zooming should be applied to the CE container, creating a consistent size of 1024 pixels along the long edge of the component. Default true |
| Zoom Pixels | Integer (optional) | If “Zoom” is set to true, specify the effective CSS zoom to apply to the long edge of the CE. Default 1024. |
Modifications to existing fields for extension setup
changes highlighted in bold italic:
| Field | Type | Description |
|---|---|---|
| Anchor | String | Specifies the “type” of extension. Add component as a valid anchor. |
Rollout Considerations
Does the addition of component extensions constitute a breaking change?
No, this is functionally additive. All existing extensions and extension-related systems should continue to function as before.
Initial Beta
Approximately a month before intended release, we will make component extensions available to create in the developer site and our backend, but CEs will be prevented from review submission. This will give developers a period of time to build and test their CEs in advance of the anticipated feature launch.
During this time, the broadcaster configuration and activation user interface will likely be temporary, incomplete, and subject to rapid change, but it should be sufficient for developers to enable their CEs while in testing mode. None of the broadcaster CE configuration UI will be visible to anyone who does not have access to a CE, and since CEs will not be reviewable during this time, that should limit visibility only to registered developers and whitelisted broadcasters for newly developed CEs.
Alternatives
Multiple full-screen overlays
Our initial instinct was to support the activation of multiple overlays as currently deployed. There are two significant issues with this approach:
- Routing input events through multiple stacked iframes proved technically infeasible. We experimented with capturing all events in a transparent overlay above all extensions and re-synthesizing them ourselves, but this approach can only go so far; synthetic “
mousemove” events do not trigger, for example, CSS “:hover” states, crippling best-practices web design. - Extensions are not required to implement viewer repositioning, and extensions are not aware of each other’s presence. Therefore there is no way to ensure that an extension’s user interface might not interfere with the interface of the one behind it.
Quadrant based video overlays
We investigated breaking the usable space in the video overlay into quadrants and allowing extensions to specify which quadrants it resides in. When evaluating this option, we looked at the overlay extensions in use today and found that this implementation would represent a breaking change for nearly all of them, forcing complete front end redesigns or encouraging the continued paradigm of a single overlay owning all four quadrants to function.
Continue supporting only one video overlay
The current state of affairs has driven almost all viewers to use one of a very small number of video overlay extensions. A few overlay extensions have become nearly ubiquitous, making it nearly impossible for developers to create smaller, more specialized extensions that broadcasters might want but are unwilling to abandon their favorite overlay for. Multiple video player extensions is therefore considered a high priority roadmap item.