RFC 0012 - Component Extension Redesign

[foilstormcrow]

Update: Please read the announcement for these changes in production which includes a guide for updating your existing component Extensions.

Summary

Twitch is redesigning the component Extension anchor in the video player in order to address usability issues and create a more consistent developer, creator, and viewer experience. We are going to be implementing a design that fixes the location of component Extensions to the side of the player, initializes them in a collapsed state, and only allows for a single component to be displayed at a time. This redesign changes the interaction model of component Extensions, and we highly encourage developers to update the user interfaces of their component Extensions accordingly. We will be rolling out these changes in early access to developers, so that they can start building within the new framework and be ready for when the design update rolls out to viewers.

Motivation

Since we launched component Extensions last year, feedback from viewers, creators, and developers have indicated to us that the experience of creating and interacting with component Extensions is confusing and hard to predict. Some of the pain points we have discovered are:

• Viewers struggle to understand what elements of a stream are – and aren’t – Extensions.
• Viewers have difficulty comprehending an Extension’s purpose or how to interact with it.
• Extension positioning is different from channel to channel.
• Broadcasters are unsure how Extensions will appear on their stream and how viewers will be able to interact with them.

While the ability to use multiple Extensions in the video was one of our most requested features following the launch of Extensions in 2017, at the end of 2018 only 12% of channels using a video Extension had more than one active simultaneously.

These challenges led us to explore alternative implementations that can make the video Extension experience more consistent and approachable across channels. Our goal with this design is to create an easy to understand and easy to use model for Extensions in the video player across all channels. We believe that by providing a more consistent and intuitive experience across Twitch, viewer engagement and broadcaster adoption of Extensions will increase, allowing Extensions to create more value for both viewers and broadcasters.

Proposed Solution

To solve these problems, we are proposing a “Extensions sidebar” design with all component Extensions hidden by default on channel load. Components will be expandable by clicking their icon in the Extensions sidebar, which is presented with a location and styling that is more prominent than the existing Extension icons in the video player’s bottom controls. Components will always display in the same location when opened – connected to the Extensions sidebar. A component can be closed by clicking on its icon again or on a standardized close button outside of the iframe. Extension information and functionality like identity linking will be available through a “cog” menu outside of the iframe when the Extension is open. A viewer will only be able to view a single component Extension at a time. Clicking a new component will automatically collapse the currently open component if there is one.

Overlay Extension functionality will remain largely unchanged and will still be visible by default on channel load. Overlay Extensions may need to account for the new non-interactive region created by the Extensions sidebar in their design.

On player controls hover

Tooltip for hovering on an Extension

An open component Extension

The overlay Extension menu

The full component Extension experience

What Changes Do I Need To Make To My Extension?

The new design eliminates the need for Component Extensions to have a “collapsed” or “minimized” state, as all components will collapse into the Extensions sidebar when they are hidden. It also no longer requires that Extensions present a “hovered” state, as it is expected that whenever a user’s cursor is over an Extension frame that they have intent to interact with it already. As a result of this, it will be expected that component Extensions utilize all of their allocated space when visible. Component Extensions that do not meet this requirement will be rejected during review.

An example of a “collapsed” state that should no longer be used by component Extensions

Component Extensions will no longer need to provide a “Close” button that collapses the Extension via the minimize() function, as this will be provided as native UI in the Twitch player.

This “X” button will no longer need to be part of the component, close functionality is now handled by the standardized header bar.

The way the sizing of a component Extension is defined will also be changing. Extensions are currently sized with height and width values that are expressed as percentages of the corresponding player dimension. This proved challenging to understand as it was hard to conceptualize how the values for height and width related to each other. We are proposing the following new scheme for specifying the size of a component:

Field Name	Description
Target Height	The desired height of the Extension expressed as a percentage of the available space in the player. “Available” space here means the space between the top and bottom controls in the player, minus the height of the new component top nav.
Aspect Ratio	A set of 2 integers (Width and Height) representing the desired aspect ratio of the Extension

For example: a component with a target height of 25% and an aspect ratio of 4:3 will render at a height of 250px and a width of 333px when the available vertical space is 1000px.

Component Extensions will always maintain the given aspect ratio and will attempt to maintain the target height provided, except in cases where the player is either extremely large or small. In those cases, they will either stop scaling with the player or be hidden respectively. The function of the “Zoom” and “Zoom Pixels” options will remain unchanged.

We will also be adding a new field to the Extension manifest titled “Viewer Description”. This field should be a short summary that explains to a viewer what your Extension does (in contrast with the existing summary and description fields, which are to be directed more at broadcasters). This description will be displayed to viewers in the Extension details menu. This field will be required for all future versions of Extensions after we launch this redesign. Existing Extensions that have not added a viewer description will fall back to displaying the existing broadcaster summary field.

The update_user_extensions API endpoint will no longer require the x and y parameters for activating component Extensions. If these parameters are included, they will simply be ignored.

Lastly, we are removing the expectation in our guidelines that Extensions in video player anchors don’t perform any background processing while their visibility is false. Given that component Extensions now will go through the initialization process while hidden and will be expected to display up-to-date UI immediately upon being shown, we feel that this policy is too restrictive. Note that we still expect Extensions in mobile views to continue to minimize background processing when hidden.

Rollout Considerations

Migration Plan and Timeline

Phase 1: Design Query Param + Dev Rig (May 15)

We will provide component Extensions with the ability to know which design they are running in with a URL query parameter legacyComponentDesign that will be passed to the Extension frame. If this parameter is present, developers should render their Extensions as they have up until now. If not, the Extension will be expected to follow the new design recommendations outlined above.

In addition to this, we will be updating the Developer Rig with a new view type for the new Component Extension design, so developers can begin testing their UI updates. The legacy Component design will be passed the legacyComponentDesign param within the Rig view as outlined above. The ability to use the new sizing scheme in your Extension manifest will not yet be available, but you will be able to specify a size for your component in the “Component (New)” view and have your component render with those values rather than the size specified in your manifest.

Phase 2: Beta Twitch Web Client + New Size Fields in Manifest (June 15 June 19)

After we roll out the Rig changes, we will allow developers to opt in to a beta version of the component Extension redesign. This will allow developers to test the migrated versions of their Extensions on the website itself. The details of how to opt in will be presented closer to this date.

This will pair with the ability to specify the size of your Extension using the new size description values in your Extension’s manifest. For a period, we will require both to be specified, and users of the production client will still see your Extension sized in terms of the old “Aspect Height” and “Aspect Width” definitions while you are able to test with the new sizing scheme in the beta client. All existing Extensions’ initial values for the new sizing scheme will be pre-set with values that match their original size as closely as possible.

Once you have updated and tested the new UI you display when the legacyComponentDesign parameter is not present, you can release your new Extension version at any time. As long as your Extension handles the parameter properly, your Extension will display your new UI when we go to launch the redesign.

Phase 3: Launch (July 15 Late July)

After the end of the two-month beta period, the new design will be released to all users. After this point, no Extensions will receive the legacyComponentDesign flag, and it will be safe to remove any logic that depends on it and deprecate Extension UIs designed to work with the old anchor design. We will also be removing the legacy design from the Developer Rig at this time. Any Extensions that have not been updated to comply with the new design policies 30 days after launch may have component support disabled until they are updated.

Drawbacks

Component Extensions now no longer display to viewers by default

We recognize that in this new framework, components will have a decreased ability to show information to viewers who take no action (by default). However, our research indicates that the consistency and ease of understanding created by the Extensions sidebar will improve overall comprehension and engagement with component Extensions in the long term. Preliminary user testing has shown us that viewers better understanding where to look for and how to interact with component Extensions when they are presented in a consistent and standardized way, generating higher overall engagement with valuable Extension experiences.

Additionally, by more clearly differentiating the functionality and use cases for overlay and component Extensions, developers can create more effective Extension experiences for each anchor and broadcasters can better understand how an Extension will change the content on their page, increasing overall willingness to use components on their channel.

Alternatives

Alternative above-the fold anchor point

As part of our research, we realized one of the primary motivations for having an extension in the video player is so that users can discover it without scrolling down into the channel’s panels section. We briefly considered creating a new anchor point that exists “above the fold” and outside of the video player, but as we researched this path, we quickly discovered the importance of continuing to have these extension experiences travel with the video and into watching modes like theater and fullscreen.

Dynamically resizing iframes

One of the major pain points we are attempting to address is viewers not understanding when a component Extension in a “collapsed” state was still overlapping with elements of an overlay Extension. We considered granting developers the ability to dynamically size a component’s frame so that this problem of invisible iframes wouldn’t block interaction with other Extension elements. We decided against this approach because of the added complexity to developers in specifying the size of their component Extensions, and because it still failed to address the problem of many extensions being unrecognizable by viewers. We believe that for component extensions to be successful, they need a consistent experience across channels in order for viewers to be able to identify a glance where to find extensions and what they add to a stream.

We appreciate and welcome any feedback or questions you might have about these changes via comments in this thread. Watch the Twitch Developers website for further updates. We will be making announcements there as well as in this thread as we roll out the various stages of this redesign.

[Breci]

Removing all the “buttons” to open extension is a great thing, it will make the streaming experience cleaner when using extensions.
But at the same time, it’s removing the interactive possibilities for extensions that don’t need to be displayed on all the player but should be there all the time.

Here are my concerns :

On my extension Live Requests I do have a notification system since the streamer can accept the viewers’ requests and then the viewer has to send the bits for it.
So I have a notification system to alert the viewer when he needs to send the bits. This system is also used to engage viewers, inform them when the streamer start something, inform them when the streamer finish their request so they can ask a new one.

What would be possible with this RFC to keep a similar system? I don’t want to send private messages to the viewers because it is intrusive and not interactive. Extension chat messages are not an option since some messages are meant to be displayed to one person only.
Part of it could be solved with RFC0011, but not the notification system.
So do you plan on having a notification system, where we can still send per viewer messages to inform them that something happened and they have to interact with the extension?

My second Extension Quests is meant to be an interactive overlay and to be displayed all the time to newcomers to quickly inform them about what the streamer is doing.
With this RFC I would have to switch to a video overlay extension. Which is way overkill for this extension since I only need 20% of the width and 50% of the height of the video player.

Is it planned to have a “displayed by default” extension?
Or a potential solution would be to still have the video components system as it is, but to only one of them at the time, like a video overlay but the video component extension would have much stricter rules in term of design UI/UX. For example, these extensions should not have a button to click to open them.
And we could still add the extension sidebar for extensions that previously required a button to open

Also some other questions:
If a viewer opens an extension and moves the mouse out of the player, will the extension still be displayed?
Will the sidebar always be on the right? Or can the streamers set it on the left, bottom, top?

[AlbericoD]

look, this is definitely a problem I encountered several times, I solved this with CSS style classes to hide and make transition effects, I want to understand the following:

when the extension is minimized, will it be a disassembled component or just a hidden component?
I say this for loading reasons on configuration objects in the channel and / or external API calls

@edit
I forgot to comment that I would like to participate in this beta, really my extension fits very much into this problem of hiding and appearing a component, one of my challenges and making my extension something almost natural to use

my id hecb122wgtrzumrv9ywwjn7wg6nglq, Battle Royale - Jump Poll

[Dist]

With the changes to component sizing, and the fact that only a single component can be visible at any one time, does this mean that potentially a component can now use 100% of the player space (eg, 100% Target Height with a 16:9 aspect ratio)?

While I don’t have plans to use the entire player space with a component, something I have been requested of is to create an extension with minimal height (~10%) but full width, for things that take the form of a progress bar, or a real-time line graph, but none of this has been possible with the current system due to the max 50% width.

[Breci]

To complete my previous answer, I mentionned the use of notification to inform viewers when an action is required from them or something happen in the extension

I made a quick gif of a suggestion of how it could be integrated with this RFC, using the same design

The idea is to have all notification from all extension at the same place. Only one can be displayed at the same time.
A notification would stay visible for a few seconds.

The notification would be sent from a method :

 twitch.ext.actions.sendNotification(content:string)

The streamer will have the possibility to mark the extension as “waiting for an action” when an action is required from the viewer, or something happened and the viewer should check it.
It would be possible using this method :

twitch.ext.actions.setActionRequired()

This methods will not return anything, like the “showBitsBalance” .
This would make an indication appear next to the extension logo to tell the viewer he can do something on this extension. For example the purple dot. Once the viewer click on the extension logo, the dot disapear.
If the extension is already open the dot doesn’t appears.

With this, we could notify viewers without breaking the UI/UX.

This system could also be later extended to other Twitch notifications like subgift, drops, etc

[Dist]

I love the idea of a helper function that is able to toggle a little indicator next to the extension icon, as it is minimally invasive (which IMO for things like notifications is essential, anything that takes away from the viewing experience is a detriment to the overall stream and having a simple coloured dot next to the extension is as minimal as you can get) and extension devs could easily have a setting to disable triggering it should the user not wish to have notifications.

Actual notification messages I’m not as sure about. There’s certainly a use for it but then there’s the question of is a single area of the screen, with static styling, suitable for all channels/extensions? There’s also the issue of what should be the default? I think the default viewer experience should be ‘Just the stream’, and that things like notifications should be opt-in. This could be tied to identity sharing so agreeing to that enables the notifications for that extension (with the option to still hide them if they wish) but even then there may be extensions that don’t require identity sharing but do want notifications.

[foilstormcrow]

look, this is definitely a problem I encountered several times, I solved this with CSS style classes to hide and make transition effects, I want to understand the following:

when the extension is minimized, will it be a disassembled component or just a hidden component?
I say this for loading reasons on configuration objects in the channel and / or external API calls

@edit
I forgot to comment that I would like to participate in this beta, really my extension fits very much into this problem of hiding and appearing a component, one of my challenges and making my extension something almost natural to use

The extension will only be hidden when it is collapsed. Its iframe will still be running and you will continue to receive context, configuration, etc.

All developers will be able to participate in the beta steps we have outlined here. We will let you know how when it’s ready

[foilstormcrow]

I love the idea of a helper function that is able to toggle a little indicator next to the extension icon, as it is minimally invasive (which IMO for things like notifications is essential, anything that takes away from the viewing experience is a detriment to the overall stream and having a simple coloured dot next to the extension is as minimal as you can get) and extension devs could easily have a setting to disable triggering it should the user not wish to have notifications.

Actual notification messages I’m not as sure about. There’s certainly a use for it but then there’s the question of is a single area of the screen, with static styling, suitable for all channels/extensions? There’s also the issue of what should be the default? I think the default viewer experience should be ‘Just the stream’, and that things like notifications should be opt-in. This could be tied to identity sharing so agreeing to that enables the notifications for that extension (with the option to still hide them if they wish) but even then there may be extensions that don’t require identity sharing but do want notifications.

Yes, we are lifting that limit on the width of components. However note a 16:9 aspect ratio using 100% target height is not going to consume the entire player, as “100%” there is going to mean the full distance between the top and bottom player controls.

[foilstormcrow]

To complete my previous answer, I mentionned the use of notification to inform viewers when an action is required from them or something happen in the extension

I made a quick gif of a suggestion of how it could be integrated with this RFC, using the same design
…

Thanks for the contribution! I can tell you that we are investigating creating a standardized in-player notification system for extensions as a followup to this work. We just don’t have firm details on the timing or specific feature set yet. Our aim right now is to solve the current pressing user issues with component extensions and then follow up with things that continue to improve the experience of the new model.

[foilstormcrow]

Is it planned to have a “displayed by default” extension?
Or a potential solution would be to still have the video components system as it is, but to only one of them at the time, like a video overlay but the video component extension would have much stricter rules in term of design UI/UX. For example, these extensions should not have a button to click to open them.
And we could still add the extension sidebar for extensions that previously required a button to open

No, there is currently no plan for a “displayed by default” option for component extensions. As mentioned in my reply to your other question, we are leaning toward the idea of a notification system for an extension that must present information to users while collapsed.

Also some other questions:
If a viewer opens an extension and moves the mouse out of the player, will the extension still be displayed?

Yes, the extension will remain open if a user opens it and mouses out of the player

Will the sidebar always be on the right? Or can the streamers set it on the left, bottom, top?

The plan for initial release is that the dock will always be on the right. The ability for a streamer to configure this is another one of our possible followup features.

[Breci]

No, there is currently no plan for a “displayed by default” option for component extensions.

So if I understand correctly, if I want to keep the purpose of my extension that works as an interactive overlay component and that need to be displayed when someone joins the stream I would have to move it to a video overlay extension, even if it needs only 20% of the width of the player?

[porcupinetv]

I’ll start by saying the motivation is clear and I think this is an elegant mechanism for new extensions. However, the new component redesign is threatening to remove 30% of our users and you should consider a better transition plan.

Here’s where we stand on this: we built Live Emotes, which lets viewers express themselves with customizable emotes on the stream. It was a launch extension and today we support millions of unique viewers every month interacting on thousands of streams.

Live Emotes supports video overlay as well as component formats, since both made sense as originally designed. There’s a major specification difference in the component redesign forcing us to stop supporting components. Specifically, being hidden by default.

• As far as our data shows, around 30% of our installation are set up as component.

• Traditionally with deprecation of big Twitch APIs developers are given time, often years. Why are extensions treated differently?

• Developers have no effective way to communicate with broadcasters about breaking changes. Suppose we drop component support and want to let broadcasters know they should switch to overlay, the only way to do it is an error message showing up instead of the component… And, during that time, we have no choice but to still falsely show up as an extension that supports component.

• We have considered redesigning the extension to support OBS Browser Source and render it on stream, then have the controls as the new component. We’d like that, but we are limited in implementing, testing, launching and then converting thousands of users on such short notice.

Looking at the proposed timeline, I urge you to reconsider the deprecation of legacy components in a short two months period. Additionally, consider better transition tools to developers. Stability is key in drawing and keeping developers.

[alexksso]

I think this is all great and makes sense.
My main concern is that our data shows that on a given channel 75% of the users never see the extension
I think it’s great to optimize the experience for the 25% who see it, but really the needle is on the 75%.
On mobile, they have no idea there even is an extension, nor that there is an action expected from them (need visual guide here).
On other platfroms, it’s just not there.

Aside from that, I find the suggestion for web extensions reasonable, kudos.