Thank you for the detailed explanations of these use cases. As mentioned in another comment, I would recommend submitting these to the Embed Feedback form cited in the original post so that they can be logged for future mitigation, depending upon when the experiment concludes.
For our last Games Done Quick event, we built a virtual crowd experience on the GDQ website as a way to bring back some of the atmosphere a live audience brings - and is also sadly lacking during the pandemic.
Experiences like this are possible because of our ability to embed the stream, because we already have a very high premium on screen real estate and a feature like this as an overlay component- or even a twitch extension- isn’t nearly as viable without a lot of refactoring.
2 Likes
Thanks for the feedback on this unique use case, certainly one that is advantageous for these pandemic times. Once the experiment is concluded, we’ll use the information that we gather from it to appropriately situate contexts like that one within the overall understanding of the creator discovery/offsite viewing advantages that the data bears out, hopefully intelligently balancing the two in the future. Ideally, this will help us build new features and controls that continue to let new experiences like this one arise.
1 Like
When do you think the experiment will end?
So instead of asking devs for their opinion first and gathering ideas from the community, you went ahead and just implemented the most intrusive update that breaks most genuine use cases?
Not to say I am surprised at the lack of foresight, but when will you learn.
We shouldn’t mistake lack of foresight for what’s really a lack of empathy and integrity.
1 Like
This would work fine for e.g. YouTube where most embeds are just a short clip, and then the user moves on to the next, but a single Twitch embed is in most cases meant to be watched for more than 1 minute.
If this change really goes through, you might as well just remove the embed altogether. I for sure will remove it from my site, as it will be completely pointless to use.
While I don’t have a direct use-case for input here, I am curious from a developer standpoint why Twitch thought implementing an experimental feature across all users without an ability to opt-out was the best choice?
Given that it directly impacts all embeds and breaks many of the above use cases, shouldn’t this have been something that’s tested for those opted into beta testing of features? This is very common practice, even your parent company utilizes this practice, along with plenty of others as it’s standard practice to do so with features that directly impact & change the required interactions with your main media player.
This is not to say there’s never a case to A/B test on a live site / experiment with minor changes in production such as style, layout, CTA, etc., but major functionality changes don’t fall in this group.
1 Like
You just compared what a e-commerce store does to a video stie. Sellers on Amazon will want to paticipate in experiements as it may increase sales, and give them an edge over other sellers on the platform.
Most of the people on this thread vary in size of traffic and use case, but uniformly no one would turn it on. Most want a option to disable it.
So, if it was opt-in, no one on this thread would opt in and then Twitch would have no data.
What you link to (loosly) refers to allowing merchants to run A/B split tests on their products on the Amazon Seller Market place.
Do I use this description/that description, do I offer free shipping here, do I move the buy button somewhere else. Amazon SellersCentral is giving choice to the merchants that use Amazon to sell their own products.
It’s a unfair comparsion as you are comparing two completely differeht things.
So you’d expect sites like multitwitch, kadger, raredrop, juked etc (some of the high traffic big guys) to opt in to these tests? Where the end result would be Reddit going “oh such and such a site is getting these popups, anyone know a site that isn’t getting popups” so Twitch has to go “do it to everyone” in order to get any test data.
Personally I agree we need a switch/option so that developers have choice, but Twitch needs to collect test data/measure effectiveness that doesn’t involve the end user, leaving site a (look on reddit for a fix) for site b instead of “paticipating” in the test by closing the dialog (if you get that test version) or moving from the embed to Twitch.
How and what to test is tough to decide, hence the test is across all embeds in order to collect enough statistically significant data.
Some of the “use cases” poeple highlight, they never want you to leave the site the embed is on, hence this is disruptive to their business.
So that, in my opinion, is why it’s not opt out.
It’s hard to test something “disruptive” on a opt-in basis.
2 Likes
That’s not entirely fair… only the people who are disrupted by this experiment come to this topic to complain. It is entirely possible (albeit unlikely) that the majority of embedded player users would not opt out. Also, ‘the percentage of people who opt out’ would probably also be a useful metric.
Yes I should of noted this in my reply. Generally it’s only the disrupted that will post as those running embeds that the experiment is not disruptive to won’t notice as the users won’t complain, the viewer will either move over to Twitch, or close the dialog or leave the site.
My “uniformly” refers to the people on this thread
Now, the counter point here is that if Twitch published a opt in experiement, there is no way to publicise it to get people to turn it on, since you don’t need a clientID to generate an embed.
So Twitch doesn’t have email address’s for every developer that is utilising Embeds. So there is no way to get sufficent penetration of the Developer Market to get people to turn it on.
If TwitchDev Tweeted about it on Twitter, how many developers will even see the Tweet? With some people I’ve been talking to about this experiement, turns out the don’t even follow TwitchDev on Twitter. (And yes the Tweet about this experiement was sent after FishStix’s tweet, and his post generated more activity, since he linked to this thread directly)
So the test would fail due to “lack of data” as no one would know about the experiement to turn it on. Which would then result in a bad business decision due to lack of data (whichever way that decision went)
Edit: 'course this is me taking the unpopular opinion route and trying to look at everything from all sides and all angles. And trying to land right down the middle
2 Likes
so Twitch has to go “do it to everyone” in order to get any test data.
I don’t know if this is accurate, our users are continuously pointing towards a site who are not currently having this happen, I think this roll out is happening on multiple levels (site and pop up configuration), which is honestly the next logical step of Twitch Embedded Player Migration - Timeline Update
We don’t know the parameters of the test, so Twitch may have exlucded that site to not have them as part of the test (as a control group) or other agreement in place between Twitch and that site.
If the former is valid test data, if the latter, who knows.
Again, trying to look at both sides here.
Have you had/noticed this happen on https://www.cohhilition.com/ ?
I’m wondering if this is another site that received an exception, which would lead me to think that streamers with personal websites are being exempt.
Edit:
As I said, it I received a pop up. My apologies. I just wish this was a more clear process as it feels very aggressive.
Haven’t tested, and technically that embed is below the minimum width that the embeds support (it’s 340 now and that one is 300). And I need to update the embed to meet this change
Course at this dimension you can’t really watch anyway and the whole point of the embed there is a easy way for viewers to know if cohh is live and follow the CTA (that is below the title)
Gotcha, this is a bit different than from a site like https://www.destiny.gg/bigscreen
I haven’t heard of anyone reporting one from this site, and it would be hard for me to believe they came to an agreement with an un-partnered streamer, which is why I think the targeting is happening on a few levels.
Edit: we specifically had a few users report that they could watch the stream there but not on our site, thus leaving our site.
So, this has a TOTALLY different experience when you are not LOGGED IN, which I didnt test out the other time.
First - In some websites, IT KILLS the stream after a period of time with ZERO way to remove the “WATCH ON TWITCH”
This is kinda turning a EMBED into a UNPAID Advertisement, If this continues, people like myself will just Switch to another Video Platform.
Some embeds have the PURPLE Screen issue, Some Show the CONTINUE TO WATCH, some freeze the screen, this test is all over the place…
Can’t wait until its over.
1 Like
As noted in this reply in this thread here
There is more than one variation that appears to be part of this test. So would seem that Twitch is also A/B split testing different varations as part of this test
At last count there are four known variations, the three in the linked reply and the one from my own (in a really small embed - Upcoming Twitch embeds experiment - #57 by BarryCarlyon)
2 Likes
I’ve tried to read through all posts here, but I couldn’t find an answer how long until the CTA appears.
If it is immediately, I am wondering how this will impact Twitter Player Cards and Twitter’s upcoming Twitter Media Studio features, where large companies and organizations will be able to embed video streams directly into Tweets.
Twitch should reach out to large events and companies such as Riot Games that has millions of views on their streams and ask them if they would appreciate (data on conversion) a CTA button on their embedded streams.
The timings for how long till it shows, and/or/if auto hides afterwards, is probably part of what is being tested.
For all we know, they already did
1 Like