Synchronize extension with the video stream

Hi, I’m working on the video overlay extension that presents some object on the video. It has to be super synchronized with the video stream. Unfortunately, as I tested the twitch video player, the video has very random delay - which is understandable to some extent. But even if I open the same twitch chanell in 2 tabs on the same computer, there will be a noticable delay between to tabs (up to 10 seconds). This makes my case very hard to work one. The best solution would be to somehow get the information about current timestamp of the stream inside the extension, and use a buffer to postpone events before they happend on stream.

Is there any way to get the stream timestamp inside the extension, when extension is loaded?

See this first post for example code

Video latency - Using the context event you can get the hlsLatencyBroadcaster which gives you how many seconds the viewer is behind the live stream. Using this latency you can pad your receive buffer so that your extension messages are in sync with the video. In my case each second of data includes a set of 10 data samples that have a 100 ms duration each. So I’m enforcing that the receive buffer always has (10 data samples for every second of latency).

Thanks @BarryCarlyon. Do you have any information about the algorithm they use to compute this “hls Latency Broadcaster” variable? I am trying to reverse engineer this thing, with no success. My guess it compares the time of stream start with the current date, and timestamp of the last video chunk… but I can’t confirm that - going through minified JS code is a nightmare.

HLS Latency Broadcaster is the delay between live and the user give or take. No comparing stream start it’s just the timestamp difference between the timestamp in the chunk and the current local time, I believe.

In addition, so be careful:

3. Limitations. You must comply with all instructions and requirements in any integration documents, guidelines, policies, or other documentation that we provide or make available to you. You will not reverse engineer, disassemble or decompile the Program Materials or otherwise compromises Twitch’s rights in the Program Materials

Sure, I simply can’t see where this timestamp comes from.
Nevertheless, investigation stops now. Thanks for warning me.

For all we know it’s mathed server side and sent in the payload or just “chunk timestamp - local time”