Access to stream video as screenshot

Is there some way from the extension to get access to the stream video as a snapshot of point in time (screenshot)? I would like to react to what is under the mouse pointer, but don’t know a way to achieve that. Thanks!


I know of a few development things that sorta work but nothing that really works to do what you want it to do. It’s surprisingly “expensive” client side too.

Ok, I was afraid of that. Care to share those development things so can get some ideas at least?

I wonder how hearthstone extension does the mouse over hover thing to show card details then…

Something on the computer tells the overlay what cards in in what positions and the cards slots are in fixed locations on the screen, then you just need bounding boxes. No need to “read” the screen.

Not sure if I can find them again, will look if I get time

if we’re talking about the broadcaster getting information as to what the viewers are doing in the extension, this would have to be done on a per-extension basis by the extension’s developer.

Since Extensions use WebSockets, the Extension could simply report back to the Extension’s back-end server, and then report the aggregated information back to a front-end panel that the broadcaster uses.

Not specifically Extension related, but our custom channel bot has a giveaway feature that, via SocketIO and TMI.js, is able to communicate with the chat room (TMIjs), a Dashboard client i wrote (SocketIO) and our on-stream widgets (also SocketIO). We can see lists of entries, have controls for the giveaway itself, and all giveaway information/events are piped to the widgets in OBS to show live stats while the giveaway is running.

Would it at least be possible to somehow get access to the time code of the current video frame during a click? For example if an extension needs to report click location info associated to the exact frame for further image processing? (I assume this is also related to Can you access the broadcaster's video stream?)