Twitch's embedded iframe. How to get its audio?

[imsamantha]

I’m building a React website to capture a Twitch stream, obtain its audio, and pass it to a speech-to-text function to generate subtitles. However, I’m struggling to make it work properly. Can someone help me with this?

I’m not able to retrieve the audio from it. If I try to console.log the twitchIframe.contentDocument , it returns null

Here is the function I created to record audio from the stream:

const startRecording = () => {
    const twitchIframe = twitchIframeRef.current;
    const audioChunks = audioChunksRef.current;

    console.log('caiu', twitchIframe.contentDocument)

    const onLoadHandler = () => {
      const iframeDocument = twitchIframe.contentDocument;

      console.log('caiu', iframeDocument);

      if (!iframeDocument) {
        return;
      }

      const videoElement = iframeDocument.body.querySelector('video');
      if (!videoElement) {
        return;
      }

      const stream = videoElement.captureStream();
      mediaRecorderRef.current = new MediaRecorder(stream);

      mediaRecorderRef.current.ondataavailable = (event) => {
        if (event.data.size > 0) {
          audioChunks.push(event.data);
          sendAudioToGoogleSpeechToText(event.data);
        }
      };

      mediaRecorderRef.current.onstop = () => {
        finalizeSpeechToTextConversion();
      };

      // Start recording
      mediaRecorderRef.current.start();

      twitchIframe.removeEventListener('load', onLoadHandler);
    };

    twitchIframe.addEventListener('load', onLoadHandler);

    twitchIframe.addEventListener('ended', () => {
      if (
        mediaRecorderRef.current &&
        mediaRecorderRef.current.state === 'recording'
      ) {
        mediaRecorderRef.current.stop();
      }
    });
  };

And here is how I’m embedding the iframe:

<iframe
          ref={twitchIframeRef}
          frameBorder="0"
          onLoad={startRecording}
          style={{ marginTop: 60, width: '60%', height: 500 }}
          title="Twitchs"
          src={iframeSrc}
          className="responsive-iframe"
        ></iframe>

The iframeSrc URL looks like this: Twitch

[BarryCarlyon]

The optimal way to do this is to do it on the Streamers side in the streaming software as you can get a clean feed of the microphone.

@imsamantha

I’m not able to retrieve the audio from it. If I try to console.log the twitchIframe.contentDocument , it returns null

Not sure if embeds support doing what you are trying to do due to cross domain iFrame security, which is what you have tripped here

[imsamantha]

Thanks for the fast response, Barry. I’ll aproach what you sugested!