Webhook handling event sub reward redemption

API
1

Hello everyone,

I’m currently experiencing difficulties handling a high volume of Twitch webhooks sent simultaneously to my application.

Problem Description:

My application receives Twitch webhooks via a PHP script. These webhooks trigger SQL queries to a MySQL database. During peak activity, I can receive over 1000 webhooks within a few seconds. This leads to the following issues:

Some SQL queries fail with the “MySQL server has gone away” error.
I suspect I’m not receiving all webhooks sent by Twitch.
I encounter “Broken pipe” errors when using redis-cli via SSH.
Configuration:

Shared hosting with O2Switch.
PHP script for receiving webhooks.
MySQL database.
Using Redis as a queue.
Accessing Redis via a Unix socket.
Questions:

Are there any known limits regarding the number of webhooks Twitch can send simultaneously to a single endpoint?
Are there any best practices recommended by Twitch for handling high volumes of webhooks?
Could performance issues on Twitch’s side explain the loss of some webhooks?
Do you have any recommendations for using Redis in this scenario?
Additional Information:

I’ve implemented a Redis queue to try and manage the load.
I’ve optimized my SQL queries.
I’m monitoring my application and Redis logs.
Thank you in advance for your assistance.

(It’s my first post on the forum I’m sorry if I don’t respect conventions)

2

A queue is the correct thing to do here.

The webhook processor 200 OK’s as fast as possible and queues the job after sending the OK.

SQS/Redis theres a number of solutions that could do the job.

But what you have changed over to doing seems optimal.

This is likely the issue.

Shared hosting on a VPS leading to exhasuation

It’s also possible your provider is anti DDoS-ing in error.

If that was the case others would be complaining about lost data.

3

Thank you !
We will try to test it on other hosting (maybe a dedicated one and faster one)