EventSub is expecting an unspecified Content-Type in verify response

hfrarfhf · January 3, 2022, 12:10am

I am trying to send a valid challenge response but they always fail because the Content-Type header field does not match what is expected:

$ twitch event verify-subscription subscribe -F https://xxxxxx/test -s dfsion43jdghjyt5mry
✔ Valid response. Received challenge 026c15d4-1c35-9ded-09e2-0905f1611983 in body
✗ Invalid content-type header. Received type text/html; charset=utf-8
✔ Valid status code. Received status 200

If I do not include the Content-Type header it will also fail:

$ twitch event verify-subscription subscribe -F https://xxxxxx/test -s dfsion43jdghjyt5mry
✔ Valid response. Received challenge 2231fa97-91fe-9bbb-f938-2153bc17b5cc in body
✗ Invalid content-type header. Received type
✔ Valid status code. Received status 200

In total I’ve tried text/plain, text/html, application/json, and application/octet, basically experimenting with different values. They all fail.

Everything is working correctly except for the Content-Type. This isn’t mentioned in the docs so I don’t understand why it’s even being considered.

I tried ignoring what the twitch-cli was saying and attempted to subscribe to the event, but those still failed, presumably for the same reason. (I’m not aware of a way to retrieve the failure reason.)

At this point I am stuck and don’t know what to do.

Any help would be appreciated.

Thanks.

BarryCarlyon · January 3, 2022, 12:45am

Something is appending a charset=utf-8 to your intended content-type, in your code.

Which is invalid for the CLI and eventsub.

The API will only report that it failed, not why. Thats what the CLI is for in this case.

You could (in error) conceivable return the signature as JSON in error with a JSON header, when it should be plain text. So it’s a “make sure the dev didn’t make a mistake”

Verify needs 200 + the challenge in plain text
Events need 200 ok or 204 No Content. (No Content will mean no content-type header to muck about with)

hfrarfhf · January 3, 2022, 1:50am

Nothing is appending a charset in my code. The Twitch CLI is doing that.

If I construct the response with this C code:

char *challenge = get_challenge_from_request();
char response[256] = {0};

memcpy(response, "HTTP/1.1 200 OK\r\nContent-Length: 36\r\nContent-Type: text/html\r\n\r\n", 64);
memcpy(response + 64, challenge, 36);

write(socketfd, response, strlen(response));

I get this response:

$ twitch event verify-subscription subscribe -F https://xxxxxx/test -s dfsion43jdghjyt5mry
✔ Valid response. Received challenge 7087e388-ff86-e86e-2763-dc3beb3aa02c in body
✗ Invalid content-type header. Received type text/html; charset=utf-8
✔ Valid status code. Received status 200

There is no charset being set anywhere in my code.

BarryCarlyon · January 3, 2022, 1:54am

The CLI only reports the headers that your server returns.
The CLI itself doesn’t add headers.

You can test header response your senf by makign a curl request to your own server or using a HTTP Testing tool such as postman or insomnia and confirm it’s not the CLI at fault, since the CLI doesn’t add headers it only reports what your server returns.

This could be being added by nginx/apache or whatever is in front of your c code (if anything).

As you can see in the code as it’s open source

https://github.com/twitchdev/twitch-cli/search?q=charset%3Dutf-8

https://github.com/twitchdev/twitch-cli/search?q=utf-8&type=code

There is no reference to this in the code

twitchdev/twitch-cli/blob/be0c5691f2d855ef945dd002a727c7e3105aa1f0/internal/events/verify/subscription_verify.go#L96


      
          			color.New().Add(color.FgGreen).Println(fmt.Sprintf(`✔ Valid response. Received challenge %s in body`, challenge))
          			r.IsChallengeValid = true
          		} else {
          			color.New().Add(color.FgRed).Println(fmt.Sprintf(`✗ Invalid response. Received %s as body, expected %s`, respChallenge, challenge))
          			r.IsChallengeValid = false
          		}
          
          		if resp.Header.Get("Content-Type") == "text/plain" {
          			color.New().Add(color.FgGreen).Println(fmt.Sprintf(`✔ Valid content-type header. Received type %v`, resp.Header.Get("Content-Type")))
          		} else {
          			color.New().Add(color.FgRed).Println(fmt.Sprintf(`✗ Invalid content-type header. Received type %v`, resp.Header.Get("Content-Type")))
          		}
          
          		if resp.StatusCode >= 200 && resp.StatusCode <= 299 {
          			color.New().Add(color.FgGreen).Println(fmt.Sprintf(`✔ Valid status code. Received status %v`, resp.StatusCode))
          			r.IsStatusValid = true
          		} else {
          			color.New().Add(color.FgRed).Println(fmt.Sprintf(`✗ Invalid status code. Received %v, expected a 2XX status`, resp.StatusCode))
          			r.IsStatusValid = false
          		}
          	}

and follow the code backwards. The CLI doesn’t modify headers it only reports the headers your server declares.

And since you are having the same problem with “real” eventsub, it’s your server not the CLI/Twitch.

As Real eventsub isn’t modifying the headers either.

If you provide your callback I can test it myself in insomnia and confirm it

hfrarfhf · January 3, 2022, 2:02am

After digging through this I found out it wasn’t my code but Cloudflare adding the charset.

I don’t understand why Twitch cannot support a charset being specified, nor why a completely nonsensical error message is being used.

This would have taken less than a minute to figure out with a proper error message, and frankly Twitch should be able to support UTF-8.

Oy.

BarryCarlyon · January 3, 2022, 2:04am

The CLI is open source so you could device and submit a PR to resolve/adjust the error reporting.

As to supporting a charset in the content type that’ll be a uservoice I imagine to modify how eventsub functions

hfrarfhf · January 3, 2022, 2:05am

I already have a job and nothing about using the Twitch API so far has made me want to keep using it, let alone work on it for free.

lu4t · January 31, 2022, 12:12pm

utf-8 is added by default, and cannot be changed, read:

github.com/expressjs/express

No way to disable charset in Content-Type header?

opened 02:46PM - 29 Nov 17 UTC

closed 03:22PM - 29 Nov 17 UTC

tomasruud

question

I'm developing an API that will replace another API implemented in another langu…age. Because the clients for this API are implemented badly, they don't accept content types other than "application/json" this means that "application/json; charset=utf-8" breaks them. Currently, I've been trying to use this middleware before my routes to prevent the charset from being appended: ```js app.use((req, res, next) => { res.setHeader('Content-Type', 'application/json'); return next(); }); ``` I usually end responses like this `res.status(404).json('Not found')`, but with the middleware I'm using, it doesn't seem to work. I've tried to look at the Express code, and from what I can tell, if I use the Express function `set()`, `header()` or `type()` the charset automatically gets appended. So when I use the native `setHeader()` function I thought that would work. When I use the native method, it doesn't seem like it should be overridden, because the `res.json()` function checks if the header is already set before trying to set it, which also uses the native `getHeader()` function. Is there any way I can get around this, without changing how reponses are sent or am I doing something wrong?

Couri_G · February 20, 2022, 4:11pm

Hey
Are there any plans for making changes so that UTF-8 can be accepted? I’m using laravel and there’s no way to remove charset=UTF-8 because it’s added last minute if it’s missing from content-type.

BarryCarlyon · February 20, 2022, 4:26pm

Live EventSub doesn’t have this issue, it’s just a bug in the CLI

Which already have a PR merged to resolve

github.com/twitchdev/twitch-cli

Potential fix for the content type header issue

twitchdev:main ← BarryCarlyon:contenttype

opened 01:08PM - 31 Jan 22 UTC

BarryCarlyon

+19 -3

By submitting this pull request, I confirm that my contribution is made under th…e terms of the Apache 2.0 license. ## Problem/Feature This fixes #138 ## Description of Changes: - parse the content type header using the mime library - test the content type when present - report the charset if a charset is returned in the content-type header Example output: ``` ✔ Valid response. Received challenge [omitted] in body ✔ Valid content-type header. Received type text/plain with charset utf-8 ✔ Valid status code. Received status 200 ``` ## Checklist - [x] My code follows the [Contribution Guide](https://github.com/twitchdev/twitch-cli/blob/master/CONTRIBUTING.md#Requirements) - [x] I have self-reviewed the changes being requested - [x] I have made comments on pieces of code that may be difficult to understand for other editors - [x] I have updated the documentation (if applicable)

system · March 22, 2022, 4:27pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.