Are buckets really fully refilling every 60 seconds?

Hey everyone! Newbie in here so don’t mind me if I commit any mistake…

I did a small script to gather videos related (up to the 6 pages) to a few games every minute, and everything looks fine for the first iteration where I’m actually starting with 800 points. But every other iteration, where I’m waiting for full 60 seconds after the last query (previously waiting until “ratelimit-reset”'s value), my points are not fully restoring (it restores to around 6xx). I was only able to do this continuously with a timestep of 2 minutes… Is there something I’m missing ?

Also, as a side question: gathering 600 videos (6 pages) related to a game will cost me 6 points, correct? Any further queries done past that, that could result http 429 would not have any effect to it…? I’m asking this because after a full iteration, I get the following stats:

{'time_until_reset': 46.38017225265503,
'total_requests': 674,
'remaining_points': '183'}

So unless the points are being refilled every second, the math here doesn’t add up, I think.

*Notes: I checked and I’m sure that I’m properly using the bearer token, and that my rate is 800 points. Furthermore, the key I’m using is totally virgin and there are no other queries being done other than within my script.

The bucket starts at 800, and is constantly refilling at a rate of 1 every 0.075s (so 800 per min) up to that maximum of 800.

If you use < 1 request every 0.075s your remaining will be ~799, as the bucket is refilling faster than you’re using it. If you make more than 1 request ever 0.075s the bucket will progressively empty (faster the more frequently you make requests) until it hits 0, at which point you’re effectively throttled to 1 request every 0.075s as any attempted beyond that will hit a 429 response.

If you burst beyond 1 request every 0.075s and the bucket starts to empty, it’ll progressively fill up again when you slow to less requests than that rate.

1 Like

@Dist So during the time that I leave my script awaiting for 60 seconds, shouldn’t I be able to have it fully restored? I don’t know why this is not the case for me.

But yeah, before I had this kind of knowledge, I was using threads to burst them all at once. Although now I am indeed limiting it and stopping when the remaining hits 0 (preventing any 429s), then I just sleep() for 60 seconds, and try it again. Unfortunately, as I mentioned, I don’t get back to 800 points.

If you reach 0, and wait for 60 seconds, yes you should be back to 800 requests remaining as long as you have nothing else making requests.

If you have other scripts or apps, that are using App Access Tokens from the same client id then they will all share the same rate limit.

1 Like

Alright, thanks @Dist ! Unfortunately, I’m really sttrugling to find why after my 60 seconds delay I’m not back to max, even after assuring there’s no unhandled thread lying around. I will do it iteratively and see if I get the expected result.

Thanks once again!

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.