API throttling error margin

A throttling counter calculates the throttling limit by measuring the counter value every second and updating a moving average of 5 seconds.

Edge servers within the network share updated counter values with a delay ranging from 1 to 3 seconds. This latency on counter updates and the use of a moving average may affect the throttling limit calculation and result in allowing requests over or denying requests under the current throttling limit.

Depending on the distribution of your traffic on the network, the error margin over the last minute may range from a median of less than 10% for steady traffic rates up to about 20% of large sudden changes in traffic rates.

In rare cases, a burst of traffic may result from extra latency on a widely-distributed region of edge servers sharing its counter updates for a low throttling limit. If the updates from the under-synchronized region significantly increase the undercounted moving average causing it to exceed the throttling limit, edge servers will keep denying incoming requests over the throttling limit until the moving average stabilizes and drops below the specified limit.