Timeout back off

A large number of test object downloads timing out could stall other tests. To prevent this, a test that times out is assigned a back off interval, which is an increment to the test interval. Each time the test times out, the back off interval is increased. The initial value for the back off interval is equal to the test interval, and each time the test times out the back off interval is multiplied by 1.5, up to a maximum of 15 minutes. As an example, if a test with a test interval of three minutes has been timing out for a while, the liveness testing agents will perform the test only once every 18 minutes. If a non-timeout error occurs, the back off interval remains unchanged. As soon as the test succeeds, the back off interval is reset to zero.

Note: If you need to bring a recently resurrected server back into rotation immediately, but this is being delayed by the timeout back off, you can reset the back off. Changing a test parameter, such as the timeout interval, will discard the old test and create a new test with a zero back off. As an example, you could change the timeout from 25 to 26, and then back to 25.