Results Service

There are a number of ways tests might be modified to avoid putting too much stress on the results service. For example, you don’t want to generate a large number of unique errors, such as having a session ID in an error text. The system does not allow more than 32,000 unique errors, and the practical limit for most implementations is going to be less than that.

Another place to look is Container elements, such as Clips, Transactions and Groups, which cause the system to calculate statistics about each of these containers while the test is running. Heavy nesting of very fast running containers can create a lot of overhead. For example, a Clip which contains a Transaction which contains a Group which contains another Transaction which contains a single, fast running HTTP message, will generate a lot of metrics, which may, depending on the size and duration of the test, put a lot of stress on the system.

In addition, there are features and settings to help put the brakes on before a server is overwhelmed.

Max URLs: A large number of unique URL's could also stress the results service. The setting Writer.MaxURLDimensions governs the maximum unique URL's for analytics. If this number is exceeded then all other URL's will be set to "Other". Reducing this number will help reduce memory usage. The default is 30,000.

Alternatively, the composition has a property which governs how URL's are tracked for analytic purposes. Setting this to "Protocol and Host Names Only" can greatly reduce the memory overhead with computing statistics for URL's.

Max Property Values: A large number of unique property values can also use up resources. Creating custom property values that have the "save value for analytics" checkbox checked causes the system to use memory for each property value. Checking this box if the property values contain something unique, such as session IDs, can cause excessive memory to be consumed. The Writer.MaxPropertyValueDimensions setting governs the number of unique dimensions. If the number goes above setting, which is default at 30,000, remaining values will be set to other.

Results Throttle: The Results Service can consider itself to be under stress primarily due to exceeding one of two potential limits. Similar to the load generator Monitor.Unhealthy.Action, which kicks in under certain monitored limits, the writing of results to the database will be throttled under specific conditions. If the Database starts to become overwhelmed it will force the Consolidators to slow down, having them write to the local Results Consolidator disk, if necessary. The two settings that will trigger the throttle are:

  • Database: The HealthCheck.Database.Max.Problem.Percent has a default 30%. If the percentage of database update transactions that exceed a normal time limit over a 5 minute period exceeds this limit then the system considers itself under stress.
  • Memory: The HealthCheck.FreeMem.Threshold has a default of 10%. If the system has less than this percent of free memory then it will consider itself under stress. You can also set the frequency at which this threshold is checked.

While this will minimize the chance that the RSDB will experience a failure, it also means the dashboards will slow down and it will likely take quite a while to complete writing the results. Also, a very fast spike can still overwhelm the results server. If this occurs, along with looking at the composition of your test, you may want to deploy a larger RSDB.