Monitor Memory: you can set CloudTest to take action if a load generator (Maestro) exceeds a threshold for memory consumption. This is set in the Maestro Service of your main or any load generator using the setting Monitor.FreeMem.Threshold.
You can choose the server you want to modify by going to the server list, choosing the server, and then going to the Settings tab.
You can set it after you launch a grid, or if you plan to use the same settings on a consistent basis you can set it in the appropriate Server Classes. You can also duplicate one or more out of the box Server Class by going to Server Classes and create a new Server Class with one or more persistent settings.
By default, the threshold is 3. You can update this by setting the amount of available free memory, as a percentage of total memory, below which the Maestro will be considered “Unhealthy”. At 3 that means if memory goes above 97% used the action chosen in the Monitor.UnhealthyAction setting. Memory is checked after each garbage collection.
For Monitor.UnhealthyAction there are two choices: DRAIN and STOP. If you choose STOP the composition in which that load generator is running will stop, the same as if you hit the Stop button in CloudTest.
DRAIN will only stop server(s) found to be unhealthy, in this case that exceed the memory threshold. DRAIN lets the currently playing virtual users finish their jobs, however, no new VUs are created. This will happen for all comps running on the load generator since the monitor does not know which comp is causing the issue. If the setting is DRAIN, you will be notified in the spinner in the top right and in the event log that unhealthy servers are being stopped.
If you set the value to -1 the check will be disabled. You may do that if you expect to experience periodic spikes that you know won’t bring down the load generators, but you’ll probably want to keep an eye on them during the test.
Monitor Load Average: you can set CloudTest to take action on a load generator (Maestro) by assessing the total number of threads waiting for a defined period of time: 1, 2, 5 or 15 minutes. For single-CPU systems that are CPU bound, one can think of load average as a percentage of system utilization during the respective time period. For systems with multiple CPUs, one must divide the number by the number of processors in order to get a comparable percentage.
This is set in the Maestro Service of your main or any load generator using the setting Monitor.LoadAvg.xMinThreshold settings. The setting is specific to the class of machine and the number of CPUs, and has a higher tolerance when measured over shorter periods. The setting default setting is per CPU:
- 1 Min - 50
- 2 Min - 25
- 5 Min – 15
- 15 Min - 5
The above is multiplied by the number of CPUs. For example, and AWS Large has 2 virtual CPUs so it’s settings will be 100, 30, and 10.
As with the memory monitor, if the number of threads in the setting is waiting for the duration of the setting then the action chosen in the Monitor.UnhealthyAction setting will be invoked, either to DRAIN the affected server(s) or STOP the test.
As with the memory settings, -1 will disable the load average check. Consider increasing the 5 and 15 minute averages to 20 and 12 respectively before disabling this check