What is Soak Testing? Definition, Meaning, Examples

The image below depicts a testing cycle that shows at which stage the Soak Testing (Type of Performance Test) is performed on an application.

In this type of testing, what basically monitored is the memory utilization by an application in a system. It is testing at a system level, to find whether the system will stand up to a very high volume of usage and to see what would happen outside its design expectations.

In this tutorial, you will learn-

• Why do Soak Testing?
• When to do Soak Testing?
• Soak Testing Strategy
• Characteristics of Soak Testing
• EXAMPLES of Soak Testing
• Common issues observed during Soak Testing

Why do Soak Testing?

A system may behave normally when used for 2 hours, but when the same system is used continuously for 10 hours or more than that then it may fail or behave abnormally/randomly/it may crash. To predict such failure Soak Testing is performed.

When to do Soak Testing?

Soak Testing should be done in the following scenarios: -

1. Before the built is deployed to the client i.e. prior to the release of any application on a specific platform, it needs to go through a successful series of load tests at high or equivalent traffic levels. After that soak testing is performed. It helps us to determine how to run any particular application for an extended period. If issues like memory leaks/memory corruption are found during the period i.e. when it is on Soak, then it should be immediately reported.
2. The best time to do a soak testing is over the weekends as an application need to be in a running state for as long as over a day or night. It totally depends on the limitations of the testing situation. Soak tests are one of the most important compliance requirements which need to be very strictly followed by every company.

Soak Testing Strategy

Long Session Soak Testing is a strategy where a system is under load for a longer time.

A simple example is where the user stays logged into a system for many hours executing a number of business transactions. In this way, a lot of data gets created. There can be lots of load on the system/database server which can result in stalling/crashing of the system/database server.

Under Long Session Soak Testing, multiple day's (say 30 days) activities are performed in a restrained time frame (say 2 days). The number of transaction in this restrained timeframe should match or surpass multiple days' worth of transactions. The focus should be on the number of transactions processed. The most important part of Soak Testing is to check the available memory in the CPU and the amount of the memory that will be in usage. We need to record the memory usage at the start and the end of a soak test. If necessary, then the memory usage of facilities such as Java Virtual Machines are also important and needs to be monitored.

Below are few more checks that need to be done by any user/tester before they begin with Soak Testing:

a) Monitor the database resource consumption.

b) Monitor the server resource consumption (ex- CPU usage).

c) Soak test should run with realistic user concurrency.

Characteristics of Soak Testing

A standard Soak Testing Method should have the following characteristics: -

• The duration of most Soak Test is often determined by the time available.
• Any application must run without any interruption if it requires an extended period of time.
• It should cover all the scenarios that are agreed upon by the stakeholders.
• Mostly every system has a regular maintenance window time period and the time between such window periods is a key driver for determining the scope of a Soak Test.

EXAMPLES of Soak Testing

• In the case of banking domain when there is a large amount of data from merchants, the tester will put the system under load continuously for 70hrs to 150 hrs to check how the application behaves during this loading period.
• Suppose there are 33,000 logins, which needs to be put through the system, it represents seven and a half days of activity. In this case, a 60- 70 hours of Soak Test can be started by Friday evening around 6 pm which can be completed by Monday morning at 6 am. Only with such a test, it will be possible to observe any degradation of performance under the controlled conditions.
• In the case of Video games, Mobile applications, etc. involve leaving the game or application in a running state for a prolonged time period, in various modes of operation- such as idling, paused at the title screen and so on to find out whether an application can handle the continuously expected load.

Common issues observed during Soak Testing

1. Memory allocation (memory leaks that would eventually result in a memory crisis or rounding errors that manifest only over time).
2. Database resource utilization (Failure to close database cursors under some conditions which would eventually result in the entire system stalling).
3. It can also lead to performance degradation, i.e. to ensure that the response time after a long period of sustained activity is as good as it was at the beginning of the test.
4. Failure to close connections between tiers of a multi-tiered system under some circumstances which could stall some or all modules of the system.
5. The gradual degradation of a response time of some functions as internal data-structures become less efficient during a long test.

Summary

• In Software Engineering, Soak testing is done to determine if the application under test can sustain the continuous load.
• It is a type of performance test.
• It helps the system to determine whether it will stand up to a very high volume of usage
• In this type of testing, what basically monitored is the memory utilization by an application in a system
• Checks that need to be done by any user/tester before they begin with Soak Testing include
  • Monitor the database resource consumption.
  • Monitor the server resource consumption (ex- CPU usage).
  • Soak test should run with realistic user concurrency.

This article is contributed by Pallavi De