When somebody asks “What is testing?”, There are many answers to it. My preferred answer is; “comparing the outputs according to the expected one and checking whether the requirements are fulfilled or not.” That’s an easy answer when it comes to functional testing. What about a non-functional test such as performance testing where requirements are not defined clearly? Let’s find out together, how to define KPI’s in performance testing…
Everybody wants their website to respond fast with %100 stability. Your site or app can be fast, but it can misinterpret a request during a high load and order the wrong shoe size for your customer or vice versa. We want both. We want stability and 24/7 availability within acceptable time frames.
During the execution of performance test projects, defining KPIs (Key Performance Indicator) is quite challenging. But this is the factor that determines the quality of the product. There are many KPIs that need to be taken into account. Response time, hit number, error per sec are some of them. Let’s take a look at each one.
Response time tells us how long a process takes to complete a specific request. In case you are familiar with statistics, we want response times to have a normal distribution when we analyze them in a graph. Most of the requests should have the mean value and some of them should be in the outlier zone.
The x-axis is the number of requests.
Y-axis is the response time.
Most of the requests have the response time between 0.3 and 0.5 sec and mean 0.4. A few of them are outliers. We hope not to see a stable line parallel to X-axis which signals the instability of the system.