10 Suggestions for Choosing the Number of Concurrent Users for Your Test

10 Suggestions for Choosing the Number of Concurrent Users for Your Test

Do you know how many virtual users should you use for your website or app performance tests? If your answer is “No”, this is the right blog post for you to read. Even if you know Apache JMeter and Gatling, they can’t tell you how many concurrent users should be sending requests to the URL you’re testing.

This is because the number of users to run a test is a business and product decision based on past user scenario patterns, future expectations, marketing initiatives, and product requirements.

So, if you’re a tester, here are a few suggestions to get you started for a performance test:

1. Check the Server Logs

Logs are essential for every software personnel. The server logs will contain the history of web requests, including the client’s IP address, date and time, and page requests. You may use this information to create a picture of how many real users visited your website each time and what activities they took while surfing.

2. Compare Server Log Information With the APM Data

APM tools analyze user data in-depth and help with the identification of trends and scenario profiling. Check if anything is missing or inconsistent between the picture you got from the logs and the one you got from your APM tools, and you’ll have a better knowledge of how users acted on your website. Google Analytics is another option for these purposes. Also, you can calculate Ramp Up Time with G4 Google Analytics since it gives numbers of all visitors and engaged visitors with their durations.

It’s also crucial to spot patterns in website growth. For example, if your traffic is consistently increasing by 10%, consider and increase the number of users in your test correspondingly.

3. Keep An Eye Out For Bottlenecks And High-Stress Points

Investigate areas with the highest user activity, keep track of distinct trends, and see if your system was about to reach its limits at any time. Include those statistics in your testing because they represent possible crashing points in real-time.

4. Check Product Requirements

Your product manager most likely determined the capacity of your website or app when the test was planned, and as it is upgraded and improved. This choice was influenced by the company’s business strategy and aims. Verify the numbers with the product manager to ensure that they are included in the testing. Future plans are likely to be based on those numbers, so make sure that your app or website can handle the traffic. If they are unable to do so, plans should be made appropriately.

5. Check If There Are Any Upcoming Campaigns or Major Events

Campaigns, events, and holidays can all result in a surge in traffic and a corresponding boost in sales for your organization. However, if your website is down, this will not be possible. If something is scheduled, check with the marketing and IT departments, and boost the number of users you are testing in preparation for these events.

6. A Thread Won’t Be The Same With A Real-Time User

Let’s think about an end-to-end scenario for an e-commerce website. A user logs in, goes to the main page, searches for a product, sees the results, and goes to the detail page of one option. Then the user will continue with adding to the cart and payment steps. A real-time user would be executing this scenario slowly. Because during the flow; he/she will think about his/her budget, try to find the most charming option then fill in the credit card info slowly. But a thread can execute this scenario much faster than a real-time user if you don’t add “think time” between requests in your script. So, 100 concurrent threads can provide a bigger load compared to 100 real-time concurrent users. So, keep that difference in mind.

7. Start by Putting 80% Of The Capacity Into The Test

Running your load test with the maximum capacity at the beginning will not be a good idea. The best practice is to improve the load gradually. Run the test with 80 percent of the capacity after you’ve decided on the numbers and keep an eye on your KPIs and how your system responds. Make sure that everything is entirely stable; memory capacity is low, CPU usage is low, and the system recovers quickly from surges. With 80 percent of the maximum load, everything should be running smoothly. If anything feels unreliable, you can be sure that you won’t be able to rely on it at 100 percent.

8. Gradually Increase to 100% If The Test Was Successful

If the test with 80 percent capacity was successful, proceed to the 100 percent test. If this isn’t the case, look for bottlenecks and errors, and make the necessary improvements and bug fixes. And then, it’s time to put your system under the test again. This is the predicted number of visitors to your website based on prior usage trends, trend research, product requirements, and anticipated events. Perform a memory leak check, as well as a CPU utilization check to see whether your server is acting unexpectedly or has made any errors. If something isn’t working right now, it’s unlikely that it will work in real-time. Make the necessary improvements. Rerun the tests to ensure that your adjustments were applied correctly, and no performance degradation occurred.

9. Push Your System Beyond the Limits

Under the specified loads, your system performs admirably — excellent work! But be prepared for the unexpected! What if a competitor’s website goes down and you get a ton of traffic? What if a well-known figure makes a statement that generates a lot of interest in your product? Be prepared. The truth of the matter is that no system can be prepared for an endless number of users, but you can learn about your system’s flaws.

To do so, run stress and soak tests on your system and try to make it crash. After that, examine the data to determine where your weak points are. When you have this information, you can decide whether to make changes to your infrastructure or code. If something goes wrong in real-time, you’ll know exactly where to investigate to fix the problem.

10. Review The Results of Earlier Tests

If you have run similar tests on your system before, reviewing these test results can provide you with lots of useful information for choosing several concurrent users. If your last test had faced 10.000 concurrent users at maximum capacity and couldn’t handle the load, your first goal may be reaching 10.000 concurrent users without any major issues. But if you were too far from 10.000 -let’s, say 6.000-, you can rerun the same tests to see if stats like total hit count, average throughput, and response time have some progress.

We hope these suggestions help you develop your performance scenarios. If you’re curious about how Loadium works, request a demo here.