Requirement Gathering

Requirement Gathering

  1:12 AM    Jitin Chadha

---

Capturing the client needs for a Performance testing project

Step 1: It is very important to know the Application's Communication protocol because it will be a very important factor to finalize the tool to be used for the activity.

Most of the web based applications are built to work with HTTP/S or now a days SOA is getting very popular hence you may need to go for websevice (API) testing.

Other Things worth knowing are (will be very Important in case of Bottleneck identification/Performance Reingeneering) 

• The technology used for application development (like .net, PHP, Java/J2EE etc)

• Database used (Like MS Sql Server, Oracle,MySQL, Postgresql) and on which platform they are deployed (like Windows, Linux etc)

• Application/web server (like IIS, Apache, Tomcat, WebLogic, WebSphere) and on which platform they are deployed (like Windows, Linux etc)

• Other components used in the application architecture (Like Load balancers, MongoDB, Solr search etc)

• Check if there is encryption used for sending and/or receiving the request/response.

• Check if there is Captcha used in any of the In-Scope workflows (properly implemented Captcha cannot be automated hence you need to ask them to remove/disable it during the testing)

• Any batch process/Cron/Scheduler that is to be considered.

Do ask for the versions of the components wherever applicable also

Do not assume, ask the development team and get a confirmation.

For HTTP/s or webservice testing the popular tools (the tools about which i am aware :) ) are Apache Jmeter (open source), HP Loadrunner (commercial, very costly), Neotys Neoload (Commercial, medium cost), IBM RPT ie. Rational Performance tester (Commercial, medium cost), VSTS i.e Microsoft Visual studio(Commercial, cost gets compensated because the development team may already have the licenses).

Other tools used in the industry are:  Opensta (open source), Grinder (Open source)

Step 2: Understand the aim of the client for getting this activity done. This will help you bridge the gap between what client expects and what you deliver.

The image shown below correctly depicts the understanding gaps

Broadly there are three possible reasons (Please add in comments if you can share some more situations)

a) The application is Live and there is some Issue that is hampering the application performance on the live (Production) environment.

Go for Performance Bottleneck Identification and Performance Benchmarking Exercise

The aim should be to make sure that the scenario where the issue occurs is replicated while testing.

Here you need to be very careful in analyzing the problem that is present on Live servers. 

Is it during the high user load time, Is there any conflict between two activities accessing same resources like DB tables, Is it due to some Batch process (Crons, schedulers).

Quick fix:

If the situation is very grave then ask the team owning the servers to monitor the hardware resource consumption (CPU, Memory, Disk I/O) on the server machines.

Ask them to increase the hardware temporarily to overcome the urgent situation (on cloud based environment this is very easy while in case of physical servers they can even go for rented servers till the time application tuning completes)

b) The application is ready for Go live and they want to check how much load the application will be able to handle.[to make Go live or No Go Live decision]

Go for Performance Benchmarking Exercise

This is a very simple activity where you need to measure the current load handling capacity (based on response time SLAs) and the break point of the application.

The most important thing will be to capture the expected user load We will discuss about the user load model later. 

c) A product is being developed and the client wants to make sure that there will be no performance issue when they go and sell it to there customer [this is mostly true for applications with high transactions like in e-commerce, travel or telecom domain]

Go for Performance Bottleneck Identification and Performance Re-Engineering Exercise (finally do provide the performance benchmarks)

In this situation you need to have a sound technical knowledge or have support from technical experts to identify the issues and resolve them.

Step 3: Next step is to Identify the business scenarios to be considered for performance testing. (this is to help you in getting the flows prioritized as testing whole of the application is almost impossible)

Below are some of the factors to be kept in mind while deciding if a flow should be considered or not:

a) Business flows on which a high number of users are expected to work simultaneously.

b) Business flows on which volume (no of records processed or displayed) is high like report generation.

c) Business flows having high visibility (the dashboard seen by the CEO of the company should not be slow).

Step 4: Prepare a User load model

A properly prepared user load model will give you the exact idea of how the users will be using the application as a live site this will also be a check point to see if the scenarios identified (described at Step 3) are properly decided.

For starters, You can use this very simplified table to prepare the workload model.

Sample Workload Model

 *% user Distribution = (Expected Active users during peak hour for that flow/Total expected users)*100
e.g: % user Distribution for login and Logout =(100/302)*100=33.11258

This matrix gives you the real life usage scenario The importance of capturing these statistics correctly is very high because

If the statistics captured are too high than the actual then you may end up procuring a very large server hardware out of which only a small portion gets consumed after the go live.
On the other side if the targets set are too low then the users will start complaining of slowness (If the gap is too high your site may even go down) on the first day itself.


An important factor to keep in mind is the database size, the relevant tables should have equal to or more than the expected number of records stored in them.

How to capture the statistics for User Load model

The marketing/product team should know the target audience as well as the number of targeted reach of the application in the product roadmap.[from them you can get information like "we are expecting to reach 6000 potential users in next 3 months"]
and for getting the breakdown of users across the flows the business analysts can help you in getting the expected user load breakdown.