SYSTEM TESTING

3) SYSTEM TESTING

It is end-to-end testing wherein testing environment is similar to the production environment.

Here, we navigate through all the features of the software and test if the end business / end feature works. We just test the end feature and don’t check for data flow or do functional testing and all.

Let us consider an example to explain System Testing.

Let us consider Citibank wants a software for overdraft feature. It asks IFlex company to develop the software and it provides CRS to develop the feature. The CRS contains how the overdraft feature works,

The difference between personal loan and overdrafts is – personal loans, loans can be provided upto 20 times more than the monthly income and also takes a long time to approve by the manager for personal loan. Whereas, in Overdraft, the loan amount is twice the monthly income and takes hardly a day to be approved by the manager. For ex, if a customer of Citibank wants a overdraft loan of Rs 20,000 over his monthly income of Rs 10,000. The manager approves the loan. Let us say that the interest rate is 2% and the activation fee for the first time is Rs 250. When the customer repays the loan at the end of the month, then the total amount he pays is – 20,000 + (2% of 20,000) + Activation fee ( 250 ) = 20,000 + 400 + 250 = 20,650Rs. 

Now, for the 2^nd time if the same customer wants another overdraft loan – then no activation fee is taken. Now the customer applies for another overdraft loan of Rs20,000. This time the amount he has to repay is – 20,000 + ( 2% of 20,000 ) = 20,400Rs.

The development team develops the software which looks something like this

The development team develops the required software as shown above. The first figure represents the software that can be accessed by the manager only. The 2^nd figure represents the software that can be accessed by the bank’s customers.

Let us consider system testing now. We test for interest calculation when the customer takes overdrafts for the 1^st time and when he takes overdrafts for the 2^nd time.

Scenario 1

1) Login as A – Apply for OD Rs 20000 – Click on Apply – Logout

2) Login as manager – Approve OD of A – Logout

3) Login as A – Check OD Balance – Rs 20000 should be deposited – Logout

4) Change the server date to next 30days

5) Login as A – Check OD Balance – 20000 + 400 + 250 = 20650 – Logout

6) Login as manager – click on Deposit – Deposit Rs 650 – Logout

7) Login as A – Repay OD amount – Check OD balance – Rs 0

8) Login as manager – Click on Deposit – Deposit Rs 20000 to A’s account - logout

9) Login as A – Apply for OD Rs 20000 – Click on Apply – Logout

10) Login as manager – Approve OD of A – Logout

11) Login as A – Check OD Balance – Rs 20000 should be deposited – Logout

12) Change the server date to next 30days

13) Login as A – Check OD Balance – 20000 + 400 = 20400 – Logout

14) Login as manager – Deposit 400 – logout

15) Login as A – repay OD amount – Check OD balance – Rs 0

Scenario 2 – now we test another scenario where in – let us consider that the bank gives an offer that states that – a customer who takes Rs 50000 as OD for the first time will not be charged activation fee and activation fee will not be refunded when he takes another OD for the 3^rd time – we have to test for 3test scenarios – wherein we have to take OD of Rs 50000 for the first time and check for OD Repay Balance after applying for another OD for 3^rd time.

Scenario 3 – now we take in other scenario – let us consider that the software is being used normally by all customers – suddenly Citibank decides to lower the Activation fee to Rs 125 for new customers – we have to test OD for new customers and see if its accepting only Rs 125.

But, then we get a requirement conflict – Suppose the customer has applied for Rs 20000 as OD with the existing Activation Fee for Rs 250. Before the manager is yet to approve it, the bank lowers the activation fee to Rs 125. Now we have to test what Activation Fee is charged for the OD of the Pending customer – In this case, the testing team cannot assume anything – they have to contact the Business Analyst or the Client and find out what they want in such a case.

Thus, if they (Client) give 1 set of requirement, we must come up with maximum possible scenarios.  

Let us consider another example – insurance domain

When we buy a car, we have to obtain an insurance on that. Let us consider that Bajaj Allianz Insurance company is the one which provides this car insurance. The insurance policy works in this way – for the 1^st year when the car is bought, the insurance to be paid is Rs10000. For the 2^nd year, the insurance must be renewed at Rs10000 again. For the 3^rd year, if no claim has been made, then that customer is offered a discount of Rs1500 and the he must have to renew the insurance at Rs8500 only. If an insurance claim has been made, then the insurance must be renewed at Rs10000 only. Bajaj Allianz Insurance wants the s/w to be developed which works as above. Thus it gives Wipro the CRS of above, to develop the s/w.

The development team develops the s/w as shown below,

The above s/w works in this way. When the insurance agent logs in to the home page, he clicks on New Insurance and creates a new insurance policy for the new customer and fills up all the details and the new customer is assigned a IID (Insurance ID) 100. He then pays the insurance amount Rs10,000. After 1year, when the time has come for renewal, then the agent logs in and clicks on Renew Insurance and enters the IID and renews insurance for Rs 10,000. The different test scenarios for the above are,

Scenario 1 :

1) Login as Agent – click on New Insurance – Create IID 100 and Amount Rs10000

2) Change server date by 1year

3) Login as agent – Click on Renew Insurance – and pay amount Rs 10,000

4) Change server date to 1year

5) Login as Agent – Renew Insurance – IID 100 – Insurance Amount must be Rs 8500 since no claim has been made – thus the test is pass

Scenario 2 :

Same as 1) , 2) , 3)

4) Before you renew for the 3^rd year, Claim Insurance – Change server date to 1year

5) Login as Agent – Renew IID 100 – Rs 10000 – No discount should be made because of the claim made above.

Scenario 3 :

Same as 1), 2), 3) and 4) of 1^st scenario

5) Before you renew 3^rd time, - Click on Claim IID 100 , Rs 15,000 – Try to claim the amount – We shouldn’t be able to claim this because the insurance has expired.

Let us consider another example – Advertisements on the Internet

When we open a website, say www.yahoo.com , we see an ad posted on the above – top of the homepage – it remains there for a few seconds before it disappears – this management of ads is done by something known as AMS – Advertisement Management System. Now we do s/w testing for this domain.

The below s/w works in the following way – let’s say that Nokia wants to showcase an ad on Feb 14^th at exactly 9:00AM on Yahoomail’s homepage for the whole of Asia region. The sales manager logs into the website and makes a request for an ad dated for the above day. He attaches a file ( preferably a video or image file of the Ad ) and submits the application. The following day AMS manager (Advertisement Management System ) of Yahoomail logs on and checks the pending ad requests. He checks the Nokia ad and checks if the space is available for the concerned day and time. If yes, then he estimates the cost of putting up the ad – at 10$ per second, the ad cost estimates upto 100$ for 10seconds. He then clicks on payment request and sends the estimate cost along with the request for payment to the Nokia manager. The Nokia manager logs onto Ad status – and checks the payment request and he makes the payment as per all the details and clicks on Submit and Pay. As soon as Yahoo’s AMS manager receives the payment, he deploys the Ad at the concerned date and time on Yahoomail’s Homepage. 

Read More

INTEGRATION TESTING

2) INTEGRATION  TESTING

Testing the data flow or interface between two features is known as integration testing.

Now let us consider the example of banking s/w as shown in the figure above ( amount transfer ).

Scenario 1 – Login as A to amount transfer – send 100rs amount – message should be displayed saying ‘amount transfer successful’ – now logout as A and login as B – go to amount balance and check balance – balance is increased by 100rs – thus integration test is successful.

Scenario 2 – also we check if amount balance has decreased by 100rs in A

Scenario 3 – click on transactions – in A and B, message should be displayed regarding the data and time of amount transfer

Thus in Integration Testing, we must remember the following points,

1) Understand the application thoroughly i.e, understand how each and every feature works. Also understand how each and every feature are related or linked to each other.

2) Identify all possible scenarios

3) Prioritize all the scenarios for execution

4) Test all the scenarios

5) If you find defects, communicate defect report to developers

6) Do positive and negative integration testing. Positive – if there is total balance of 10,000 – send 1000rs and see if amount transfer works fine – if it does, then test is pass. Negative – if there is total balance of 10,000 – send 15000rs and see if amount transfer happens – if it doesn’t happen, test is pass – if it happens, then there is a bug in the program and send it to development team for repairing defects.

Let us consider gmail software as shown above. We first do functional testing for username and password and submit and cancel button. Then we do integration testing for the above. The following scenarios can be considered,

Scenario 1 – Login as A and click on compose mail. We then do functional testing for the individual fields. Now we click on send and also check for save drafts. After we send mail to B, we should check in the sent items folder of A to see if the sent mail is there. Now we logout as A and login as B. Go to inbox and check if the mail has arrived.

Scenario 2 – we also do integration testing for spam folders. If the particular contact has been marked as spam, then any mail sent by that user should go to spam folder and not to the inbox.

We also do functional testing for each and every feature like – inbox,sent items etc .

Let us consider the figure shown above.

We first do functional testing for all the text fields and each and every feature. Then we do integration testing for the related features. We first test for add user and list user and delete user and then edit user and also search user.

Points to remember,

1) There are features we might be doing only doing functional testing and there are features we might be doing both integration and functional testing. It depends on features. 

2) Prioritizing is very important and we should do it at all the stages which means – open the application and decide which feature to be tested first. Go to that feature and decide which component must be tested first. Go to that component and decide what value to be entered first. Don’t apply same rule everywhere!!. Testing logic changes from feature to feature.

3) Focus is important i.e, completely test 1 feature and then only move onto another feature.

4) Between 2 features, we might be doing only positive integration testing or we might be doing both positive and negative integration testing. It depends on the feature.

There are two types of integration testing,

Incremental Integration Testing :

  

Take two modules. Check if data flow between the two is working

fine. If it is, then add one more module and test again. Continue like

this. Incrementally add the modules and test the data flow between

the modules.

There are two ways,

a) Top-down Incremental Integration Testing

b) Bottom – up Incremental Integration Testing

Top-down Incremental Integration Testing:

Bottom-up Integration Testing :

Non – incremental Integration Testing

We use this method when,

a) When data flow is very complex

b) When it is difficult to identify who is parent and who is child.

It is also called Big – Bang method.

Example for Incremental Integration Testing:

In the above example. The development team develops the s/w and send it to the CEO of the testing team. The CEO then logs onto the s/w and creates the username and password and send a mail to a manager and tells him to start testing the s/w. The manager then edits the username and password and creates an username and password and send it to the engineer for testing. This hierarchy from CEO to Testing Engineer is top-down incremental integration testing.

Similarly, the testing engineer once he finishes testing sends a report to the manager, who then sends a report to the CEO. This is known as bottom-up incremental integration testing.

The above example displays a homepage of a gmail inbox. When we click on inbox link, we are transferred to the inbox page. Here we have to do non-incremental integration testing because there is no parent and child process here.

Stub is a dummy module which just receives data and generates a whole lot of expected data, but it behaves like a real module. When a data is sent from real module A to stub B, then B just accepts the data without validating and verifying the data and it generates expected results for the given data.

The function of a driver is it checks the data from A and sends it to stub and also checks the expected data from stub and sends it to A. Driver is one which sets up the test environment and takes care of communications, analyses results and sends the report. We never use stubs and drivers in testing.

In WBT, bottom-up integration testing is preferred because writing drivers is easy. In black-box testing, no preference and depends on the application.  

Read More

BLACK BOX TESTING

It is verifying the functionality ( behavior ) against requirement specifications.

Types of Black Box Testing

1) FUNCTIONAL TESTING

Also called component testing. Testing each and every component thoroughly (rigorously) against requirement specifications is known as functional testing.

For ex, let us consider that Citibank wants a s/w for banking purpose and it asks the company Iflex to develop this s/w. The s/w is something as shown below. When the user clicks his valid user name and enters his password, then he is taken into the homepage. Once inside the homepage, he clicks on amount transfer and the below page is displayed. He enters his valid account number and then the account number to which the money is to be transferred. He then enters the necessary amount and clicks on transfer. The amount must be transferred to the other account number.

Now in black box testing, the test engineer tests the s/w against requirements and checks if the s/w is working correctly as per requirements. 

This is how the requirements given by the client looks like (figure below). It is usually a word document file. Let us consider that Citibank gives a 80pg SRS in MS-WORD format. The test engineer then looks at the requirements and correspondingly checks the s/w.

Now the test engineer does all possible tests on the 2 account numbers. Now, he proceeds with the testing of Amount transfer. These are the following tests he conducts for testing the amount field,

He enters the following data in the amount field,

a) – 100           X                      b) 100$                        X                      c)100.50          X

d) Hundred rupees only        X          e) 100 blank space 0               X

f) 100                           g)0.001                        X

For all the above cases except for f) , it should throw an error message. If it doesn’t throw, then there is a bug in the s/w and the s/w must be sent to the development team to repair the defect.

Thus, during testing, we must remember the following points,

a) We must always start testing the application with the valid data. In the above example for amount transfer, we see that we have entered the valid data 100 only in the 6^th test. This should not be done, because if the valid data itself is not taken correctly, then we need not have to waste our time checking for the invalid data

b) If the application works for valid data, only then we must start testing for invalid data

c) If the application is not working for 1 of the invalid values, then we can continue testing for all the other invalid values and then submit the test report of all the defects for invalid values.

d) In testing, we should not assume or propose requirement. If we have any queries, talk to the one who knows the requirements very well and clarify the queries.

e) We must not do over-testing ( testing for all possible junk values ) or under-testing ( testing for only 1 or 2 values ). We must only try and do optimize testing (testing for only the necessary values- both invalid and valid data).

f) We must do both positive testing (testing for valid data) and negative testing (testing for invalid data).

The characterstics of a good requirement are,

1) Unitary ( cohesive ) – the requirement addresses 1 and only 1 thing

2) Complete – the requirement is fully stated in 1 place with no missing information

3) Consistent – the requirement does not contradict any other requirement and is fully consistent with all authoritative external documentation

4) Non-Conjugated ( Atomic ) – the requirement is atomic i.e, it does not contain certain conjunctions. Ex – “the postal code field must validate American and Canadian postal codes” should be written as two separate requirements : 1) “The postal code field must validate American Postal codes” and 2) “The postal code field must validate Canadian Postal codes”.

5) Traceable – the requirement meets all or part of a business need as stated by stakeholders and authoritatively documented

6) Current – the requirement has not been made obsolete by the passage of time

7) Unambiguous – the requirement is concisely stated without recourse to technical jargon, acronyms etc. it expresses objective facts, not subjective opinions. It is subjective to one and only one interpretation.

8) Mandatory – the requirement represents a stakeholder defined characterstic the absence of which will result in a deficiency that cannot be ameliorated

9) Verifiable – the implementation of the requirement can be determined through one of 4 possible methods – inspection, demonstration, test or analysis.

Read More

WHITE BOX TESTING (WBT)

Entire WBT is done by developers. It is the testing of each and every line of code in the program. Developers do WBT, sends the s/w to testing team. The testing team does black box testing and checks the s/w against requirements and finds any defects and sends it to the developer. The developers fixes the defect and does WBT and sends it to the testing team. Fixing defect means the defect is removed and the feature is working fine.

Test engineers should not be involved in fixing the bug because,

1) if they spend time in fixing the bug, they lose time to catch some more other defects in the s/w

2) fixing a defect might break a lot of other features. Thus, testers should always identify defects and developers should always be involved in fixing defects.

WBT consists of the following tests :

a) Path testing

Write flow graphs and test all the independent paths.

Writing flow graphs means – flow graphs means representing the flow of the  program, how each program is interlinked with one another.

Test all independent paths – Consider a path from main( ) to function 7. Set the parameters and test if the program is correctly in that path. Similarly test all other paths and fix defects.

b) Condition testing

Test all the logical conditions for both true and false values i.e, we check for both “if” and “else” condition.

If( condition) - true

{

                …….

                …….

}

Else - false

{

                …..

                …..

}

The program should work correctly for both conditions i.e, if condition is true, then else should be false and vice-versa

c) Loop testing

Test the loops(for, while, do-while, etc) for all the cycles and also check for terminating condition if working properly and if the size of the condition is sufficient enough.

For ex, let us consider a program where in the developer has given about 1lakh loops.

{

                While ( 1,00,000 )

                …….

                …….

}

We cannot test this manually for all 1lakh cycles. So we write a small program,

Test A

{

                ……

                ……             }

Which checks for all  1lakh loops. This Test A is known as unit test. The test program is written in the same language as the source code program. Developers only write the test program.

Let us consider the above case as shown in the figure. Suppose the project consists of Requirement A,B,C,D,E. Then the developer writes the Codes A,B,C,D,E for the corresponding requirements. The program consists of 100s of lines of code.

As  developers do WBT, they test the 5 programs line by line of code for defects. If in any the program, there is a defect, the developers identify the defect and rectifies it, and they again have to test the program and all the programs again. This involves a lot of time and effort and slows down the project completion time.

Let us consider another case. Suppose the client asks for requirement changes, then the developers have to do the necessary changes and test the entire 5 programs again. This again involves a lot of time and effort.

This drawback can be corrected in the following manner.

We write a test program for the corresponding program. The developers write these test programs in the same language as the source code. The developers then run these test programs also called as unit test programs. These test programs connect to the main programs and run as the programs. Thus if there is any requirement change or defects in the program, then the developers simply make the changes either in the test program / main program and run the test program as a whole. 

The collection of test programs is known as test suite.

As early as we catch the bug, cost of fixing the bug will be less. If we delay in catching the bug, cost of fixing the bug increases exponentially.

Let us consider a program for the calculator. Let us consider the addition function.

The other test programs are,

Y = callAdd.myAdd (10.5, 5.2)

If ( Y = 15.7)

{

            Print (“test is pass”)

}

Else

{

            Print(“test is fail”)

}

Similarly we do for,

Z = 9,00,000 and 1,00,000

P = -10, -5

In all the above test programs, the variables of X, Y, Z, P are calling the function add.java and getting the add. We do the addition manually and check for the remaining results. We can see that the test programs are more tedious and lengthy and time consuming.

Thus, we have readymade programs.

Like, for example, we have a ready-made program named Junit which has a feature named Assert which runs the above and compares the 2 values and automatically prints pass or fail.

Thus we can replace the test programs with these simple programs,

Import Junit

X = callAdd.myAdd (10, 5)

            Call Assert (X, 15)

It automatically prints pass or fail. Similarly we can do for the other tests for Y, Z and P.

d) From memory point of view of testing

The size of code of a program increases because,

i) The logic used by the programmer may vary. If one developer writes a code of 300kb file size, then another developer may write the same program using different logic and write of code of 200kb file size.

ii ) Reuse of code is not there : for example, if for 5 programs of the same s/w, the 1^st ten lines of the program is the same. Then we can write these ten lines as a separate function and write a single function call using which the 5programs can access this function. Also, if any defect is there we can simply change the line of code in the function rather than all the programs.

iii ) Developers declare so many variables, functions which may never be used in any part of the program. Thus, the size of the program increases.

For ex,

Int i = 10 ;

Int j = 15 ;

String S = “hello” ;

. . . . .

. . . . .

. . . . .

.  . . .

. . . . .

Int a = j ;

Create user

{

            . . . .

            . . . .                 100 lines of code

            . . . .

} 

In the above program, we can see that the integer i has never been called anywhere in the program and also the function create user has  never been called anywhere in the program. Thus this leads to memory wastage.

We cannot recognize this error manually by checking the program because of the large program. Thus we have ready-made tools to check for unnecessary variables and functions.

Program A Int i = 10 Int j = 15 String S = “hello” . . . . . . . . Int a = j Create user ( ) {             . . . .             . . . .             . . . . }

We have a tool by name Rational Purify

Programs A, B, and C is given as input to D. D goes into the programs and checks for unused variables. It then gives the result. The developer can then click on the various results and call or delete the unused variables and functions.

This tool is specific only for C, C++ languages. For other languages, we have other similar tools.

iv ) The developer doesn’t use already existing inbuilt functions and sits and writes the entire function using his logic. Thus leads to waste of time and also delays.

Let us consider there is an already inbuilt function sort ( ). Instead the developer sits and writes his own program mysort ( ). This leads to waste of time and effort. Instead he could have used a single function call sort ( ).

e ) Test for response time/ speed/ performance of the program

The reasons for slow performance could be,

i ) Logic used

ii ) Switch case – Do not use nested if, instead use switch case

iii ) Also there must be appropriate use of “or” and “and” for conditional cases.

As the developer is doing WBT, he sees that the program is running slow or the performance of the program is slow. The developer cannot go manually through the code and check which line of the code is slowing the program.

We have a tool by name Rational Quantify to do this job automatically. As soon as all the programs are ready. This tool will go into the program and runs all the programs. The result is shown in the result sheet in the form of thick and thin lines. Thick lines indicate that the piece of code is taking longer time to run. When we double-click on the thick line, then the tool automatically takes us to the line/piece of code which is also colored differently. We can modify that code and again use rational quantify. When the sequence of lines are all thin, we know that the performance of the program has improved.

Developers do a lot of WBT automatically rather than manually because it saves time.

Difference between White Box Testing and Black Box testing

1) White Box Testing                       2) Black Box Testing

a) 1) Done by developers

    2) Done by test engineers

b) 1) Look into the source code and test the logic of the code

     2) Verifying the functionality of the application against requirement specifications

c)  1) Should have knowledge of internal design of the code

     2) No need to have knowledge of internal design of the code

d) 1) Should have knowledge of programming

     2) No need to have knowledge of programming

Read More