Friday 24 January 2014

GREY BOX TESTING

Grey Box testing


Grey Box testing is a technique to test the application with limited knowledge of the internal workings of an application. In software testing, the term “the more you know the better” carries a lot of weight when testing an application.
Mastering the domain of a system always gives the tester an edge over someone with limited domain knowledge. Unlike black box testing, where the tester only tests the application’s user interface, in grey box testing, the tester has access to design documents and the database. Having this knowledge, the tester is able to better prepare test data and test scenarios when making the test plan.


Advantages

  • Offers combined benefits of black box and white box testing wherever possible.
  •  Grey box testers don’t rely on the source code; instead they rely on interface definition and functional specifications.
  • Based on the limited information available, a grey box tester can design excellent test scenarios especially around communication protocols and data type handling.
  • The test is done from the point of view of the user and not the designer.

Disadvantages


  • Since the access to source code is not available, the ability to go over the code and test coverage is limited.
  • The tests can be redundant if the software designer has already run a test case.
  • Testing every possible input stream is unrealistic because it would take an unreasonable amount of time; therefore, many program paths will go untested.

Differences Between terms in Software Testing

Differences Between Audit and Inspection

Audit: A systematic process to determine how the actual testing process is conducted within an organization or a team. Generally, it is an independent examination of processes which are involved during the testing of software. As per IEEE, it is a review of documented processes whether organizations implements and follows the processes or not. Types of Audit include the Legal Compliance Audit, Internal Audit, and System Audit.


Inspection: A formal technique which involves the formal or informal technical reviews of any artifact by identifying any error or gap. Inspection includes the formal as well as informal technical reviews. As per IEEE94, Inspection is a formal evaluation technique in which software requirements, design, or code are examined in detail by a person or group other than the author to detect faults, violations of development standards, and other problems.
Formal Inspection meetings may have following process: Planning, Overview Preparation, Inspection Meeting, Rework, and Follow-up.


Difference between Testing and Debugging

Testing: It involves the identification of bug/error/defect in the software without correcting it. Normally professionals with a Quality Assurance background are involved in the identification of bugs. Testing is performed in the testing phase.

Debugging: It involves identifying, isolating and fixing the problems/bug. Developers who code the software conduct debugging upon encountering an error in the code. Debugging is the part of White box or Unit Testing. Debugging can be performed in the development phase while conducting Unit Testing or in phases while fixing the reported bugs.

Difference Between Testing, Quality Assurance and Quality Control

Most people are confused with the concepts and difference between Quality Assurance, Quality Control and Testing. Although they are interrelated and at some level they can be considered as the same activities, but there is indeed a difference between them. Mentioned below are the definitions and differences between them:


Difference Between Verification and Validation




Difference Between Black Box Testing White Box Testing and Grey Box Testing





User Acceptance Testing (UAT)

What is User Acceptance Testing?
-----------------------------------

User Acceptance Testing (UAT) - also called beta testing, application testing, and/or end user testing - is a phase of software development in which the software is tested in the "real world" by the intended audience or a business representative. Whilst the technical testing of IT systems is a highly professional and exhaustive process, testing of business functionality is an entirely different proposition.

User Acceptance Testing (UAT) is one sure way to reduce or eliminate change requests, and drastically reduce project costs.

Advantages of UAT

•       Ensuring that the application behaves exactly as expected.
•       Reducing the total cost of ownership.
•       Reducing the cost of developing the application.


Tasks of User Acceptance Testing
When performing UAT, there are seven (7) basic steps to ensure the system is tested thoroughly and meets the business needs.

1 – Analyze Business Requirements
2 – Identify UAT Scenarios
3 – Define the UAT Test Plan
4 – Create UAT Test Cases
5 – Run the Tests
6 – Record the Results
7 – Confirm Business Objectives are met


Alpha Testing

This test is the first stage of testing and will be performed amongst the teams (developer and QA teams). Unit testing, integration testing and system testing when combined are known as alpha testing. During this phase, the following will be tested in the application:
  • Spelling Mistakes
  • Broken Links
  • Cloudy Directions
  • The Application will be tested on machines with the lowest specification to test loading times and any latency problems.

Beta Testing

This test is performed after Alpha testing has been successfully performed. In beta testing a sample of the intended audience tests the application. Beta testing is also known as pre-release testing. Beta test versions of software are ideally distributed to a wide audience on the Web, partly to give the program a "real-world" test and partly to provide a preview of the next release. In this phase the audience will be testing the following:
  • Users will install, run the application and send their feedback to the project team.
  • Typographical errors, confusing application flow, and even crashes.
  • Getting the feedback, the project team can fix the problems before releasing the software to the actual users.
  • The more issues you fix that solve real user problems, the higher the quality of your application will be.
  • Having a higher-quality application when you release to the general public will increase customer satisfaction.




Summary
Whether your organization designates the functional role associated with testing as a Business Analyst, Tester, or Quality Assurance professional, User Acceptance Testing done well will engage those responsible very early on in the project development cycle. DevelopMentor Business Analysis curriculum offers many opportunities for learning the skills required of those responsible for UAT.

Thursday 23 January 2014

DEFECT TRACKING


DEFECT TRACKING

Defect : If a feature is not working according to the requirement, it is called a defect.
Deviation from requirement specification is called as defect.

Developer develops the product – test engineer starts testing the product – he finds a defect – now the TE must send the defect to the development team.

He prepares a defect report – and sends a mail to the Development lead saying “bug open.
Development lead looks at the mail and at the bug – and by looking at the bug – he comes to know to which development engineer developed that feature which had a bug – and sends the defect report to that particular developer and says “bug assigned”.

The development engineer fixes the bug – and sends a mail to the test engineer saying “bug fixed” – he also “cc mail” to the development lead.
Now the TE takes the new build in which the bug is fixed – and if the bug is really fixed – then sends a mail to the developer saying “bug closed” and also “cc mail” to the development lead.

Every bug will have an unique number.
If the defect is still there – it will be sent back as “bug reopen”. 

DEFECT LIFE CYCLE



DEFECT REPORT

Defect ID – it is an unique number given to the defect

Test Case Name – whenever we find a defect, we send the defect report and not the test case to the developer. For defect tracking, we only track the defect report and not the test case. Test case is only for reference for the TE. We always only send the defect report whenever we catch a bug.

Given below is – how a defect report looks like




SOFTWARE TEST LIFE CYCLE (STLC)

SOFTWARE TEST LIFE CYCLE (STLC)
Testing itself has many phases i.e is called as STLC.
STLC is part of SDLC
Defect Life Cycle is a part of STLC


Requirement is the input for testing.

Test Plan – is a document which derives all future activities of the project. All future testing activities is planned and put into a document and this document is known as Test Plan. It contains – number of engineers needed for the project, who should test which feature, how the defects must be communicated to the development team, when we should start and finish writing test cases, executing test cases, what are the types of testing we use to test for the application etc.

Write test case – we write test cases for each feature. These test cases are reviewed, and after all mistakes are corrected and once the test cases are approved – then they are stored in the test case repository.

Traceability Matrix – it is a document which ensures that every requirement has a test case .
Test cases are written by looking at the requirements and test cases are executed by looking at the test cases. If any requirement is missed i.e, test cases are not written for a particular requirement, then that particular feature is not tested which may have some bugs. Just to ensure that all the requirements are converted, traceability matrix is written. This is shown below,

Defect Tracking – any bug found by the testing team is sent to the development team. This bug has to be checked by the testing team if it has been fixed by the developers.

Test Execution Report :- Send it to customer – contains a list of bugs(major, minor and critical), summary of test pass, fail etc and when this is sent, according to the customer – the project is over.
TER is prepared after every test cycle and sent to development team, testing team, management and customer(depends if it is a fixed bid project or time & material bid project).
The last TER of the last test cycle is always sent to the customer. And this means that the project is over-according to the customer.

Retrospect meeting – (also called Post Mortem Meeting / Project Closure Meeting)
The Test Manager calls everyone in the testing team for a meeting and asks them for a list of mistakes and achievements in the project.
This is done by test lead or test manager. Here, the manager documents this retrospect meeting and stores it in QMS (Quality Management System). It is a folder, where inside this folder, there is another folder called Retrospect folder and here this excel sheet document is stored. When we get new project, while we write the test plan – we will open this retrospect file and will try and implement the good practices and correct the mistakes.

REQUIREMENTS COLLECTION / SYSTEM STUDY

The requirements can be in any of the following forms,
·         CRS (Customer Requirement Specification)
·         SRS (System Requirement Specification)
·         FS (Functional Specification)
·         If we don’t have requirements and if we are given only the application, then we do exploratory testing.
·         Use case

Use Case
Use case is a pictorial representation of requirements. It explains how the end user interacts with the application. It gives all possible ways of how the end user uses the application.

TEST PLAN
Test plan is a document which drives all future testing activities.
Test plan is prepared by Test manager(20%), Test Engineer(20%) and by Test Lead(60%).

There are 15 sections in a test plan. We will look at each one of them below:-

1) Objective 
It gives the aim of preparing test plan i.e, why are we preparing this test plan. 

2) SCOPE

3) TESTING METHODOLOGIES

For example, 
Smoke testing                        Functional testing                  Integration testing
System testing                       Adhoc testing                         Compatibility testing
Regression testing                Globalization testing             Accessibility testing
Usability testing                   Performance testing

4) APPROACH

The way we go about testing the product in future,
a)      By writing high level scenarios
b)      By writing flow graphs

5) ASSUMPTIONS
When writing test plans, certain assumptions would be made like technology, resources etc.

6) RISKS
If the assumptions fail, risks are involved.

7) CONTINGENCY PLAN OR MITIGATION PLAN OR BACK-UP PLAN
To overcome the risks, a contingency plan has to be made. Atleast to reduce the percentage from 100% to 20%

8) ROLES AND RESPONSIBILITIES


9) SCHEDULES :-
This section contains – when exactly each activity should start and end? Exact date should be mentioned and for every activity, date will be specified.

10) DEFECT TRACKING
In this section, we mention – how to communicate the defects found during testing to the development team and also how development team should respond to it. We should also mention the priority of the defect – high, medium, low.

11) Test Environment
It describes the environment where the test has to be carried out.
It will give details about the software, hardware and the procedure to install the software.

12) Entry and Exit Criteria

Entry Criteria
a) WBT should be over
b) Test cases should be ready
c) Product should be installed with proper test environment
d) Test data should be ready

e) Resources should be availablea) WBT should be over

Exit Criteria
1)Based on %age test execution
2)Based on %age test pass

3) Based on severity

13) TEST AUTOMATION

At this stage it is decided that 
1)Which features should be automated
2) Features not to be automated
3)Which is the automation tool you are planning to use
4) What is the automation framework you are planning to use

14) DELIVERABLES

It is the output from the testing team. It contains what we will deliver to the customer at the end of the project. It has the following sections :-
v   Test Plan
v  Test Cases
v  Test Scripts
v  Traceability Matrix
v  Defect Report
v  Test Execution Report
v  Graphs and Metrics
v  Release Note

15)TEMPLATES
This section contains all the templates for the documents which will be used in the project. Only these templates will be used by all the test engineers in the project so as to provide uniformity to the entire project.

The various documents which will be covered in the Template section are,
·         Test Case
·         Traceability Matrix
·         Test Execution Report
·         Defect Report
·         Test Case Review Template


TEST DESIGN

At this stage the Test plan is implemented.The Test case are designed for each functionality at this stage

TRACEABILITY MATRIX

Traceability Matrix is a document which has got the mapping between requirements and test cases. We write TM to make sure that every requirement has got atleast 1 test case.

Advantages of Traceability Matrix
·         Ensures that every requirement has atleast 1 test case
·         If suddenly requirement is changed – we will be knowing which is the exact test case or automation script to  be modified
·         We will come to know which test case should be executed manually and which are to be done automatically.

TEST EXECUTION

Here, we test the product.
We test repeatedly for 40 – 60 cycles. We do all types of testing on the application. Test Execution is the phase where we spend 80% of our time on the project. Only 20% is spent on the remaining stages.

TEST REPORT
At this stage the Defect is raised and assigned to the test manager who will assign it to the responsible person to resolve the defect.