The Approach (Part3)

The day to day activities and the existing scenarios and processes have to be studied and  the  list of observations across the teams development, QA, product, technical support and customer support teams should be plotted down.

Also, the team members experience and skill set should be discussed and a set of best practices and processes should be enumerated and a feasibility analysis should made.

The collected data needs to be analyzed and bounced in sessions of brain storming and later compared them with the collected set of heuristics.

As long as the team is involved in the change,  and their concerns are addressed with their solutions, the team will eagerly switch over to the new processes and their motivation level will be high, brimming with ideas.

Action Items

As per the metrics that are collected to identify the problem areas, the changes should be implemented in a phased out manner

Analysis of the collected metrics could be done by the following actions (the list is not all inclusive)

§  Measurable metrics like

-      Defect Finding Rate

-      Defect Fixing Rate

-      Defect distribution across components

-      Defect cause distribution chart

-      Closed defect distribution

-      Code coverage

-      Number of testcases executed to find number of critical or major defects

-      Defects found outside of the testcases etc., 

§  Evaluate the present methods and procedures to identify these metrics or to fetch new ones with new processes to improve the quality quotient of efficiency

§  Consult any Standards, as per the organization's need for certification like TMM (I) etc,(if required)  the processes need to be inline with those standards

§  Collect a set of heuristics from various members based on their inputs, a feasibility analysis for processes of best fit should be arrived and a baseline should be declared.  Speak to consultants and people from similar industry, books, articles and research materials to pitch in for the areas where more expertise is required.

Checklists

When the processes are in put in place, a check process is required and the following are the list of checklists that can be deployed to carry out the validation and track the progress of the changes that are to be deployed.

§  The present state to be recorded as a snapshot to compare against the results post the processes change implementations

§  The awareness of the new changes to be introduced to the team with the help of many sessions, documents and discussions

§  Checklists, review points etc to be put in place to ensure it is being following by all the members.

§  Common board for the members to state any improvements or suggestions to fine tune the baseline set

§  Once the initialization is done, the status is to be updated by the members along with any difficulties and suggestions

§  The progress to be tracked through various status details, charts and graphs

§  Feedback or survey mechanisms to be in place to measure the value add that resulted out of this exercise and keep the processes up to date

§  The outcome to be reviewed periodically and the changes should be proposed to get implemented and the cycle to continue till the processes reach end of life

§  Postmortem or root-cause analysis to be done at the end of every project

§  Assign owners to each task to monitor the progress at the micro level and the end report to be consolidated to review the outcomes

Contd... (Part 2)

§ The structure of the QA plan and the level of effort estimation could be random

§  Handling the requirement changes and the code drops are mostly on demand

§  Lack of plan for preconditions, testdata etc

§  Most importantly, all the relevant documents including the product specifications etc are either mailed or stored in a common location

§  Test measurement - in terms of the number of defects and the number of testcases and hence difficult to chart out a test measurement strategy with the existing metrics

§  Estimating or arriving at the value of test and QA measurement in terms of ROI with respect to processes, particularly in testing is difficult

 §  Managing and communicating the status of a test effort was cumbersome

§  Usage of graphs and charts are done using excel sheets for different projects and hence a correlation would be difficult to achieve

§  Metrics like Defect Finding Rate, Defect Fixing Rate, Defect distribution across components, Defect cause distribution chart, Closed defect distribution have to be manually calculated each time.

§  ROI depended on the actual and estimated level of effort from various teams and it was difficult to find the support evidences.

Pitfalls without a Well formed QA Team (Part2)

Let us try and identify the pitfalls, when such a solution is not in place. We shall include the individual's skill set in the later point of the discussion. The below given is not going to be an exhaustive list. But these are some of the problems that we face in lieu of absence of best practices incorporated in the system.

§  The QA team could be very naive in its processes, methods, techniques and metrics.

§  Development probably could be more or less completed by the time the requirements reaches the QA team. The changes in requirements might be known only when the testcases contradicted the workflows. Since QA was not involved in the requirements stage, the following were the issues that the team will encounter

-      Critical defects defects will go unnoticed or may be closed as ‘By design’

-      Loss of time in re-designing or fixing issues

-      Significant turnaround time from development team and QA team

-      Large amount of Re-work

-      Issues had to be release noted as they are found in the last min
-   Losing the big picture
-   Failure to gauge the impact on the existing baseline 
-   Losing the long term sight
-  Push the dates further or release it as it is, only to be bombarded with the customer tickets

§  There may not be a common agreed upon format of test artifacts or deliverable or they might be the ones that was created ages ago, probably when the software was in its nascent stage. and now the current road map, objectives and the product features and the product positioning have changed. Also, the functional behavior and technologies are very different now.

§  Each member has their own way of doing things and they are very random .

§  There are little standards and procedures or templates in place. or they  were not reviewed for its gaps...

§  Microsoft excel still serves as Test case management, test execution and test report tool. Hence updating of the sheet by multiple members is a task in itself

§   Files are maintained either in remote or local fileserver across all releases.

§  Defect Tracking is done with a primitive tool that had limited features

§  QA Project plan, tracking and metrics are very disjointed and incorporating the learning from the previous project could be tough.

§  Identification of regression suite

§  There was no common place where the product, development, QA, customer support and management can get their respective queries answered.
§  Over a period of time, there is a chance that work will get monotonous and repeated updation of data in different sheets, can bring down the morale of the team and result in lack of motivation.
§  This could result in intellectual wastage 

Thus, the need for identifying, introducing, streamlining, improving the QA processes, methods, techniques and metrics is inevitable

Setting it Right - Foreword (Part1)

A naive QA team has random processes and the outcome is a chaos. The question here is how to streamline them?

I have put in my efforts to come up with an article from my past and current experiences.

It is going to be an enriching experience for me as well as the readers through this article, where a lot of thought process has been put to work and has been reflected and bounced over many times to extract the fine nuances and present this in a form that can serve as a guide for the aspiring quality professionals.

The article will aim to present a holistic picture of streamlining the QA that will include defining, deploying, identifying gaps, improving the processes, planning, execution, measurement and reporting. 

The reader may find discussions that highlight the importance of the active participation from all teams QA, dev, product, etc. Inputs from them and effective communication are crucial for the successful installation of the proposed changes and improvements 

So, how are we going to have a simple, easy to deploy  solution?
It starts with, identifying, introducing, streamlining, improving the processes, methods, techniques, reporting and metrics for any QA Team

Sorry Mr. Michael Bolten, I disagree.

When it comes to quality there are a lot of variations that we keep hearing.
Quality control, testing, quality assurance, quality assistance etc.
There was a presentation, where the focus was on the above "The difference"
The point that got emphasized is, that testers are the gate keepers, and they have no say in decision making!
Further it was added that one cannot assure quality unless one writes either code or defines requirements!!
So, the conclusion made was "tester cannot assure quality, they can only assist"
The highlight was quality control is only for the manufacturing industry it seems!!!
Then what about test driven development, dev-qa pair wise development etc...
Several internet reference were quoted regarding this and the first word that came to my mind was RIDICULOUS!
When you know the system end to end, and when changes come in, you'll question a lot about it.
About its impact, touch points, effects of the changes with respect to the context, feasibility in terms of usability, UI, predictability, consistency, performance etc.
How would it affect the existing end-users who are on the base-lined code?
It is not only the question of testing the application. Only then such doubts will arise.
When you ask the questions, you are verifying the requirements and product team benefits out of it. Also, the design gets improved.
Then how are you not helping to assure quality? Nobody is stopping you from upgrading yourself from tester to quality engineer.
For some people quality assurance is auditing, process maturity etc.
When you can question, analyze, improve, solve, learn, validate, verify, check, test, streamline, emulate, simulate, present, report etc and  address the various dimensions attributing quality, why do you want to restrict yourself? Go for it! Build your own collective skill set and common sense.
Everybody has their own definition and views on each of them. And remember they are OPINIONS.
You have every right to bounce and reflect your thoughts on the same and do some retrospection on your own.
Know the industry standard best practices, the heuristics, the empirical methods, models, research materials etc
Match it up with the work that you do and what really is expected. Happy value adding.

Defect Template

I have come up with the following defect template to ensure the uniformity among the members who are part of the testing/validation team.

Summary

1.     Issue summary should be crisp [E.g. user with option A  is not listed under the report ABC]

2.     Pre-fix the summary with the Release Index, if any. [E.g. SFXRelease - ]

Description

Test Case ID:

Customer: , Browser name, Version:

Steps to Duplicate

Pre-requisites (If any): [E.g. Option A should be available for the customer for which the user for feature 1 is being created]

Checklist for Field Validation

The following is the checklist that can be used to validate the fields for any given UI. This should be helpful for any application that has predominant number of fields.