Practice or Process?

It is tricky being a practice or a process person. There is a subtle difference though
Some one said, " I don’t like processes, as I am a responsible person, leave it to my own way, I'll get it done!"
Later, he comes back and shares his view points on how to make the job easier by providing some insights.
He wanted those insights to be followed by everybody!! Wouldn’t that become a process, if that is enforced to maintain uniformity?
Instead, it would be lot more easier to follow, when the same person is willing to take ideas from others and fine tune his contribution.
That is how processes evolve. It is everybody's duty to revisit them to identify the gaps or evaluate the relevance and change them.
For a practice person, it is a different game. They'll tend to derive and follow the best practices and evaluate them time to time.
They'll never wait for an enforcement and will always volunteer and contribute to the betterment of the outcome.

User Story Template

Testplan constitution

Scope Layout

There are a lot of mind map tools that are available that can help us in preparing the scope layout.
But topology is a primitive method can serve us, when such tool is not in place.

It presents a pictorial scope of requirements under consideration for the current product/project/module

Topological Space

1. Collection/group of classifications with certain attributes together with Module names for the product

2. A geometric object with n number of attributes that can be n dimensional

3. The expansion of the object has to be continuous

Topological Space

1. Collection/group of classifications with certain attributes together with Module names for the product

2. A geometric object with n number of attributes that can be n dimensional

3. The expansion of the object has to be continuous

Fig Topological Space

Purpose

1. Pictorial layout/overview of requirements for various modules and scenarios

2. Track Status

E

D

S

                          Dataset                             TestCases                               Environment

G

Y

R

                          Not Started                    Awaiting Review                     Completed

 Fig Track Status

3. Possible coverage of requirements for the tests                   

4. Possible to incorporate the changes in the requirements

5. Possible place-holders for the items to be tracked later

Sample

Contd... (Part3)

The process improvements are useful only when success can be measured. The outcome should benefit all the stakeholders including top management, Product, development, QA  and CPS teams which should be a value add for the members.

The following are the metrics that would be used to measure the success of the process improvement changes

§  Uniformity in writing testcases

-      This would be easier to process the entire testcase suite from the excel to convert to any format that the test management tool requires.

§  Have an updated list of tests at any given instance

-      This kind of version control in place avoids the redundancy of the tests when the requirements change.

-      Also, this reduces the time taken to make the changes as the structure is well organized

§  Review with the checklists that is published

-      The avoids any major gaps in identifying all the scenarios that are required to test that particular feature

§  Defect tracking

-      The consolidation of the results or the snapshot of the project can be provided live at any moment.

-      Easier to know the state of the project with the charts and graphs and other important metrics that the tool provides.

-      This reduces a lot of manual labor and the time can be effectively spent for QA purposes

§  Number of critical defects found at the earliest

-      As the testcases are written based on the outlined methodologies in line with the published topology and assigned with priority the critical defects are found in the initial phase thereby avoiding the last minute emergency fixes

-      This enables us to stick to the deadline

§  The work is legible and a new member acquiring the knowledge transfer has a quick turn-around time

-      This reduces the time to ramp up the new members

§  Reduced number of defects from the customer

-      Reduced re-work after the release of the project to the customer increases customer satisfaction and reduces the overhead incurred .

Continuous improvement process should be engaged , for periodic review to find the gaps in them or to modify and replace them as required.

The main business objective is to deliver a quality product to the customer by meeting the customer’s expectations within the stipulated time with less overhead. The return on investment was put forward in the following terms

-      Finding the gaps in the requirements at the earliest

-      Identifying the design faults

-      Matching the planned effort with the actual

-      Finding the critical issues at the earliest

-      Reduced re-work

-      Quick turn-around time

-      Reduced cost by using tools effectively

-      Better visibility to the top management

-      Increased customer Satisfaction  

§  The identified processes should be incorporated along with the testplan and sessions were conducted to educate the team about the processes.

§  The member’s performance to be linked to adherence of the processes

§  Process improvement effort from the teams to be part of the recognitions

§  Assigned ownerships for the tasks

§  Day in a life scenarios were demonstrated while creating the process awareness among the members.

§  Recommendations, learnings and improvements to be carried over to the next project in the pipeline

§  Provision to engage the members to come up with continuous improvement processes, to reduce the turn-around time.

§  To acknowledge and reward the rightful candidates

The Approach (Part3)

The day to day activities and the existing scenarios and processes have to be studied and  the  list of observations across the teams development, QA, product, technical support and customer support teams should be plotted down.

Also, the team members experience and skill set should be discussed and a set of best practices and processes should be enumerated and a feasibility analysis should made.

The collected data needs to be analyzed and bounced in sessions of brain storming and later compared them with the collected set of heuristics.

As long as the team is involved in the change,  and their concerns are addressed with their solutions, the team will eagerly switch over to the new processes and their motivation level will be high, brimming with ideas.

Action Items

As per the metrics that are collected to identify the problem areas, the changes should be implemented in a phased out manner

Analysis of the collected metrics could be done by the following actions (the list is not all inclusive)

§  Measurable metrics like

-      Defect Finding Rate

-      Defect Fixing Rate

-      Defect distribution across components

-      Defect cause distribution chart

-      Closed defect distribution

-      Code coverage

-      Number of testcases executed to find number of critical or major defects

-      Defects found outside of the testcases etc., 

§  Evaluate the present methods and procedures to identify these metrics or to fetch new ones with new processes to improve the quality quotient of efficiency

§  Consult any Standards, as per the organization's need for certification like TMM (I) etc,(if required)  the processes need to be inline with those standards

§  Collect a set of heuristics from various members based on their inputs, a feasibility analysis for processes of best fit should be arrived and a baseline should be declared.  Speak to consultants and people from similar industry, books, articles and research materials to pitch in for the areas where more expertise is required.

Checklists

When the processes are in put in place, a check process is required and the following are the list of checklists that can be deployed to carry out the validation and track the progress of the changes that are to be deployed.

§  The present state to be recorded as a snapshot to compare against the results post the processes change implementations

§  The awareness of the new changes to be introduced to the team with the help of many sessions, documents and discussions

§  Checklists, review points etc to be put in place to ensure it is being following by all the members.

§  Common board for the members to state any improvements or suggestions to fine tune the baseline set

§  Once the initialization is done, the status is to be updated by the members along with any difficulties and suggestions

§  The progress to be tracked through various status details, charts and graphs

§  Feedback or survey mechanisms to be in place to measure the value add that resulted out of this exercise and keep the processes up to date

§  The outcome to be reviewed periodically and the changes should be proposed to get implemented and the cycle to continue till the processes reach end of life

§  Postmortem or root-cause analysis to be done at the end of every project

§  Assign owners to each task to monitor the progress at the micro level and the end report to be consolidated to review the outcomes

Contd... (Part 2)

§ The structure of the QA plan and the level of effort estimation could be random

§  Handling the requirement changes and the code drops are mostly on demand

§  Lack of plan for preconditions, testdata etc

§  Most importantly, all the relevant documents including the product specifications etc are either mailed or stored in a common location

§  Test measurement - in terms of the number of defects and the number of testcases and hence difficult to chart out a test measurement strategy with the existing metrics

§  Estimating or arriving at the value of test and QA measurement in terms of ROI with respect to processes, particularly in testing is difficult

 §  Managing and communicating the status of a test effort was cumbersome

§  Usage of graphs and charts are done using excel sheets for different projects and hence a correlation would be difficult to achieve

§  Metrics like Defect Finding Rate, Defect Fixing Rate, Defect distribution across components, Defect cause distribution chart, Closed defect distribution have to be manually calculated each time.

§  ROI depended on the actual and estimated level of effort from various teams and it was difficult to find the support evidences.

Pitfalls without a Well formed QA Team (Part2)

Let us try and identify the pitfalls, when such a solution is not in place. We shall include the individual's skill set in the later point of the discussion. The below given is not going to be an exhaustive list. But these are some of the problems that we face in lieu of absence of best practices incorporated in the system.

§  The QA team could be very naive in its processes, methods, techniques and metrics.

§  Development probably could be more or less completed by the time the requirements reaches the QA team. The changes in requirements might be known only when the testcases contradicted the workflows. Since QA was not involved in the requirements stage, the following were the issues that the team will encounter

-      Critical defects defects will go unnoticed or may be closed as ‘By design’

-      Loss of time in re-designing or fixing issues

-      Significant turnaround time from development team and QA team

-      Large amount of Re-work

-      Issues had to be release noted as they are found in the last min
-   Losing the big picture
-   Failure to gauge the impact on the existing baseline 
-   Losing the long term sight
-  Push the dates further or release it as it is, only to be bombarded with the customer tickets

§  There may not be a common agreed upon format of test artifacts or deliverable or they might be the ones that was created ages ago, probably when the software was in its nascent stage. and now the current road map, objectives and the product features and the product positioning have changed. Also, the functional behavior and technologies are very different now.

§  Each member has their own way of doing things and they are very random .

§  There are little standards and procedures or templates in place. or they  were not reviewed for its gaps...

§  Microsoft excel still serves as Test case management, test execution and test report tool. Hence updating of the sheet by multiple members is a task in itself

§   Files are maintained either in remote or local fileserver across all releases.

§  Defect Tracking is done with a primitive tool that had limited features

§  QA Project plan, tracking and metrics are very disjointed and incorporating the learning from the previous project could be tough.

§  Identification of regression suite

§  There was no common place where the product, development, QA, customer support and management can get their respective queries answered.
§  Over a period of time, there is a chance that work will get monotonous and repeated updation of data in different sheets, can bring down the morale of the team and result in lack of motivation.
§  This could result in intellectual wastage 

Thus, the need for identifying, introducing, streamlining, improving the QA processes, methods, techniques and metrics is inevitable

Setting it Right - Foreword (Part1)

A naive QA team has random processes and the outcome is a chaos. The question here is how to streamline them?

I have put in my efforts to come up with an article from my past and current experiences.

It is going to be an enriching experience for me as well as the readers through this article, where a lot of thought process has been put to work and has been reflected and bounced over many times to extract the fine nuances and present this in a form that can serve as a guide for the aspiring quality professionals.

The article will aim to present a holistic picture of streamlining the QA that will include defining, deploying, identifying gaps, improving the processes, planning, execution, measurement and reporting. 

The reader may find discussions that highlight the importance of the active participation from all teams QA, dev, product, etc. Inputs from them and effective communication are crucial for the successful installation of the proposed changes and improvements 

So, how are we going to have a simple, easy to deploy  solution?
It starts with, identifying, introducing, streamlining, improving the processes, methods, techniques, reporting and metrics for any QA Team

Sorry Mr. Michael Bolten, I disagree.

When it comes to quality there are a lot of variations that we keep hearing.
Quality control, testing, quality assurance, quality assistance etc.
There was a presentation, where the focus was on the above "The difference"
The point that got emphasized is, that testers are the gate keepers, and they have no say in decision making!
Further it was added that one cannot assure quality unless one writes either code or defines requirements!!
So, the conclusion made was "tester cannot assure quality, they can only assist"
The highlight was quality control is only for the manufacturing industry it seems!!!
Then what about test driven development, dev-qa pair wise development etc...
Several internet reference were quoted regarding this and the first word that came to my mind was RIDICULOUS!
When you know the system end to end, and when changes come in, you'll question a lot about it.
About its impact, touch points, effects of the changes with respect to the context, feasibility in terms of usability, UI, predictability, consistency, performance etc.
How would it affect the existing end-users who are on the base-lined code?
It is not only the question of testing the application. Only then such doubts will arise.
When you ask the questions, you are verifying the requirements and product team benefits out of it. Also, the design gets improved.
Then how are you not helping to assure quality? Nobody is stopping you from upgrading yourself from tester to quality engineer.
For some people quality assurance is auditing, process maturity etc.
When you can question, analyze, improve, solve, learn, validate, verify, check, test, streamline, emulate, simulate, present, report etc and  address the various dimensions attributing quality, why do you want to restrict yourself? Go for it! Build your own collective skill set and common sense.
Everybody has their own definition and views on each of them. And remember they are OPINIONS.
You have every right to bounce and reflect your thoughts on the same and do some retrospection on your own.
Know the industry standard best practices, the heuristics, the empirical methods, models, research materials etc
Match it up with the work that you do and what really is expected. Happy value adding.

Defect Template

I have come up with the following defect template to ensure the uniformity among the members who are part of the testing/validation team.

Summary

1.     Issue summary should be crisp [E.g. user with option A  is not listed under the report ABC]

2.     Pre-fix the summary with the Release Index, if any. [E.g. SFXRelease - ]

Description

Test Case ID:

Customer: , Browser name, Version:

Steps to Duplicate

Pre-requisites (If any): [E.g. Option A should be available for the customer for which the user for feature 1 is being created]

Checklist for Field Validation

The following is the checklist that can be used to validate the fields for any given UI. This should be helpful for any application that has predominant number of fields.