Basic funda of QA (Testing )

"No Standards for Testing only the Best Practices"

Thursday, October 23, 2008

Testing Metrics Stuff.

"Metrics are a system of parameters or ways of quantitative and periodic assessment of a process that is to be measured, along with the procedures to carry out such measurement and the procedures for the interpretation of the assessment in the light of previous or comparable assessments. Metrics are usually specialized by the subject area, in which case they are valid only within a certain domain and cannot be directly benchmarked or interpreted outside it."

Metrics are measurements. It is as simple as that. We use them all the time in our everyday lives. Entangling them in wordy definitions is just intended to make them seem more mysterious and technical than they really are.

So what sorts of things do we measure in our daily lives and how do we use them? Shopping for food is a good place to start. At the meat counter, there is a choice of cuts of different kinds of meat, all at different prices. If we just look at the total price, we may be misled. A nice round steak might cost $10.00 while a round roast might cost $8.00 even though it weighs the same as the steak. So to get the best value for our money we tend to look at the price per unit weight. This is a microcosm of the field of metrics.

There are two basis types of metrics. The first type is the elemental or basic measurement such as weight, length, time, volume, and in this example, cost. The second type is derived, normally from the elemental measurements. At the meat counter, the derived metric is dollars/weight (VIZ. $7.49/kg). This is called a normalized metric.

Generally speaking, normalized metrics are the most useful because they allow us to make comparisons between things that are different. Some other examples are miles/gallon, dollars/gallon, dollars/share, dollars/hr, and dollars/square foot to give but a few.

We also see metrics in sports. In hockey its shots on goal and plus/minus ratio. In baseball its batting average and errors per game. All of these numbers are provided in newspapers and sports magazines and if they disappeared there would be a great uproar among fans.

"When you can measure what you are speaking about and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of the meager and unsatisfactory kind." - Lord Kelvin

Now Lord Kelvin wasn't right about everything he spoke about. He predicted that heavier than air flight was impossible. But about metrics, he was dead right.

We as shoppers apply this principle whenever we go to the market. If a cut of meat is marked $10.00 but has no weight assigned, we are likely to look for something else. The same would apply if the weight were given but no price. This is just plain old ordinary common sense. Yet we may go though our professional lives without using metrics to guide us in our work. Maintaining a "meager and unsatisfactory" knowledge about the way you earn your living is probably not the best approach.

Types :

Characteristics of Effective Test Metrics
Ideally, identifying test metrics takes place at the beginning of the project, so incorporation into the appropriate activities is easy. The test metrics you wish to collect need to be:

· quantifiable,
· easy to collect,
· simple,
· meaningful,
· non-threatening.

What is metrics?

A standard of measurement. Software metrics are the statistics describing the structure or content of a program. A metric should be a real objective measurement of something such as number of bugs per lines of code.

There are several test metrics identified as part of the overall testing activity in order to track and measure the entire testing process. These test metrics are collected at each phase of the testing life cycle /SDLC and analyzed and appropriate process improvements are determined and implemented as a result of these test metrics that are constantly collected and evaluated as a parallel activity together with testing both for manual and automated testing irrespective of the type of application. The test metrics can be broadly classified into the following three categories such as:

1. Project Related Metrics – such as Test Size, # of Test Cases tested per day –Automated (NTTA), # of Test Cases tested per day –Manual (NTTM), # of Test Cases created per day – Manual (TCED), Total number of review defects (RD), Total number of testing defects (TD), etc
2. Process Related Metrics – such as Schedule Adherence (SA), Effort Variance (EV), Schedule Slippage (SS), Test Cases and Scripts Rework Effort, etc.
3. Customer related Metrics – such as Percentage of defects leaked per release (PDLPR), Percentage of automation per release (PAPR), Application Stability Index (ASI), etc.

what are different types of metrics to measure software quality? and explain them briefly?

Metric is a mathematical number that shows a relationship between two variables. It is a quantitative measure of the degree to which a system, component or process possesses a given attribute. Software Metrics are measures that are used to quantify the software, software development resource and software development process.

Types of Metrics :

Metric generally
classified into 2 types.
· Process Metric
· Product Metric
Process Metric: Metric used to measure the characteristic of the methods, techniques and tools employed in developing, implementing and maintaining the software system.
Product Metric: Metric used to measure the characteristic of the documentation and code
The metrics for the test process would include status of test activities against the plan, test coverage achieved so far, among others. An important metric is the number of defects found in internal testing compared to the defects found in customer tests, which indicate the effectiveness of the test process itself.


What type of metrics would you use?

QAM:Quality Assurance Matrix
TMM:Test
Management Matrix
PCM:Process Compatibility
Matrix

What is Test Matric?

Test metrics A document showing the relation ship between test requirements and test cases.

No comments: