Basic funda of QA (Testing )

"No Standards for Testing only the Best Practices"

Thursday, October 23, 2008

Testing Metrics Stuff.

"Metrics are a system of parameters or ways of quantitative and periodic assessment of a process that is to be measured, along with the procedures to carry out such measurement and the procedures for the interpretation of the assessment in the light of previous or comparable assessments. Metrics are usually specialized by the subject area, in which case they are valid only within a certain domain and cannot be directly benchmarked or interpreted outside it."

Metrics are measurements. It is as simple as that. We use them all the time in our everyday lives. Entangling them in wordy definitions is just intended to make them seem more mysterious and technical than they really are.

So what sorts of things do we measure in our daily lives and how do we use them? Shopping for food is a good place to start. At the meat counter, there is a choice of cuts of different kinds of meat, all at different prices. If we just look at the total price, we may be misled. A nice round steak might cost $10.00 while a round roast might cost $8.00 even though it weighs the same as the steak. So to get the best value for our money we tend to look at the price per unit weight. This is a microcosm of the field of metrics.

There are two basis types of metrics. The first type is the elemental or basic measurement such as weight, length, time, volume, and in this example, cost. The second type is derived, normally from the elemental measurements. At the meat counter, the derived metric is dollars/weight (VIZ. $7.49/kg). This is called a normalized metric.

Generally speaking, normalized metrics are the most useful because they allow us to make comparisons between things that are different. Some other examples are miles/gallon, dollars/gallon, dollars/share, dollars/hr, and dollars/square foot to give but a few.

We also see metrics in sports. In hockey its shots on goal and plus/minus ratio. In baseball its batting average and errors per game. All of these numbers are provided in newspapers and sports magazines and if they disappeared there would be a great uproar among fans.

"When you can measure what you are speaking about and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of the meager and unsatisfactory kind." - Lord Kelvin

Now Lord Kelvin wasn't right about everything he spoke about. He predicted that heavier than air flight was impossible. But about metrics, he was dead right.

We as shoppers apply this principle whenever we go to the market. If a cut of meat is marked $10.00 but has no weight assigned, we are likely to look for something else. The same would apply if the weight were given but no price. This is just plain old ordinary common sense. Yet we may go though our professional lives without using metrics to guide us in our work. Maintaining a "meager and unsatisfactory" knowledge about the way you earn your living is probably not the best approach.

Types :

Characteristics of Effective Test Metrics
Ideally, identifying test metrics takes place at the beginning of the project, so incorporation into the appropriate activities is easy. The test metrics you wish to collect need to be:

· quantifiable,
· easy to collect,
· simple,
· meaningful,
· non-threatening.

What is metrics?

A standard of measurement. Software metrics are the statistics describing the structure or content of a program. A metric should be a real objective measurement of something such as number of bugs per lines of code.

There are several test metrics identified as part of the overall testing activity in order to track and measure the entire testing process. These test metrics are collected at each phase of the testing life cycle /SDLC and analyzed and appropriate process improvements are determined and implemented as a result of these test metrics that are constantly collected and evaluated as a parallel activity together with testing both for manual and automated testing irrespective of the type of application. The test metrics can be broadly classified into the following three categories such as:

1. Project Related Metrics – such as Test Size, # of Test Cases tested per day –Automated (NTTA), # of Test Cases tested per day –Manual (NTTM), # of Test Cases created per day – Manual (TCED), Total number of review defects (RD), Total number of testing defects (TD), etc
2. Process Related Metrics – such as Schedule Adherence (SA), Effort Variance (EV), Schedule Slippage (SS), Test Cases and Scripts Rework Effort, etc.
3. Customer related Metrics – such as Percentage of defects leaked per release (PDLPR), Percentage of automation per release (PAPR), Application Stability Index (ASI), etc.

what are different types of metrics to measure software quality? and explain them briefly?

Metric is a mathematical number that shows a relationship between two variables. It is a quantitative measure of the degree to which a system, component or process possesses a given attribute. Software Metrics are measures that are used to quantify the software, software development resource and software development process.

Types of Metrics :

Metric generally
classified into 2 types.
· Process Metric
· Product Metric
Process Metric: Metric used to measure the characteristic of the methods, techniques and tools employed in developing, implementing and maintaining the software system.
Product Metric: Metric used to measure the characteristic of the documentation and code
The metrics for the test process would include status of test activities against the plan, test coverage achieved so far, among others. An important metric is the number of defects found in internal testing compared to the defects found in customer tests, which indicate the effectiveness of the test process itself.


What type of metrics would you use?

QAM:Quality Assurance Matrix
TMM:Test
Management Matrix
PCM:Process Compatibility
Matrix

What is Test Matric?

Test metrics A document showing the relation ship between test requirements and test cases.

Wednesday, October 22, 2008

Difference between QTP 8.2 and QTP9.0 / QTP 9.2

In this post, i just presents few differences between 8.2 and 9.2 versions.

Mercury Screen Recorder:

Capture your entire run session in a movie clip or capture only the segments with errors, and then view your movie from the Test Results window.

Dynamic Management of Object Repositories:

QuickTest now has a new Repositories Collection reserved object that you can use to programmatically manage the set of object repositories that are associated with an action during a run session.

Over and above features provided with QTP 8.2 , QTP 9.0 provides following features:

Object Repository Manager:

You can use the Object Repository Manager to manage all of the shared object repositories in your organization from one, central location. This includes adding and defining objects, modifying objects and their descriptions, parameterizing test object property values, maintaining and organizing repositories, and importing and exporting repositories in XML format.

You can open multiple object repositories at the same time. Each object repository opens in its own resizable document window. This enables you to compare the content of the repositories, to copy or move objects from one object repository to another, and so forth.

Object Repository Merge Tool:

You can use the Object Repository Merge Tool to merge the objects from two shared object repositories into a single shared object repository. You can also use the Object Repository Merge Tool to merge objects from the local object repository of one or more actions or components into a shared object repository.

When you merge objects from two source object repositories, the content is copied to a new, target object repository, ensuring that the information in the source repositories remains unchanged.

If any conflicts occur during the merge, for example, if two objects have the same name and test object class, but different test object descriptions, the relevant objects are highlighted in the source repositories, and the Resolution Options pane details the conflict and possible resolutions.

Multiple Object Repositories per Action or Component:

QuickTest provides several options for storing and accessing test objects. You can store the test objects for each action or component in its corresponding local object repository, which is unique for each action and component. You can also store test objects in one or more shared object repositories that can be used in multiple actions and components. Alternatively, you can use a combination of objects from the local object repository and one or more shared object repositories. You choose the combination that matches your testing needs.

XML Object Repository Format:

QuickTest now enables you to import and export object repositories from and to XML format. This enables you to modify object repositories using the XML editor of your choice and then import them back into QuickTest. You can import and export files either from and to the file system or a Quality Center project (if QuickTest is connected to Quality Center).

Function Library Editor:

QuickTest now has a built-in function library editor, which enables you to create and edit function libraries containing VBScript functions, subroutines, modules, and so forth, and then call their functions from your test or component.

Handling Missing Actions and Resources:

Whenever a testing document (test, component, or application area) contains a resource that cannot be found, QuickTest opens the Missing Resources pane and lists the missing resource(s). For example, a test may contain an action or a call to an action that cannot be found; a testing document may use a shared object repository that cannot be found; or a testing document may use a object repository parameter that does not have a default value. In all of these cases, QuickTest indicates this in the Missing Resources pane, enabling you to map a missing resource to an existing one, or remove it from the testing document, as required.

Source: What's New in Quick Test Professional--QTP Documentation