Monday, July 20, 2009

Monday, December 8, 2008

ISTQB FOUNDATION LEVEL - Documents

Thursday, October 23, 2008

"Metrics are a system of parameters or ways of quantitative and periodic assessment of a process that is to be measured, along with the procedures to carry out such measurement and the procedures for the interpretation of the assessment in the light of previous or comparable assessments. Metrics are usually specialized by the subject area, in which case they are valid only within a certain domain and cannot be directly benchmarked or interpreted outside it."

Metrics are measurements. It is as simple as that. We use them all the time in our everyday lives. Entangling them in wordy definitions is just intended to make them seem more mysterious and technical than they really are.

So what sorts of things do we measure in our daily lives and how do we use them? Shopping for food is a good place to start. At the meat counter, there is a choice of cuts of different kinds of meat, all at different prices. If we just look at the total price, we may be misled. A nice round steak might cost $10.00 while a round roast might cost $8.00 even though it weighs the same as the steak. So to get the best value for our money we tend to look at the price per unit weight. This is a microcosm of the field of metrics.

There are two basis types of metrics. The first type is the elemental or basic measurement such as weight, length, time, volume, and in this example, cost. The second type is derived, normally from the elemental measurements. At the meat counter, the derived metric is dollars/weight (VIZ. $7.49/kg). This is called a normalized metric.

Generally speaking, normalized metrics are the most useful because they allow us to make comparisons between things that are different. Some other examples are miles/gallon, dollars/gallon, dollars/share, dollars/hr, and dollars/square foot to give but a few.

We also see metrics in sports. In hockey its shots on goal and plus/minus ratio. In baseball its batting average and errors per game. All of these numbers are provided in newspapers and sports magazines and if they disappeared there would be a great uproar among fans.

"When you can measure what you are speaking about and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of the meager and unsatisfactory kind." - Lord Kelvin

Now Lord Kelvin wasn't right about everything he spoke about. He predicted that heavier than air flight was impossible. But about metrics, he was dead right.

We as shoppers apply this principle whenever we go to the market. If a cut of meat is marked $10.00 but has no weight assigned, we are likely to look for something else. The same would apply if the weight were given but no price. This is just plain old ordinary common sense. Yet we may go though our professional lives without using metrics to guide us in our work. Maintaining a "meager and unsatisfactory" knowledge about the way you earn your living is probably not the best approach.

Types :

Characteristics of Effective Test Metrics
Ideally, identifying test metrics takes place at the beginning of the project, so incorporation into the appropriate activities is easy. The test metrics you wish to collect need to be:

· quantifiable,
· easy to collect,
· simple,
· meaningful,
· non-threatening.

What is metrics?

A standard of measurement. Software metrics are the statistics describing the structure or content of a program. A metric should be a real objective measurement of something such as number of bugs per lines of code.

There are several test metrics identified as part of the overall testing activity in order to track and measure the entire testing process. These test metrics are collected at each phase of the testing life cycle /SDLC and analyzed and appropriate process improvements are determined and implemented as a result of these test metrics that are constantly collected and evaluated as a parallel activity together with testing both for manual and automated testing irrespective of the type of application. The test metrics can be broadly classified into the following three categories such as:

1. Project Related Metrics – such as Test Size, # of Test Cases tested per day –Automated (NTTA), # of Test Cases tested per day –Manual (NTTM), # of Test Cases created per day – Manual (TCED), Total number of review defects (RD), Total number of testing defects (TD), etc
2. Process Related Metrics – such as Schedule Adherence (SA), Effort Variance (EV), Schedule Slippage (SS), Test Cases and Scripts Rework Effort, etc.
3. Customer related Metrics – such as Percentage of defects leaked per release (PDLPR), Percentage of automation per release (PAPR), Application Stability Index (ASI), etc.

what are different types of metrics to measure software quality? and explain them briefly?

Metric is a mathematical number that shows a relationship between two variables. It is a quantitative measure of the degree to which a system, component or process possesses a given attribute. Software Metrics are measures that are used to quantify the software, software development resource and software development process.

Types of Metrics :

Metric generally classified into 2 types.
Â· Process Metric
Â· Product Metric
Process Metric: Metric used to measure the characteristic of the methods, techniques and tools employed in developing, implementing and maintaining the software system.
Product Metric: Metric used to measure the characteristic of the documentation and code
The metrics for the test process would include status of test activities against the plan, test coverage achieved so far, among others. An important metric is the number of defects found in internal testing compared to the defects found in customer tests, which indicate the effectiveness of the test process itself.

What type of metrics would you use?

QAM:Quality Assurance Matrix
TMM:Test Management Matrix
PCM:Process Compatibility Matrix

What is Test Matric?

Test metrics A document showing the relation ship between test requirements and test cases.

Wednesday, October 22, 2008

Difference between QTP 8.2 and QTP9.0 / QTP 9.2

In this post, i just presents few differences between 8.2 and 9.2 versions.

Mercury Screen Recorder:

Capture your entire run session in a movie clip or capture only the segments with errors, and then view your movie from the Test Results window.

Dynamic Management of Object Repositories:

QuickTest now has a new Repositories Collection reserved object that you can use to programmatically manage the set of object repositories that are associated with an action during a run session.

Over and above features provided with QTP 8.2 , QTP 9.0 provides following features:

Object Repository Manager:

You can use the Object Repository Manager to manage all of the shared object repositories in your organization from one, central location. This includes adding and defining objects, modifying objects and their descriptions, parameterizing test object property values, maintaining and organizing repositories, and importing and exporting repositories in XML format.

You can open multiple object repositories at the same time. Each object repository opens in its own resizable document window. This enables you to compare the content of the repositories, to copy or move objects from one object repository to another, and so forth.

Object Repository Merge Tool:

You can use the Object Repository Merge Tool to merge the objects from two shared object repositories into a single shared object repository. You can also use the Object Repository Merge Tool to merge objects from the local object repository of one or more actions or components into a shared object repository.

When you merge objects from two source object repositories, the content is copied to a new, target object repository, ensuring that the information in the source repositories remains unchanged.

If any conflicts occur during the merge, for example, if two objects have the same name and test object class, but different test object descriptions, the relevant objects are highlighted in the source repositories, and the Resolution Options pane details the conflict and possible resolutions.

Multiple Object Repositories per Action or Component:

QuickTest provides several options for storing and accessing test objects. You can store the test objects for each action or component in its corresponding local object repository, which is unique for each action and component. You can also store test objects in one or more shared object repositories that can be used in multiple actions and components. Alternatively, you can use a combination of objects from the local object repository and one or more shared object repositories. You choose the combination that matches your testing needs.

XML Object Repository Format:

QuickTest now enables you to import and export object repositories from and to XML format. This enables you to modify object repositories using the XML editor of your choice and then import them back into QuickTest. You can import and export files either from and to the file system or a Quality Center project (if QuickTest is connected to Quality Center).

Function Library Editor:

QuickTest now has a built-in function library editor, which enables you to create and edit function libraries containing VBScript functions, subroutines, modules, and so forth, and then call their functions from your test or component.

Handling Missing Actions and Resources:

Whenever a testing document (test, component, or application area) contains a resource that cannot be found, QuickTest opens the Missing Resources pane and lists the missing resource(s). For example, a test may contain an action or a call to an action that cannot be found; a testing document may use a shared object repository that cannot be found; or a testing document may use a object repository parameter that does not have a default value. In all of these cases, QuickTest indicates this in the Missing Resources pane, enabling you to map a missing resource to an existing one, or remove it from the testing document, as required.

Source: What's New in Quick Test Professional--QTP Documentation

Monday, September 22, 2008

Software Test Automation Myths and Facts

Introduction

Today software test automation is becoming more and more popular in both C/S and web environment. As the requirements keep changing (mostly new requirements are getting introduced on daily basis) constantly and the testing window is getting smaller and smaller everyday, the managers are realizing a greater need for test automation. This is good news for us (people who do test automation). But, I am afraid this is the only good news.

Myths & Facts

A number of articles and books are written on different aspects of Software Test Automation. I like to discuss some of these myths and will try to point out the facts about these myths. I also like to discuss some of my observations and hopefully point out possible solutions. These are based on my experience with a number of automation projects I was involved and the information I collected from people worked on automation.

Find more bugs: Management often think that by doing automation they should be able to find more bugs. It’s a myth. Let’s think about it for a while. The process of automation involves writing test cases. Generally test cases are written by test engineers who are familiar with the application they are working w.r.t .specficiations . The test cases are then given to the automation engineers. In most cases the automation engineers are not very familiar with the test cases they are automating. From test cases to test scripts, automation does not add anything in the process to find more bugs. The test scripts will work only as good as the test cases when comes to finding bugs. So, it’s the test cases that find bugs, not the test scripts. So, the number of bugs found depends on the effectiveness of writing test cases with respect to specifications.

Eliminate or reduce manual testers: In order to justify automation, some point out that they should able to reduce the number of manual testers in the long run and thus save money in the process. Absolutely not true. Reduction of manual testers is not any of the objectives of test automation. Here is why – as I have pointed out earlier that the test scripts are only as good as the test cases and the test cases are written primarily by manual testers. They are the ones who know the application inside out. If the word gets out (it usually does) that the number of manual testers will be reduced by introducing automation. But effective usage of a tester’s time can be achieved and automation will reduce the test repetition cost and save the test execution time.

Observations

I have met a number of QA managers who are frustrated with their automation. According to them the tool is not doing what it is supposed to do. Here is a true story, the client (I had the opportunity to work with them for some time) found out that the tool they have just bought does not support the application they are testing (I am not making it up). How can this happen! – It does happen more often than one would think. I will get back on this when I discuss possible solutions. A manager of one of the major telecom companies that I had a recent interview with told me that after three years and more than a million dollar he is still struggling with automation. This is pretty sad and I get the feeling that he is not alone.

Solutions/Suggestions

Let’s discuss some of the reasons for this frustration and some of the solutions to this problem.

- Unrealistic expectations: Most managers have their first encounter with any automation tool when they look at the demo and everything looks nice and simple.

But everything is not so nice and simple when you try to use the tool with your application. The vendors will only tell you the things you want to hear (how easy to use, how simple to set up, how it will save time and money, how it will help you find more bugs etc.). This builds a false set of hopes and expectations.

- Lack of planning: A great deal of planning is required from selection to implementation of the tool. “Evaluating Tools” by Elisabeth Hendrickson is a very good article on step by step process of selecting a tool. She talks about “Tool Audience” as one of the steps. This would be an ideal way to select a tool. It may not happen in every place because of the everyday workload of the people involved. But the participation of the users in the process is very important, because they are the ones who will use the tool day in and day out. I am almost certain that what happened to one of my clients (the tool they have bought did not support the application they were testing) would not have happened if the users were involved in the selection process.

- Lack of a process: Lack of a process may also contribute to failure of automation. Most places do have some kind of process in place. In most cases (although it differs from place to place) developers write code against a set of requirements. If the requirement does not call for a change in GUI then, there should not be any change in GUI. But if the GUI keep changing constantly from one release to another without any requirement for that change then, there is a problem in the process. You may have the best tool and the best (for your environment) architecture is in place and you will still have problems with your automation because of a faulty process.

FUNDAS ABOUT TEST AUTOMATION FRAMEWORKS :

Historically, test automation has not met with the level of success that it could. Time and again test automation efforts are born, stumble, and die. Most often this is the result of misconceived perceptions of the effort and resources necessary to implement a successful, long-lasting automation framework. Why is this, we might ask? Well, there are several reasons.

The commercial automation tools have been chiefly marketed for use as solutions for testing an application. They should instead be sought as the cornerstone for an enterprise-wide test automation framework. And, while virtually all of the automation tools contain some scripting language allowing us to get past each tool’s failings, testers have typically neither held the development experience nor received the training necessary to exploit these programming environments

Test automation must be approached as a full-blown software development effort in its own right. Without this, it is most likely destined to failure in the long term.

In order to make the most of our test strategy, we need to make it reusable and manageable. To that end, there are some essential guiding principles we should follow when developing our overall test strategy:

Test automation is a fulltime effort, not a sideline.
The test design and the test framework are totally separate entities.
The test framework should be application-independent.
The test framework must be easy to expand, maintain, and perpetuate.
The test strategy/design vocabulary should be framework independent.
The test strategy/design should remove most testers from the complexities of the test framework.

Data Driven Automation Frameworks

Over the past several years there have been numerous articles done on various approaches to test automation. Anyone who has read a fair, unbiased sampling of these knows that we cannot and must not expect pure capture and replay of test scripts to be successful for the life of a product. We will find nothing but frustration there

1.2.1 Data Driven Scripts

Data driven scripts are those application-specific scripts captured or manually coded in the automation tool’s proprietary language and then modified to accommodate variable data. Variables will be used for key application input fields and program selections allowing the script to drive the application with external data supplied by the calling routine or the shell that invoked the test script.

Variable Data, Hard Coded Component Identification:
These data driven scripts often still contain the hard coded and sometimes very fragile recognition strings for the window components they navigate. When this is the case, the scripts are easily broken when an application change or revision occurs. And when these scripts start breaking, we are not necessarily talking about just a few. We are sometimes talking about a great many, if not all the scripts, for the entire application.

1.2.2 Keyword or Table Driven Test Automation

Nearly everything discussed so far defining our ideal automation framework has been describing the best features of "keyword driven" test automation. Sometimes this is also called "table driven" test automation. It is typically an application-independent automation framework designed to process our tests. These tests are developed as data tables using a keyword vocabulary that is independent of the test automation tool used to execute them. This keyword vocabulary should also be suitable for manual testing, as you will soon see.

Action, Input Data, and Expected Result ALL in One Record:
The data table records contain the keywords that describe the actions we want to perform. They also provide any additional data needed as input to the application, and where appropriate, the benchmark information we use to verify the state of our components and the application in general.

Reusable Code, Error Correction and Synchronization:
Application-independent component functions are developed that accept application-specific variable data. Once these component functions exist, they can be used on each and every application we choose to test with the framework.

1.2.3 Hybrid Test Automation (or, "All of the Above")

The most successful automation frameworks generally accommodate both keyword driven testing as well as data driven scripts. This allows data driven scripts to take advantage of the powerful libraries and utilities that usually accompany a keyword driven architecture.

The framework utilities can make the data driven scripts more compact and less prone to failure than they otherwise would have been. The utilities can also facilitate the gradual and manageable conversion of existing scripts to keyword driven equivalents when and where that appears desirable.

1.2.4 Commercial Keyword Driven Frameworks

Some commercially available keyword driven frameworks are making inroads in the test automation markets. These generally come from 3^rd party companies as a bridge between your application and the automation tools you intend to deploy. They are not out-of-the-box, turnkey automation solutions just as the capture\replay tools are not turnkey solutions.

Keyword Driven Automation Framework Model

The model focuses on implementing a keyword driven automation framework. It does not include any additional features like tracking requirements or providing traceability between automated test results and any other function of the test process. It merely provides a model for a keyword driven execution engine for automated tests.

The commercially available frameworks generally have many more features and much broader scope. Of course, they also have the price tag to reflect this.

I) Project Guidelines

The test strategy will allow each test to include the step to perform, the input data to use, and the expected result all together in one line or record of the input source.

Implement a framework that will integrate keyword driven testing and traditional scripts, allowing both to benefit from the implementation.

Implement the framework to be completely application-independent since it will need to test at least 4 or 5 different applications once deployed.

The framework will be fully documented and published.

The framework will be publicly shared on the intranet for others to use and eventually (hopefully) co-develop.

31) How to make QTP to recognise the activeX controls

1) In the Expert View, you can use the Object property to activate the method for an ActiveX control. The list of available methods depends on the ActiveX control.

2)QuickTest records and runs steps on ActiveX controls as it does on any other object.

Using the Insert>Step Option , we can activate ActiveX control methods, retrieve and set the values of properties and check the object exists.

It is recommended that to begin recording session before opening the application containing the ActiveX controls on which you want to record.

32) what to do if the tree view is not recognised by QTP

U've something called "Object Mapping" under Object Identification. This is only for Standard Windows Class. U can map the Tree View to WinTreeView and then try recording. It works.

33) what is meant by external files in QTP.

.vbs files are called as external files.

.Vbs,Excel,dlls and any other files that for importing,Processing and Resulting can be treated as External files in QTP

34) what is keyword driventest ?

Keyword driven test has been introduced in QTP 8.0. Its like tree view. Keyword driven implies that the script can be created easily through keyword View.

35) How can an object from a per action repository be called to another per action repository?

It is possible through shared repository only

36)How you write scripts in QTP? What's the main process in QTP? How do you run scripts in QTP?

Main process in QTP is Recording, stores the properties in object repository then Running the script and then Test Results.

Recording: QTP 'looks' at the object on which we are recording and stores it as a test object, determining in which test object class it fits like standard window dialog box or web button etc. Then for each test object class, QTP has list of mandatory properties that it always learns. When we record an object, QTP learns these default property values, and then 'looks' at the rest of the objects in the page , to distinguish and identify the object uniquely.If not it adds assistive properties , one ny one, to the description, until it has compiled a unique description.If no assistive properties are not available , it adds a special 'ordinal identifier' such as objects location on the screen.

Running the scripts: While running the script, QTP searches for a run time object that exactly matches the description of the test object it learned while recording. If it is not matching, QTP uses ' smart identification' mechanism to identify the object.

We can run the scripts from Test>Run After running the script we can see the Test Results also .

37) What is the command in QTP to invoke IE Brow?

InvokeApplication "The path of the browser EXE file"

The following example uses the InvokeApplication function to open Internet Explorer.

InvokeApplication "E:\Program Files\Plus!\Microsoft Internet\IEXPLORE.EXE"

38) I am new to QTP, please tell me how to invoke an application in QTP.

For Ex:
In winrunner we use syntax's like "web_url_valid, web_browser_invoke", like the same way i want in QTP.

Else please let me know where can i find these syntaxs in QTP. Use SytemUtil Object's Run Method

ex:SystemUtil.Run "iexplore" will open the IE Browser.

39) How would u manipulate the script so that when the test is run it takes a new login name?

External datatable can be used for the login name so that it takes a new login name whenever the test is run. You can parameterize the values in the Gobal data table sheet, whatever the number of rows you enter in this data table will instruct QuickTest to run same number of new login name you've enter. You need to do only parameterize the login window object in object repository and there you need to specify datatable parameter name or else qtp specify default papramter name

40) How can i add a action (external action) programatically?

1) its not possible :( 2)if action is reusable then only u can call only in other test with "runaction" command and in folder option u hv to gv information regarding test from where u r takin action.u can do it programatically also. 3)You can add an external Action programatically using the Command

RunAction ActionName, [IterationMode , IterationRange]

Before you can use the RunAction statement in the Expert View for an external action, you must first call or copy the external action into your test by choosing Insert > Copy of Action or Call to Action. If the external action does not exist in your test, the RunAction statement is not recognized.

Example

The following example calls the SearchFlight action, and runs all iterations of the action.

Call RunAction "SearchFlight", rngIterations, rngAll

41) what is meant by SOURCE CONTROL ?

It is used to hold all the bulids of diff versions

42) how and what kind of Vb functions do u use in qtp?

1. User Defined2. Pre Defined

43) how can u discribe the basic flow of automation with conditional and programatic logic?

I think ur Question is Executing of operators flow in the automation code if the question is that then my answer is

For example:

z = 78 * (96 + 3 + 45)

There are five operators in this expression: =, *, (), +, and another +. According to the rules of operator precedence, they are evaluated in the following order: (), +, +, *, =.

1. Evaluation of the expression within the parentheses occurs first. Within the parentheses, there are two addition operators. Since the addition operators both have the same precedence, they are evaluated from left to right. 96 and 3 are added together first, then 45 is added to this total, resulting in a value of 144.

2. Multiplication occurs next. 78 is multiplied by 144, resulting in a value of 11232.

3. Assignment occurs last. 11232 is assigned to z.

44) HOW CAN I IMPLEMENT ERROR HANDLING IN QTP,I KNOW WITH RECOVERY MANAGER BUT HOW PLZ GIVE ME DETAILED TO HOW TO HANDLE GIVING AN EXAMPLE?

U can do it thru Recovery Manager..

Eg... Suppose there is an Edit box called Uname n PWD... Just type in uname n don't enter in PWD..

It displays a pop up msg called plz,,enter PWD... Then stop recording..

Go to Recovery MGR and call POPUP exception handling./../

45) Give one example where you have used Regular Expression?

1)for the date format "dd/mm/yyyy" the equivalent regular expression would be

Set regExp_Term = New RegExp
regExp_Term.pattern = "11/11/1981"
validation=regExp_Term.test("date to be validated")

if validation="True" Then

-----------------

End If

2)We can use Regular Expression where ever required, like in my Web Testing, my browser name changes oftnely, so I used Regular expression in Name property like...

Set Browser= (Title:=Browser.*)

3) generaly regular expressions r used when the data is dynamically changing . for example take a flights application in the QTP samples. we can use regular expression for the Date field ,FaxOrder no,etc

QA (Testing) FUNDAS

Basic funda of QA (Testing )