What is Automation?
Automation is a Microsoft technology that makes it possible to access software objects inside one application from other applications. These objects can be easily created and manipulated using a scripting or programming language such as VBScript or VC++. Automation enables you to control the functionality of an application programmatically.
An object model is a structural representation of software objects (classes) that comprise the implementation of a system or application. An object model defines a set of classes and interfaces, together with their properties, methods and events, and their relationships.
Essentially all configuration and run functionality provided via the QuickTest interface is in some way represented in the QuickTest automation object model via objects, methods, and properties. Although a one-on-one comparison cannot always be made, most dialog boxes in QuickTest have a corresponding automation object, most options in dialog boxes can be set and/or retrieved using the corresponding object property, and most menu commands and other operations have corresponding automation methods.
You can use the objects, methods, and properties exposed by the QuickTest automation object model, along with standard programming elements such as loops and conditional statements to design your program.
Automation programs are especially useful for performing the same tasks multiple times or on multiple tests or components, or quickly configuring QuickTest according to your needs for a particular environment or application.
For example, you can create and run an automation program from Microsoft Visual Basic that loads the required add-ins for a test or component, starts QuickTest in visible mode, opens the test or component, configures settings that correspond to those in the Options, Test or Business Component Settings, and Record and Run Settings dialog boxes, runs the test or component, and saves the test or component.
You can then add a simple loop to your program so that your single program can perform the operations described above for multiple tests or components.
You can also create an initialization program that opens QuickTest with specific configuration settings. You can then instruct all of your testers to open QuickTest using this automation program to ensure that all of your testers are always working with the same configuration.
Like the tests or components you design using QuickTest, creating a useful QuickTest automation program requires planning, design time, and testing. You must always weigh the initial investment with the time and human-resource savings you gain from automating potentially long or tedious tasks.
Any QuickTest operation that you must perform many times in a row or must perform on a regular basis is a good candidate for a QuickTest automation program.
Initialization programs—You can write a program that automatically starts QuickTest and configures the options and the settings required for recording on a specific environment.
Maintaining your tests or components—You can write a program that iterates over your collection of tests and components to accomplish a certain goal. For example:
Updating values—opening each test or component with the proper add-ins, running it in update run mode against an updated application, and saving it in order to update the values in all of your tests and components to match the updated values in your application.
Applying new options to existing tests or components—When you upgrade to a new version of QuickTest, you may find that the new version offers certain options that you want to apply to your existing tests and components. You can write a program that opens each existing test and component, sets values for the new options, then saves and closes it.
Calling QuickTest from other applications—You can design your own applications with options or controls that run QuickTest automation programs. For example, you could create a Web form or simple Windows interface from which a product manager could schedule QuickTest runs, even if the manager is not familiar with QuickTest.
When to use the Automation tools:
Great effort is required in firstly automating the test cases and maintaining them thereafter. Hence some sort of cost-benefit analysis is quite helpful before investing money and putting efforts into it.
The test engineer should evaluate whether the likely benefits of automation are fitting within the boundaries of the required improvement criteria and whether the pursuit of automated testing on the project is still logical & falling within the framework of organizational needs.
Automation is suited to following types of testing with specific intentions:
1) Functional Testing – on operations which perform as per the expectations.
2) Regression Testing – on the behavior of the system which has not been changed.
3) Exception Testing – thereby forcing error conditions in the system.
4) Stress Testing – to determine the absolute capacities of the application and operational infrastructure.
5) Performance Testing – to provide assurance that the performance of the system will be adequate for both batch runs and online transactions in relation to business projections and requirements.
6) Load Testing – to determine the points at which the capacity and performance of the system become degraded to the situation that hardware or software upgrades would be required.
Benefits of Automated Testing:
1) Reliable: Tests perform precisely the same operations each time they are run, thereby eliminating human error. Hence it produces a reliable system.
2) Repeatable: You can test how the software reacts under repeated execution of the same operations.
3) Programmable: You can program sophisticated tests that bring out hidden information from the application.
4) Comprehensive: You can build a suite of tests that covers every feature in your application.
5) Reusable: You can reuse tests on different versions of an application, even if the user interface changes.
6) Better Quality Software: It Improves the quality of the testing effort. Because you can run more tests in less time with fewer resources
7) Fast: It reduces the test effort by minimizing the schedule. Automated Tools run tests significantly faster than human users.
8) Economical: As the number of resources for regression test are reduced.
Choosing the right tools for the job and targeting the right areas of the organization to deploy them can only realize these benefits. The right areas where the automation fit must be chosen.
Disadvantages of Automation Testing:
Though the automation testing has many advantages, it has its own disadvantages too. Some of the disadvantages are:
# Proficiency is required to write the automation test scripts.
# Debugging the test script is major issue. If any error is present in the test script, sometimes it may lead to deadly consequences.
# Test maintenance is costly in case of playback methods. Even though a minor change occurs in the GUI, the test script has to be re-recorded or replaced by a new test script.
# Maintenance of test data files is difficult, if the test script tests more screens.
Ultimately how to decide – Whether to go in for Automated Testing or Manual Testing?
1) Based upon the frequency of use of Test Cases:
Automating a test case requires almost 3-4 times the effort as compared to manual execution of it once. To draw benefit out of significant investment in the automation tool, we should be able to execute the automation script the maximum number of times or at least 7-8 times. There is no worthwhile idea of going in for an automation tool for short-term product, which can be easily managed by manual means. For products involving many rounds of testing, the use of an automation tool is understandable.
2) Time comparison:
Automated versus Manual Testing: Generally an automation script runs much faster as compared to manual execution. However post execution activities are considerably time consuming. After running the automation script fully; time consuming activities are the analysis of test results & investigation or identification of actions causing failure at defined checkpoints. Whereas in case of manual execution of the test script, there is no need of any separate time for analysis of the results, since the actions causing failure at checkpoints become already known.
Hence automated testing is viable only if, the combined time spent on running the automation script as well as doing post automation analysis of test results is significantly less in comparison to the time spent on manual execution. However nicely developed automation scripts do not need constant monitoring and can run without any manual intervention. In such a case automation can be greatly productive & can cut down the time of running the scripts.
However when large number of regression issues are there after fixing the bugs, automation is the best alternative. Manual testing can be extremely time consuming in such cases.
3) Reusability of Automation Scripts:
The automated testing is viable in case automation scripts are reusable in future as well. One may be compelled to think again in favor of automation if considerable effort is expected to be made in upgrading the automation scripts. The return on investment made on the automation tool can be maximized by reusability of the automation scripts with small modifications.
4) Stability of the Product under Test:
Automation scripts are not advisable for use on software product, which itself is not adequately stable. Frequent changes in automation script are not desirable, thus unless the product acquires enough stability during its development cycle, there is no point in automation unless we are operating in an agile environment.
5) Adaptability of test cases for automation:
A statement that, "All test cases can easily be automated" – is not true. Many a times we land up with some type of test cases, which are not worth automating. Hence there is no point in wasting automation effort over such test cases. For complicated product with tightly integrated bunch of many applications, running the automation script again & again in the event of stoppage of the test case, can become a pain in the neck. In such cases tester would certainly prefer to run the script manually & save considerable amount of time as compared to running the automated script.
6) Exploitation of automation tool:
If we want to go in for automated testing of a product, we must draw full benefit out of the test tool to draw the maximum return on its investment. The automation tool should be deployed for performing less complicated / time consuming & repetitive tasks. This will help the test engineers in concentrating their time & energy on other significantly important tasks. There is no point in automating highly complex test cases, which can easily be executed manually.
Automation tool should be deployed to test the breadth of the application under test. However manual test engineers can handle the in-depth testing more efficiently.
Manual testing can not be totally eliminated in any way. Automation scripts have both advantage and disadvantages in the sense that they perform actions exactly in the way they are coded to perform without any deviation. For even a slight deviation, the script needs to be changed. Running the same automated script again & again will not detect more bugs. However to detect more & more bugs, we need to move across the side & have little deviation from the flow, & such actions can be better accomplished by manual testing.
When a test case is perfectly designed & made to execute from end to end automatically through the tool without, it automatically verifies all predefined check points without any manual intervention.
7) User simulation over large web application: For simulation of several virtual / dummy users interacting over a large web application, load testing automated tools like LoadRunner etc. can be easily deployed to establish the load bearing functionality of the application. Such load testing by manual means is extremely difficult. Thus in spite of many disadvantages & harms from automated scripts, the automation testing is adapted widely all over the world.
Great effort is required in firstly automating the test cases and maintaining them thereafter. Hence some sort of cost-benefit analysis is quite helpful before investing money and putting efforts into it.
The test engineer should evaluate whether the likely benefits of automation are fitting within the boundaries of the required improvement criteria and whether the pursuit of automated testing on the project is still logical & falling within the framework of organizational needs.
Automation is suited to following types of testing with specific intentions:
1) Functional Testing – on operations which perform as per the expectations.
2) Regression Testing – on the behavior of the system which has not been changed.
3) Exception Testing – thereby forcing error conditions in the system.
4) Stress Testing – to determine the absolute capacities of the application and operational infrastructure.
5) Performance Testing – to provide assurance that the performance of the system will be adequate for both batch runs and online transactions in relation to business projections and requirements.
6) Load Testing – to determine the points at which the capacity and performance of the system become degraded to the situation that hardware or software upgrades would be required.
Benefits of Automated Testing:
1) Reliable: Tests perform precisely the same operations each time they are run, thereby eliminating human error. Hence it produces a reliable system.
2) Repeatable: You can test how the software reacts under repeated execution of the same operations.
3) Programmable: You can program sophisticated tests that bring out hidden information from the application.
4) Comprehensive: You can build a suite of tests that covers every feature in your application.
5) Reusable: You can reuse tests on different versions of an application, even if the user interface changes.
6) Better Quality Software: It Improves the quality of the testing effort. Because you can run more tests in less time with fewer resources
7) Fast: It reduces the test effort by minimizing the schedule. Automated Tools run tests significantly faster than human users.
8) Economical: As the number of resources for regression test are reduced.
Choosing the right tools for the job and targeting the right areas of the organization to deploy them can only realize these benefits. The right areas where the automation fit must be chosen.
Disadvantages of Automation Testing:
Though the automation testing has many advantages, it has its own disadvantages too. Some of the disadvantages are:
# Proficiency is required to write the automation test scripts.
# Debugging the test script is major issue. If any error is present in the test script, sometimes it may lead to deadly consequences.
# Test maintenance is costly in case of playback methods. Even though a minor change occurs in the GUI, the test script has to be re-recorded or replaced by a new test script.
# Maintenance of test data files is difficult, if the test script tests more screens.
Ultimately how to decide – Whether to go in for Automated Testing or Manual Testing?
1) Based upon the frequency of use of Test Cases:
Automating a test case requires almost 3-4 times the effort as compared to manual execution of it once. To draw benefit out of significant investment in the automation tool, we should be able to execute the automation script the maximum number of times or at least 7-8 times. There is no worthwhile idea of going in for an automation tool for short-term product, which can be easily managed by manual means. For products involving many rounds of testing, the use of an automation tool is understandable.
2) Time comparison:
Automated versus Manual Testing: Generally an automation script runs much faster as compared to manual execution. However post execution activities are considerably time consuming. After running the automation script fully; time consuming activities are the analysis of test results & investigation or identification of actions causing failure at defined checkpoints. Whereas in case of manual execution of the test script, there is no need of any separate time for analysis of the results, since the actions causing failure at checkpoints become already known.
Hence automated testing is viable only if, the combined time spent on running the automation script as well as doing post automation analysis of test results is significantly less in comparison to the time spent on manual execution. However nicely developed automation scripts do not need constant monitoring and can run without any manual intervention. In such a case automation can be greatly productive & can cut down the time of running the scripts.
However when large number of regression issues are there after fixing the bugs, automation is the best alternative. Manual testing can be extremely time consuming in such cases.
3) Reusability of Automation Scripts:
The automated testing is viable in case automation scripts are reusable in future as well. One may be compelled to think again in favor of automation if considerable effort is expected to be made in upgrading the automation scripts. The return on investment made on the automation tool can be maximized by reusability of the automation scripts with small modifications.
4) Stability of the Product under Test:
Automation scripts are not advisable for use on software product, which itself is not adequately stable. Frequent changes in automation script are not desirable, thus unless the product acquires enough stability during its development cycle, there is no point in automation unless we are operating in an agile environment.
5) Adaptability of test cases for automation:
A statement that, "All test cases can easily be automated" – is not true. Many a times we land up with some type of test cases, which are not worth automating. Hence there is no point in wasting automation effort over such test cases. For complicated product with tightly integrated bunch of many applications, running the automation script again & again in the event of stoppage of the test case, can become a pain in the neck. In such cases tester would certainly prefer to run the script manually & save considerable amount of time as compared to running the automated script.
6) Exploitation of automation tool:
If we want to go in for automated testing of a product, we must draw full benefit out of the test tool to draw the maximum return on its investment. The automation tool should be deployed for performing less complicated / time consuming & repetitive tasks. This will help the test engineers in concentrating their time & energy on other significantly important tasks. There is no point in automating highly complex test cases, which can easily be executed manually.
Automation tool should be deployed to test the breadth of the application under test. However manual test engineers can handle the in-depth testing more efficiently.
Manual testing can not be totally eliminated in any way. Automation scripts have both advantage and disadvantages in the sense that they perform actions exactly in the way they are coded to perform without any deviation. For even a slight deviation, the script needs to be changed. Running the same automated script again & again will not detect more bugs. However to detect more & more bugs, we need to move across the side & have little deviation from the flow, & such actions can be better accomplished by manual testing.
When a test case is perfectly designed & made to execute from end to end automatically through the tool without, it automatically verifies all predefined check points without any manual intervention.
7) User simulation over large web application: For simulation of several virtual / dummy users interacting over a large web application, load testing automated tools like LoadRunner etc. can be easily deployed to establish the load bearing functionality of the application. Such load testing by manual means is extremely difficult. Thus in spite of many disadvantages & harms from automated scripts, the automation testing is adapted widely all over the world.
What automating testing tools are you familiar with?
Win Runner , Load runner, QTP , Silk Performer, Test director, Rational robot, QA run.
2. How did you use automating testing tools in your job?
1. For regression testing
2. Criteria to decide the condition of a particular build
3. Describe some problem that you had with automating testing tool.
The problem of winrunner identifying the third party controls like infragistics control.
4. How do you plan test automation?
1. Prepare the automation Test plan
2. Identify the scenario
3. Record the scenario
4. Enhance the scripts by inserting check points and Conditional Loops
5. Incorporated Error Handler
6. Debug the script
7. Fix the issue
8. Rerun the script and report the result.
5. Can test automation improve test effectiveness?
Yes, Automating a test makes the test process:
1.Fast
2.Reliable
3. Repeatable
4.Programmable
5.Reusable
6.Comprehensive
6. What is data - driven automation?
Testing the functionality with more test cases becomes laborious as the functionality grows. For multiple sets of data (test cases), you can execute the test once in which you can figure out for which data it has failed and for which data, the test has passed. This feature is available in the WinRunner with the data driven test where the data can be taken from an excel sheet or notepad.
7. What are the main attributes of test automation?
software test automation attributes :
Maintainability - the effort needed to update the test automation suites for each new release
Reliability - the accuracy and repeatability of the test automation
Flexibility - the ease of working with all the different kinds of automation test ware
Efficiency - the total cost related to the effort needed for the automation
Portability - the ability of the automated test to run on different environments
Robustness - the effectiveness of automation on an unstable or rapidly changing system
Usability - the extent to which automation can be used by different types of users
8. Does automation replace manual testing?
There can be some functionality which cannot be tested in an automated tool so we may have to do it manually. therefore manual testing can never be repleaced. (We can write the scripts for negative testing also but it is hectic task).When we talk about real environment we do negative testing manually.
9. How will you choose a tool for test automation?
choosing of a tool depends on many things ...
1. Application to be tested
2. Test environment
3. Scope and limitation of the tool.
4. Feature of the tool.
5. Cost of the tool.
6. Whether the tool is compatible with your application which means tool should be able to interact with your application
7. Ease of use
10. How you will evaluate the tool for test automation?
We need to concentrate on the features of the tools and how this could be beneficial for our project. The additional new features and the enhancements of the features will also help.
11. What are main benefits of test automation?
FAST ,RELIABLE,COMPREHENSIVE,REUSABLE
12. What could go wrong with test automation?
1. The choice of automation tool for certain technologies
2. Wrong set of test automated
13. How you will describe testing activities?
Testing activities start from the elaboration phase. The various testing activities are preparing the test plan, Preparing test cases, Execute the test case, Log the bug, validate the bug & take appropriate action for the bug, Automate the test cases.
14. What testing activities you may want to automate?
1. Automate all the high priority test cases which needs to be executed as a part of regression testing for each build cycle.
15. Describe common problems of test automation.
The common problems are:
1. Maintenance of the old script when there is a feature change or enhancement
2. The change in technology of the application will affect the old scripts
16. What types of scripting techniques for test automation do you know?
5 types of scripting techniques:
Linear
Structured
Shared
Data Driven
Key Driven
17. What are principles of good testing scripts for automation?
1. Proper code guiding standards
2. Standard format for defining functions, exception handler etc
3. Comments for functions
4. Proper error handling mechanisms
5. The appropriate synchronization techniques
18. What tools are available for support of testing during software development life cycle?
Testing tools for regression and load/stress testing for regression testing like, QTP, load runner, rational robot, winrunner, silk, testcomplete, Astra are available in the market. -For defect tracking BugZilla, Test Runner are available.
19. Can the activities of test case design be automated?
As I know it, test case design is about formulating the steps to be carried out to verify something about the application under test. And this cannot be automated. IHowever, I agree that the process of putting the test results into the excel sheet.
20. What are the limitations of automating software testing?
Hard-to-create environments like "out of memory", "invalid input/reply", and "corrupt registry entries" make applications behave poorly and existing automated tools can't force these condition - they simply test your application in "normal" environment.
21. What skills needed to be a good test automator?
1.Good Logic for programming.
2. Analytical skills.
3.Pessimistic in Nature.
22. How to find that tools work well with your existing system?
1. Discuss with the support officials
2. Download the trial version of the tool and evaluate
3. Get suggestions from people who are working on the tool
23. Describe some problem that you had with automating testing tool.
1. The inability of winrunner to identify the third party control like infragistics controls
2. The change of the location of the table object will cause object not found error.
3. The inability of the winrunner to execute the script against multiple languages.
24. What are the main attributes of test automation?
Maintainability, Reliability, Flexibility, Efficiency, Portability, Robustness, and Usability - these are the main attributes in test automation.
25. What testing activities you may want to automate in a project?
Testing tools can be used for :
* Sanity tests(which is repeated on every build),
* stress/Load tests(U simulate a large no of users,which is manually impossible) &
* Regression tests(which are done after every code change)
26. How to find that tools work well with your existing system?
To find this, select the suite of tests which are most important for your application. First run them with automated tool. Next subject the same tests to careful manual testing. If the results are coinciding you can say your testing tool has been performing.
27. How will you test the field that generates auto numbers of AUT when we click the button 'NEW" in the application?
We can create a textfile in a certain location, and update the auto generated value each time we run the test and compare the currently generated value with the previous one will be one solution.
28. How will you evaluate the fields in the application under test using automation tool?
We can use Verification points(rational Robot) to validate the fields .Ex.Using objectdata,objectdata properties VP we can validate fields.
29. Can we perform the test of single application at the same time using different tools on the same machine?
No. The Testing Tools will be in the ambiguity to determine which browser is opened by which tool.
31. What is 'configuration management'?
Configuration management is a process to control and document any changes made during the life of a project. Revision control, Change Control, and Release Control are important aspects of Configuration Management.
32. How to test the Web applications?
The basic difference in web testing is here we have to test for URL's coverage and links coverage. Using WinRunner we can conduct web testing. But we have to make sure that Webtest option is selected in "Add in Manager". Using WR we cannot test XML objects.
33. what are the problems encountered during the testing the application compatibility on different browsers and on different operating systems
Font issues,alignment issues
34. how exactly the testing the application compatibility on different browsers and on different operating systems is done
Please Send Your Suggestion
35. How testing is proceeded when SRS or any other document is not given?
If SRS is not there we can perform Exploratory testing. In Exploratory testing the basic module is executed and depending on its results, the next plan is executed.
36. How do we test for severe memory leakages ?
By using Endurance Testing .
Endurance Testing means checking for memory leaks or other problems that may occur with prolonged execution.
Win Runner , Load runner, QTP , Silk Performer, Test director, Rational robot, QA run.
2. How did you use automating testing tools in your job?
1. For regression testing
2. Criteria to decide the condition of a particular build
3. Describe some problem that you had with automating testing tool.
The problem of winrunner identifying the third party controls like infragistics control.
4. How do you plan test automation?
1. Prepare the automation Test plan
2. Identify the scenario
3. Record the scenario
4. Enhance the scripts by inserting check points and Conditional Loops
5. Incorporated Error Handler
6. Debug the script
7. Fix the issue
8. Rerun the script and report the result.
5. Can test automation improve test effectiveness?
Yes, Automating a test makes the test process:
1.Fast
2.Reliable
3. Repeatable
4.Programmable
5.Reusable
6.Comprehensive
6. What is data - driven automation?
Testing the functionality with more test cases becomes laborious as the functionality grows. For multiple sets of data (test cases), you can execute the test once in which you can figure out for which data it has failed and for which data, the test has passed. This feature is available in the WinRunner with the data driven test where the data can be taken from an excel sheet or notepad.
7. What are the main attributes of test automation?
software test automation attributes :
Maintainability - the effort needed to update the test automation suites for each new release
Reliability - the accuracy and repeatability of the test automation
Flexibility - the ease of working with all the different kinds of automation test ware
Efficiency - the total cost related to the effort needed for the automation
Portability - the ability of the automated test to run on different environments
Robustness - the effectiveness of automation on an unstable or rapidly changing system
Usability - the extent to which automation can be used by different types of users
8. Does automation replace manual testing?
There can be some functionality which cannot be tested in an automated tool so we may have to do it manually. therefore manual testing can never be repleaced. (We can write the scripts for negative testing also but it is hectic task).When we talk about real environment we do negative testing manually.
9. How will you choose a tool for test automation?
choosing of a tool depends on many things ...
1. Application to be tested
2. Test environment
3. Scope and limitation of the tool.
4. Feature of the tool.
5. Cost of the tool.
6. Whether the tool is compatible with your application which means tool should be able to interact with your application
7. Ease of use
10. How you will evaluate the tool for test automation?
We need to concentrate on the features of the tools and how this could be beneficial for our project. The additional new features and the enhancements of the features will also help.
11. What are main benefits of test automation?
FAST ,RELIABLE,COMPREHENSIVE,REUSABLE
12. What could go wrong with test automation?
1. The choice of automation tool for certain technologies
2. Wrong set of test automated
13. How you will describe testing activities?
Testing activities start from the elaboration phase. The various testing activities are preparing the test plan, Preparing test cases, Execute the test case, Log the bug, validate the bug & take appropriate action for the bug, Automate the test cases.
14. What testing activities you may want to automate?
1. Automate all the high priority test cases which needs to be executed as a part of regression testing for each build cycle.
15. Describe common problems of test automation.
The common problems are:
1. Maintenance of the old script when there is a feature change or enhancement
2. The change in technology of the application will affect the old scripts
16. What types of scripting techniques for test automation do you know?
5 types of scripting techniques:
Linear
Structured
Shared
Data Driven
Key Driven
17. What are principles of good testing scripts for automation?
1. Proper code guiding standards
2. Standard format for defining functions, exception handler etc
3. Comments for functions
4. Proper error handling mechanisms
5. The appropriate synchronization techniques
18. What tools are available for support of testing during software development life cycle?
Testing tools for regression and load/stress testing for regression testing like, QTP, load runner, rational robot, winrunner, silk, testcomplete, Astra are available in the market. -For defect tracking BugZilla, Test Runner are available.
19. Can the activities of test case design be automated?
As I know it, test case design is about formulating the steps to be carried out to verify something about the application under test. And this cannot be automated. IHowever, I agree that the process of putting the test results into the excel sheet.
20. What are the limitations of automating software testing?
Hard-to-create environments like "out of memory", "invalid input/reply", and "corrupt registry entries" make applications behave poorly and existing automated tools can't force these condition - they simply test your application in "normal" environment.
21. What skills needed to be a good test automator?
1.Good Logic for programming.
2. Analytical skills.
3.Pessimistic in Nature.
22. How to find that tools work well with your existing system?
1. Discuss with the support officials
2. Download the trial version of the tool and evaluate
3. Get suggestions from people who are working on the tool
23. Describe some problem that you had with automating testing tool.
1. The inability of winrunner to identify the third party control like infragistics controls
2. The change of the location of the table object will cause object not found error.
3. The inability of the winrunner to execute the script against multiple languages.
24. What are the main attributes of test automation?
Maintainability, Reliability, Flexibility, Efficiency, Portability, Robustness, and Usability - these are the main attributes in test automation.
25. What testing activities you may want to automate in a project?
Testing tools can be used for :
* Sanity tests(which is repeated on every build),
* stress/Load tests(U simulate a large no of users,which is manually impossible) &
* Regression tests(which are done after every code change)
26. How to find that tools work well with your existing system?
To find this, select the suite of tests which are most important for your application. First run them with automated tool. Next subject the same tests to careful manual testing. If the results are coinciding you can say your testing tool has been performing.
27. How will you test the field that generates auto numbers of AUT when we click the button 'NEW" in the application?
We can create a textfile in a certain location, and update the auto generated value each time we run the test and compare the currently generated value with the previous one will be one solution.
28. How will you evaluate the fields in the application under test using automation tool?
We can use Verification points(rational Robot) to validate the fields .Ex.Using objectdata,objectdata properties VP we can validate fields.
29. Can we perform the test of single application at the same time using different tools on the same machine?
No. The Testing Tools will be in the ambiguity to determine which browser is opened by which tool.
31. What is 'configuration management'?
Configuration management is a process to control and document any changes made during the life of a project. Revision control, Change Control, and Release Control are important aspects of Configuration Management.
32. How to test the Web applications?
The basic difference in web testing is here we have to test for URL's coverage and links coverage. Using WinRunner we can conduct web testing. But we have to make sure that Webtest option is selected in "Add in Manager". Using WR we cannot test XML objects.
33. what are the problems encountered during the testing the application compatibility on different browsers and on different operating systems
Font issues,alignment issues
34. how exactly the testing the application compatibility on different browsers and on different operating systems is done
Please Send Your Suggestion
35. How testing is proceeded when SRS or any other document is not given?
If SRS is not there we can perform Exploratory testing. In Exploratory testing the basic module is executed and depending on its results, the next plan is executed.
36. How do we test for severe memory leakages ?
By using Endurance Testing .
Endurance Testing means checking for memory leaks or other problems that may occur with prolonged execution.
AOM : Automation Object Model
Automating QuickTest Operations
Just as you use QuickTest to automate the testing of your applications, you can use the QuickTest Professional automation object model to automate your QuickTest operations. Using the objects, methods, and properties exposed by the QuickTest automation object model, you can write programs that configure QuickTest options and run tests or components instead of performing these operations manually using the QuickTest interface.
Automation programs are especially useful for performing the same tasks multiple times or on multiple tests or components, or quickly configuring QuickTest according to your needs for a particular environment or application.
You can use the QuickTest Professional automation object model to write programs that automate your QuickTest operations. The QuickTest automation object model provides objects, methods, and properties that enable you to control QuickTest from another application.
Am I doing Too Little or Too Much Automation.
We shall be able to count a large number of possible test cases for any system although fact may be that we might be having too little a time to execute only a few of them. Even if this number may be small, still we would expect to detect majority of our bugs out of these whatever small number of test cases.
Hence identification of test cases to be created & to be executed is an important activity. Random selection of our test cases is not a good approach for testing. Hence for the development of good test cases, a carefully designed approach is necessary
Am I trying to do too much of Automation:Testers usually commit this common mistake of doing too much of automation & that also too soon. Intuitively, testers would be tempted to feel that having more number of automated tests better it would be. But this would be wrong. This belief would leave us with plenty of poorly automated tests, which shall become quite difficult & uneconomical to maintain.
If we go on adding more & more automated tests we shall land up with unwanted duplication, redundancy & increased cost of their maintenance. Hence it is better to start with small numbers of good but diversified tests and automate them. Here it is ideal to begin with say 10-15 tests involving 2-3 hours of interactive testing.
Am I automating the Wrong Tests:
It is not wise to attempt to automate every test case just because the benefit of automating some of the tests is outweighing the cost of automating them. In fact some of the test cases cannot be automated however if some dedicated tester pulls up his strings, can get success in automating them at a very high cost but with no tangible benefit.
However after gaining some reasonable experience of automation testing one can predict with reasonable accuracy the time it will take to automate a particular test. The decision as to which of the test cases to be automated and which of these to be automated first, is quite crucial & is based upon the likely pay back.
Attributes of test case, which can be likely candidate for automation, are as under:1) Any test which can be expected to run several times. It is quite natural that more the times a test case is made to usefully run the more beneficial an automated version of it will be. Regression tests, which are made to run every time a new version of software is created, are best suitable for automation.
2) Input verification tests, for example checking that an edit box accepts values in a valid range only. Such tests are quite boring by nature & prone to error for manual operation.
3) Tests, which are expensive to execute manually, for example multi-user tests, which take a long, time to run.
4) Tests, which are difficult to execute manually, for example test cases which are timing critical or are complex to execute.
5) Tests, which require persons having special knowledge, like business knowledge or system's knowledge can be good candidates for automation.
Attributes of test case, which are unlikely to be candidate for automation, are as under:
1) Any test which will not be expected to run many times. If a test case is not run many times just because there had been no need to run it often (rather than because it could be costly to run it often manually) then it should not be automated.
2) Test cases, which are not important, will definitely not find important bugs. Naturally, If a bug is not important it is not wise to invest time & money in finding it, especially when such bugs are not going to be fixed.
3) Usability test, which cannot be automated because usability, is an issue related to human interactions.
4) Any test which is difficult to automate. Test cases, which would take a lot of effort to automate, are generally not worth automating.
5) Any test which is expensive to maintain. Even if a test case is quite simple to automate but if it is vulnerable to changes in the software it would not be worth automating, because it would be very costly to maintain it.
6) A test case which might have a lot of value in itself, but if it duplicates some or all of the value of an existing test case then the value of the new one gets reduced to almost zero. Such test cases are also not worth automating.
Best Lessons learnt:
1) Where full automation is not warranted, consider partial automation.
For example, it may be difficult to automate the execution of a particular complex test case but it may be possible and beneficial to automate the comparison of some of the results of the test case with the expected results. Alternatively, where the execution of test case cannot be automated may be due to some technical reasons, but it may be possible and beneficial to automate some parts of it like data preparation and clear-up.
2) There is a great learning curve in the beginning, hence it is best not to automate many test cases to start with because they may not be as good & fit for automation, compared to the ones we could automate later on after having learnt more about good practices.
3) It is better to focus on relatively few tests, trying out different implementations and assessing their relative strengths and weaknesses before automating large numbers of tests.
Remember that many case histories have made startling revelation, that over 70% of bugs are found by manual testing & not by automated testing in-spite of the fact that the automated tests had been designed & developed with great efforts of many years. Hence think well before automating your tests.
Hence identification of test cases to be created & to be executed is an important activity. Random selection of our test cases is not a good approach for testing. Hence for the development of good test cases, a carefully designed approach is necessary
Am I trying to do too much of Automation:Testers usually commit this common mistake of doing too much of automation & that also too soon. Intuitively, testers would be tempted to feel that having more number of automated tests better it would be. But this would be wrong. This belief would leave us with plenty of poorly automated tests, which shall become quite difficult & uneconomical to maintain.
If we go on adding more & more automated tests we shall land up with unwanted duplication, redundancy & increased cost of their maintenance. Hence it is better to start with small numbers of good but diversified tests and automate them. Here it is ideal to begin with say 10-15 tests involving 2-3 hours of interactive testing.
Am I automating the Wrong Tests:
It is not wise to attempt to automate every test case just because the benefit of automating some of the tests is outweighing the cost of automating them. In fact some of the test cases cannot be automated however if some dedicated tester pulls up his strings, can get success in automating them at a very high cost but with no tangible benefit.
However after gaining some reasonable experience of automation testing one can predict with reasonable accuracy the time it will take to automate a particular test. The decision as to which of the test cases to be automated and which of these to be automated first, is quite crucial & is based upon the likely pay back.
Attributes of test case, which can be likely candidate for automation, are as under:1) Any test which can be expected to run several times. It is quite natural that more the times a test case is made to usefully run the more beneficial an automated version of it will be. Regression tests, which are made to run every time a new version of software is created, are best suitable for automation.
2) Input verification tests, for example checking that an edit box accepts values in a valid range only. Such tests are quite boring by nature & prone to error for manual operation.
3) Tests, which are expensive to execute manually, for example multi-user tests, which take a long, time to run.
4) Tests, which are difficult to execute manually, for example test cases which are timing critical or are complex to execute.
5) Tests, which require persons having special knowledge, like business knowledge or system's knowledge can be good candidates for automation.
Attributes of test case, which are unlikely to be candidate for automation, are as under:
1) Any test which will not be expected to run many times. If a test case is not run many times just because there had been no need to run it often (rather than because it could be costly to run it often manually) then it should not be automated.
2) Test cases, which are not important, will definitely not find important bugs. Naturally, If a bug is not important it is not wise to invest time & money in finding it, especially when such bugs are not going to be fixed.
3) Usability test, which cannot be automated because usability, is an issue related to human interactions.
4) Any test which is difficult to automate. Test cases, which would take a lot of effort to automate, are generally not worth automating.
5) Any test which is expensive to maintain. Even if a test case is quite simple to automate but if it is vulnerable to changes in the software it would not be worth automating, because it would be very costly to maintain it.
6) A test case which might have a lot of value in itself, but if it duplicates some or all of the value of an existing test case then the value of the new one gets reduced to almost zero. Such test cases are also not worth automating.
Best Lessons learnt:
1) Where full automation is not warranted, consider partial automation.
For example, it may be difficult to automate the execution of a particular complex test case but it may be possible and beneficial to automate the comparison of some of the results of the test case with the expected results. Alternatively, where the execution of test case cannot be automated may be due to some technical reasons, but it may be possible and beneficial to automate some parts of it like data preparation and clear-up.
2) There is a great learning curve in the beginning, hence it is best not to automate many test cases to start with because they may not be as good & fit for automation, compared to the ones we could automate later on after having learnt more about good practices.
3) It is better to focus on relatively few tests, trying out different implementations and assessing their relative strengths and weaknesses before automating large numbers of tests.
Remember that many case histories have made startling revelation, that over 70% of bugs are found by manual testing & not by automated testing in-spite of the fact that the automated tests had been designed & developed with great efforts of many years. Hence think well before automating your tests.
Judiciously testing all possible permutations of such components creates a highly complex testing situation with hundreds or thousands of testing scenarios. Under such situations there comes a need for automating the testing process with the help of automation framework approach, which can help in achieving detailed testing with great reduction in testing time.
It can never be a workable idea to automate all the test cases. Hence it is important to scientifically understand the areas which can be automated. Remember that an ad-hoc approach to test automation can in fact, lead to longer testing time and poor quality irrespective of the name & fame of the testing tool selected by you.
Now let us understand what is 'Test Automation Framework'?
Test Automation Framework is a set of assumptions, concepts and practices which provide necessary support for the automated software testing.
The main advantage of such a framework is the low cost for maintenance. If there is change to any test case then only the test case file needs to be updated and the Driver Script and Startup script will remain the same. There's no need to update the scripts in case of changes to the application.
Types of Automation Framework:
1) Modularity driven framework: It requires the creation of small, independent scripts that represent modules, sections, and functions of the application-under-test. These small scripts are then used in a hierarchical fashion to construct larger tests, realizing a particular test case. It applies the principle of abstraction or encapsulation in order to improve the maintainability and scalability of automated test suites.
2) Data driven framework: It involves bunch of several interacting test scripts clubbed with their related data results. In this framework, variables are used for both input values and output verification values: navigation through the program, reading of the data sources, and logging of test status and information are all coded in the test script. It is quite suitable framework for use in RFT using Data-pools. This approach reduces coding effort to a great extent in case of large test cases, which otherwise could be quite time-consuming & cumbersome.
3) Keyword driven framework: It involves automated tests, which have inherent reusability and therefore ease of maintenance of tests that have been created at a high level of abstraction. It divides test creation into two stages like
a) Planning Stage: Involving analysis of the requirements for the application to determine which operations and objects have to be tested. E.g. an application having web based questionnaire will require a large amount of text entries.
b) Implementation Stage: It differs according to the tool or framework used. Generally automation engineers implement a framework that provides keywords like "check" and "enter". Testing engineers (who don't have to know the coding) write the test cases based on the keywords defined in the planning stage that have been implemented by the engineers. The test is executed using a driver who reads the keywords and executes the corresponding code.
Keyword Driven Framework methodology requires more planning and a longer initial time-investment than going directly to the test creation stage and recording a test, it does make the test creation and test maintenance stages more efficient and keeps the structure of individual tests more readable and easier to modify.
4) Hybrid framework: Is a combination of three frameworks. This type of frameworks evolve over a passage of time and across multiple projects. It is the most successful automation frameworks, which generally accommodates both Keyword-driven testing as well as Data-driven testing. This allows data driven scripts to take advantage of the powerful libraries and utilities that usually accompany a keyword driven architecture.
In this case, the framework utilities can make the data driven scripts more compact and less prone to failure. The utilities can also facilitate the gradual and manageable conversion of existing scripts to keyword driven scripts as & when required. On the other hand, the framework can use scripts to perform some tasks that might be too difficult to re-implement in a pure keyword driven approach, or where the keyword driven capabilities are not yet in place.
Ten Steps for Test Automation Framework Methodology:
1) Identification of the Scope of Testing: Company oriented, Product oriented, Project Oriented.
2) Identification of the Needs of Testing: Identify Types of testing e.g. FT, Web Services etc. and application / modules to be tested.}
3) Identification of the Requirements of Testing: Find out the nature of requirements, identify type of actions for each requirement & identify high priority requirements.
It can never be a workable idea to automate all the test cases. Hence it is important to scientifically understand the areas which can be automated. Remember that an ad-hoc approach to test automation can in fact, lead to longer testing time and poor quality irrespective of the name & fame of the testing tool selected by you.
Now let us understand what is 'Test Automation Framework'?
Test Automation Framework is a set of assumptions, concepts and practices which provide necessary support for the automated software testing.
The main advantage of such a framework is the low cost for maintenance. If there is change to any test case then only the test case file needs to be updated and the Driver Script and Startup script will remain the same. There's no need to update the scripts in case of changes to the application.
Types of Automation Framework:
1) Modularity driven framework: It requires the creation of small, independent scripts that represent modules, sections, and functions of the application-under-test. These small scripts are then used in a hierarchical fashion to construct larger tests, realizing a particular test case. It applies the principle of abstraction or encapsulation in order to improve the maintainability and scalability of automated test suites.
2) Data driven framework: It involves bunch of several interacting test scripts clubbed with their related data results. In this framework, variables are used for both input values and output verification values: navigation through the program, reading of the data sources, and logging of test status and information are all coded in the test script. It is quite suitable framework for use in RFT using Data-pools. This approach reduces coding effort to a great extent in case of large test cases, which otherwise could be quite time-consuming & cumbersome.
3) Keyword driven framework: It involves automated tests, which have inherent reusability and therefore ease of maintenance of tests that have been created at a high level of abstraction. It divides test creation into two stages like
a) Planning Stage: Involving analysis of the requirements for the application to determine which operations and objects have to be tested. E.g. an application having web based questionnaire will require a large amount of text entries.
b) Implementation Stage: It differs according to the tool or framework used. Generally automation engineers implement a framework that provides keywords like "check" and "enter". Testing engineers (who don't have to know the coding) write the test cases based on the keywords defined in the planning stage that have been implemented by the engineers. The test is executed using a driver who reads the keywords and executes the corresponding code.
Keyword Driven Framework methodology requires more planning and a longer initial time-investment than going directly to the test creation stage and recording a test, it does make the test creation and test maintenance stages more efficient and keeps the structure of individual tests more readable and easier to modify.
4) Hybrid framework: Is a combination of three frameworks. This type of frameworks evolve over a passage of time and across multiple projects. It is the most successful automation frameworks, which generally accommodates both Keyword-driven testing as well as Data-driven testing. This allows data driven scripts to take advantage of the powerful libraries and utilities that usually accompany a keyword driven architecture.
In this case, the framework utilities can make the data driven scripts more compact and less prone to failure. The utilities can also facilitate the gradual and manageable conversion of existing scripts to keyword driven scripts as & when required. On the other hand, the framework can use scripts to perform some tasks that might be too difficult to re-implement in a pure keyword driven approach, or where the keyword driven capabilities are not yet in place.
Ten Steps for Test Automation Framework Methodology:
1) Identification of the Scope of Testing: Company oriented, Product oriented, Project Oriented.
2) Identification of the Needs of Testing: Identify Types of testing e.g. FT, Web Services etc. and application / modules to be tested.}
3) Identification of the Requirements of Testing: Find out the nature of requirements, identify type of actions for each requirement & identify high priority requirements.
4) Evaluation of the Test Automation Tool: Evaluation checklist, Identify the candidate tools available in the market, Sample run, rate & select the tools, Implementation & Training
5) Identification of the Actions to be automated: Actions, Validations & requirements supported by the Tool
6) Design of the Test Automation Framework: Framework guidelines, validations, Actions Involved, Systems involved, Tool Extensibility Support, Customs messages & UML Documentation.
7) Design of the Input Data Bank: Types of Input file. Input files – Categorization & Design of file prototypes.
8) Development of the Automation Framework: Development of script based upon framework design, Driver scripts, Worker Scripts, Record / Playback, Screen / Window / Transaction, Action / Keyword & Data Driven.
9) Population of Input Data Bank: Different Types of data Input, Populate data from different data sources, Manual input of data and Parent – Child data hierarchy.
10) Configuration of the Schedulers: Identify scheduler requirements & configure the schedulers.
Benefits of Test Automation Framework Approach:Test Automation Framework built with systematic approach yields following benefits:
# Ensures consistency
# Significant reduction in the amount of code to develop & maintain thereby reducing the testing cycle time.
# Comprehensive coverage against requirements.
# Use of a "Common Standard" across the organization / Product team / Project team
# Maximizes reusability of test scripts ( Utility Functions)
# Provides a structured for test library having systematic maintenance of automation scripts
# Data Pooling
# Protects non-technical testers from the code
8) Development of the Automation Framework: Development of script based upon framework design, Driver scripts, Worker Scripts, Record / Playback, Screen / Window / Transaction, Action / Keyword & Data Driven.
9) Population of Input Data Bank: Different Types of data Input, Populate data from different data sources, Manual input of data and Parent – Child data hierarchy.
10) Configuration of the Schedulers: Identify scheduler requirements & configure the schedulers.
Benefits of Test Automation Framework Approach:Test Automation Framework built with systematic approach yields following benefits:
# Ensures consistency
# Significant reduction in the amount of code to develop & maintain thereby reducing the testing cycle time.
# Comprehensive coverage against requirements.
# Use of a "Common Standard" across the organization / Product team / Project team
# Maximizes reusability of test scripts ( Utility Functions)
# Provides a structured for test library having systematic maintenance of automation scripts
# Data Pooling
# Protects non-technical testers from the code
What are the steps during implementing the tests with Keyword Driven Methodology?
No comments:
Post a Comment