Problem solve Get help with specific problems with your technologies, process and projects.

Save time -- automate

Everyone tests their software, but you could be on the cutting edge with automated software testing.

Save time -- automate
Elfriede Dustin

Sure, you do software testing. But is the testing automated? This tip, by Elfriede Dustin, the author of Automated Software Testing: Introduction, Management, and Performance (Addison Wesley), discusses some of the prerequisites for automating software testing. This tip is excerpted from InformIT.

The test team initiates the test tool introduction process by analyzing the organization's current test process. Generally, some method of performing tests is in place, and therefore the exercise of defining the process may actually result in process improvement. In any case, process improvement begins with process definition.

The test process must be documented in such a way that it can be communicated to others. If the test process is not documented, it can't be communicated or executed in a repeatable fashion. If the test process can't be communicated or isn't documented, it's less likely to be implemented. In addition, if the process isn't documented, it can't be consciously and uniformly improved. On the other hand, if a process is documented it can be measured and therefore be improved.

If the organization's overall test process is not yet documented, or is documented but outdated or inadequate, the test team may need to adopt part or all of an existing test process. As the organization's test process, the test team may adopt the Automated Test Lifecycle Methodology (ATLM) outlined in Automated Software Testing: Introduction, Management, and Performance. When defining or tailoring a test process, it may prove useful for the test engineer to review the organization's product-development or software-development process document, when available.

The test engineer needs to analyze the existing development and testing process. During this analytical phase, the test engineer determines whether the current test process meets the following defined prerequisites:

  • The testing goals and objectives have been defined.
  • The testing strategies have been defined.
  • The required tools are available to implement planned strategies.
  • A testing methodology has been defined.
  • The test process is communicated and documented.
  • The test process is being measured.
  • The test process implementation is audited.
  • Users are involved throughout the test program.
  • The test team is involved from the beginning of the system development lifecycle.
  • Testing is conducted parallel to the system development lifecycle.
  • The schedule allows for process implementation.
  • The budget allows for process implementation.
  • The organization is seeking to comply with industry quality and process maturity guidelines (CMM, ISO).

The purpose of analyzing the organization's test process is to identify test goals, objectives, and strategies that may be inherent in the test process. These elements of test planning are the cornerstones for which a project's test program develops. The purpose of documenting the test tool introduction process is to ensure that the test team has a clearly defined strategy for implementing automated testing, so that the team can fully leverage the functionality and time-saving features of the automated test tool.

The additional time and cost associated with the documentation and implementation of a test tool introduction process is sometimes an issue. A well-planned and well-executed process will pay for itself many times over by ensuring a higher level of defect detection and fielded software fixes, shortening product-development cycles, and providing labor savings. A test team will perform well if it is disciplined in defining test goals and reflecting the test goals within defined processes, the skills of test team staff, and the selection of a test tool. This kind of discipline, exercised incrementally, supports the test team's (and the entire organization's) advancement in quality and maturity from one level to the next.

To read this entire tip, click over to InformIT. You have to register there, but it's free.

Did you like this tip? Email us and let us know.

Related Book

Effective Methods for Software Testing, 2nd Ed.
Author : William Perry
Publisher : John Wiley & Sons
Published : Feb 2000
Summary :
Software testing is an essential part of developing computer systems and applications--you've got to make sure that programs work as they were intended to. If they don't, you've got unhappy customers and company employees, lost productivity, and lost revenue. A company can even face lawsuits if its software doesn't work as advertised. There are a lot of methods of software testing--not all are effective. Effective Methods of Software Testing, Second Edition, selects proven testing techniques and organizes them into a complete testing program that will ensure that the software works in the way it was designed. Testing is done at many levels. In some companies, there are people who have software testing as a job responsiblity. In other cases, it may be a programmer or software developer who has to take over some testing functions. This book is intended mainly for software testers, but it certainly is useful for others with testing responsibilities.

Dig Deeper on Enterprise infrastructure management

Join the conversation


Send me notifications when other members comment.

Please create a username to comment.

Automation is a good step for when you know specifically what you are looking for, and want to prove it is there. It's a wonderful addition to any regression testing strategy, and it helps tremendously with where you know what you expect to see. the challenge with automation is that it cant make judgment calls or be genuinely curious. that requires up front exploration, and thus far, I would say the risk of software automating the exploration process is pretty slim. My personal mantra is "explore, learn, process what we learn, then automate the steps to get back there and confirm it each time". The explore part, though, starts with fresh eyes, and a curious mind, two tools that automation cannot yet emulate.
Michael has made fantastic points about what automation is useful for.

I wanted to mention that it is extremely rare for automation to save time (or money) for a development org. With exception of one off tools and one time uses of scripts to load data and whatnot, automation is basically another development project. It has to be developed, vetted, maintained, and cared for. 
When you utomate the testing process, the common errors/bugs and flaws most likely will be found. It will help in some ways when there is not enough manpower to do internaly. Personally I do most of  my testing as if I were back in grade school. Trying things so far off from what should be obvious and I find a lot of issues most people do not catch. The key here is automated testing applications were designed with some form of logic in mind. Try testing as if no logic is involved and you may be surprised what you find.
Following up on this, there is a terrific Ruby Rogues Podcast that talks about the confirmation bias that goes into software programming and the ways that we start to accept the so called logic of the system even at the language design level. It's very relevant to this topic, and worth a listen, even if you do not program in ruby. See for show notes or to listen.
Woah, let's not be to hasty. Its easy to look at the fast execution time of automation and say, yes we need that so we can test more.  Few people spend sufficient time considering, what is really valuable to automate as opposed to trying to throw an entire bucket of scenarios at them.)  

The author of the linked article spends a lot of time talking about documenting the existing test processes, and that's certainly important, but there is a lot of value in automation that goes beyond the traditional 'do it like a user does it' way some automation is crafted from end to end.  Yet the reason so many automation efforts fail, is because little time is spent setting real objectives for the testing.  It seems so easy to just say 'this is the feature we are building this sprint, you, test automator, go automate it.'  

For one thing, the right hooks need built into the feature as it is being constructed.  Without good locators, you run the risk of serious brittleness in an end to end test.  What's more, there is also the risk of turning a blind eye to the value of the human eye combined with an effective brain, and many usability issues are missed, precisely because no human eye finds it, until the customer encounters it on time.