The Earlier A Bug Is Found, The Less Expensive It Is To Fix.

OTIVO’s Web Site Testing Process

• Phase 1: Project Kickoff
  Finalize objectives, bug reporting, testing specifications, list of browser configurations, and other functionality requirements. We usually test using a variety of operating systems and browsers and devices and gadgets. In 2009, most of our projects required testing in Firefox, Safari, MSIE, WinXP, WinVista, Windows7, and MacOS10 (Leopard and Snow Leopard) as well as a few browsers on mobile devices (iPhone, Android/Droid, Blackberry). We can test, if requested, in Opera, Chrome, Android browser applications, and other browsers. We also have older operating systems and browsers available for testing, such as WinNT, Win2000, Win98, and MacOS9. Windows 3.1, Commodore 64 testing and Amiga testing are not available (tee-hee).
• Phase 2: Test Plan and Strategy
  Develop test plan and strategy, and set up bug database project to manage bug reporting. We use our own Web-based bug database or any available bug database that you’re already using. (We’re fond of JIRA and bugzilla and recommend you implement one or the other for your own use)
• Phase 3: Testing and Bug Reports
  Test Web site and application according to test plan, including browser compatibility testing. Report bugs in bugbase. Email daily status reports to client with information about completed testing, the next day’s planned testing, reported bugs, and outstanding issues.
• Phase 4: Bug Regression
  Regress bugs as they are fixed to insure fix is complete and hasn’t created any new bugs.
• Phase 5: Final Report
  Develop and deliver test completion report.

Frequently Asked Questions

1. How much does it cost? What do I get?
   Most projects are done on an hourly basis with an estimated cost and number of hours determined upfront for a specific set of deliverables. OTIVO charges $70-130/hour for testing and bug reporting and $100-180/hour for testing management. We are very efficient (ask us for references!).

   Sample statement of work for a Web QA project (PDF download; Acrobat Reader required)

2. Why should we outsource instead of testing in-house?
   In a word: Objectivity.

   Familiarity breeds blindness. When you work on and develop a project, you often can’t see or find obvious bugs, omissions, and errors. Our testers ramp up and become familiar with a Web site while they’re testing so they have a fresh view of the application, elements, and interface.

3. Why not rely on developers to test their own pages and elements
   Again, objectivity, and cross-platform-friendliness.

   Developers typically develop and test on the same platform while they are working. They also see the same page or element a bajillion times, so if you put it in front of their face with a typo on the third line or an incorrect calculation, they could easily miss it.

4. Why rely on lengthier, more costly test plans to test a Web site, instead of just pounding on the site (gorilla testing)?
   Especially with content publishing and electronic commerce applications, it’s important to test as much of the site and application’s range of functionality as possible. Gorilla testing (pounding on a site or application in as many ways as the tester can think of without an organized test plan) is useful, but the methodical process that creates a test plan usually covers more of the possible functionality than randomly pounding on the site.

More of OTIVO’s Services: Usability Testing and Research