Brooke Kelly answers the question: “What is QA?” and breaks down the process of Quality Assurance when it comes to your website.
Quality Assurance: the ever important practice of making sure things work. Where QA lacks the innovative experimentation and flashy creativity of development and design, or the leadership of project management, it makes up for in consistency, organization, and the incredibly rewarding feeling of knowing, with complete confidence, when the final product is ready for the client. For those of us who are detail oriented, obsessively methodized, and really good at thinking outside the box (or as I prefer to say, “coming up with ways to break things”), QA is the perfect role. (You know who you are.)
For everyone else working for a digital agency or software development company, however, QA might not seem so cut and dry, or easy to understand, especially if you’re a newcomer to the industry. “What exactly are they doing all day if they aren’t manipulating code, designing websites, or marketing and managing the company?”
One of the best ways to start learning about QA, in order to understand what exactly it is that we do, is to first learn how to speak like one of us. No, I’m not about to lead you through some bizarre initiation ritual chant (repeat after me: “Always-think-like-a-user, always-think-like-a-user…”), I’m going to walk you through some common “QA lingo”, so that you too may speak like a Professional Quality Analyst. Whether you’re a seasoned veteran of the IT world, or you’ve just started out in the field, reviewing QA terminology can never hurt (after all, I’m a proponent of “QA-ing” our own brains as often as possible).
QA Lingo: What are we Talking About?
There are three main “jobs” a QA analyst is expected to perform: planning for tests, implementing tests, and reporting on tests.
When it comes to ensuring that your company is following as many QA best practices as possible, the planning stage is critical, and cannot be skipped: after all, how can you run a successful and efficient test – especially during a time sensitive build or alpha period – without first strategizing, planning, and creating proper documentation?
- Test Strategy: Many digital agencies and software development companies encourage an initial document called the Test Strategy, which is a high-level overview of the QA approach for a given testing period. This document often looks at: project scope, test methodologies (deciding whether or not manual testing, automated testing, or some combination of the two will be used), the testing schedule, completion criteria, materials, and test set-up. Some companies prefer to combine this document with more detailed test plans, others keep them separate, and still others simply prefer a verbal planning meeting with notes taken over writing a physical document. It often varies by project size and scope.
- Test Plan (which outline Test Cases): A QA Test plan is a document that outlines every item that needs to be tested on a given build release day or alpha testing period (there are often two types of test plans per release: the build-specific test plan and the regression test plan – we’ll get to this a little later). Test plans are step-by-step guides that are meant to lead potential testers through each validation or “step-to-reproduce”, sort of like a video game walkthrough or Google Maps directions. Test Plans are written based on the project business requirements determined by the client and the project manager(s) and/or business analyst(s) at the start of the project. Regardless of whether you will be running a manual test or an automated test, having a step-by-step plan in place before the actual testing day or period is essential for optimum execution – not to mention, the documentation remains as historical proof that your company not only tested their own work, but knew exactly how to do so.
- Test Matrices: Test matrices go hand-in-hand with test plans. They are spreadsheets that take the test plan validations and test cases and organize them into a simple, easy-to-follow “check-list” format.
Example section of a test matrix
Test matrices are also excellent for easily comparing differences between scenarios. For example, if a website has a form that triggers different options based on user age, the test matrix would outline a range of age-specific test cases (as per the original test plan’s suggestion), to make sure all form options are tested accordingly and to allow comparison between results. The matrix also prioritizes test categories and provides reporting on test coverage. Tracking data is a very important QA best practice because it helps you identify improvements for future testing.
After the planning period, it’s time to execute our tests. There a several types of QA tests, and each one has an important role in improving the overall quality of a product.
- Build-Specific Testing (also called Alpha Testing): These are tests that specifically focus on new releases. This could range from an entirely new project to a simple addition to an already-existing software or website. Each build or “alpha” test has a corresponding build test plan and test matrix to help organize the testing process. Alpha Testing also refers to any testing period that is internal; testing done by the client is usually referred to as UAT (User Acceptance Testing) or Beta Testing, and comes right before the production release (especially if working within an Agile sprint-based framework).
- Regression Testing: Regression Testing, at its simplest, is based entirely on verification: making sure that what existed before a new addition didn’t break because of the new addition. It is a quality control measure to ensure that “new” or modified code follows all requirements and that unmodified or “old” code still “does its job.” The best QA teams often have a Master Regression Test Plan and Matrix in place for every existing project their company actively maintains or starts, that can be tested against during a new release. Regression testing is vital for quality assurance, and should never be skipped!
- Compatibility Testing: Testing to make sure something works WITH something else, simultaneously. This includes testing softwares running at the same time, cross-browser testing, and cross-device testing, to ensure something works within a wide range of contexts. Users are buying all kinds of devices nowadays: make sure your company knows their clients well, and understands what devices, browsers, and softwares they are using so that all products are accurately tested for compatibility.
- Smoke Testing: This is actually a slang term originally coined by programmers in reference to hardware testing. As told by Webopedia: “When testing software in development, the joke is if it is tried on a new piece of hardware for the first time and it does not catch on fire, it is a successful test.” Smoke testing is high-level overview testing: a “general sweep” of a newly launched website, for example, that only checks for major errors/issues. Smoke testing is used on an as-needed or Ad Hoc basis.
Finally, when the tests are complete, it’s up to the QA department to repor
t on their testing and share their results/experience with the rest of the company. In truth, all three QA “jobs” tie together: Planning and documentation help guide testers and make the testing phase more efficient, but they also provides historical data for reporting and post-production process analysis.
- Defect Tracking: This is arguably THE MOST important task a quality analyst needs to complete during and after a testing period. Defect tracking is, as the name states, keeping track of all issues or bugs that pop up during a build or beta testing period. This includes finding the defects during testing, reporting them to the lead developers of the project, and re-testing after the required maintenance. Many companies use external defect tracking programs like JIRA and Bugzilla to help with this process; others simply keep track of defects in their own internally managed spreadsheets or programs. Paying attention to defect tracking from a data analysis standpoint is also important: say you found 50 defects during beta testing, and the client only found 20 defects during UAT (10 of which were new, and 10 of which you found in that initial batch of 50). You now have historical data showing just how thorough your company was – and information on exactly where you need to improve for next time.
- Bug Postmortem (sometimes also called the Build Postmortem): Usually, this is a meeting that is held after production (or sometimes directly after the testing period) to discuss the results of the defect tracking with the rest of the company (development, management, design, etc); some companies, though, prefer to simply have a monthly QA processes discussion, and go over all the numbers for several projects at once, rather than have a postmortem meeting after every production release. No matter how your company chooses to do it, taking a close look at the matrix reporting and defect tracking results is vital for discussing how the company can improve overall, and making sure that the testing process improves in efficiency, while also making sure less defects are caught next time both internally and externally. The goal is always for there to be zero defects found during both Alpha and Beta (UAT) testing! It’s a lofty goal, but one we always want to strive for.
Feeling Like A Quality Analyst Yet?
Now that we’ve gone over some key QA lingo, you’re practically one of us! Whether you’re a project manager, developer, designer, marketer or newly inducted quality analyst, you’re that much more aware of what goes into quality assurance. The most important thing to remember, though, is that we’re a team: so no matter where your role lies within your own IT company, understanding every part of the software development, website creation, and digital marketing process puts you that much more ahead of the game.
For even more QA Lingo, check out this handy list!
Ready to dive in? Here’s my next post, how-to “Save Time Testing Submissions with Selenium IDE.” Enjoy!
Not ready, and want help? That’s what we’re here for! Contact us and let’s talk about how Delphic can support your business.