How do you maintain the quality of your software while making your testing efforts more efficient?

Back in my first post, I lamented the way that testing and code quality seemed to be little more than lip service in my prior employment experience and how I strove to change that perspective at my current company. I am very lucky to have the opportunity to attempt these changes and I feel that there have been many successes. Yet even with that opportunity, one thing doesn’t change: there is SO MUCH that needs to be done. So how do you maintain the quality of your software while making your testing efforts more efficient? Spoiler alert: there is no silver bullet. However, here are some things that help us streamline testing.

Having a plan for testing is essential to help streamline the process. It can be as informal as a checklist or as detailed as a complete list of tests, but defining the direction and scope of testing ahead of time is a huge time saver. You may want to see if there are some more efficient test automation tools that can be of use for this project and future ones too. It could help you streamline the process and get the results you need. Beware of putting too much effort into planning. The amount of planning should match the impact of the work. There is nothing more important than time, and spending thirty minutes filling out test planning paperwork for a ten-second test kills productivity, enthusiasm, and momentum.

Pull Requests and Code Review

Testers should be looking at the changes made to code, attempting to understand those changes, and asking questions when they don’t. This will help confirm if the test plan is accurate or needs revision prior to testing. It will establish a dialogue with developers that can reveal updated or missed requirements that can lead to unnecessary delays in testing. Moreover, you might choose to hire remote developers and testers for your business software development instead of creating an in-house team of software engineers. In that situation, as testers and developers build their relationships, testers improve their code understanding and learn new methods of what and how to test.

Regression Testing First

As with planning, not every test effort will require regression testing. But when regression testing is part of the testing plan, do it first. That way, if you find that something that previously worked is broken, it’s as early in the process as you can make it. Too often, a majority of the testing time is spent testing the changes first, and then regression testing is done. This approach usually means that regression testing is short-changed. If regressions are found late in the cycle, there may not be enough time to correct them before release, leaving product owners no choice but to “accept” them to get their new feature released according to their timelines.

Multiple Testing Environments

This might seem obvious, but it’s surprising how frequently companies only have one, or even just part of one, test environment. Here’s a good return-on-investment (ROI) exercise to support more than one testing environment: If important (and potentially revenue-generating) features are delayed because they are stuck in the testing queue, do the investigating and find out how much the “delay” is “costing” the company to help make your point. Having the ability to test in parallel reduces queue time and time spent in test set up and context shifting. Even a partial second environment can have a significant impact on streamlining testing.

“Tetris-ing”

Yes, this is a reference to the 80s block stacking game, but the analogy is valid. This tactic is most beneficial when you have more than one test environment, but it could still apply if you have just one. Not every feature will require a complete environment for testing. This is especially true for systems that have both front-end and back-end components. Identifying those features that do not require an end-to-end test environment and deploying them simultaneously means that two (or sometimes more) features can get tested at the same time. And as above, parallelism streamlines testing.

“NOT” Testing

(Unconventional Testing)

Production Testing – Don’t shake your head like that. Read a bit first. This may not be possible for some companies, but for others, it should be seriously considered. Testing takes time and time is money. All testing has a cost that should provide a “Return on Investment”. If the risk is very low, it can be more cost-effective to roll out a change to a single production machine, a small group of production machines, or the whole system and then monitor the effects than to configure a testing environment and run tests. An example would be an ETL query that has been modified to include or exclude a type or status in the result set. In addition, you can eliminate the need to run costly join queries or track down data between sources with a reverse ETL query. New integrations and Proof of Concept (POC) work also fall into this category since the work is essentially “on spec,” with the expectation that it will provide benefits in the long run.

Over-The-Shoulder Testing – Or you could call it “Developer-Tester Peer Testing”. Test environments don’t always have the resources that production environments do, especially for data heavy jobs. When this is the case, we often do a one-on-one code demonstration, where the developer walks through the existing and updated process(es) with the tester and must provide before and after data, either to show that it hasn’t changed, or that it has changed according to the new requirements. This type of testing should be planned, and the tester should posit various use cases, scenarios or possible issues (missing data, bad data, no connection, no directory, etc.) and the developer should be able to show or demonstrate the code that addresses them. This type of testing is where developer-tester relationships are built.

Developer Testing – And before you start yelling: no, I am not advocating or in any way suggesting that developers can replace testers. But testers should realize that good developers do test, and great developers test a lot. They write unit tests, they generate files and do differentials on the outputs, and they validate data inputs and outputs. And if a tester doesn’t know that those tests were performed, they will test them again. This is a duplication of effort. We advocate developers documenting their unit tests and any testing they’ve done, so instead of repeating those tests, testers “validate” them. Or they can “reject” them (after a discussion) if they feel the testing doesn’t cover what was planned. They then perform any additional planned tests (like regression tests). The objective is to reduce the duplication of testing effort.

Automation

Or more accurately, testing with tools. All testers use tools to test. Some use more complicated tools than others. Look for opportunities to remove manual and repetitive test operations, whether it is generating test data, validating test data, or full-blown automation suites. If you are making your first foray into automation test suites, resist the urge to begin with your user interface, especially if it’s is evolving. You’ll end up spending more time maintaining tests than using them. Instead, look to validate your APIs and other backend systems. The initial investment is more, but the longevity and robustness of your tests will provide a much better ROI.

Central Library

In our business, we have many customers making requests and receiving responses from us. Most use a standard format, but quite a few need custom requests, custom responses, or both. We maintain sample requests and expected responses in a central area. Developers, testers and even account managers use these samples to either baseline existing code before making changes, verify that code changes haven’t negatively affected expected responses or for troubleshooting. Having a central repository saves time and reduces duplication of data. Your business may also have common data or processes shared by many groups and could benefit from a central repository.

Test Management System

This is an extension of the central repository idea that is specific to testing. We use a test management system to house all of our tests, whether automated or manual, in one location. Tests can be curated and grouped by descriptions and keywords to map to coverage areas. This means that regression testing can be limited to only the areas affected by the changes under test rather than running a complete regression every time. One copy means only one place that tests need to update when features change and only one place to look when searching for test documentation with new team members.

The examples above all have one or more common threads: return on investment (ROI – or as I like to ask, “Is the juice worth the squeeze?”), reduce or eliminate duplication of effort, and pragmatism. It also offers employees ways to embrace many of the concepts mentioned in my previous post on the Quality Mindset, including heightening awareness, extending and encouraging trust and ownership, and accepting risk. Each company’s needs and circumstances are different, so feel free to change up what you’ve read here or to use it as a springboard to come up with your own method of streamlining your testing.