1. What is testing?

The content here is under the Attribution 4.0 International (CC BY 4.0) license

A note before starting

Hello there, if you are here it is probably because you want to understand more what is software testing and its contexts, this is great because the first part of this series about software testing goes exactly in this direction. However, the content that follows does not aim at being an extensive guide, it is focused on a variety of sources that define software testing and what it is in the software development context.

Introduction

Have you ever wondered what the practice of testing is? The practice of testing among us in our everyday actions. When we wake up, when we go to work or even the act of opening a door. Does that work? how long it will last? What happens if I try to pass without opening it?

If we take the place of a seller who wants to sell a door, customers will ask those questions and even more before buying it. This hypothetical scenario gives the ground to start narrowing down the context in which the testing activity takes place. Can we find the correct answer to those questions? Well, in the door example, we can look at what the definition of test is at the core of the word and see what we can do. According to dictionary.com test is described as follows:

1. the means by which the presence, quality, or genuineness of anything is determined; a means of trial.
2. the trial of the quality of something: to put to the test.
3. a particular process or method for trying or assessing.

Rapidly we can adjust the definition to fit our door scenario. For example, in a hypothetical scenario, the first step would be to submit the product through a series of tests, such as the material resistance, durability, lifetime and so on. This would give us the numbers needed to understand the different characteristics of the door. However, it does not mean that this is tested in all scenarios possible and all cases in the door will be used. In other words, the essence of a test is simply to guarantee that a given element does what it is supposed to do, not more or less just what it needs to do.

If we start transitioning from our door example in software development the same holds. Software out there needs to go through a battery of tests to make sure that the software works in different environments, different programming language versions and even different devices, Wikipedia defines software testing as:

Software testing is an investigation conducted to provide stakeholders with information about the quality of the product or service under test. Software testing can also provide an objective, independent view of the software to allow the business to appreciate and understand the risks of software implementation. Test techniques include the process of executing a program or application with the intent of finding software bugs (errors or other defects), and to verify that the software product is fit for use.

Traditional software development teams often delegate the testing part to a QA (Quality Assurance) area, the purpose is to guarantee the quality of the software before submission to the client. This model was inspired by the waterfall model of development and, more recently, this has been shifting to shift-left the testing activities and incorporate them while the application is being developed rather than later.

More broadly, the definition of software testing according to Sommerville [1] is the task of showing that software does what it is intended to do, and discovering its defects before putting the software into use. Still, according to the same author, we can divide the objective testing software in two parts:

1. Demonstrate to the client and developer that the software meets the specified requirements
2. Expose some unwanted behavior

Through these two definitions, we have an overview of what we find in the process of software testing. The first topic aims to demonstrate to the client what is being built and exactly what was previously specified. The customer in this meaning does not necessarily need to be a person, as it often becomes an entity, such as a company.

The second topic has the opposite objective, exposing undesirable behavior and even possible failures given a given set of data. Behavior This must also be explicitly stated in the software specifications, even if it is undesirable.

Verification and Validation

The task of verification and validation is a slightly more comprehensive view of the test software, but it is directly related to topic 1 of the previous section.

Verification refers to the task that ensures that the software implements correctly the specifications. Validation refers to different tasks that ensure that the software is built according to the customer’s needs (requirements) [1] [2].

According to Sommerville [1] and Pressman [2], it is necessary to have expected and non-expected behaviors in the documentation of the system in question, however, the literature shows that in addition to testing, software requirements suffer from being a subject that requires an abstraction of the system on the part of those who exercise it, which can be costly for who does not have prior experience.

This leaves a gap not just in one aspect of the software development cycle software, but in two: testing and requirements. This leads to the hypothesis that one is directly connected to the other to deliver software that meets the customer’s expectations.

Requirements dictate what must be written by the programmer, what the results are, and what are the expectations expected by the customer. Then the testing phase uses this information to check what was developed, thus guaranteeing the veracity that the software is doing what it sets out to do. The accuracy of software requirements and essential for the creation and execution of the test, once the requirement is wrong, the test generated to verify will also be wrong.

Unit testing

Unit testing is the type of test that is created and executed by the programmer at the time in which the software code is written, this type of testing is the opposite of black box testing and white box which is often carried out by a professional dedicated to this task, and that it does not write the code in question. Sommerville (2017) defines unit testing as the process of testing different components of a program, such as methods and classes.

Unit testing (from the English Unit Test) became popular through the TDD methodology (in Portuguese: Test-Driven Development) where the programmer starts writing the test code before the actual software code (JANZEN, 2005).

TDD

According to Kent Beck [3] TDD is a bit nebulous, TDD is an awareness of the gap between decision and feedback during the programming phase. Beck mentions the gap between feedback and the program, aiming at the difficulty for programmers in having a quick response at the moment they are writing the code. Often the writing process by the programmer follows three basic steps: The first is writing the code, the second is testing and the third is refactoring in case something is wrong or there is a need to change the code. Here the hypothesis is created that with the evolution of graphical interfaces and the evolution of flows for carrying out tasks in the systems, the programmer is at the forefront of executing extra steps to get to the point where your code can be tested, thus delaying feedback if the code is working as expected and directly impacting late delivery of necessary changes to the software. The TDD mantra defined by Kent Beck follows three distinct phases:

• Red - First write the test code, thinking about how it must behave
• Green - Make the test pass with the smallest change possible
• Blue - Refactor, remove any duplicate code and make small improvements to standards

The first phase of TDD (red) is the first step to be considered by the programmer when developing with unit testing, for some it results It’s strange to write the test code before the “production” code. Production code is a term used for the code that will effectively build the desired algorithm to perform a given task. The barrier here is in a paradigm shift, which perhaps for those with more experience in programming more complex. Because with the years of the program accumulated, vices are accumulated and developed, making the experience of using a new paradigm costly.

On the other hand, years of experience can help the programmer to make this change, with experience comes mastery of a programming language and consequently the task of writing tests becomes less complex, as the focus will be on writing the code test, without the need to make external queries to resolve a syntax question.

Then comes the second stage of the cycle, writing the production code that does the test passes with the smallest possible change. It is important to highlight: “the smallest change is possible”, as this step focuses exclusively on having the test written by the step previous pass. This stage is what Beck defines as the baby step.

A baby step is the act of making a small change that achieves the goal as well as a baby’s steps, which are short but make the baby walk and reach his goal.

Particularly the bay step phase is one of the most valuable techniques using TDD, changing the smallest part of the code necessary for the test to pass provides the programmer with a moment of self-reflection while programming.

Experience at this stage can also cause an undesirable effect on the cycle, as More experienced programmers tend to use a high abstraction. Even without the need for the same. This behavior is called over-engineering

The various pyramids

Test pyramid

So far in this introduction we haven’t classified or suggested any type of approach when it comes to the different types of tests. However, to tackle business needs, different types of tests are required. The test pyramid described in the book Succeeding with Agile by Mike Cohn [4] and cited by Ham Vocke on Martin’s Fowler blog, suggests the following:

• item having a solid base of unit tests, which ideally run as fast as possible and provide fast feedback
• item in the middle we have integration tests that can be slower than unit tests but provide feedback if smaller pieces are working as they should
• item and last but not least the end-to-end tests (depicted as UI Tests), also references as tests that act as if they were a user (be it a human or another system/program)

Test Trophy

Kent C Dodds in 2018 made a tweet that described another shape for testing strategy targetting javascript applications. I would narrow that to specifically frontend applications that use testing-library. In his blog post entitled “The Testing Trophy and Testing Classifications”, he elaborates on the reasons behind the need to write more integration tests. They are as follows:

1. Code that is written for web apps mostly interacts with the browser
2. Testing library supports this approach and was made with that philosophy

Since then, the testing library has also been mentioned in the technology radar and moved to adoption in May 2020, with such a move a wide range of developers started to adopt the library. As an author of different projects myself, I decided to move to the testing library in 2021 and even looked out for an easier migration from enzyme:

From 2021 I have created projects that mainly use the testing library as a tool for testing in the frontend, some of them are:

• GitHub Analytics - Web app to see the contributors, commits and tech stack used in the GitHub repository
• JSON tool - it allows you to quickly format JSON content with a click of a button
• Text tool - it provides utilities that developers need to interact with text daily

These are a few open projects that I am allowed to share, in closed-source projects, most of them used the testing library.

Diamond

Type of tests

• Black-box
• White-box
• Load testing

Additional resources

• This playlist is a walk-through TDD, starting the basics and leveling up on each video. I tried to build a video sequence that makes sense and hopefully will have a chronological timeline that progresses bit by bit, from basics to more advanced concepts.

References

1. [1]I. Sommerville, “Software Engineering 10th Edition (International Computer Science),” Essex, UK: Pearson Education, pp. 1–808, 2016.
2. [2]R. S. Pressman, Software engineering: a practitioner’s approach. Palgrave macmillan, 2005.
3. [3]Beck, “Test Driven Development: By Example.” Addison-Wesley Longman Publishing Co., Inc., 2002.
4. [4]M. Cohn, Succeeding with agile: software development using Scrum. Pearson Education, 2010.

Go back