Istqb Certified Tester Foundation Level CTFL · Free Practice Question Medium
Question 25
You're a QA engineer testing a new mobile game. For the past few weeks, you've been diligently running a set of automated tests that cover core gameplay mechanics. These tests have successfully identified several bugs related to character movement, item interaction, and level progression. However, lately, the tests haven't uncovered any new defects despite repeated runs. You suspect there might be additional bugs lurking, but the development team is confident that the game is stable based on the passing tests. You believe it's time to revisit the test strategy.
Which of the following principles explains your concern?
-
A
Testing shows the presence, not the absence, of defects.
-
B
Exhaustive testing is impossible.
-
C
Tests wear out
-
D
Defects cluster together.
Reveal correct answer
Correct answer: C
Explanation
As per syllabus page 17 & 18:
1.3.Testing Principles
A number of testing principles offering general guidelines applicable to all testing have been suggested over the years. This syllabus describes seven such principles.
1. Testing shows the presence, not the absence of defects. Testing can show that defects are present in the test object, but cannot prove that there are no defects. Testing reduces the probability of defects remaining undiscovered in the test object, but even if no defects are found, testing cannot prove test object correctness.
2. Exhaustive testing is impossible. Testing everything is not feasible except in trivial cases. Rather than attempting to test exhaustively, test techniques, test case prioritization, and risk-based testing, should be used to focus test efforts.
3. Early testing saves time and money. Defects that are removed early in the process will not cause subsequent defects in derived work products. The cost of quality will be reduced since fewer failures will occur later in the SDLC. To find defects early, both static testing and dynamic testing should be started as early as possible.
4. Defects cluster together. A small number of system components usually contain most of the defects discovered or are responsible for most of the operational failures. This phenomenon is an illustration of the Pareto principle. Predicted defect clusters, and actual defect clusters observed during testing or in operation, are an important input for risk-based testing.
5. Tests wear out. If the same tests are repeated many times, they become increasingly ineffective in detecting new defects. To overcome this effect, existing tests and test data may need to be modified, and new tests may need to be written. However, in some cases, repeating the same tests can have a beneficial outcome, e.g., in automated regression testing.
6. Testing is context dependent. There is no single universally applicable approach to testing. Testing is done differently in different contexts.
7. Absence-of-defects fallacy. It is a fallacy (i.e., a misconception) to expect that software verification will ensure the success of a system. Thoroughly testing all the specified requirements and fixing all the defects found could still produce a system that does not fulfill the users’ needs and expectations, that does not help in achieving the customer’s business goals, and that is inferior compared to other competing systems. In addition to verification, validation should also be carried out.
A.
While this principle is generally true, it doesn't directly address your specific concern. You've already found defects, but you're worried the current tests might be missing some.
B.
This principle emphasizes the impracticality of testing everything. While relevant for test design, it doesn't explain why the existing tests might be less effective over time.
C.
This principle perfectly explains your concern. Repeatedly running the same tests can make them less effective at finding new defects. Your suspicion that additional bugs exist despite passing tests aligns with this principle.
D.
In the scenario, you've already identified defects related to core gameplay mechanics, which could be indicative of clustering in that area. However, your concern isn't about identifying the location of defects (which might already be happening). You suspect the existing tests, designed to cover those core mechanics, are becoming ineffective at finding new issues within the same area.
Discussion
Think the marked answer is wrong, or have a better explanation? Share it below — comments appear after review.
