[testing] Should black box or white box testing be the emphasis for testers?

Which type of testing would you say should be the emphasis (for testers/QAs), and why?

A quick set of definitions from wikipedia:

Black box testing

• takes an external perspective of the test object to derive test cases. These tests can be functional or non-functional, though usually functional. The test designer selects valid and invalid input and determines the correct output. There is no knowledge of the test object's internal structure.

White box testing

• uses an internal perspective of the system to design test cases based on internal structure. It requires programming skills to identify all paths through the software. The tester chooses test case inputs to exercise paths through the code and determines the appropriate outputs. In electrical hardware testing, every node in a circuit may be probed and measured; an example is in-circuit testing (ICT).

Just to clarify a bit more, I realize that both are important, but, usually, they are separate between dev and QA.

Is internal knowledge important for the tester/QA? I've heard arguments that testing with this knowledge in mind enables them to better test for problems, but I've also heard arguments that this knowledge can distract from functional needs and promote "testing to the code" rather than to the intended solution.

What constitutes, "internal knowledge?" Does knowing that such-and-such algorithm was used to solve a problem qualify or does the tester have to see every line of code for it to be "internal?"

I think in any test case, there should be expected results given by the specification used and not determined by how the tester decides to interpret the specification as this can lead to issues where each thinks they are right and blaming the other for the problem.

In my experience most developers naturally migrate towards white box testing. Since we need to ensure that the underlying algorithm is "correct", we tend to focus more on the internals. But, as has been pointed out, both white and black box testing is important.

Therefore, I prefer to have testers focus more on the Black Box tests, to cover for the fact that most developers don't really do it, and frequently aren't very good at it.

That isn't to say that testers should be kept in the dark about how the system works, just that I prefer them to focus more on the problem domain and how actual users interact with the system, not whether the function SomeMethod(int x) will correctly throw an exception if x is equal to 5.

Black Box

1 Focuses on the functionality of the system Focuses on the structure (Program) of the system

2 Techniques used are :

· Equivalence partitioning

· Boundary-value analysis

· Error guessing

· Race conditions

· Cause-effect graphing

· Syntax testing

· State transition testing

· Graph matrix

Tester can be non technical

Helps to identify the vagueness and contradiction in functional specifications

White Box

Techniques used are:

· Basis Path Testing

· Flow Graph Notation

· Control Structure Testing

1. Condition Testing

2. Data Flow testing

· Loop Testing

1. Simple Loops

2. Nested Loops

3. Concatenated Loops

4. Unstructured Loops

   Tester should be technical

   Helps to identify the logical and coding issues.

Black Box Testing: Black Box testing is just observation no need Internal Knowledge or structure of software product. just putting valid and Invalid data input and expecting the correct result. here tester find the defect but unable to Find the Location of defect.black box testing done in all testing level.

Black box testing tecniques are: 1. Equivalance partition 2. Boundary Value Analysis 3. Decision table 4. State Transition Diagram 4. Use case diagram

White Box Testing: White box is testing it requires the knowledge of internal logic and structure of software product. here we will check the loop, condition and branch. here we find not only the defect but also and location of defect.

White box Testing Techniques: 1. Statement Coverage 2. Decision Coverage 3. Branch Coverage 4. Path Coverage.

In my experience most developers naturally migrate towards white box testing. Since we need to ensure that the underlying algorithm is "correct", we tend to focus more on the internals. But, as has been pointed out, both white and black box testing is important.

Therefore, I prefer to have testers focus more on the Black Box tests, to cover for the fact that most developers don't really do it, and frequently aren't very good at it.

That isn't to say that testers should be kept in the dark about how the system works, just that I prefer them to focus more on the problem domain and how actual users interact with the system, not whether the function SomeMethod(int x) will correctly throw an exception if x is equal to 5.

White Box Testing equals Software Unit Test. The developer or a development level tester (e.g. another developer) ensures that the code he has written is working properly according to the detailed level requirements before integrating it in the system.

Black Box Testing equals Integration Testing. The tester ensures that the system works according to the requirements on a functional level.

Both test approaches are equally important in my opinion.

A thorough unit test will catch defects in the development stage and not after the software has been integrated into the system. A system level black box test will ensure all software modules behave correctly when integrated together. A unit test in the development stage would not catch these defects since modules are usually developed independent from each other.

• *Black box testing: Is the test at system level to check the functionality of the system, to ensure that the system performs all the functions that it was designed for, Its mainly to uncover defects found at the user point. Its better to hire a professional tester to black box your system, 'coz the developer usually tests with a perspective that the codes he had written is good and meets the functional requirements of the clients so he could miss out a lot of things (I don't mean to offend anybody)
• Whitebox is the first test that is done in the SDLC.This is to uncover bugs like runtime errors and compilation errrors It can be done either by testers or by Developer himself, But I think its always better that the person who wrote the code tests it.He understands them more than another person.*

"Both" has been stated above, and is the obvious answer...but IMO, white box testing goes far beyond developer unit testing (althoughI suppose it could depend on where you draw the line between white and black). For example, code coverage analysis is a common white box approach - i.e. run some scenarios or tests, and examine the results looking for holes in testing. Even if unit tests have 100% cc, measuring cc on common user scenarios can reveal code that may potentially need even more testing.

Another place where white box testing helps is examining data types, constants and other information to look for boundaries, special values, etc. For example, if an application has an input that takes a numeric input, a bb only approach could require the tester to "guess" at what values would be good for testing, whereas a wb approach may reveal that all values between 1-256 are treated one way, while larger values are treated another way...and perhaps the number 42 has yet another code path.

So, to answer the original question - both bb and wb are essential for good testing.

It's a bit of an open door, but in the end both are about equally important.

What's worse?

1. software that does what it needs to do, but internally has problems?

2. software that is supposed to work if you look at the sources, but doesn't?

My answer: Neither is totally acceptable, but software cannot be proven to be 100% bugfree. So you're going to have to make some trade-offs. Option two is more directly noticable to clients, so you're going to get problems with that sooner. On the long run, option one is going to be problematic.

QA should focus on Black box testing. The main goal of QA is to test what the system does (do features meet requirements ?), not how it does it.

Anyway it should be hard for QA to do white box testing as most of QA guys aren't tech guys, so they usually test features through the UI (like users).

A step further, I think developpers too should focus on Black box testing. I disagree with this widespread association between Unit testing and White box testing but it may be just a question a vocabulary/scale. At the scale of a Unit test, the System Under Test is a class/method which has contract (through its signature) and the important point is to test what it does, not how. Moreover White box testing implies you know how the method will fill its contract, that seems incompatile with TDD to me.

IMHO if your SUT is so complex that you need to do white box testing, it's usually time for refactoring.

• *Black box testing: Is the test at system level to check the functionality of the system, to ensure that the system performs all the functions that it was designed for, Its mainly to uncover defects found at the user point. Its better to hire a professional tester to black box your system, 'coz the developer usually tests with a perspective that the codes he had written is good and meets the functional requirements of the clients so he could miss out a lot of things (I don't mean to offend anybody)
• Whitebox is the first test that is done in the SDLC.This is to uncover bugs like runtime errors and compilation errrors It can be done either by testers or by Developer himself, But I think its always better that the person who wrote the code tests it.He understands them more than another person.*

• Black box testing you dont see the system under test inner workings.
• White box testing you have full view into the system under test. Here are a few pictures showing this.

Testers need to focus on the testing pyramid. You will want to understand unit tests, integration tests, and end to end tests. Each of these can be performed both with black box and white box testing. Start small and work your way up learning the different types and when to use each. remember you cant always test everything.

It's a bit of an open door, but in the end both are about equally important.

What's worse?

1. software that does what it needs to do, but internally has problems?

2. software that is supposed to work if you look at the sources, but doesn't?

My answer: Neither is totally acceptable, but software cannot be proven to be 100% bugfree. So you're going to have to make some trade-offs. Option two is more directly noticable to clients, so you're going to get problems with that sooner. On the long run, option one is going to be problematic.

Here is my 5 cents:

As a developer, I mostly write tests for methods (in a class) as white box tests, simple because I do not want my test to break just because I change the inner works of my code.

I only want to my tests to break if my method behavior changes (e.g. returns a different result than before).

Over the last 20 years of development, I simple got tired of doing up-to double work just because my unit tests was strongly tied to the code and I need to maintain both application code and test code.

I think making decoupling code (also when you code tests) is a very good practice.

Another 5 cents: I hardly never use mocking frameworks, because when I find it necessarily to mock something I prefer to decouple my code instead - and yes in many cases that is very possible (especially if you are not working in legacy code) :-)

White Box Testing equals Software Unit Test. The developer or a development level tester (e.g. another developer) ensures that the code he has written is working properly according to the detailed level requirements before integrating it in the system.

Black Box Testing equals Integration Testing. The tester ensures that the system works according to the requirements on a functional level.

Both test approaches are equally important in my opinion.

A thorough unit test will catch defects in the development stage and not after the software has been integrated into the system. A system level black box test will ensure all software modules behave correctly when integrated together. A unit test in the development stage would not catch these defects since modules are usually developed independent from each other.

• Usually the white-box testing is not possible for testers. Thus the only viable answer for testers is to emphasize black-box approach.

• However, with aspect-oriented-programming and design-by-contract methodology, when the testing goals are programmed into the target code as contracts (seen from the static view of a program), and/or when the testing temporal logic is programmed into the code as cross-cuts (dynamic view of the test logic), white-box testing would become not only possible but also a preferred take for testers. Given that said, it will need be an expertise-demanding take, the testers need to be not only good testers, but also good programmers or more than good programmers.

Black Box

1 Focuses on the functionality of the system Focuses on the structure (Program) of the system

2 Techniques used are :

· Equivalence partitioning

· Boundary-value analysis

· Error guessing

· Race conditions

· Cause-effect graphing

· Syntax testing

· State transition testing

· Graph matrix

Tester can be non technical

Helps to identify the vagueness and contradiction in functional specifications

White Box

Techniques used are:

· Basis Path Testing

· Flow Graph Notation

· Control Structure Testing

1. Condition Testing

2. Data Flow testing

· Loop Testing

1. Simple Loops

2. Nested Loops

3. Concatenated Loops

4. Unstructured Loops

   Tester should be technical

   Helps to identify the logical and coding issues.

*Black-Box testing: If the source code is not available then test data is based on the function of the software without regard to how it was implemented. -strong textExamples of black-box testing are: boundary value testing and equivalence partitioning.

*White-Box testing: If the source code of the system under test is available then the test data is based on the structure of this source code. -Examples of white-box testing are: path testing and data flow testing.

Black Box Testing: Black Box testing is just observation no need Internal Knowledge or structure of software product. just putting valid and Invalid data input and expecting the correct result. here tester find the defect but unable to Find the Location of defect.black box testing done in all testing level.

Black box testing tecniques are: 1. Equivalance partition 2. Boundary Value Analysis 3. Decision table 4. State Transition Diagram 4. Use case diagram

White Box Testing: White box is testing it requires the knowledge of internal logic and structure of software product. here we will check the loop, condition and branch. here we find not only the defect but also and location of defect.

White box Testing Techniques: 1. Statement Coverage 2. Decision Coverage 3. Branch Coverage 4. Path Coverage.

Black Box Testing is a software testing method in which the internal structure/ design/ implementation of the item being tested is NOT known to the tester. White Box Testing is a software testing method in which the internal structure/ design/ implementation of the item being tested is known to the tester.

What constitutes, "internal knowledge?" Does knowing that such-and-such algorithm was used to solve a problem qualify or does the tester have to see every line of code for it to be "internal?"

I think in any test case, there should be expected results given by the specification used and not determined by how the tester decides to interpret the specification as this can lead to issues where each thinks they are right and blaming the other for the problem.

Here is my 5 cents:

As a developer, I mostly write tests for methods (in a class) as white box tests, simple because I do not want my test to break just because I change the inner works of my code.

I only want to my tests to break if my method behavior changes (e.g. returns a different result than before).

Over the last 20 years of development, I simple got tired of doing up-to double work just because my unit tests was strongly tied to the code and I need to maintain both application code and test code.

I think making decoupling code (also when you code tests) is a very good practice.

Another 5 cents: I hardly never use mocking frameworks, because when I find it necessarily to mock something I prefer to decouple my code instead - and yes in many cases that is very possible (especially if you are not working in legacy code) :-)

QA should focus on Black box testing. The main goal of QA is to test what the system does (do features meet requirements ?), not how it does it.

Anyway it should be hard for QA to do white box testing as most of QA guys aren't tech guys, so they usually test features through the UI (like users).

A step further, I think developpers too should focus on Black box testing. I disagree with this widespread association between Unit testing and White box testing but it may be just a question a vocabulary/scale. At the scale of a Unit test, the System Under Test is a class/method which has contract (through its signature) and the important point is to test what it does, not how. Moreover White box testing implies you know how the method will fill its contract, that seems incompatile with TDD to me.

IMHO if your SUT is so complex that you need to do white box testing, it's usually time for refactoring.

What constitutes, "internal knowledge?" Does knowing that such-and-such algorithm was used to solve a problem qualify or does the tester have to see every line of code for it to be "internal?"

I think in any test case, there should be expected results given by the specification used and not determined by how the tester decides to interpret the specification as this can lead to issues where each thinks they are right and blaming the other for the problem.

It's a bit of an open door, but in the end both are about equally important.

What's worse?

1. software that does what it needs to do, but internally has problems?

2. software that is supposed to work if you look at the sources, but doesn't?

My answer: Neither is totally acceptable, but software cannot be proven to be 100% bugfree. So you're going to have to make some trade-offs. Option two is more directly noticable to clients, so you're going to get problems with that sooner. On the long run, option one is going to be problematic.

• *Black box testing: Is the test at system level to check the functionality of the system, to ensure that the system performs all the functions that it was designed for, Its mainly to uncover defects found at the user point. Its better to hire a professional tester to black box your system, 'coz the developer usually tests with a perspective that the codes he had written is good and meets the functional requirements of the clients so he could miss out a lot of things (I don't mean to offend anybody)
• Whitebox is the first test that is done in the SDLC.This is to uncover bugs like runtime errors and compilation errrors It can be done either by testers or by Developer himself, But I think its always better that the person who wrote the code tests it.He understands them more than another person.*

I only partially agree with the top rated answer for this question. Which type of testing would you say should be the emphasis (for testers/QAs), and why?

1. I agree that: "Black box testing should be the emphasis for testers/QA."
2. I agree that White box testing should be the emphasis for developers, but I don't agree that White Box testing is just unit tests.

I agree with the definition here which states that White Box Testing method is applicable to the following levels of software testing:

• Unit Testing: For testing paths within a unit
• Integration Testing:For testing paths between units
• System Testing: For testing paths between subsystems

*Black-Box testing: If the source code is not available then test data is based on the function of the software without regard to how it was implemented. -strong textExamples of black-box testing are: boundary value testing and equivalence partitioning.

*White-Box testing: If the source code of the system under test is available then the test data is based on the structure of this source code. -Examples of white-box testing are: path testing and data flow testing.

Simple...Blackbox testing is otherwise known as Integration testing or smoke-screen testing . This is mostly applied in a distributed environment which rely on event-driven architecture. You test a service based on another service to see all possible scenarios. Here you cannot completely forecast all possible output because each component of the SOA/Enterprise app are meant to function autonomously. This can be referred to as High-Level testing

while

White box testing refers to unit-testing. where all expected scenarios and output can be effectively forecasted. i.e Input and expected output.This can be referred to as Low-level testing

I only partially agree with the top rated answer for this question. Which type of testing would you say should be the emphasis (for testers/QAs), and why?

1. I agree that: "Black box testing should be the emphasis for testers/QA."
2. I agree that White box testing should be the emphasis for developers, but I don't agree that White Box testing is just unit tests.

I agree with the definition here which states that White Box Testing method is applicable to the following levels of software testing:

• Unit Testing: For testing paths within a unit
• Integration Testing:For testing paths between units
• System Testing: For testing paths between subsystems

Black Box Testing is a software testing method in which the internal structure/ design/ implementation of the item being tested is NOT known to the tester. White Box Testing is a software testing method in which the internal structure/ design/ implementation of the item being tested is known to the tester.

White Box Testing equals Software Unit Test. The developer or a development level tester (e.g. another developer) ensures that the code he has written is working properly according to the detailed level requirements before integrating it in the system.

Black Box Testing equals Integration Testing. The tester ensures that the system works according to the requirements on a functional level.

Both test approaches are equally important in my opinion.

A thorough unit test will catch defects in the development stage and not after the software has been integrated into the system. A system level black box test will ensure all software modules behave correctly when integrated together. A unit test in the development stage would not catch these defects since modules are usually developed independent from each other.

Simple...Blackbox testing is otherwise known as Integration testing or smoke-screen testing . This is mostly applied in a distributed environment which rely on event-driven architecture. You test a service based on another service to see all possible scenarios. Here you cannot completely forecast all possible output because each component of the SOA/Enterprise app are meant to function autonomously. This can be referred to as High-Level testing

while

White box testing refers to unit-testing. where all expected scenarios and output can be effectively forecasted. i.e Input and expected output.This can be referred to as Low-level testing

• Black box testing you dont see the system under test inner workings.
• White box testing you have full view into the system under test. Here are a few pictures showing this.

Testers need to focus on the testing pyramid. You will want to understand unit tests, integration tests, and end to end tests. Each of these can be performed both with black box and white box testing. Start small and work your way up learning the different types and when to use each. remember you cant always test everything.

"Both" has been stated above, and is the obvious answer...but IMO, white box testing goes far beyond developer unit testing (althoughI suppose it could depend on where you draw the line between white and black). For example, code coverage analysis is a common white box approach - i.e. run some scenarios or tests, and examine the results looking for holes in testing. Even if unit tests have 100% cc, measuring cc on common user scenarios can reveal code that may potentially need even more testing.

Another place where white box testing helps is examining data types, constants and other information to look for boundaries, special values, etc. For example, if an application has an input that takes a numeric input, a bb only approach could require the tester to "guess" at what values would be good for testing, whereas a wb approach may reveal that all values between 1-256 are treated one way, while larger values are treated another way...and perhaps the number 42 has yet another code path.

So, to answer the original question - both bb and wb are essential for good testing.

• Usually the white-box testing is not possible for testers. Thus the only viable answer for testers is to emphasize black-box approach.

• However, with aspect-oriented-programming and design-by-contract methodology, when the testing goals are programmed into the target code as contracts (seen from the static view of a program), and/or when the testing temporal logic is programmed into the code as cross-cuts (dynamic view of the test logic), white-box testing would become not only possible but also a preferred take for testers. Given that said, it will need be an expertise-demanding take, the testers need to be not only good testers, but also good programmers or more than good programmers.

• *Black box testing: Is the test at system level to check the functionality of the system, to ensure that the system performs all the functions that it was designed for, Its mainly to uncover defects found at the user point. Its better to hire a professional tester to black box your system, 'coz the developer usually tests with a perspective that the codes he had written is good and meets the functional requirements of the clients so he could miss out a lot of things (I don't mean to offend anybody)
• Whitebox is the first test that is done in the SDLC.This is to uncover bugs like runtime errors and compilation errrors It can be done either by testers or by Developer himself, But I think its always better that the person who wrote the code tests it.He understands them more than another person.*

It's a bit of an open door, but in the end both are about equally important.

What's worse?

1. software that does what it needs to do, but internally has problems?

2. software that is supposed to work if you look at the sources, but doesn't?

My answer: Neither is totally acceptable, but software cannot be proven to be 100% bugfree. So you're going to have to make some trade-offs. Option two is more directly noticable to clients, so you're going to get problems with that sooner. On the long run, option one is going to be problematic.

In my experience most developers naturally migrate towards white box testing. Since we need to ensure that the underlying algorithm is "correct", we tend to focus more on the internals. But, as has been pointed out, both white and black box testing is important.

Therefore, I prefer to have testers focus more on the Black Box tests, to cover for the fact that most developers don't really do it, and frequently aren't very good at it.

That isn't to say that testers should be kept in the dark about how the system works, just that I prefer them to focus more on the problem domain and how actual users interact with the system, not whether the function SomeMethod(int x) will correctly throw an exception if x is equal to 5.