Progressive Development

Zany Adventures in Software Engineering with Maven and Motley

Motley says: "Test both private and public methods"

Motley says: "Test both private and public methods"

  • Comments 6

Summary

 

Motley: To get high code coverage, we should be testing both public and private methods

 

Maven: Only test public methods. Testing private methods gets in the way of refactoring

______________________________

 

[Context:  Maven checks in on Motley's unit testing practices and notices something odd]

 

Maven: Hey Mot - how is the unit testing thing going?

 

Motley: Pretty good, I must say. I'm leveraging Test-Driven Development (TDD) as much as possible and it has really improved my code coverage. Check out all these tests!

 

Maven: Wow - that is pretty good! In fact, that's more tests than I was expecting. Are you literally testing everything?

 

Motley: You betcha. I have test cases around every method I write.

 

Maven: Even the private methods?

 

Motley: Duh. Of course! I gotta get that 100% code coverage, man! That was your idea don't forget.

 

Maven: Okay, I am willing to bet you are doing TDD without the refactoring part of the process. Am I right?

 

Motley: Well, um, I guess. I'm trying to do a bit but it's such a pain in the butt. But wait, how can you tell just by looking at my tests???

 

Maven: Well, if you are testing private methods that typically leads to a slightly larger number of tests, because a good practice is one test (or more) per method that you are testing. If you just test public methods, you can typically get the same code coverage with a slightly lower number of tests.

 

Motley: So what? So I have a few more tests. It doesn't harm anything.

 

Maven: Ah, but it does. Testing private methods makes it really hard to refactor. You see, refactoring  does not usually affect a public interface but instead the implementation behind that interface, i.e. the private methods. With TDD, you need to be free to refactor, or you defeat one of its primary purposes - to drive a better design. If you need to update tests every time you make a refactoring change to your implementation because you are testing private methods directly, it really gets in the way and you are not going to do it. I am willing to bet that's why you are skipping an extremely important part of the TDD process.

 

Motley: The private methods do get in the way of refactoring, but code coverage is everything!

 

Maven: With TDD you don't write a line of code without having a test in place first. When you do refactoring, you rerun your tests to ensure you haven't broken anything. You should theoretically end up with the same coverage even just testing via public methods. If you can't hit your private methods through your public methods, why are they even there? Besides, the code coverage number in itself is not super interesting. What is more interesting is the analysis of why coverage numbers are low. This can lead you to more tests that you may otherwise miss. Don't go overboard trying to get to 100% - it's not worth it. Some C# language constructs, like foreach, have extra compiler-generated code in the intermediate language, which makes it real tough to get 100%. It's not worth the effort. Oh, and by the way, when I say "public", I typically also mean protected methods (public for subclasses) and even internal methods (public within the assembly).

 

Motley: So you're saying I should just test public methods and code coverage will just come if I use TDD. You're also saying that getting to that magical 100% code coverage number is often not worth the extra effort, and although code coverage is a good measure of your tests, the analysis is more important.

 

Maven: You should write a book, Mot. I could not have put it better myself.

 

Motley: I'm a quick learner, as you can see from my stellar deliverables. I can even teach you a few things, like how to be likeable in a social atmosphere.

 

Maven: What are you saying?!?

______________________________

 

Maven's Pointer: You may ask, "How do I test private methods anyway? They are private!". In .NET, there are several ways to test private methods should you have a rare case where you definitely need to test a private:

  • Leverage a test framework that uses reflection to call methods. The System.Reflection APIs can access private members of a class
  • Use Visual Studio 2005 (and later) for unit testing. It will generate a test context object when you use the wizard to create a test that lets you access privates.
  • Create a friend assembly using the InternalsVisibleTo attribute that lets you access internal methods at least.
  • Create/Run a tool that flips some bits in the .NET assembly metadata to change the scope of classes and methods all to public (you are not testing the original compiled assembly in this case though).
  • Create/Run a tool that generates a proxy for your class and accesses members via reflection. Your tests then use the proxy instead of the original class.
  • Use preprocessor definitions to change the scope of the private methods to public in a special build.

 

Maven's Resources:  How to Test Private and Protected Methods in .NET provides similar arguments for/against testing private methods, and talks about a subset of the above list for testing private methods.

  • Great article!

    I've had some situations where I've wanted to unit test private methods. I used the following approach.

    I mark the class to be tested as "partial". I create a directory in the project called "Test" or similar. Here I create another partial class which is named as the previous class. The source file is decorated with #if TEST #endif, so that the whole file is ignored when the TEST-symbol is undefined. Then the partial in the Test-directory is provided with the appropriate attributes (for example Test from NUnit, etc). Any default constructors not defined in the real sourcefile can also be defined in the test-file. Example:

    File 1 (main implementation):

    partial class ToBeTested {

      // ...

    }

    File 2 (unit tests):

    #if TEST

    [TestFixture]

    partial class ToBeTested {

      // Add default construction if needed

      [Test]

      public void TestBlah() {

         // Access privates in ToBeTested defined in other file

      }

    }

    #endif

    This gives you full access to all internals of the class you're testing. Problems with the approach include:

     - The public interface of a class is changed when compiling with the TEST-symbol. For me this problem is minimal as I like to prefix my tests, but unintended default construction can be tricky to notice and error prone. So compile every so often with the TEST-symbol off.

     - Last time I checked it didn't work with generic classes. It might have been just me.

    The approach isn't ideal, but it's fairly simple to do. And as you say, normally you probably don't need to do this.

  • Thanks for the tip Lars! That's an interesting use of partial classes, and definitely a technique to add to the list. I never liked dealing with a multitude of #define's, but this is a fairly clean way to separate test code from production at build time.

    But again, as you said, hopefully we can avoid all this and not test the privates, but a useful technique if the scenario arises. Thanks for the addition!

  • I agree with Maven that 100% code coverage isn't practical but during development we need to keep an eye on coverage.  Some code paths could open up a whole lot of not covered code.  Watching the code coverage during development will expose these areas.

    It's always painfull to go back later in the project cycle to add tests because the coverage is too low.

  • Definitely do keep an eye on coverage, and larger numbers are better. Just don't spend that extra time trying to get from 98% to that magical 100% number when you are already sufficiently covering your code with tests. A good team will set a threshold for the coverage number that must be hit prior to check-in. Start with something like 60% or 70% and move up from there. If your team does TDD, start with 80% and increase as the team gets more comfortable with unit testing and coverage metrics.

  • I often wonder to myself, if my private methods are getting hairy enough to need testing and coverage, perhaps they should be extracted and promoted to a public interface of their own.

    Haven't gone there yet but the idea is always lurking at the back of my mind.

    --

    Jason

  • Private methods definitely need testing, but they should be testable via your public methods. If you cannot test the private methods through the public methods, why are they there? If it is difficult to test the private methods due to some weird configuration or expression combinations, and a public interface makes sense for clients, then yes, refactor it to public. Be careful with blindly making APIs public though - you increase your security attack surface area and complicate your object model. It has to make sense for the end user (developer).

Page 1 of 1 (6 items)
Leave a Comment
  • Please add 5 and 5 and type the answer here:
  • Post