Windows Phone Unit Tests in Visual Studio 2012 Update 2

Windows Phone Unit Tests in Visual Studio 2012 Update 2

Rate This
  • Comments 27

Unit Testing is an important part of the application development life cycle for any platform. Visual Studio users no longer need to depend on 3rd party frameworks for unit tests on Windows Phone. The Windows Phone Unit Test framework delivered in CTP2 of Visual Studio Update 2 provides a consistent experience for developers to author, run and debug tests for Windows Phone 8 platform. This article walks you through the steps to start unit testing on Windows Phone.

Prerequisites

  1. Visual Studio 2012
  2. Windows Phone SDK 8.0
  3. Visual Studio 2012 Update 2 CTP 2

Note: You need a 64 bit Windows 8 physical machine (with Hyper-V enabled) to run unit tests on Windows Phone Emulator.

One Framework – Multiple Platforms

You can now write Unit Tests for Windows Phone in C# by creating a new “Windows Phone Unit Test App” project under Visual C#/Windows Phone. Tests are executed on the target device by installing the Unit Test app on the phone.

image

Figure 1: New Windows Phone Unit Test Template

Whether you are writing classic unit tests, Windows Store unit tests, or Windows Phone unit tests – Visual Studio automatically adds all the necessary references and you are ready to start writing tests using the familiar MS Test framework. You no longer need to learn multiple frameworks to author unit tests for different platforms.

image

Figure 2: Author tests using MSTest framework

Running Tests from Visual Studio

The Phone UT framework is designed as an adapter on top of the extensible Visual Studio 2012 unit testing platform (similar to other adapters like nUnit, xUnit). With this, the whole Visual Studio Test Driven Development experience is now extended to Windows Phone. Tests are discovered and displayed in the familiar Test Explorer window. Right from within the IDE, users can group/filter tests using traits, turn on continuous test runner to view test results after each build, and debug or run selected (or failed) set of tests.

clip_image001

Figure 3: Phone Unit Tests discovered

clip_image002

Figure 4:Set Target Device

Visual Studio uses the target device set on the Run palette to install the unit test app, run tests, and return results. Supported targets include Emulators (only available when Hyper-V is enabled) and Windows Phone 8 devices physically connected to the machine. Developers can author, run, and debug tests on any of the target devices without leaving the IDE. While unit tests are running on the device, a 'Tests Running' screen will be shown.

image

Figure 5: Unit Tests running on device

Running Tests from Command Line

As I mentioned earlier, since Phone UT is designed as an adapter to unit testing platform, you can use vstest.console.exe to execute unit tests from command line. You can also choose to export the results to Visual Studio Test Results File using /Logger:trx.

image

Figure 6: Run tests from command line

image

Figure 7: Test Results from TRX logger

By default Emulator WVGA 512MB is the target device to execute tests, use run settings to change target device (Sample run settings attached).

image

Figure 8: Run Settings to set target device

FAQs and Known Issues

  • Phone Unit Test does not support data collectors either on the physical device or emulator.
  • Can I write unit tests in native C++?
    In CTP2 only C# is supported. VB# and C++ are not supported for now.
  • Which versions of Windows Phone is supported?
    You can only author unit tests for apps that target Windows Phone 8. Windows Phone 7, 7.5, 7.8 are not supported.
  • Which Visual Studio SKUs do I need to enable this feature?
    Windows Phone Unit Tests works on all editions - Express, Test Professional, Professional and Ultimate.
  • After installing all the prerequisites, I still do not see Windows Phone Unit Test template
    Workaround: Install the mentioned prerequisites in the specified order, all on the system drive
  • Device was disconnected or interrupted while unit tests were in progress, what do I do?
    Workaround: When a device is disconnected or interrupted (for example, by an incoming call or the camera button), the test run fails and vstest may crash. Manually uninstall all the unit test applications from the device, reconnect the device, and try running tests again.
  • I get an error message while debugging unit tests.
    Workaround: Manually stop the debugging session at the end of a test run and start debugging selected tests again.
  • Phone Unit Tests that have a native dependency cannot be run from command line
  • After Test Runs are complete, I sometimes still see Unit Test applications installed on device
    Workaround: Manually uninstall all the unit test applications from the device.
  • Known Issue: Users who have a non English version of Windows Phone SDK Tools 8.0 will not be able to create new Windows Phone Unit Test projects.
  • Known Issue: When a test fails, the stack trace does not contain line numbers indicating where the test failed, and users cannot navigate to code from stack trace information.
Attachment: phone-tests.runsettings
Leave a Comment
  • Please add 3 and 3 and type the answer here:
  • Post
  • Any plans to bring this to WP7? FYI, this doesn't send a very positive message...

  • What's the point in creating and deploying test app to run unit tests? Usually tests cover some business logic or algorithms and are fine to run as any other test. Can you share some scenarios where this feature may be useful?

  • Thanks for bringing this much-needed feature to Visual Studio!

    @andreycha: Theoretically you are right, but in reality just too many methods/code paths of a phone application depend in some way or another on namespaces that are only available on the phone/emulator. Except for the smallest algorithmic methods none of this code would be runnable and testable in VS directly.

  • Great post Vinay!

    Congrats on getting your first one out!

  • @friederbluemle I see. So creating of test app is just a way to run that code in proper environment?

  • @Richard Szalay: Owing to architectural constraints, this remains a WP8 only feature.

    @andreycha: Yes - ideal unit tests should mock all Phone APIs to be able to run tests in VS. In absence of that, the next best thing is to run tests on the device itself.

    @Chuck: Thanks! :)

  • Trying this out... and couldn't get the test to run (after a long pause, the test runner stops and then leave the tests in "Not Run Tests" group). So I went to try the command line instead (what's up with it saying Windows Store apps instead of Phone, btw? :)), and it gives following error (am trying on device - I don't currently have Hyper-V to use emulators):

    Starting test execution, please wait...

    Warning: Using Isolation mode to run unit tests for Windows Store apps. Use /InI

    solation switch to suppress this warning.

    Warning: Diagnostic data adapters are not supported when executing unit tests fo

    r Windows Store apps. Remove diagnostic data adapters settings from settings.

    Error: Failed to initialize client proxy: could not connect to .

    Error: The socket connection was aborted. This could be caused by an error proce

    ssing your message or a receive timeout being exceeded by the remote host, or an

    underlying network resource issue. Local socket timeout was '00:00:59.9990004'.

  • Hi Vinay!

    Can one let the windows phone unit tests run in a Team Foundation Build Server (maybe inside of vstest.console.exe)?

    Regards,

    Alexander

  • @Vinay.Krishna Any chance you could elaborate on those "architectural constraints"? Since the tests run on the phone, I assume you're just hosting an HTTP service to communicate the results. I've done this for the WP7 (wp7ci on Codeplex) so I know it's possible, albeit without the VS integration.

  • @Wiyono: Since you are running tests on a processor without SLAT capability, the Target defaults to Device. Can you set Platform to ARM in Build -> Configuration Manager and give it a try? Also, while running tests from CLI, provide the arm xap and test settings with Target set to device. (I'm assuming your device is already unlocked with a developer license). And we are fixing up the names in messages.

    @Alexander Soutchilin: You cannot run Phone Unit Tests as part of Team Build yet - we have this in our backlog. However when you execute tests from CLI, you can use /logger:TfsPublisher to publish results to Build Summary Page.

    @Richard Szalay: More than the communication channel, it's the .NET CLR for Windows Phone 8 that made it possible for us to port our existing unit testing solution to WP8 platform.

  • Thanks Vinay, turns out I overlooked to set Platform to ARM (or use the ARM xap for CLI) instead, indeed. Nice one! (although perhaps, error experience could've been more helpful...) Now that it works on device I have found it to be more reliable also it seems (I have since tried it on emulators on another machine and it seems flaky that way in that sometimes not all tests are run and I have to manually select and run each of those, despite having selected Run All - note they are not skipped). But thanks, I have a better workflow now that this works!

  • Regarding the known issue : "Phone Unit Tests that have a native dependency cannot be run from command line". Will this issue be fixed when update 2 is released?

  • Does the unit test framework provide a way to collection code coverage numbers?

  • @RobN75- Yes, referencing native dependency has been addressed in upcoming release.

    @Julius Ang- No, WP8 Unit Test framework doesn't support code coverage collection.

  • Are UITestMethodAttribute and async Assert.ThrowsException coming to this WP8 test framework too? Perhaps CTP3 (please)? :)

Page 1 of 2 (27 items) 12