Windows Phone Unit Tests in Visual Studio 2012 Update 2

Windows Phone Unit Tests in Visual Studio 2012 Update 2

Rate This
  • Comments 27

Unit Testing is an important part of the application development life cycle for any platform. Visual Studio users no longer need to depend on 3rd party frameworks for unit tests on Windows Phone. The Windows Phone Unit Test framework delivered in CTP2 of Visual Studio Update 2 provides a consistent experience for developers to author, run and debug tests for Windows Phone 8 platform. This article walks you through the steps to start unit testing on Windows Phone.

Prerequisites

  1. Visual Studio 2012
  2. Windows Phone SDK 8.0
  3. Visual Studio 2012 Update 2 CTP 2

Note: You need a 64 bit Windows 8 physical machine (with Hyper-V enabled) to run unit tests on Windows Phone Emulator.

One Framework – Multiple Platforms

You can now write Unit Tests for Windows Phone in C# by creating a new “Windows Phone Unit Test App” project under Visual C#/Windows Phone. Tests are executed on the target device by installing the Unit Test app on the phone.

image

Figure 1: New Windows Phone Unit Test Template

Whether you are writing classic unit tests, Windows Store unit tests, or Windows Phone unit tests – Visual Studio automatically adds all the necessary references and you are ready to start writing tests using the familiar MS Test framework. You no longer need to learn multiple frameworks to author unit tests for different platforms.

image

Figure 2: Author tests using MSTest framework

Running Tests from Visual Studio

The Phone UT framework is designed as an adapter on top of the extensible Visual Studio 2012 unit testing platform (similar to other adapters like nUnit, xUnit). With this, the whole Visual Studio Test Driven Development experience is now extended to Windows Phone. Tests are discovered and displayed in the familiar Test Explorer window. Right from within the IDE, users can group/filter tests using traits, turn on continuous test runner to view test results after each build, and debug or run selected (or failed) set of tests.

clip_image001

Figure 3: Phone Unit Tests discovered

clip_image002

Figure 4:Set Target Device

Visual Studio uses the target device set on the Run palette to install the unit test app, run tests, and return results. Supported targets include Emulators (only available when Hyper-V is enabled) and Windows Phone 8 devices physically connected to the machine. Developers can author, run, and debug tests on any of the target devices without leaving the IDE. While unit tests are running on the device, a 'Tests Running' screen will be shown.

image

Figure 5: Unit Tests running on device

Running Tests from Command Line

As I mentioned earlier, since Phone UT is designed as an adapter to unit testing platform, you can use vstest.console.exe to execute unit tests from command line. You can also choose to export the results to Visual Studio Test Results File using /Logger:trx.

image

Figure 6: Run tests from command line

image

Figure 7: Test Results from TRX logger

By default Emulator WVGA 512MB is the target device to execute tests, use run settings to change target device (Sample run settings attached).

image

Figure 8: Run Settings to set target device

FAQs and Known Issues

  • Phone Unit Test does not support data collectors either on the physical device or emulator.
  • Can I write unit tests in native C++?
    In CTP2 only C# is supported. VB# and C++ are not supported for now.
  • Which versions of Windows Phone is supported?
    You can only author unit tests for apps that target Windows Phone 8. Windows Phone 7, 7.5, 7.8 are not supported.
  • Which Visual Studio SKUs do I need to enable this feature?
    Windows Phone Unit Tests works on all editions - Express, Test Professional, Professional and Ultimate.
  • After installing all the prerequisites, I still do not see Windows Phone Unit Test template
    Workaround: Install the mentioned prerequisites in the specified order, all on the system drive
  • Device was disconnected or interrupted while unit tests were in progress, what do I do?
    Workaround: When a device is disconnected or interrupted (for example, by an incoming call or the camera button), the test run fails and vstest may crash. Manually uninstall all the unit test applications from the device, reconnect the device, and try running tests again.
  • I get an error message while debugging unit tests.
    Workaround: Manually stop the debugging session at the end of a test run and start debugging selected tests again.
  • Phone Unit Tests that have a native dependency cannot be run from command line
  • After Test Runs are complete, I sometimes still see Unit Test applications installed on device
    Workaround: Manually uninstall all the unit test applications from the device.
  • Known Issue: Users who have a non English version of Windows Phone SDK Tools 8.0 will not be able to create new Windows Phone Unit Test projects.
  • Known Issue: When a test fails, the stack trace does not contain line numbers indicating where the test failed, and users cannot navigate to code from stack trace information.
Attachment: phone-tests.runsettings
Leave a Comment
  • Please add 2 and 2 and type the answer here:
  • Post
  • Awesome.

    But please explain why I would run my unit tests on the phone? It looks like I would use this for integration tests, in which I would prefer to run as much of my code as possible on an environment as close to the final device as possible. In contrast, in a unit test I would mock out any dependencies that I cannot control and test smaller units. With this framework I cannot use Rhino Mocks and probably also not SpecFlow.

    Please elaborate on the intended scope of use for this framework!

  • @Brian Elgaard - Yes - ideal unit tests should mock all Phone APIs to be able to run tests in VS. In absence of that, the next best thing is to run tests on the device itself. Please read earlier comments from andreycha and friederbluemle.

    @Wiyono - Thanks for your excitement. Unfortunately both UITestMethod and async Assert.ThrowsException will not be available in Update 2. Please vote for this idea on visualstudio.uservoice.com to help us prioritize our future backlog.

  • Thanks. Voted up (with maximum count) :). Meanwhile, I just came across another issue now. Knowing that there is a limitation currently with native dependency (as you mentioned, although I actually found not just on command line but also from the test explorer), I have created AnyCPU configuration that is built without native dependencies. However I now found that the framework doesn't seem to like it? It just hangs waiting for something it seems - I could see the test app deployed fine to the emulator and the app opens briefly and closes, and the test still keep waiting indefinitely. Any idea?

  • This is kind of useless if I cannot use Fakes. Any plan to support Fakes ?

  • I guess implementing your own logger will be possible in the future versions? Howabout unlocking the device automatically so that I can have fully unattended test execution in CI?

    I would like to get this used in CI but the two above features are a must before we can do it.

  • Hi Vinay. When will the limitation of "tests that have a native dependency cannot be run from command line" be addressed? Thanks.

  • What's the best way to write unit tests for old school asynchronous methods like the below using VS 2012 Update 2 and MsTest?

    Something like:

    var request = new MyOldRequest();

    request.Completed += result => Assert.IsTrue(result);

    request.Failed += error => Assert.Fail(error.ToString());

    request.Execute();

  • Seems like the AnyCpu configuration does not work, only x86. Why?

  • Hi, Unit test for windows phone 8 is very useful for us. And I want to integrate with CI server (Jenkins), but it shows

    "Could not start test run for unit tests for Windows Phone app: Unit tests for Windows Store apps cannot be run from a service or non interactive process. Please run unit tests from an interactive process.."

    I can run unit test with wp8 device by using command vstest.console, but cannot run it through Jenkins.

    Any ideas to run with Jenkins?

    Thanks a lot. ^_^

  • Can unit test cases be run manually.

    The unit test app is getting installed, then test are running and after that is getting uninstalled if we run unit tests from either IDE/command prompt.

    If I am running manually, the app is getting stuck at showing "Tests running."

    My requirement is that after the test cases have been run, a file will get generated in the isolated storage which I need. But in this case, as unit test app is getting uninstalled after the test cases have run, I am not getting any file.

  • Pajace, did you find a solution for your problem?

  • I've read this article and tried to get running. Windows Phone 8.0 SDK is installed, but I do not see any Test App Projects. Is it, because I have Windows Phone 8.0 SDK - DEU (German version)? What am I missing to get the templates. They don't appear in VS 2012 and 2013 either.

Page 2 of 2 (27 items) 12