Welcome to MSDN Blogs Sign in | Join | Help

SysTest quick start

Overview

I am a big fan of TestDrivenDevelopment and I am trying to follow this approach for all my development. Unfortunately DynamicsAX didn't have any UnitTest framework and thus it was difficult to use TestDrivenDevelopment. It was clear that we needed something in DynamicsAX and that's why we now have SysTest.

SysTest

SysTest is a UnitTest framework. If you are familiar with any other UnitTest framework then you will feel comfortable with SysTest.

Main features/goals:

  • it is very fast
  • it offers rich set of "assert" methods
  • it supports tests throwing exceptions (expected exceptions)
  • it supports transactions (tests can be placed inside a database transaction that is aborted at the end)
  • it supports company accounts (tests can be placed inside a separate company account that is deleted at the end of the test)
  • it supports fixture setup and tear down that are called automatically by the framework
  • it supports suites of tests
  • it supports setup and tear down method for suite that is run before the first method and after the last test method
  • it supports suites of suites
  • it supports code coverage
  • it has a wide variety of listeners for DynamicsAX environment
  • toolbar for quick test suite execution

Examples

Let's demonstrate the framework on examples. Let's create a unit test to test int2str conversion method. First we have to create our test class:

public class MySimpleTest extends SysTestCase
{
}

Now if you use toolbar (Tools > Development Tools > Unit Test > Show toolbar), then enter name of your test MySimpleTest and click Run. You immediately see "0 run, 0 failed". It's that simple. Ok, it maybe simple because it doesn't do much. Let's add a test validating that the function can successfully convert positive and negative numbers. To do so just add new method named with test prefix that is public, returns void and doesn't take any parameter. In the method check all your asserts.

public class MySimpleTest extends SysTestCase
{
    public void testConversion()
    {
        this.assertEquals('123', int2str(123));
        this.assertEquals('-2', int2str(-2));
    }
}

When you run the test in a toolbar runner for example you will immediately see "1 run, 0 failed". Great! Let's add more tests to our test class. Let's see how the framework handles tests that throw exceptions but those exceptions are expected. Because of that we don't want to fail the test method. There are two ways to do that. First one is that you will code the exception handling yourself:

    public void testExpectedException()
    {
        try
        {
            throw Exception::Error;
        }
        catch(Exception::Error)
        {
            return;
        }
        this.fail('An expected exception wasn\'t thrown!');
    }

As you can see when we expect an exception and it is not thrown then it is considered a failure. SysTest offers a better way to do the same.

    public void testExpectedException()
    {
        this.parmExceptionExpected(true);
        throw Exception::Error;
    }

Exceptions are more complex in DynamicsAX. Usually we don't just throw an error exception but we throw an error message. How do we handle that in SysTest?

public void testExpectedError() { ; this.parmExceptionExpected(true, 'Oops -- error message'); throw error('Oops -- error message'); } }

When you run the test you will get "3 run, 0 failed". Let's now see what happens when a failure occurs. We will begin by adding a failing test.

    public void testFailure()
    {
        this.assertEquals('0', int2str(123));
    }

This is clearly a failing test. When we run the test class now we get "3 run, 1 failed". Toolbar displays all failures in Infolog window. For our test class we would see the following window:



On the other hand we don't have to rely on toolbar. Let's say that we want to see all messages (not just failures) and let's say that we want to write them into DynamicsAX Print window. First let's write our own runner. For a runner we have to specify the whole suite we want to run. That's easy and all you have to do is to create instance of SysTestSuite class and pass your test class name to SysTestSuite constructor. To run the suite call run method. Here is one little problem. Suite itself is not interested in results at all. The problem is that we are and if we want to know the result at the end we have to create special SysTestResult object and pass it to the suite when it runs our test.

    public static void main(Args _params)
    {
        SysTestSuite suite = new SysTestSuite(classstr(MySimpleTest));
        SysTestResult result = new SysTestResult();
        ;
        suite.run(result);
        print result.getSummary();
    }

The code above creates the suite for our test class and executes all the tests. At the end it prints the summary report to the Print window. If we want to print all messages (failures, information, but also when a test is started or completed) then we have to register the corresponding listener.

    public static void main(Args _params)
    {
        SysTestSuite suite = new SysTestSuite(classstr(MySimpleTest));
        SysTestResult result = new SysTestResult();
        ;
        result.addListener(new SysTestListenerPrint()); // <- LISTENER REGISTERED
        suite.run(result);
        print result.getSummary();
    }

In total our whole class looks like this:

public class MySimpleTest extends SysTestCase
{
public void testConversion()
{
;
this.assertEquals('123', int2str(123));
this.assertEquals('-2', int2str(-2));
}
public void testExpectedException()
{
;
this.parmExceptionExpected(true);
throw Exception::Error;
}
public void testExpectedError()
{
;
this.parmExceptionExpected(true, 'Oops -- error message');
throw error('Oops -- error message');
}
public void testFailure()
{
;
this.assertEquals('0', int2str(123));
}

public static void main(Args _params)
{
SysTestSuite suite = new SysTestSuite(classstr(MySimpleTest));
SysTestResult result = new SysTestResult();
;
result.addListener(new SysTestListenerPrint()); // <- LISTENER REGISTERED
suite.run(result);
print result.getSummary();
}
}

That's all. If you want more information then stay tuned. In the next post I will try to describe individual features of SysTest in more details.

Published Friday, August 25, 2006 9:32 AM by dpokluda
Filed under: ,

Comment Notification

If you would like to receive an email when updates are made to this post, please register here

Subscribe to this post's comments using RSS

Comments

# re: SysTest quick start

Monday, August 28, 2006 6:50 PM by pyll
Good job on documenting how this works in Ax.  When I was first looking at the SysTest* code, it looked like it wasn't functional.  However, after delving further it looked better.

I'm glad that it works.

As for the code snippets you used above, they seemed to work, except for the main method.  I'm not sure if it is because I just threw the code into a job and ran it, but running it this way seems to produce no failures, eventhough the toolbar shows that there are errors...

Any ideas?

Thanks!

# re: SysTest quick start

Thursday, August 31, 2006 10:13 AM by Björn
It's great to see support for unit testing in the standard application.  It sure beats my feeble attempt of porting JUnit to Axapta 3.0.

What still bothers me is that the standard application is not well suited for unit testing.  This is a problem when trying to customize the standard code in a TDD fashion.  You need a lot of setup code and data to test even minor modifications.

It's great for completely new modules though.

# re: SysTest quick start

Thursday, August 31, 2006 4:35 PM by mugz
what portion of existing ax code is covered by unit tests?

does MS internally do TDD and in what scale?

How many tests are in the Ax4 codebase?

# re: SysTest quick start

Thursday, September 07, 2006 1:23 AM by Tests in Ax4 codebase
I've been playing with one of the pre-releases of 4.0 and don't think there are any unit tests in the standard codebase.  Not even for the test framework itself.

# re: SysTest quick start

Friday, September 08, 2006 12:15 AM by dpokluda
Sorry for such a late reply. I've been out of the town. Here are the answers to your questions:

Re Pyll: Just add the main method to your test class. Then you can open the class -- this would execute the main method.

Re Bjorn: You are right. Standard (legacy) applications were not designed with testability in mind. Therefore it might be hard to test those.

Re Mugz: Core of SysTest was done using TDD. There are other teams that are using TDD (like test automation framework team for example).

Re Tests in Ax4 codebase: We have already thousands of unit tests covering mostly the new functionality. For 4.1 development we have even higher bar and there must be certain code coverage before you can check-in the code (otherwise the check-in would be refused). These tests are not shipped with the product and so you will not find them.

I'm planning to post another post either later this week or early next week. :-)

# re: SysTest quick start

Friday, September 08, 2006 10:37 AM by Björn
Too bad no tests are included.  It would be good to have them both for testing if a modification breaks existing functionality, as well as having an example how to use existing code.

Can you tell us why they aren't shipped with the final release?

# re: SysTest quick start

Friday, September 08, 2006 12:56 PM by dpokluda
Re Bjorn: I totally understand what you're saying and I agree that it might be a help for all the partners if they would have some of our tests. On the other hand I also understand others saying that the value of shipping the tests wouldn't be so big since with all the modifications (made by partners) most of the tests would either not compile or fail anyway. Partners would either have to invest heavily to update all the tests (and repeat that investment with every release) or to investigate every failure to find out whether it is a valid failure or not.

On top of that you have to understand that there is different bar for code quality for tests. Therefore we don't really want to publish/ship those classes with the product. I remember that there were some thoughts about shipping our test automation framework with one of our test suites but since test automation framework doesn't meet our shipping code quality bar we never did that.

I think it's great we're shipping the framework now and people can define and write their own set of tests. We might also try to define a basic set of tests and ship those later but that's not something I can or will decide. :-)

# SafeMap &laquo; SysDictCoder

Friday, September 08, 2006 3:22 PM by SafeMap « SysDictCoder

# re: SysTest quick start

Friday, September 08, 2006 3:45 PM by Björn
Thanks for the clarification, David.  Those are of course good arguments.  I'm just one of those guys who wouldn't mind dealing with broken tests and "low quality" code :)  IMO it would increase overall quality in the long run.

I agree that it's great to finally have a testing framework. And it's looking very promising. No complaints there. Can't wait to use it on a real project.

# re: SysTest quick start

Friday, October 20, 2006 4:57 AM by Dom

Thanks alot for writing this article~~!

By the way, I agree with Bjorn regarding availability of some samples.

# re: SysTest quick start

Wednesday, March 07, 2007 1:58 PM by dpokluda

I understand what you're saying and I also think it would be useful for you to have some of our tests. Try to be heard and maybe the policy will change. :-)

# re: SysTest quick start

Wednesday, March 14, 2007 3:01 PM by ...

Great site! Good luck to it's owner!

# re: SysTest quick start

Friday, March 16, 2007 3:02 PM by ...

pagine piuttosto informative, piacevoli =)

# re: SysTest quick start

Sunday, March 18, 2007 6:28 AM by ...

luogo grande:) nessun osservazioni!

# re: SysTest quick start

Monday, March 19, 2007 9:27 PM by ...

Great site! Good luck to it's owner!

# re: SysTest quick start

Tuesday, August 12, 2008 1:23 PM by Ashok Kumar

I am a newbie to axapta, Please give me the clarity on SysTest Framework.

# re: SysTest quick start

Friday, December 12, 2008 4:38 AM by Anh Mai

Do you have any idea about tool to test for kernel class (use C++ language).

Because i think this tool is used for test on X++ layer.

Thanks!

# David Pokluda s blog SysTest quick start | Paid Surveys

# David Pokluda s blog SysTest quick start | Outdoor Ceiling Fans

# David Pokluda s blog SysTest quick start | Outdoor Ceiling Fans

# David Pokluda s blog SysTest quick start | home lighting

Leave a Comment

(required) 
required 
(required) 

  
Enter Code Here: Required
 
Page view tracker