mfp's two cents

...on Dynamics AX Development!
  • mfp's two cents

    Garbage Collection and RPC calls in X++

    • 0 Comments

    Dynamics AX is a 3-tier application that evolved from a 2-tier application.  Yes, this is right, the first versions of Axapta was solely a client side application communicating with the database. In version 2.0 the middle tier was introduced. The X++ language got a few new keywords, client and server, and the AX run-time provided the smartest marshaling of objects across the tiers on the planet. The marshaling is guaranteed to work in virtually any object graph you can instantiate. You can have client-side classes holding references to instances of server-side classes, which contains references back to other client-side objects, which references … you get the idea. All you have to do as a developer is to decorate your classes as client, server or called-from.

    You don’t have to worry about any low level details about how the instances are communicating across the wire. The key word in the previous sentence is “have” – stuff will just work, but unless you are very careful you may end up creating an extremely chatty (i.e. lot of RPC calls on the wire) implementation.  Recently I’ve seen two cases of well-intended changes that on the surface looked right, but both caused an explosion of RPC calls. Both were implemented by smart guys, and I wanted to point them to an article explaining the problem – and I realized that article didn’t exist. Until now.

    Garbage collection in AX

    The garbage collector (GC) AX is responsible for releasing memory consumed by object instances no longer in use. In .NET the GC is indeterministic, it runs when it “feels like” running, typically when the system has CPU capacity and is low on memory.   In contrast the GC in AX is deterministic – it runs every time an object goes out of scope.

    Consider this small example:

    static void GCJob1(Args _args)
    {
        MyServerClass myServerClass;
        //Create instance
        myServerClass = new MyServerClass();
        //Release instance
    }
    image

    Jobs run on the client tier, so this will create an instance of the MyServerClass and release it again. MyServerClass is a trivial class with RunOn=Server.

    If we enable Client/Server trace under Tools | Options | Development, and run the job, we get:

    Create instance
    Call Server: object: MyServerClass.new()
    Release instance
    Call Server: destruct class


    Notice this: The client-tier reference to the server instance, is keeping the instance alive. When the reference goes out-of-scope, then the GC takes over and calls the server to free the server memory.

    Island detection

    The GC is not just looking for instances without references – often object graphs are more complicated. To release memory that is no longer needed, the GC is looking for groups of objects without any external references – or in popular lingo: Islands. This search is potentially harmful to the performance of your application. The GC must traverse all members of any object that goes out of scope – regardless of their tier.

    Let’s build out the example by introducing a client side class that is referenced by the server instance.

    static void GCJob2(Args _args)
    {
        MyServerClass myServerClass;
        MyClientClass myClientClass;
        //Create client instance
        myClientClass = new MyClientClass();
        //Create server instance
        myServerClass = new MyServerClass();
        //Make server instance reference client instance
        myServerClass.parmMyClientClass(myClientClass);
        //Release instances
    }
    image
    Now; when myServerClass goes out-of-scope then the GC will start analyzing the object graph, and it will discover an island consisting of our two objects – despite they are on different tiers, and it will release the memory consumed.
    image

    Pretty smart – but not for free!

     

    This is the resulting RPC traffic from the above job:

    Create client instance
    Create server instance
    Call Server: object: MyServerClass.new()
    Make server instance reference client instance
    Call Server: object: MyServerClass.parmMyClientClass()
    Call Client: set class loop dependencies
    Call Client: test class loop dependencies
         
    Release instances
    Call Server: destruct class
    Call Client: destruct class

    Now suddenly we jumped from 2 RPC calls to 6! What happened? We met the GC! 

    • The first 2 calls are expected, they are a direct consequence of the methods invoked on the server tier.
    • The next 2 calls are set/test class loop dependencies. Both of these are consequences of the parm method. The set call is a result of the assignment inside the parm method. It tells the client that the server now holds a reference to the client side object. The test call is the GC looking for islands, but not finding any. When the parameter (containing the client-side object) goes out-of-scope at the end of the parm method, then GC looks for islands. As the server side class holds a client side member, then the traversal of the object graph requires a trip to the client.  
    • The last 2 calls are cleaning up. Notice that destruction is a chain reaction. First the server is called to destruct the server side object, then the server calls back to the client to destruct the client-side object.

     

    A real life example

    Consider a server side class that is looping over some data, and for each, say, row, it spins up another class instance on the server to do some calculations.  This is all server side, and perfect. So let’s add client-side member to the mix.

    class MyServerClass
    {
        MyClientClass myClientClass;
        
        public void run()
        {
            int i;
            MyServerHelper myServerHelper;
        
            //Create client-side member
            myClientClass = new MyClientClass();
        
            //Loop over some data
            for (i=1; i<=10; i++)
            {
                myServerHelper = new MyServerHelper();
                myServerHelper.parmMyServerClass(this);
                myServerHelper.calc();
            }
        }
    }
    image

    The alarming result is 10 client calls – or one per iteration in the loop – despite the loop only contains server side logic.

    Create client-side member
    Call Client: object: MyClientClass.new()
    Call Client: set class loop dependencies
    Loop over some data
    Call Client: set class loop dependencies
    Call Client: set class loop dependencies
    Call Client: set class loop dependencies
    Call Client: set class loop dependencies
    Call Client: set class loop dependencies
    Call Client: set class loop dependencies
    Call Client: set class loop dependencies
    Call Client: set class loop dependencies
    Call Client: set class loop dependencies
    Call Client: set class loop dependencies

    The assignment inside the parm method forces the AX runtime to traverse the object graph, and the object graph contains a client side instance.

    The alert reader would have recognized this as the Runbase pattern. The client side class is the operation progress bar. In Dynamics AX 2009 the operation progress bar framework regressed, as a client side reference was introduced - exposing all thousands of consumers to this specific problem. This got fixed in Dynamics AX 2012 R3.

    Symmetrical implementation

    The implementation of the GC and the supporting runtime is symmetrical on each tier – you can recognize them in action, when you come across these calls in the Trace Parser. Remember; they are always a consequence of the exercised X++ logic. I.e. something that can be addressed if required.

    Call Client: set class loop dependencies
    Call Client: test class loop dependencies
    Call Client: destruct class

    Call Server: set class loop dependencies
    Call Server: test class loop dependencies
    Call Server: destruct class

     

    Wrapping up

    There is only one way of understanding the impact the GC has on your implementation: Measure it!   The best tool for measurement is the Trace Parser. Alternatively, the Client/Server trace in Tools | Options | Development can be used – it will show all the RPC calls in the Message Window.

    The rule-of-thumb as a developer is to avoid class members that are living on the opposite tier. This will ensure your object graphs are single tiered, and it will make the life of the runtime and the GC much simpler, and your applications equally faster.

    There are situations where cross tier members seem unavoidable. However, there are techniques to avoid them, and achieve the same functional results. Take a look in the SysOperationProgress class in AX4 or AX 2012 R3 for an example.

     

    Code samples are attached. They are provided AS-IS and confers no rights.

  • mfp's two cents

    10,000 feet overview of Dynamics AX Inventory Cost Management

    • 0 Comments

    The past few years I’ve ventured into the SCM code base of AX, and I most admit I found the physical operations easier to comprehend than their financial counterparts. Perhaps it is just the terminology, or perhaps its because I’m “just” an engineer. At any rate I discovered I’m not the only one struggling – so here is an high level overview of the Cost Management domain and how it is handled in Dynamics AX.

    Background
    A text-book condensed into a paragraph

    A company is profitable if it can sell its goods for a higher price than they paid for them. This difference is called the Gross profit, it can be found on the Income statement, and is the difference between Sales and Cost of goods sold (COGS). In other words, COGS directly influences the company’s profitability and the amount of tax to pay.  The General accepted accounting principles (GAAP) describes several Costing principles, i.e. ways COGS can be measured. As the costing principle(s) used can have a significant impact on the company’s financial performance, there must be full disclosure to which costing principle(s) is used. For these reasons companies are inclined to stay with their current costing principle. The history of accounting for costs predates computer science and ERP systems. Several of the costing principles in use today reflects this fact – they are designed to be applied periodically (in the past by a book keeper with a pen in his hand).  The inputs to the costing calculations are called cost elements. They include Direct cost (purchase price of finished goods or raw material) and Indirect cost (such as labor and equipment).

    Dynamics AX’s Inventory Models

    In Dynamics AX the Costing principles are called Inventory models. An Inventory model prescribes how Direct cost is determined. Indirect cost is always estimated. These estimations are provided by the Costing sheet.

    In the following consider this sequence of event:

    1. Monday:          Purchase of 1 pcs @ $10
    2. Tuesday:          Purchase of 1 pcs @ $12
    3. Wednesday:     Sale of 1 pcs (sales price is irrelevant)
    4. Thursday:         Purchase of 2 pcs @ $15 each.
    5. Friday:              Inventory Close

     

    AX supports these Inventory models:

    • Normal costing accounts the costs as they occurred, however various circumstances can later influence the posting, such as back-dating. An example of back-dating could be that the Thursday purchase is register as if it occurred on Tuesday. At the end of the period the Inventory Close process will take care of any differences.

      AX supports 5 variants:
      • FIFO – First in first out; In the example the cost is $10
      • LIFO – Last in last out over the entire next period; In the example the cost is $15
      • LIFO Date – Last in last out (until the sale); In the example the cost is $12
      • Weighted average – over the entire period; In the example the cost $13. ((10+12+2*15)/4)
      • Weighted average date – in the period until the sale; In the example the cost is $11. ((10+12)/2)

    • Standard cost is based on estimates for both Direct and Indirect costs. This means it can be calculated during planning and thus is a powerful operational management tool. The BOM calculation is used to estimate the costs. AX supports several versioning of the estimates – they are called Costing versions. This makes Standard cost a perpetual model, where cost always can be determined during the sale. In the example above the cost is whatever estimate is provided by the active costing version, e.g. $9. 

    • Moving average is similar to Weighted average date. However; it is a perpetual model, where cost always is determined during the sale. This means that back-dating will not impact the posted costs, and Inventory Closing is not required. In the example; the cost is $11.


    Links

    AX 2009 Costing models: http://www.microsoft.com/en-us/download/details.aspx?id=1041

    Moving average: http://technet.microsoft.com/en-us/library/hh597235.aspx

    More background: http://www.accountingcoach.com/inventory-and-cost-of-goods-sold/explanation

  • mfp's two cents

    X++ Debugging Tips and Tricks #4 - Reading uncommitted data

    • 2 Comments

     X   Debugging - tall 2

    This next trick in the X++ Debugging Tip and Tricks series is invaluable when stuck deep in a debugging session, and want to verify that the SQL statements being executed has the intended behavior.

    For illustrative purposes, I’ve created a small job. It updates the customer group to “40” for the customer with account number “4001”. This happens inside a transaction. I’ve also added a breakpoint statement to jump to the debugger inside the transaction.

    When running the job the debugger opens:

    image

    Now, if we go the SQL Server Management Studio and execute a select statement to validate that the field was correctly updated, we will notice that it still has the old value:

    image

    This should not be a surprise, as the transaction has not been committed yet. However; that shouldn’t block our noble purpose. By adding the keyword “nolock” to the query, we instruct SQL to read the uncommitted data, and voila, we get the results we expect:

    image

    One additional piece of advice is related to more complicated SQL statements: You have to add the nolock keyword to every joined table.

    This can be a true life saver when debugging. In fact I used this heavily the past few weeks where I've been working on the new Warehouse Management module in R3.

  • mfp's two cents

    X++ Pedal To The Metal

    • 7 Comments

    This is a post I’ve been waiting to write for quite a while – but it had to wait until R3 became available.

    T-Shirt-Fast Xpp The SysExtension framework offers some great capabilities, but unfortunately it also comes with a performance penalty (in R2). A while ago I set out to understand where the time was spent, and hoping to optimize it. As an engineer this is a typical task with an expected outcome: A few days spent, an optimization of 10-20 percent is found. This time the task turned out to completely consume me for the better part of a week, I learned some important lessons, and the outcome exceeded my wildest imagination. This blog shares my findings.

    Starting point

    To be able to measure the impact of any changes I build a very simple test harness to exercise the SysExtension framework. A two level deep class hierarchy and one attribute that decorated the sub-class. I then compiled everything to IL, and wrote a small job to measure how many class instances I could spin up per second. This velocity measurement was around 3,400 classes/second.

    First success

    Debugging through the code I quickly learned that a lot of things were going on. This included creating a key for the attribute for various caches. This key was constructed via reflection on the attribute class. I avoided using reflection by introducing a new interface (SysExtensionIAttribute) and I fixed a number of other minor issues. Now the velocity jumped to 40,000 classes/second.

    First physical limit

    Is this an acceptable velocity? Well, how fast can it possibly be? The logic is in essence just creating a class via reflection, so I did a measurement of DictClass.MakeObject(). This could give me 84,000 classes/second. Slightly about double of the my current implementation. After some investigation I discovered two expensive calls: “new DictClass()” and “dictClass.makeObject()”. Can you spot what they have in common?  They both requires a call into the native AOS libraries. In other words an interop call. I tried various other calls into the AOS, such as “TTSBegin” (only the first is hitting the DB), and “CustParameters::Find()” (again, only the first one is hitting the DB). To my surprise the velocity of these calls where comparative to DictClass.MakeObject(). The interop overhead outweighs what the method is actually doing.  In other words, there is a limit to how many native AOS methods you can call per second. Let us call this velocity: Speed-of-sound.

    Ultimate physical limit

    Being a bit intrigued I measured the fastest and rawest possible implementation: “new MyClass()”. This would run strictly in IL, no overhead of any kind, the result was a whooping 23,800,000 classes/second.  Let us call this velocity: Speed-of-light. In the words of Barney Stinton: “Challenge accepted!” 

    Final success

    To achieve this kind of velocity the code must run 100% as IL. No calls into native AOS code. Period. Naturally there are APIs in .NET allowing for dynamically creation of class instances – they are slower than a direct instantiation, but still much faster than calling native AOS code. One other challenge was that SysExtension also can execute as pCode, and then a call into IL would cause a similar slow interop – just in the opposite direction. After a few iterations I had an implementation that would not cause an interop calls, regardless of if the code runs as IL or pCode. Take a look at SysExtensionAppClassFactory.getClassFromSysExtAttribute in R3 for details. I was pleased with the velocity: 661,000 classes/second. Or about 200 times faster than R2. Or about 15 times faster than a call to CustParameters::Find(). 

    Problem solved: The SysExtension framework no longer has performance issues.

    image

    Conclusion

    For a long time we have been hunting for SQL and RPC calls when looking for performance. RPC calls are expensive, as communication between two components (Client and Server) occurs. Just like SQL calls are expensive as the Server communicates with SQL (and waits for the reply). We still need to hunt for unnecessary RPC and SQL calls! Nothing changed; except that a third culprit has been identified: Native AOS calls. Relatively speaking the cost of calls into native AOS code is insignificant when compared to RPC or SQL calls – in the absence of these, the impact is measurable and significant.

    This is an ERP system, so there will always be SQL calls. So why be concerned with the performance of X++ code?  Well, if you can minimize the time between SQL calls, then you will also limit the time SQL holds locks, and you will experience better overall performance and scalability. After all, do you want you code to run with the speed-of-sound or speed-of-light?

    Update 11-05-2014:
    Here is the test harness I used: PrivateProject_SysExpProject.xpo

  • mfp's two cents

    Dynamics AX R3 is here, on-prem and in the cloud!

    • 0 Comments

    For business and IT leaders in today’s digitally-connected world who want to deliver amazing customer experiences, your solution – Microsoft Dynamics AX 2012 R3 – is now available in 36 countries!

    Brand new modules: Call center, Transportation Management and Warehouse Management - and new and improved capabilities in General Ledger, Inventory costing, Master planning, Procurement and sourcing, Retail, Production control, Public sector, Project and Trade. See all the details here: What's new in R3.

    If you like to take R3 for a spin - it has never been easier. You can deploy an Azure instance of R3 for demo purposes in a matter of minutes. Read more here: Deploy Microsoft Dynamics AX 2012 R3 on Azure using Lifecycle Services

    You can download R3 from CustomerSource or PartnerSource today.

     

  • mfp's two cents

    Dynamics AX 2012 R3 Launch Events

    • 2 Comments

    image

    It is today! Make sure to join the online Launch Event.

    • Session 1: 11:00 AM SGT (Asia: GMT +8)
    • Session 2: 15:00 GMT/16:00 CET (Europe)
    • Session 3: 12:00 PM CDT (North America: GMT -6)

     

    Read more here.

  • mfp's two cents

    Microsoft Dynamics AX WMS/TMS Workshop

    • 2 Comments

    image

    Today and tomorrow the SCM team from the Microsoft Dynamics AX R&D division is hosting a workshop on the upcoming Warehouse and Transportation management functionality in Dynamics AX 2012 R3. 140 partners and customers are gather from over 80 different companies at DGI Byen in Copenhagen.

    We have 2 busy days ahead of us with short presentations followed by hands-on-labs. No one will leave Copenhagen without a first hands experience of WMS and TMS including:

    • Reservation Hierarchies,
    • New Inventory dimensions: License Plate and Inventory Status,
    • Product configuration,
    • Warehouse setup of Location, Zones and Profiles,
    • Warehouse flow,
    • Outbound warehouse processes using mobile devices,
    • Warehouse integration with Discrete manufacturing and Kanban,
    • Transportation setup of Carrier, Rating Profiles, Rate Master, Override Charges, and Routes and Routing Constraints.
    • Freight reconciliation
  • mfp's two cents

    Register for the Dynamics AX 2012 R3 Launch Event

    • 0 Comments

     

     

    During the event, we will show our new solutions that will help you better engage with your customers.

    Join the CFO of Kathmandu, the CIO of Ashley Furniture and the IT director of Kent Corporation as they discuss their “reimagined customer experience” and the success they’re achieving with Microsoft Dynamics AX. We’ll share a “digital view” of a dynamic business solution (Microsoft Dynamics AX 2012 R3) that helps you engage with your customers on their terms, run dynamics operations and expand rapidly to better serve your customers.             

    • Hear why Principal Analyst Nigel Montgomery of Gartner talking about how Business Decision Makers respond to today’s rapidly changing market conditions.
    • Discover how organizations are simplifying their supply chain, giving timely business insight to the board room and increasing employee productivity.
    • See our new capabilities in action and how mobile, devices and the cloud are helping businesses be highly responsive to evolving customer preferences and changing market conditions.

    Now is the time to transform your company into a dynamic business that delivers amazing customer experiences!

    Date: April 10, 2014

    Time(s):

    • Session 1: 11:00 AM (Asia/Singapore)
    • Session 2: 3:00 PM (Europe/London)
    • Session 3: 12:00 PM (Central Standard Time)

    Register here.

  • mfp's two cents

    Inside Microsoft Dynamics AX 2012 R3

    • 0 Comments
    The 4th book in the “Inside Microsoft Dynamics AX” series will be released June 25th 2014.
    image

    Fully updated for Microsoft Dynamics AX 2012 R3!

    Dig into the architecture and internals of Microsoft Dynamics AX 2012 R3 - with firsthand insights from the team that designed and developed it. Deepen your knowledge of the underlying frameworks, components, and tools - and deliver custom ERP applications with the extensibility and performance your business needs. Useful for Microsoft Dynamics AX solution developers at all levels, this guide will provide max benefit to those who understand OOP, relational database, and Transact-SQL concepts.

     

    And this time it comes with a picture of a tool on the cover that I’ve actually used before – but I must admit (so far) never in a work related situation!

  • mfp's two cents

    Microsoft Dynamics AX R3 is coming

    • 0 Comments
    AX Logo

    Today is a big day for the engineering teams at Microsoft working on R3. We are reaching the important ZBB milestone. ZBB is one of the final internal checkpoints before releasing. It signals that there are zero active product defects known and planned for the release. Reaching ZBB is by no means an easy win. It requires maintaining a bug fix rate higher than the bug discovery rate for a lengthy period of time. It also requires significant coordination and team collaboration to have all teams reach this criteria at the same time.   

    One part of reaching ZBB is to have zero best practice errors across the entire code base. I’m happy to announce that R3 will ship with zero best practice errors despite the significant increase in code base.

    Once again; I’m awestruck by the energy, effort, diligence and dedication demonstrated by the Microsoft Dynamics teams to bring this important release to market.

    More to follow, as the release draws nearer…

Page 1 of 19 (184 items) 12345»

mfp's two cents

...on Dynamics AX Development!