mfp's two cents

...on Dynamics AX Development!
  • mfp's two cents

    SQL–More memory and CPU is not always a win

    • 1 Comments
    image

    All computer programs will run better when adding more memory and CPU cycles – right?  Not necessarily true.

    Assuming everything else is equal, then more memory and CPU will be a win; however, all computer systems have finite resources, also memory and CPU. Granting more to one program (or service), will take it away somewhere else.

    SQL is no exception, in fact it will happily consume all the resources you grant it – at the risk of starving other systems, including the OS.

    MaxServerMemory

    You can control how much memory SQL can use through the MaxServerMemory property. Setting it too low, means you are throttling SQL – setting it too high and you are throttling the OS.  Despair not, help is near: Here is a blog post written by Tara Shankar Jana – with a script, that will give you the optimal setting.

    CPU – Priority boost

    It may be tempting to give the SQL process a priority boost. But don’t do this! Ever! Doing this will starve any other process (including the OS processes), and it will not make the system perform better – in most cases it will be significantly worse.

    Here is a blog post by Arvind Shyamsundar on the topic: Priority boost details and why its not recommended

    If you want to disable priority boost, you can do it using this sql script:

    sp_configure 'priority boost', 0
    reconfigure with override
    

    One more thing…

    Since you are reading this, you probably want to get the best performance out of SQL on the hardware available. Check one more thing: The power plan!  You likely bought the hardware for your server to use it, so make sure to set the power plan to High performance – also on any VM hosts.

    Cindy Gross has written a blog on the topic.

     

    What has this got to do with Dynamics AX?

    Nothing and everything. The three guidelines above apply to any use of SQL Server – including when SQL is used with Dynamics AX.

    I recently visited an AX customer with performance problems. It turned out that SQL was granted 100% of the memory on the box, it was set to run with priority boost, and the power plan was set to balanced. The first two due to best intentions, the last due to this being the default setting. This starved the OS for resources, making overall performance of the system unpredictable – some simple queries would take seconds to complete, and blocks were observed too. Getting these settings right fundamentally changed the behavior – it was like night and day.

    Kudos to Tara for educating me on these topics.

  • mfp's two cents

    Damgaard Data turns 30 this month

    • 0 Comments

    <nostalgia>

    Version 2 just published a nice article about Damgaard Data – the company behind DanMax, C4, C5, XAL and Axapta.

    You can read it here: In Danish and in English

    </nostalgia>

  • mfp's two cents

    Trace Parser: NULL::inner–explained

    • 0 Comments

    If you have analyzed AX traces in the Trace Parser, you most likely came across something like this during your efforts:

    image

    First time I saw it, I was puzzled – an object named “NULL” with a method named “inner”, I’ve never heard of that before – what is it? I started asking around, no one seemed to know. Bing didn’t help me either. I did a search in our source files, it gave me more hits than I wanted to explorer. About to give up, it struck me, that the name is not too bad for a method that doesn’t exists on an object and is inside the caller.

    So the answer is simple and straight forward. "NULL::inner” is used for all embedded methods in X++ when shown in the trace parser. Navigating to the caller in the Trace Parser also clearly shows that an embedded method exists (with the name “doEscape()”).

    image

    As a side note, I can mention that the Call Stack window in the X++ Debugger doesn’t include embedded methods.

  • mfp's two cents

    Garbage Collection and RPC calls in X++

    • 3 Comments

    Dynamics AX is a 3-tier application that evolved from a 2-tier application.  Yes, this is right, the first versions of Axapta was solely a client side application communicating with the database. In version 2.0 the middle tier was introduced. The X++ language got a few new keywords, client and server, and the AX run-time provided the smartest marshaling of objects across the tiers on the planet. The marshaling is guaranteed to work in virtually any object graph you can instantiate. You can have client-side classes holding references to instances of server-side classes, which contains references back to other client-side objects, which references … you get the idea. All you have to do as a developer is to decorate your classes as client, server or called-from.

    You don’t have to worry about any low level details like how the instances are communicating across the wire. The key word in the previous sentence is “have” – stuff will just work, but unless you are very careful you may end up creating an extremely chatty (i.e. lot of RPC calls on the wire) implementation.  Recently I’ve seen two cases of well-intended changes that on the surface looked right, but both caused an explosion of RPC calls. Both were implemented by smart guys, and I wanted to point them to an article explaining the problem – and I realized that article didn’t exist. Until now.

    Garbage collection in AX

    The garbage collector (GC) AX is responsible for releasing memory consumed by object instances no longer in use. In .NET the GC is indeterministic, it runs when it “feels like” running, typically when the system has CPU capacity and is low on memory.   In contrast the GC in AX is deterministic – it runs every time an object goes out of scope.

    Consider this small example:

    static void GCJob1(Args _args)
    {
        MyServerClass myServerClass;
        //Create instance
        myServerClass = new MyServerClass();
        //Release instance
    }
    image

    Jobs run on the client tier, so this will create an instance of the MyServerClass and release it again. MyServerClass is a trivial class with RunOn=Server.

    If we enable Client/Server trace under Tools | Options | Development, and run the job, we get:

    Create instance
    Call Server: object: MyServerClass.new()
    Release instance
    Call Server: destruct class


    Notice this: The client-tier reference to the server instance, is keeping the instance alive. When the reference goes out-of-scope, then the GC takes over and calls the server to free the server memory.

    Island detection

    The GC is not just looking for instances without references – often object graphs are more complicated. To release memory that is no longer needed, the GC is looking for groups of objects without any external references – or in popular lingo: Islands. This search is potentially harmful to the performance of your application. The GC must traverse all members of any object that goes out of scope – regardless of their tier.

    Let’s build out the example by introducing a client side class that is referenced by the server instance.

    static void GCJob2(Args _args)
    {
        MyServerClass myServerClass;
        MyClientClass myClientClass;
        //Create client instance
        myClientClass = new MyClientClass();
        //Create server instance
        myServerClass = new MyServerClass();
        //Make server instance reference client instance
        myServerClass.parmMyClientClass(myClientClass);
        //Release instances
    }
    image
    Now; when myServerClass goes out-of-scope then the GC will start analyzing the object graph, and it will discover an island consisting of our two objects – despite they are on different tiers, and it will release the memory consumed.
    image

    Pretty smart – but not for free!

     

    This is the resulting RPC traffic from the above job:

    Create client instance
    Create server instance
    Call Server: object: MyServerClass.new()
    Make server instance reference client instance
    Call Server: object: MyServerClass.parmMyClientClass()
    Call Client: set class loop dependencies
    Call Client: test class loop dependencies
         
    Release instances
    Call Server: destruct class
    Call Client: destruct class

    Now suddenly we jumped from 2 RPC calls to 6! What happened? We met the GC! 

    • The first 2 calls are expected, they are a direct consequence of the methods invoked on the server tier.
    • The next 2 calls are set/test class loop dependencies. Both of these are consequences of the parm method. The set call is a result of the assignment inside the parm method. It tells the client that the server now holds a reference to the client side object. The test call is the GC looking for islands, but not finding any. When the parameter (containing the client-side object) goes out-of-scope at the end of the parm method, then GC looks for islands. As the server side class holds a client side member, then the traversal of the object graph requires a trip to the client.  
    • The last 2 calls are cleaning up. Notice that destruction is a chain reaction. First the server is called to destruct the server side object, then the server calls back to the client to destruct the client-side object.

     

    A real life example

    Consider a server side class that is looping over some data, and for each, say, row, it spins up another class instance on the server to do some calculations.  This is all server side, and perfect. So let’s add a client-side member to the mix.

    class MyServerClass
    {
          MyClientClass myClientClass;

          public void run()
          {
              int i;
              MyServerHelper myServerHelper;
              //Create client-side member
              myClientClass = new MyClientClass();

              //Loop over some data
              for (i=1; i<=10; i++)
              {
                  myServerHelper = new MyServerHelper();
                  myServerHelper.parmMyServerClass(this);
                  myServerHelper.calc();
              }
          }
      }
    image

    The alarming result is 10 client calls – or one per iteration in the loop – despite the loop only contains server side logic.

    Create client-side member
    Call Client: object: MyClientClass.new()
    Call Client: set class loop dependencies
    Loop over some data
    Call Client: set class loop dependencies
    Call Client: set class loop dependencies
    Call Client: set class loop dependencies
    Call Client: set class loop dependencies
    Call Client: set class loop dependencies
    Call Client: set class loop dependencies
    Call Client: set class loop dependencies
    Call Client: set class loop dependencies
    Call Client: set class loop dependencies
    Call Client: set class loop dependencies

    The assignment inside the parm method forces the AX runtime to traverse the object graph, and the object graph contains a client side instance.

    The alert reader would have recognized this as the Runbase pattern. The client side class is the operation progress bar. In Dynamics AX 2009 the operation progress bar framework regressed, as a client side reference was introduced - exposing all thousands of consumers to this specific problem. This got fixed in Dynamics AX 2012 R3.

    Symmetrical implementation

    The implementation of the GC and the supporting runtime is symmetrical on each tier – you can recognize them in action, when you come across these calls in the Trace Parser. Remember; they are always a consequence of the exercised X++ logic. I.e. something that can be addressed if required.

    Call Client: set class loop dependencies
    Call Client: test class loop dependencies
    Call Client: destruct class

    Call Server: set class loop dependencies
    Call Server: test class loop dependencies
    Call Server: destruct class

     

    Wrapping up

    There is only one way of understanding the impact the GC has on your implementation: Measure it!   The best tool for measurement is the Trace Parser. Alternatively, the Client/Server trace in Tools | Options | Development can be used – it will show all the RPC calls in the Message Window.

    The rule-of-thumb as a developer is to avoid class members that are living on the opposite tier. This will ensure your object graphs are single tiered, and it will make the life of the runtime and the GC much simpler, and your applications equally faster.

    There are situations where cross tier members seem unavoidable. However, there are techniques to avoid them, and achieve the same functional results. Take a look in the SysOperationProgress class in AX4 or AX 2012 R3 for an example.

     

    Code samples are attached. They are provided AS-IS and confers no rights.

  • mfp's two cents

    10,000 feet overview of Dynamics AX Inventory Cost Management

    • 0 Comments

    The past few years I’ve ventured into the SCM code base of AX, and I most admit I found the physical operations easier to comprehend than their financial counterparts. Perhaps it is just the terminology, or perhaps its because I’m “just” an engineer. At any rate I discovered I’m not the only one struggling – so here is an high level overview of the Cost Management domain and how it is handled in Dynamics AX.

    Background
    A text-book condensed into a paragraph

    A company is profitable if it can sell its goods for a higher price than they paid for them. This difference is called the Gross profit, it can be found on the Income statement, and is the difference between Sales and Cost of goods sold (COGS). In other words, COGS directly influences the company’s profitability and the amount of tax to pay.  The General accepted accounting principles (GAAP) describes several Costing principles, i.e. ways COGS can be measured. As the costing principle(s) used can have a significant impact on the company’s financial performance, there must be full disclosure to which costing principle(s) is used. For these reasons companies are inclined to stay with their current costing principle. The history of accounting for costs predates computer science and ERP systems. Several of the costing principles in use today reflects this fact – they are designed to be applied periodically (in the past by a book keeper with a pen in his hand).  The inputs to the costing calculations are called cost elements. They include Direct cost (purchase price of finished goods or raw material) and Indirect cost (such as labor and equipment).

    Dynamics AX’s Inventory Models

    In Dynamics AX the Costing principles are called Inventory models. An Inventory model prescribes how Direct cost is determined. Indirect cost is always estimated. These estimations are provided by the Costing sheet.

    In the following consider this sequence of event:

    1. Monday:          Purchase of 1 pcs @ $10
    2. Tuesday:          Purchase of 1 pcs @ $12
    3. Wednesday:     Sale of 1 pcs (sales price is irrelevant)
    4. Thursday:         Purchase of 2 pcs @ $15 each.
    5. Friday:              Inventory Close

     

    AX supports these Inventory models:

    • Normal costing accounts the costs as they occurred, however various circumstances can later influence the posting, such as back-dating. An example of back-dating could be that the Thursday purchase is register as if it occurred on Tuesday. At the end of the period the Inventory Close process will take care of any differences.

      AX supports 5 variants:
      • FIFO – First in first out; In the example the cost is $10
      • LIFO – Last in last out over the entire next period; In the example the cost is $15
      • LIFO Date – Last in last out (until the sale); In the example the cost is $12
      • Weighted average – over the entire period; In the example the cost $13. ((10+12+2*15)/4)
      • Weighted average date – in the period until the sale; In the example the cost is $11. ((10+12)/2)

    • Standard cost is based on estimates for both Direct and Indirect costs. This means it can be calculated during planning and thus is a powerful operational management tool. The BOM calculation is used to estimate the costs. AX supports several versioning of the estimates – they are called Costing versions. This makes Standard cost a perpetual model, where cost always can be determined during the sale. In the example above the cost is whatever estimate is provided by the active costing version, e.g. $9. 

    • Moving average is similar to Weighted average date. However; it is a perpetual model, where cost always is determined during the sale. This means that back-dating will not impact the posted costs, and Inventory Closing is not required. In the example; the cost is $11.


    Links

    AX 2009 Costing models: http://www.microsoft.com/en-us/download/details.aspx?id=1041

    Moving average: http://technet.microsoft.com/en-us/library/hh597235.aspx

    More background: http://www.accountingcoach.com/inventory-and-cost-of-goods-sold/explanation

  • mfp's two cents

    X++ Debugging Tips and Tricks #4 - Reading uncommitted data

    • 2 Comments

     X   Debugging - tall 2

    This next trick in the X++ Debugging Tip and Tricks series is invaluable when stuck deep in a debugging session, and want to verify that the SQL statements being executed has the intended behavior.

    For illustrative purposes, I’ve created a small job. It updates the customer group to “40” for the customer with account number “4001”. This happens inside a transaction. I’ve also added a breakpoint statement to jump to the debugger inside the transaction.

    When running the job the debugger opens:

    image

    Now, if we go the SQL Server Management Studio and execute a select statement to validate that the field was correctly updated, we will notice that it still has the old value:

    image

    This should not be a surprise, as the transaction has not been committed yet. However; that shouldn’t block our noble purpose. By adding the keyword “nolock” to the query, we instruct SQL to read the uncommitted data, and voila, we get the results we expect:

    image

    One additional piece of advice is related to more complicated SQL statements: You have to add the nolock keyword to every joined table.

    This can be a true life saver when debugging. In fact I used this heavily the past few weeks where I've been working on the new Warehouse Management module in R3.

  • mfp's two cents

    X++ Pedal To The Metal

    • 7 Comments

    This is a post I’ve been waiting to write for quite a while – but it had to wait until R3 became available.

    T-Shirt-Fast Xpp The SysExtension framework offers some great capabilities, but unfortunately it also comes with a performance penalty (in R2). A while ago I set out to understand where the time was spent, and hoping to optimize it. As an engineer this is a typical task with an expected outcome: A few days spent, an optimization of 10-20 percent is found. This time the task turned out to completely consume me for the better part of a week, I learned some important lessons, and the outcome exceeded my wildest imagination. This blog shares my findings.

    Starting point

    To be able to measure the impact of any changes I build a very simple test harness to exercise the SysExtension framework. A two level deep class hierarchy and one attribute that decorated the sub-class. I then compiled everything to IL, and wrote a small job to measure how many class instances I could spin up per second. This velocity measurement was around 3,400 classes/second.

    First success

    Debugging through the code I quickly learned that a lot of things were going on. This included creating a key for the attribute for various caches. This key was constructed via reflection on the attribute class. I avoided using reflection by introducing a new interface (SysExtensionIAttribute) and I fixed a number of other minor issues. Now the velocity jumped to 40,000 classes/second.

    First physical limit

    Is this an acceptable velocity? Well, how fast can it possibly be? The logic is in essence just creating a class via reflection, so I did a measurement of DictClass.MakeObject(). This could give me 84,000 classes/second. Slightly about double of the my current implementation. After some investigation I discovered two expensive calls: “new DictClass()” and “dictClass.makeObject()”. Can you spot what they have in common?  They both requires a call into the native AOS libraries. In other words an interop call. I tried various other calls into the AOS, such as “TTSBegin” (only the first is hitting the DB), and “CustParameters::Find()” (again, only the first one is hitting the DB). To my surprise the velocity of these calls where comparative to DictClass.MakeObject(). The interop overhead outweighs what the method is actually doing.  In other words, there is a limit to how many native AOS methods you can call per second. Let us call this velocity: Speed-of-sound.

    Ultimate physical limit

    Being a bit intrigued I measured the fastest and rawest possible implementation: “new MyClass()”. This would run strictly in IL, no overhead of any kind, the result was a whooping 23,800,000 classes/second.  Let us call this velocity: Speed-of-light. In the words of Barney Stinton: “Challenge accepted!” 

    Final success

    To achieve this kind of velocity the code must run 100% as IL. No calls into native AOS code. Period. Naturally there are APIs in .NET allowing for dynamically creation of class instances – they are slower than a direct instantiation, but still much faster than calling native AOS code. One other challenge was that SysExtension also can execute as pCode, and then a call into IL would cause a similar slow interop – just in the opposite direction. After a few iterations I had an implementation that would not cause an interop calls, regardless of if the code runs as IL or pCode. Take a look at SysExtensionAppClassFactory.getClassFromSysExtAttribute in R3 for details. I was pleased with the velocity: 661,000 classes/second. Or about 200 times faster than R2. Or about 15 times faster than a call to CustParameters::Find(). 

    Problem solved: The SysExtension framework no longer has performance issues.

    image

    Conclusion

    For a long time we have been hunting for SQL and RPC calls when looking for performance. RPC calls are expensive, as communication between two components (Client and Server) occurs. Just like SQL calls are expensive as the Server communicates with SQL (and waits for the reply). We still need to hunt for unnecessary RPC and SQL calls! Nothing changed; except that a third culprit has been identified: Native AOS calls. Relatively speaking the cost of calls into native AOS code is insignificant when compared to RPC or SQL calls – in the absence of these, the impact is measurable and significant.

    This is an ERP system, so there will always be SQL calls. So why be concerned with the performance of X++ code?  Well, if you can minimize the time between SQL calls, then you will also limit the time SQL holds locks, and you will experience better overall performance and scalability. After all, do you want you code to run with the speed-of-sound or speed-of-light?

    Update 11-05-2014:
    Here is the test harness I used: PrivateProject_SysExpProject.xpo

  • mfp's two cents

    Dynamics AX R3 is here, on-prem and in the cloud!

    • 0 Comments

    For business and IT leaders in today’s digitally-connected world who want to deliver amazing customer experiences, your solution – Microsoft Dynamics AX 2012 R3 – is now available in 36 countries!

    Brand new modules: Call center, Transportation Management and Warehouse Management - and new and improved capabilities in General Ledger, Inventory costing, Master planning, Procurement and sourcing, Retail, Production control, Public sector, Project and Trade. See all the details here: What's new in R3.

    If you like to take R3 for a spin - it has never been easier. You can deploy an Azure instance of R3 for demo purposes in a matter of minutes. Read more here: Deploy Microsoft Dynamics AX 2012 R3 on Azure using Lifecycle Services

    You can download R3 from CustomerSource or PartnerSource today.

     

  • mfp's two cents

    Dynamics AX 2012 R3 Launch Events

    • 2 Comments

    image

    It is today! Make sure to join the online Launch Event.

    • Session 1: 11:00 AM SGT (Asia: GMT +8)
    • Session 2: 15:00 GMT/16:00 CET (Europe)
    • Session 3: 12:00 PM CDT (North America: GMT -6)

     

    Read more here.

  • mfp's two cents

    Microsoft Dynamics AX WMS/TMS Workshop

    • 2 Comments

    image

    Today and tomorrow the SCM team from the Microsoft Dynamics AX R&D division is hosting a workshop on the upcoming Warehouse and Transportation management functionality in Dynamics AX 2012 R3. 140 partners and customers are gather from over 80 different companies at DGI Byen in Copenhagen.

    We have 2 busy days ahead of us with short presentations followed by hands-on-labs. No one will leave Copenhagen without a first hands experience of WMS and TMS including:

    • Reservation Hierarchies,
    • New Inventory dimensions: License Plate and Inventory Status,
    • Product configuration,
    • Warehouse setup of Location, Zones and Profiles,
    • Warehouse flow,
    • Outbound warehouse processes using mobile devices,
    • Warehouse integration with Discrete manufacturing and Kanban,
    • Transportation setup of Carrier, Rating Profiles, Rate Master, Override Charges, and Routes and Routing Constraints.
    • Freight reconciliation
Page 1 of 19 (187 items) 12345»

mfp's two cents

...on Dynamics AX Development!