Dynamics AX is a 3-tier application that evolved from a 2-tier application. Yes, this is right, the first versions of Axapta was solely a client side application communicating with the database. In version 2.0 the middle tier was introduced. The X++ language got a few new keywords, client and server, and the AX run-time provided the smartest marshaling of objects across the tiers on the planet. The marshaling is guaranteed to work in virtually any object graph you can instantiate. You can have client-side classes holding references to instances of server-side classes, which contains references back to other client-side objects, which references … you get the idea. All you have to do as a developer is to decorate your classes as client, server or called-from.
You don’t have to worry about any low level details about how the instances are communicating across the wire. The key word in the previous sentence is “have” – stuff will just work, but unless you are very careful you may end up creating an extremely chatty (i.e. lot of RPC calls on the wire) implementation. Recently I’ve seen two cases of well-intended changes that on the surface looked right, but both caused an explosion of RPC calls. Both were implemented by smart guys, and I wanted to point them to an article explaining the problem – and I realized that article didn’t exist. Until now.
The garbage collector (GC) AX is responsible for releasing memory consumed by object instances no longer in use. In .NET the GC is indeterministic, it runs when it “feels like” running, typically when the system has CPU capacity and is low on memory. In contrast the GC in AX is deterministic – it runs every time an object goes out of scope.
Consider this small example:
static void GCJob1(Args _args)
myServerClass = new MyServerClass();
Jobs run on the client tier, so this will create an instance of the MyServerClass and release it again. MyServerClass is a trivial class with RunOn=Server.
If we enable Client/Server trace under Tools | Options | Development, and run the job, we get:
Notice this: The client-tier reference to the server instance, is keeping the instance alive. When the reference goes out-of-scope, then the GC takes over and calls the server to free the server memory.
The GC is not just looking for instances without references – often object graphs are more complicated. To release memory that is no longer needed, the GC is looking for groups of objects without any external references – or in popular lingo: Islands. This search is potentially harmful to the performance of your application. The GC must traverse all members of any object that goes out of scope – regardless of their tier.
Let’s build out the example by introducing a client side class that is referenced by the server instance.
static void GCJob2(Args _args)
//Create client instance
myClientClass = new MyClientClass();
//Create server instance
myServerClass = new MyServerClass();
//Make server instance reference client instance
Pretty smart – but not for free!
This is the resulting RPC traffic from the above job:
Create client instance Create server instance Call Server: object: MyServerClass.new() Make server instance reference client instance Call Server: object: MyServerClass.parmMyClientClass() Call Client: set class loop dependencies Call Client: test class loop dependencies Release instances Call Server: destruct class Call Client: destruct class
Now suddenly we jumped from 2 RPC calls to 6! What happened? We met the GC!
Consider a server side class that is looping over some data, and for each, say, row, it spins up another class instance on the server to do some calculations. This is all server side, and perfect. So let’s add client-side member to the mix.
public void run()
//Create client-side member
myClientClass = new MyClientClass();
//Loop over some data
for (i=1; i<=10; i++)
myServerHelper = new MyServerHelper();
The alarming result is 10 client calls – or one per iteration in the loop – despite the loop only contains server side logic.
Create client-side member Call Client: object: MyClientClass.new() Call Client: set class loop dependencies Loop over some data Call Client: set class loop dependencies Call Client: set class loop dependencies Call Client: set class loop dependencies Call Client: set class loop dependencies Call Client: set class loop dependencies Call Client: set class loop dependencies Call Client: set class loop dependencies Call Client: set class loop dependencies Call Client: set class loop dependencies Call Client: set class loop dependencies
The assignment inside the parm method forces the AX runtime to traverse the object graph, and the object graph contains a client side instance.
The alert reader would have recognized this as the Runbase pattern. The client side class is the operation progress bar. In Dynamics AX 2009 the operation progress bar framework regressed, as a client side reference was introduced - exposing all thousands of consumers to this specific problem. This got fixed in Dynamics AX 2012 R3.
The implementation of the GC and the supporting runtime is symmetrical on each tier – you can recognize them in action, when you come across these calls in the Trace Parser. Remember; they are always a consequence of the exercised X++ logic. I.e. something that can be addressed if required.
Call Client: set class loop dependencies Call Client: test class loop dependencies Call Client: destruct class
Call Server: set class loop dependencies Call Server: test class loop dependencies Call Server: destruct class
There is only one way of understanding the impact the GC has on your implementation: Measure it! The best tool for measurement is the Trace Parser. Alternatively, the Client/Server trace in Tools | Options | Development can be used – it will show all the RPC calls in the Message Window.
The rule-of-thumb as a developer is to avoid class members that are living on the opposite tier. This will ensure your object graphs are single tiered, and it will make the life of the runtime and the GC much simpler, and your applications equally faster.
There are situations where cross tier members seem unavoidable. However, there are techniques to avoid them, and achieve the same functional results. Take a look in the SysOperationProgress class in AX4 or AX 2012 R3 for an example.
Code samples are attached. They are provided AS-IS and confers no rights.
The past few years I’ve ventured into the SCM code base of AX, and I most admit I found the physical operations easier to comprehend than their financial counterparts. Perhaps it is just the terminology, or perhaps its because I’m “just” an engineer. At any rate I discovered I’m not the only one struggling – so here is an high level overview of the Cost Management domain and how it is handled in Dynamics AX.
A company is profitable if it can sell its goods for a higher price than they paid for them. This difference is called the Gross profit, it can be found on the Income statement, and is the difference between Sales and Cost of goods sold (COGS). In other words, COGS directly influences the company’s profitability and the amount of tax to pay. The General accepted accounting principles (GAAP) describes several Costing principles, i.e. ways COGS can be measured. As the costing principle(s) used can have a significant impact on the company’s financial performance, there must be full disclosure to which costing principle(s) is used. For these reasons companies are inclined to stay with their current costing principle. The history of accounting for costs predates computer science and ERP systems. Several of the costing principles in use today reflects this fact – they are designed to be applied periodically (in the past by a book keeper with a pen in his hand). The inputs to the costing calculations are called cost elements. They include Direct cost (purchase price of finished goods or raw material) and Indirect cost (such as labor and equipment).
In Dynamics AX the Costing principles are called Inventory models. An Inventory model prescribes how Direct cost is determined. Indirect cost is always estimated. These estimations are provided by the Costing sheet.
In the following consider this sequence of event:
AX supports these Inventory models:
AX 2009 Costing models: http://www.microsoft.com/en-us/download/details.aspx?id=1041
Moving average: http://technet.microsoft.com/en-us/library/hh597235.aspx
More background: http://www.accountingcoach.com/inventory-and-cost-of-goods-sold/explanation
For illustrative purposes, I’ve created a small job. It updates the customer group to “40” for the customer with account number “4001”. This happens inside a transaction. I’ve also added a breakpoint statement to jump to the debugger inside the transaction.
When running the job the debugger opens:
Now, if we go the SQL Server Management Studio and execute a select statement to validate that the field was correctly updated, we will notice that it still has the old value:
This should not be a surprise, as the transaction has not been committed yet. However; that shouldn’t block our noble purpose. By adding the keyword “nolock” to the query, we instruct SQL to read the uncommitted data, and voila, we get the results we expect:
One additional piece of advice is related to more complicated SQL statements: You have to add the nolock keyword to every joined table.
This can be a true life saver when debugging. In fact I used this heavily the past few weeks where I've been working on the new Warehouse Management module in R3.
This is a post I’ve been waiting to write for quite a while – but it had to wait until R3 became available.
To be able to measure the impact of any changes I build a very simple test harness to exercise the SysExtension framework. A two level deep class hierarchy and one attribute that decorated the sub-class. I then compiled everything to IL, and wrote a small job to measure how many class instances I could spin up per second. This velocity measurement was around 3,400 classes/second.
Debugging through the code I quickly learned that a lot of things were going on. This included creating a key for the attribute for various caches. This key was constructed via reflection on the attribute class. I avoided using reflection by introducing a new interface (SysExtensionIAttribute) and I fixed a number of other minor issues. Now the velocity jumped to 40,000 classes/second.
Is this an acceptable velocity? Well, how fast can it possibly be? The logic is in essence just creating a class via reflection, so I did a measurement of DictClass.MakeObject(). This could give me 84,000 classes/second. Slightly about double of the my current implementation. After some investigation I discovered two expensive calls: “new DictClass()” and “dictClass.makeObject()”. Can you spot what they have in common? They both requires a call into the native AOS libraries. In other words an interop call. I tried various other calls into the AOS, such as “TTSBegin” (only the first is hitting the DB), and “CustParameters::Find()” (again, only the first one is hitting the DB). To my surprise the velocity of these calls where comparative to DictClass.MakeObject(). The interop overhead outweighs what the method is actually doing. In other words, there is a limit to how many native AOS methods you can call per second. Let us call this velocity: Speed-of-sound.
Being a bit intrigued I measured the fastest and rawest possible implementation: “new MyClass()”. This would run strictly in IL, no overhead of any kind, the result was a whooping 23,800,000 classes/second. Let us call this velocity: Speed-of-light. In the words of Barney Stinton: “Challenge accepted!”
To achieve this kind of velocity the code must run 100% as IL. No calls into native AOS code. Period. Naturally there are APIs in .NET allowing for dynamically creation of class instances – they are slower than a direct instantiation, but still much faster than calling native AOS code. One other challenge was that SysExtension also can execute as pCode, and then a call into IL would cause a similar slow interop – just in the opposite direction. After a few iterations I had an implementation that would not cause an interop calls, regardless of if the code runs as IL or pCode. Take a look at SysExtensionAppClassFactory.getClassFromSysExtAttribute in R3 for details. I was pleased with the velocity: 661,000 classes/second. Or about 200 times faster than R2. Or about 15 times faster than a call to CustParameters::Find(). Problem solved: The SysExtension framework no longer has performance issues.
For a long time we have been hunting for SQL and RPC calls when looking for performance. RPC calls are expensive, as communication between two components (Client and Server) occurs. Just like SQL calls are expensive as the Server communicates with SQL (and waits for the reply). We still need to hunt for unnecessary RPC and SQL calls! Nothing changed; except that a third culprit has been identified: Native AOS calls. Relatively speaking the cost of calls into native AOS code is insignificant when compared to RPC or SQL calls – in the absence of these, the impact is measurable and significant.
This is an ERP system, so there will always be SQL calls. So why be concerned with the performance of X++ code? Well, if you can minimize the time between SQL calls, then you will also limit the time SQL holds locks, and you will experience better overall performance and scalability. After all, do you want you code to run with the speed-of-sound or speed-of-light?
Update 11-05-2014: Here is the test harness I used: PrivateProject_SysExpProject.xpo
For business and IT leaders in today’s digitally-connected world who want to deliver amazing customer experiences, your solution – Microsoft Dynamics AX 2012 R3 – is now available in 36 countries!
Brand new modules: Call center, Transportation Management and Warehouse Management - and new and improved capabilities in General Ledger, Inventory costing, Master planning, Procurement and sourcing, Retail, Production control, Public sector, Project and Trade. See all the details here: What's new in R3.
If you like to take R3 for a spin - it has never been easier. You can deploy an Azure instance of R3 for demo purposes in a matter of minutes. Read more here: Deploy Microsoft Dynamics AX 2012 R3 on Azure using Lifecycle Services
You can download R3 from CustomerSource or PartnerSource today.
It is today! Make sure to join the online Launch Event.
Read more here.
Today and tomorrow the SCM team from the Microsoft Dynamics AX R&D division is hosting a workshop on the upcoming Warehouse and Transportation management functionality in Dynamics AX 2012 R3. 140 partners and customers are gather from over 80 different companies at DGI Byen in Copenhagen.
We have 2 busy days ahead of us with short presentations followed by hands-on-labs. No one will leave Copenhagen without a first hands experience of WMS and TMS including:
During the event, we will show our new solutions that will help you better engage with your customers.
Join the CFO of Kathmandu, the CIO of Ashley Furniture and the IT director of Kent Corporation as they discuss their “reimagined customer experience” and the success they’re achieving with Microsoft Dynamics AX. We’ll share a “digital view” of a dynamic business solution (Microsoft Dynamics AX 2012 R3) that helps you engage with your customers on their terms, run dynamics operations and expand rapidly to better serve your customers.
Now is the time to transform your company into a dynamic business that delivers amazing customer experiences!
Date: April 10, 2014 Time(s):
Fully updated for Microsoft Dynamics AX 2012 R3!
Dig into the architecture and internals of Microsoft Dynamics AX 2012 R3 - with firsthand insights from the team that designed and developed it. Deepen your knowledge of the underlying frameworks, components, and tools - and deliver custom ERP applications with the extensibility and performance your business needs. Useful for Microsoft Dynamics AX solution developers at all levels, this guide will provide max benefit to those who understand OOP, relational database, and Transact-SQL concepts.
And this time it comes with a picture of a tool on the cover that I’ve actually used before – but I must admit (so far) never in a work related situation!
Today is a big day for the engineering teams at Microsoft working on R3. We are reaching the important ZBB milestone. ZBB is one of the final internal checkpoints before releasing. It signals that there are zero active product defects known and planned for the release. Reaching ZBB is by no means an easy win. It requires maintaining a bug fix rate higher than the bug discovery rate for a lengthy period of time. It also requires significant coordination and team collaboration to have all teams reach this criteria at the same time.
One part of reaching ZBB is to have zero best practice errors across the entire code base. I’m happy to announce that R3 will ship with zero best practice errors despite the significant increase in code base.
Once again; I’m awestruck by the energy, effort, diligence and dedication demonstrated by the Microsoft Dynamics teams to bring this important release to market.
More to follow, as the release draws nearer…