Microsoft Dynamics AX Tools

To do the job right you need the right tools.

How the X++ compiler works for AX2012

Rate This
  • Comments 7

By request, today's blog attempts to explains the sequence of phases that are performed by the X++ compiler in AX 2012.

The compile process is a little bit involved, so a drawing quickly becomes quite complicated. Instead we've tried to present the explanation in two modes that compliment each other:
A. C#-ish pseudocode
B. Corresponding paragraphs of verbiage

C#-ISH PSEUDOCODE

// Phase1: Limited compile only for metadata
// (of class declarations and method signatures).

foreach(Element in AOT)
{
       Element.CompileHeaders();
}

// Phase2: Full compile for metadata and code bodies.

foreach(Element in AOT)
{
       compileSuceeded = Element.Compile();

       if (compileSuceeded == false)
       {
              ListOfBadElements.Add(Element);
       }
}

// Phase3 to PhaseN: Recompile the errored elements
// until success or error stabilization.

if (ListOfBadElements.Count > 0)
{
       while(true)
       {
              foreach(Element in ListOfBadElements)
              {
                     compileSuceeded = Element.Compile();
                     if (compileSuceeded == false)
                     {
                           NewListOfBadElements.Add(Element)
                     }
              }
             
              if (ListOfBadElements.Count == NewListOfBadElements.Count)
              {
                     break; // The list of elements with errors has been stabilized.
              }

              ListOfBadElements = NewListOfBadElements.GetCopy();
              NewListOfBadElements.Reset();
       }
}

VERBAL EXPLANATION

During a complete AOT compilation, there are at least two full compilation phases, plus the potential for N iterations of Phase3.

Phase1 is a pass over all the AOT elements. For now the compile operations concern only the class declarations and the method signatures. These limited compile operations create metadata about the types and their methods.
Reference errors are ignored during Phase1 because they are uninteresting consequences of the compile sequence. For example, the first class that is compiled might reference another class that has not quite yet been compiled, but which will be compiled a moment later in Phase1.

Phase2 is a pass over each AOT element to perform a full compile of each. The compiler validates all metadata, all references, and all X++ code. The compiler emits p-code (which the interpreter can later execute).
Any AOT element that suffers an error is added to a list of bad elements. In the explanatory pseudo code this list collection is named ListOfBadElements.

Phase3 occurs only if ListOfBadElements is not empty at the end of Phase2.
Phase3 is a pass through ListOfBadElements, so that each previously bad element can be recompiled. The elements that fail this recompile are added to a new list that is named NewListOfBadElements.
At the end of Phase3 there are three possible outcomes:

A. NewListOfBadElements.Count == 0
This means the overall compile has completed successfully. Error log will be empty. There might be warnings or informational messages (TODOs)

B. NewListOfBadElements.Count == ListOfBadElements.Count
This means the set of errors has stabilized (at a number above zero), which means the overall compile has failed. Error log will contain the errors, warnings and informational messages.

C. NewListOfBadElements.Count < ListOfBadElements.Count
This means that Phase3 made useful progress in reducing the count of errors. Therefore another phase like Phase3 must be run. This leads to Phase4, Phase5, and PhaseN for as many iterations as progress continues.
The more elements the AOT contains, the more likely it is that iterations beyond Phase3 will be needed.

AND THEN SOME...

Also note that the compile pass is doing more than crunching X++ code into executable p-code. It's performing a vast number of metadata validations and it's updating the security information by running the Security Inference process on applicable constructs to update the security information available. 

Leave a Comment
  • Please add 4 and 8 and type the answer here:
  • Post
  • So, why can't the compiler be multi-threaded for the 1st pass? If it is just capturing metadata, why can't it capture it while running in more than 1 thread?

  • @Michael Franchino:

    You are right, from an algorithmic perspective there are definitely options on the multi-threading side. The "easiest" one (not necessarily "easy" though due to intrinsic state that helps optimize the process already) is the first pass, which unfortunately isn't bringing vast improvements as it's the least of the work. If you look at the IL generation code in AX, you'll see some of this parallelism already in place.

    Retrofitting parallelism and making single-threaded implementations MT safe isn't a straight forward exercise. That's not the same as saying we're not looking at this though :)

  • Thanks for sharing this. *thumbs up* :D

  • Sounds good.. Compile has always been a real big pain, especially if the customer doesn't have the horsepower to do it.. Where I am working now, we have the compile running in about 45 minutes.. At my previous customer, it took 6 hours.

    Also, is there a way to have the compiler running on a separate thread as the client itself? Or having an external tool to do the compile so that at least you can see if it is still moving and where it is?

  • @Michael Franchino: Stay tuned, we've got more improvements in the works along those lines :)

  • Just got my new dev box going (2012 RTM).  64GB of memory and 2 SSD RAID 1 drives.. All in one. Cached raid controller.. 37 minutes to compile AX.. We timed it, pass one took 20 minutes.  

  • Michael - what is the spec (CPU details) of your new dev box?

Page 1 of 1 (7 items)

How the X++ compiler works for AX2012