In Microsoft Dynamics AX 2012 we have solved the element id problem. The solution is outlined in this post. The solution consists of major changes both having an impact on existing AX 2009 and AX 4 solutions:

1. Element IDs (like class ID and table ID) are now 32 bit.
2. Element IDs are installation specific. The same class will have different IDs on different installations.

When making IDs 32 bit we had to deprecate the TypeId() function. TypeId() took one argument the name of an Base Enum or Extended Data Type. And it returned an integer value, where the upper 16 bits were the ID, and the lower 16 bits denoted if the type was a Base Enum or an Extended Data Type. For example: TypeId(MyEDT) returned 0xC351 000B = (50001 << 16)+11. Naturally such a function wouldn't work with 32 bit IDs. It turned out, that we didn't need a replacement for the TypeId() function, as the name of Base Enums and Extended Data Types are unique across both types. The solution then simply become to update the framework methods that accepted the combined ID from TypeId() with a new name based implementation. And to make it better, it turned out that usage of TypeId() was restricted to very few frameworks. The table below shows a complete list of the usage patterns of TypeId() we found in the SYS layer, and how we updated them. When you are upgrading the compiler will detect all reference to TypeId() for you.

Dynamics AX 4/2009              Dynamics AX 2012
// Dialog related
// When adding fields to a dialog, the types are now identified by Name – not ID.
dialog.addField(typeid(MyEdt));             dialog.addField(extendedTypeStr(MyEdt)); 
dialog.addField(typeid(MyEnum));            dialog.addField(enumStr(MyEdt)); 
dialog.addFieldValue(typeid(MyEdt));        dialog.addFieldValue(extendedTypeStr(MyEdt)); 
dialog.addFieldValue(typeid(MyEnum));       dialog.addFieldValue(enumStr(MyEdt)); 
new DialogField(dialog, typeId(MyEdt));     new DialogField(dialog, extendedTypeStr(MyEdt)); 
new DialogField(dialog, typeId(MyEnum));    new DialogField(dialog, enumStr(MyEnum)); 
dialog.addField(types::string);             dialog.addField(identifierStr(str)); 
dialog.addFieldValue(types::string);        dialog.addFieldValue(identifierStr(str)); 
new DialogField(dialog, types::string);     new DialogField(dialog, identifierStr(str)); 
// Dict related
DictField.extendedTypeID() == typeid(MyEdt) DictField.typeID() == extendedTypeNum(MyEdt) 
DictType.extendedTypeID() == typeid(MyEdt)  DictType.id() == extendedTypeNum(MyEdt) 
// Args related
args.parmEnumType(typeId(myEnum))           args.parmEnumType(enumNum(myEnum)) 
//Type related
typeId2Type(typeId(myEdt))                  typeName2Type(extendedTypeStr(myEdt)) 
// Other patterns with better alternatives
// These alternatives can also be used in AX 4 / 2009
typeId(myEdt) >> 16                         extendedTypeNum(MyEdt) 
typeId(myEnum) >> 16                        enumNum(MyEnum) 
typeId2ExtendedTypeId(typeId(myEdt))        extendedTypeNum(MyEdt) 

As element IDs now are installation specific then we need to provide enough information for data export/import to convert the data correctly between the source and target system. There are 3 things to do:

1. Do not store IDs in containers,
   If IDs are stored in a non-relational format (like a container), then data export/import cannot identify them, and the IDs will be imported as-is; which means they will resolve incorrectly on the target system. A best practice rule is available to identify IDs in containers.
2. Use the right Extended Data Types on your fields (TableId, FieldId, ClassId, etc.)
   Data export will convert data using these Extended Data Types into a format that can be correctly resolved on the target system.
3. For fields containing field IDs

Set the RelatedTable property, OR Create a new field group with the field referencing the table and the field referencing the field. For data export/import to be able to correctly convert a field ID, it is required that you specify which table the field belongs to. If the field always belongs to the same table, you can use the RelatedTable property to specify it (3.1), OR if the field ID is referencing a field in an arbitrary table, you need to link the table reference and the field reference to each other. You do this by creating a new field group containing the two fields (3.2)

THIS POST IS PROVIDED AS-IS AND CONFERS NO RIGHTS.

About 3 years ago heading into the design phase of AX 2012 I wrote the Solving the element ID problem post. As the post describes element IDs in AX have constituted an inherent problem in all past AX releases. The problem is so fundamental that when I met with the original Axapta architects (at the Damgaard Data 25 years reunion party) they considered it insolvable without a major rewrite. In AX 2012 we did solve the problem. This post outlines the solution.

Design pillars

Before I dive into the solution these are our design pillars:

1. Backward compatible. We cannot introduce any inconsistencies in business data or meta data.
2. Minimal impact on customizations. The uptake of required changes must be minimal and easily absorbed when upgrading to AX 2012.
3. Completed by AX 2012 RTM. We do not desire a staged multi-release solution to this problem.

The solution part A - 32 bit IDs

First we needed to break the upper limit of 60.000 IDs. We decided to make IDs 32 bit (used to be 16 bit). The primary deciding factor for int32 (and not int64 or guid or str) was that there would be no X++ uptake cost of using an int32, as the int data type in X++ already is a 32 bit integer - in other words all X++ code (and table fields) handling/containing IDs would simply just continue to work untouched. In the SYS layer we have >500 fields containing IDs. Not having to convert these (e.g. to GUIDs), and provide upgrade scripts is highly desirable. 

As the ID for table fields already were 32 bit (16 bit for the element ID and 16 bit for the array index) we decided to keep the field IDs as is. In other words the element ID for fields are still just 16 bit. Given these are scoped by the table, there are no upper limit concerns involved here.

Besides a lot of kernel refactoring there was one significant X++ impact. The intrinsic function TypeID had to be deprecated. It returned a 32 bit integer where the lower 16 bits contained the Type (either Types::Enum or Types:UserType), and the upper 16 bits the element ID. This function was primarily used in relation to the Dialog framework. The solutionwas to change the dialog framework to use the name of the type instead of the bit mangled integer.

The solution part B - Installation specific IDs

To solve the uniqueness problem, we need to ensure that two (or more) solutions can be implemented independently, and installed on the same system without any ID conflicts. In AX 2009 the assignment of IDs happens at creation time (i.e. when the element is created in the AOT). In AX 2012 we are deferring the assignment of IDs to installation time. This means that the model file containing the elements does not contain any element IDs. The IDs are being assigned when the model is imported by AxUtil. The consequence of this is that the same class will have different IDs on different installations. We call it "Installation specific IDs".

Caution: Deleting a model and reimporting it (or a newer version of it) will randomize IDs and thus cause data integrity issues. The right procedure when upgrading a model is to not delete the model, but just import on top of the existing model.

Given the requirement to be backwards compatible we must ensure that elements released in AX 2009 (and AX 4) retained their IDs. To satisfy this requirement we introduced a new int property on all ID based elements: LegacyID, and assigned it the ID value from past releases. Now we have captured the ID the element used to have in a simple (and editable) property, and we can use it during import.

As it is vital that an element retain its ID across releases, we need to find a way to support rename scenarios. Consider the scenario where a new AX 2012 class is renamed in AX7. As the class is new it will have an installation specific ID; once the AX7 model is being imported we need to retain the ID - but if the only match criteria is the class name, and the class has been renamed, then we cannot fulfill the requirement. To solve this we introduced a new guid property on all ID based elements and all root-elements: Origin. This property is set when an element is created, and remains static for the lifetime of the elements. It is the element's fingerprint.  Besides enabling invariant IDs across releases for renamed elements the Origin property has proven valuable in data export scenarios, and rename scenarios of newly fine grained meta data (like forms). This will be covered by another blog post soon.

AxUtil will during import assign IDs based on these rules:

1. If an element already exists with the same Origin, then replace the element and reuse its ID - else
2. If an element already exists with the same Type, Name and ParentID, then replace the element and reuse its ID - else
3. If the import element has a Legacy ID, and the LegacyId is available on the target system, then add the element setting ID = LegacyID - else
4. Assign a new installation specific ID from a guaranteed free range that will not collide with any LegacyIDs (>60000 for fields, >1000000 for all other elements)

Notice: IDs are no longer tied to layers, e.g. Layers no longer have an ID range. Nor do you need the Team Server (aka. ID Server) anymore when using a version control system.

Data can be exported from one installation to another. It is vital that any ID on the source system is mapped to the corresponding ID on the target system. The data import feature has been improved so all columns containing element IDs will be converted automatically (given they are using the proper extended data type). However; we also needed to address eventual IDs in blobs (containers). Data import and model import has no insights into the structure of the blobs. The only solution to this is to replace all ID-based element references in blobs with name-based references. We have done this for:

• All kernel classes returning a container - e.g. Query.pack(), QueryRun.pack(), Map.pack(), Set.pack() etc.
• All X++ classes packing element IDs (typically Runbase derived classes). A best practice rule is in place to detect these.
• All references in meta data - e.g. when a form data source refers to a table.

Wrapping up

To solve the ID problem we had to introduce two new id properties - LegacyID and Origin. This may sound like a step in the wrong direction; however, the nature of these new properties are less restricting than that of the element ID property. In fact the only restriction on the new properties is that Origin must be unique - given Origin is a guid that is easy to satisfy. This restriction is enforced at the database level.  All in all the changes outline above has been a significant investment to solve this inherent problem in the least intrusive way possible. Yet it has an impact. My next blog post will describe the impact.

The most important message is: If you are a new Microsoft Dynamics AX 2012 customer or partner, you do not need to worry (or care) about element IDs.

Microsoft Dynamics AX 2012 is RTM. This means final build of the product has left the R&D team and will soon be general available. Today, you can already access MSDN documentation for Microsoft Dynamics AX 2012 for Developers. 

My favorite features in AX 2012:

• Models - multiple solutions in the same layer.
• X++ code running as IL.
• New X++ editor.
• Table inheritance - just like for classes.
• Layer based managed code customization - bring your C# project into the AOT.
• Role based user interface - beautifully simple and simply beautiful.
• New security model - with 80 predefined roles.
• Online help server - allowing customized help contents.

I'm proud to have been a part of the extraordinary team building this extraordinary product, executing on a 3 year plan and delivering on time. Since you are reading my blog, I feel confident this product will have a positive impact on your professional life. Starting today!

Recently; Nikolaj Bjørner (Senior Researcher, Microsoft Research) shared interesting insights and perspectives on software engineering at MDCC. Getting a regularly injection of food-for-thought is something I appreciate about my job. And as always... we are hiring.

The Research in Software Engineering (RiSE) group at Microsoft Research in Redmond works on several dimensions related to software engineering, including logical foundations, program verification, testing, analysis, empirical software analysis, compiler tools, program optimization, synthesis and model-based software design. This talk describes several of the RiSE groups activities, which are roughly three-pronged: divided between Microsoft product groups, with external developers and with academic interactions.

A significant part of our work involves tool building. The http://rise4fun.com web-site provides an interactive environment where several of the tools developed in RiSE are showcased through interactive demonstrations. I will walk through several of these tools during the talk showing by example our work on graph drawing (AGL), satisfiability modulo theories theorem proving (Z3), program verification (Boogie and VCC), test-case generation using dynamic symbolic execution (Pex), model-based design (FORMULA), symbolic analysis of regular expressions (Rex) and transducers (Bek). I will also describe and present a new tool TouchStudio (http://touchstudioapp.com) that brings a radically new software development environment to and on the Windows Phone.

It is a new programming environment and language built around the new reality of mobile devices with advanced touchscreens, sensors and cloud connectivity.

This short recording shows an alternative appliance of models. If you have already upgraded, wouldn't it be nice to not see all the "DEL_" tables, fields and indexes in the AOT? With models you can do exactly that. All the upgrade-only artifacts are stored in a separate model, and you can simply uninstall it to rid your system of the several thousand elements you really don't care about (anymore).

At the Dynamics AX 2011 Technical Conference I was asked in the hallway: "Was it a boy or girl?" I guess I look pretty perplexed, so the follow up was: "Since your blog has been quiet most of the fall, I assumed you were on paternity leave." It is true that I haven't blogged much in the fall of 2010, it is also true that I have been tending to my baby - but it was neither a boy nor a girl. It was Microsoft Dynamics AX 2012. The feature showcased in this video is what I spent most of my awake time on in October and November 2010. 

This video is also available on Channel 9.

THIS POST IS PROVIDED AS-IS AND CONFERS NO RIGHTS.

This article applies to Microsoft Dynamics AX 2012.

The error message ”Internal error 25” is a generic error message that occurs when one type is trying to be converted into another incompatible type at runtime.

This error can be triggered in a myriad of different situations. However; there are three likely root causes to this.

1. A programmatic error

In the ideal world the X++ compiler would detect any illegal X++ construct, that could lead to conversion between incompatible types. There are unfortunately a few situations where the X++ compiler doesn’t have enough information to do so. In X++ the type anytype can be used to denote any type in the system. Whenever the compiler sees a variable, parameter or return type of this type, it will not perform any type checking.

This means that the code using the anytype type must explicitly verify the variable/parameter/return type is compatible with how it is being used. Failing to do so can lead to an “Internal error 25”.

Consider this code:

It will result in an “Internal error 25”. The function any2str() accepts an anytype as parameter, so the compiler will not perform any checking. However, the implementation of any2str() doesn’t support that a System.String (CLR Object) is passed in – and thus this error. As an X++ developer the only remedy is to work around the issue. In above example by assigning the System.String to a str (X++ native type). That will work because the interpreter’s assignment implementation do support conversion of System.String to str. The code then becomes:

One might argue that the interpreter’s assignment conversion should have solved the issue when using any2str() too. That unfortunately cannot be done, as the interpreter will not perform any conversion as the receiving side claims to support anytype.

 2. Stale data

The type to convert might originate from data in the data base. If the data has been stored in one format and is being retrieved in another, it could lead to this situation.

Things to try:

1. Synchronize the data base
   Right-click AOT | Data dictionary and select “Synchronize”
   
2. Delete Usage data
   Tools | Options | Usage data. Click ‘Reset’
   
3. Debugging
   Debug the steps leading up to the error, and correct the affected data – typically by deleting it.

3. Stale pcode

The compiler produces pcode from the X++ code. This pcode is stored in the model store, and thus imported to the model store via AXUtil import (or AOD import). Given pcode often is referencing meta data from other models, the model store needs to be recompiled when new models are added or removed. Until the model store has been recompiled the interpreter may interpret metadata incorrectly, which can lead to “Internal error 25”. AXUtil will inform you about this after import, and so will Dynamics AX the first time a client is started.

The steps to take are:

1. Restart AOS
2. Start AX client
3. Select “Launch model upgrade checklist” or “Compile and Synchronize” in the form that automatically opens.

Given the nature of the problem, the internal error 25 can occur before the system has had a chance to open the form automatically. This typically occurs when one of these classes are customized: Application, Infolog or ClassFactory.  It can also occur, if elements referenced by these classes are customized.

To solve this issue follow these steps:

1. Restart AOS
2. Start AX client in safe mode
   In command line type:  AX32.exe -noauto
   The –noauto option will prevent the system from running any pcode during start up, including the pcode that is triggering the “Internal error 25”, and the pcode that is launching the compile form. The client will start directly in the developer workspace to further prevent any application logic to be run.
3. Right-click the root of AOT and select “Compile”  or compile select portions of the AOT
4. Resolve any compilation errors that could trigger a problem during start up.

This is a sneak preview of two new tools in Microsoft Dynamics AX 2012: The Type Hierarchy Context and the Type Hierarchy Browser.

Please notice:

• Microsoft Dynamics AX 2012 supports table inheritance
• These tools runs off live meta data - no need for cross reference to fuel them
• These tools improves the navigation and discoverability of meta data

This video is also available on Channel 9.

THIS POST IS PROVIDED AS-IS AND CONFERS NO RIGHTS.

Here is a sneak preview of the new X++ Editor in Microsoft Dynamics AX 2012.

Notice that the editor now features word-completion, automatic indenting, scripting, zoom, multiline editing, and much much more.

This post is also available on Channel 9.

THIS POST IS PROVIDED AS-IS AND CONFERS NO RIGHTS.

Here is a sneak preview of the new MorphX developer workspace in Microsoft Dynamics AX 2012.

Notice the clear distinction between developer and application workspaces, and how the layout of the drop down menus makes it possbile to access tools with fewer clicks. You can launch the AX client directly in development mode using: AX32.exe -development.

This post is also available on Channel 9.

THIS POST IS PROVIDED AS-IS AND CONFERS NO RIGHTS.

This long anticipated and quite comprehensive book is now available for download here.