June, 2008

  • Microsoft Dynamics NAV Team Blog

    The random events

    • 0 Comments
     Some years ago I was dealing with an issue that was driving me crazy and it took many hours to figure out what was wrong with the code. It was until I was explaining my findings to a colleague (don't you solve lots of things this way?) that it hit me, and when looking at the original (NAS) code I confirmed what was wrong.

    What was more irritating was that the system was working most of the time, and, of course, there was no way to consistently reproduce the issue.

    I will explain the issue here, hoping that people don't make the same mistake again.

    The problem

    Think about a single instance code unit which is receiving an event in order to process the incoming data (this was using the ComCom objects), and, in order to catch possible errors in the AL execution, a NAV Timer (Navision Timer) is used to reply successfully or with an error to the receiving event initiator (the COM component that started the conversation).

    Normal flow (from the NAS perspective) was:

    • ComCom2::MessageReceived would be invoked.
    • GlobalVal, a global variable in the single instance Codeunit would be filled with the incoming data to be processed.
    • NTimer would be enabled, in order to fire relatively soon (I think it was something like 20 milliseconds).
    • NTimer::Timer would fire and would get the value from GlobalVal to process the data and reply to the connection (by NAV architecture, this connection had to be a new one, as, when the MessageReceived event finishes, the ComCom connection drops).
    • If any AL error would arise within the NTimer event, the NTimer::TimerError would be triggered, which would allow us to reply an error to the calling component (using the GlobalVal to identify which connection would have to be informed about the problem).

    Sounds reasonable, and in most cases this would work great. However, people were experiencing mixed up answers especially when errors were to be reported.

    Of course, my first thought was that the component connecting to the NAS was the guilty one, as, after all, it is a multi-thread, multi-connection component (it runs as a Windows Service), and getting all this threads straight is always a tricky business. However, after inspecting the components code, and correcting some issues, the problem was still there.

    I decided it was time to get the hands really dirty and started making my own component which would simulate the system and start hammering the NAS with requests using multiple threads and inspecting the replies, and sure enough, especially when the NAS would error, the replies would contain a different ID than the original sending ID on the same virtual connection.

    So, there must be something in the NAS that does not work correctly. More code inspection, but nothing showed up until.... Hey, wait a second... We are using a global variable to save data and then reusing this data later on when the NTimer event fires, and, since the NAS is single threaded, this should all work fine, right?

    Well, of course the first thing that came into my mind was that there was a possibility that a new incoming event would be "queued" before the NTimer event would be scheduled, and this would mean that a new MessageReceived event would be triggered before the NTimer could even process the previous message. Bingo! That is the issue!

    After further investigating the NAS code, it turns out that, although the previous scenario is possible, it actually does not queue sequentially. Incoming events are basically random when the host COM cannot handle the call. I could go very deeply here, but what happens is that, when the NAS COM event is called, the NAS actually checks to see if it is already handling another incoming event. If this is the case, it signals that it cannot process the incoming event (via a RPC_E_SERVERCALL_RETRYLATER) and COM basically waits "some time" before it retries the call. The problem is that this "some time" is not known and it changes, and that is when the trouble really starts.

    Basically, what happened in our scenario was that 2 consecutive MessageReceived events will fire (before the first NTimer would) and obviously one GlobalVal would be lost and we would reply twice  to a single event (the NTimers would fire, but with the same GlobalVal).

    How to solve this

    The cure was simple. When the MessageReceived event is fired, we process the information and reply to the event on the same connection. After all, ComCom objects do report back AL errors in XML format, so, in principle, there is no need to complicate matters.

    So, after getting rid of the NTimer, and replying to the incoming request within the MessageReceived event, everything works as expected.

    Jorge Alberto Torres (jtorres)
    Software Developer Engineer

  • Microsoft Dynamics NAV Team Blog

    Jobs Update for Microsoft Dynamics NAV 5.0 SP1 is now released

    • 0 Comments

    I am pleased to announce the release of Jobs Update for Microsoft Dynamics NAV 5.0 SP1. This update is now available for you to download from Partner Source. Login credentials are required. https://mbs.microsoft.com/partnersource/downloads/releases/MicrosoftDynamicsNAV50SP1.htm

    Jobs update rollup corrects the issues that are described in KB article number 954191.

    Mtoo Norrild
    Program Manager

     

     

  • Microsoft Dynamics NAV Team Blog

    Overview of platform updates for NAV 4 SP3

    • 0 Comments

    For a table of platform updates and hotfix history for NAV 4 Sp3 with build numbers, update numbers and file contents, follow this link.

  • Microsoft Dynamics NAV Team Blog

    E-mail logging and how to get the QUEUE folder filled with message automatically in Exchange 2003 and Exchange 2007

    • 2 Comments

    The E-Mail Logging feature is a service that offers users e-mail integration between Microsoft Outlook and Microsoft Dynamics NAV. This allows the user to maintain current e-mail records in both systems. It also gives the user the ability to share and publish knowledge about external contacts.

    This is how it works: The e-mail dispatcher ("watchdog") monitors a specified mail folder, which is referred to as the Queue folder in this blog, on the Exchange Server. When an e-mail is retrieved from the Queue folder by the dispatcher, it is then placed in another folder – referred to as the Storage folder in this blog. The dispatcher logs the interaction in the Interaction Log Entry table in Navision. If the sender or the receiver is unknown to the system (no e-mail address is registered in the system), the e-mail will not be registered/logged in Navision and will not be stored in the queue or storage folder in Outlook. If the e-mail is marked with an option other than Normal, such as Private, Personal or Confidential, it will be deleted.

    There is a very old whitepaper that discusses this (Navision40_IntegrationwithOutlook_Technic.doc). It was written in the Exchange 5.5 days. This paper also discusses the way how to automate the Queue folder. Here is an extract from that paper:

    There are several ways to get an e-mail copied to the Queue folder for further handling. The first method (Message Archival) involves using the E-Mail Logging feature on the Exchange server. In order to use the other methods they must be set up in each individual client Outlook installation. These methods offer various degrees of automatic logging.

    · Message Archival. The Exchange Server can be set up to log all traffic and forward the e-mails to a specified folder or mailbox, which should then be used as the Queue folder. To configure the Exchange server to enable Message Archival, refer to the link below:

    Exchange 2000 and Exchange Server 2003
    http://support.microsoft.com/support/kb/articles/Q261/1/73.ASP

    · Setting Up Rules in Outlook. You can also set up rules in Outlook that are triggered by sending and/or receiving an e-mail. These rules are based on user-defined conditions. You can set up the rules to allow specific e-mails to be copied to the Queue folder. To ensure that all outgoing e-mails are placed in the Queue folder.

    When you define these rules, make sure that they are the first rules that are applied so that other rules do not override them.

    · Creating a Button. Another way to use the client to log your mail is to create a button you can click to have your e-mails copied automatically to the Queue folder. See the whitepaper for detailed instructions on how to create this button.

    · Setting Up an E-Mail Logging User. You can set up an individual user on the Exchange Server. Make sure the user’s Inbox is selected as the Queue folder. Other users can simply add this recipient as a “BCC” on outgoing e-mails that they want to log. Note that this only works for outgoing e-mails and should, therefore, be used in combination with the other methods described above.

    One would expect this to work in Exchange 2007 the same way it worked in Exchange 2007. Well, there is news. Message Journaling in Exchange 2007 is much powerfull then it ever was. Good news you as a NAV admin would think. Well, there are many reasons why one would use Exchange 2007 Message Journaling over Exchange 2003 Message Archival. There is however one big reason for a NAV admin NOT to use is and that is that it breaks E-mail logging because of the following reasons: "It is all about security".

    By default, all communication between computers running Exchange Server 2007 in the same Exchange organization is encrypted. This encryption includes journal reports. Exchange Server does a number of things to help reduce the risk of journal reports being tampered with:
    Secure links are used between Hub Transport servers and Mailbox servers in the Exchange 2007 organization.
    Journal reports are sent as "Microsoft Exchange" on behalf of the sender of the original message.
    Sessions between the Hub Transport server and Mailbox server are authenticated.
    Only authenticated connections are accepted when journal reports are sent between the Hub Transport servers and the Mailbox servers in the same Exchange 2007 organization.

    The workaround is to create two Transport Rules. To do so, please start up Exchange Management Console, expand Organization Configuration, Hub Transport. On the left side pane of the screen, below Actions / Hub Transport, click on New Transport Rule. Follow the wizard and generate the conditions. For the Transport Rule to be effective for E-mail logging, you need to select "sent to users inside or outside the organization" and make sure it has Inside selected and when you create the second Transport Rule, you again need to select "sent to users inside or outside the organization" but then make sure it has the Outside selected. BCC the message to the Email logging account. Using BCC will copy just the message to the folder where it can be processed by the NAS as usual. By modifying the rules yet further you can reduce the set of messages copied to the folder so that the NAS is yet more efficient than it was using exchange 2003 message logging as the number of messages to parse can be easily reduced

    This again makes the Exchange 2007 a far more better powerfull application then all other releases of Exchange Server!

    Marco Mels (mmels)
    Microsoft Dynamics NL

  • Microsoft Dynamics NAV Team Blog

    NAV 2009 - How to - Simple Form Transformation

    • 4 Comments

    This post describes the simplest possible way to transform a form into a page in NAV 2009. Both Pages and Transformation is documented in more details elsewhere. Here, I just want to make the simples possible example, just to get a simple form transformed into a simple page.

    Prerequisites:

    You need to have at least a NAV 2009 classic client and the Transformation tool. To test the page, of course you also need the Rolebased client. But to just transform a form, you don't need it.

    Import the tool:

    1)  Open a classic client, go to Object Designer and import the file TIF.fob from the folder TransformationTool\TIF Editor\
    2)  Create a new form in NAV, as simple as possible. For example a Card form based on the customer table, with just a few fields on it. This is the form you want to transform.

    Setup and use the Transformation tool:

    3)  Run form 177000 "Transformation Forms"

    4)  Click Import -> "Import PageType - FormType mapping", and select the file TransformationTool\TIF Editor\FormToPagetypeMapping.txt.

    5)  Click Functions -> Get Forms. Filter on Type = Form, and ID = the new form, then click OK. This gets the form(s) that you want to transform.

    6)  In the FormType field, select "Card" ("Form with TabControl only and a source table").


    Run the transformation:

    7)  Export the settings from form 177000: Click on Export -> "Transform Pages". You must export it to the TransformationTool folder, and it must be called TransformPages.xml.
    8)  From Object Designer, export the form (Tools -> Export) as xml.You must export it to the folder TransformationTool, and it must be called Forms.xml.

    You now have the form you want to transform, and the meta-data that tells the TranformationTool how to transform it.

    9)  Run the file FormTransformation.exe from the TransformationTool folder. This generates a new file called Pages.xml. It also logs any progress or errors in the file Transformation.log.

    Import the new page:

    10) Back in NAV (classic client): Go to Object Designer, and import the file Pages.xml, and then compile it.

    If all steps ran without any errors, you have now transformed your form into a page which can be displayed in the rolebased client.

     

    Lars Lohndorf-Larsen (Lohndorf )

    Microsoft Dynamics UK

  • Microsoft Dynamics NAV Team Blog

    Modern NAV/SQL troubleshooting II

    • 1 Comments

    Please refer to this post about what I mean with "modern troubleshooting".

    This post describes methods that work on any version of SQL Server, including SQL2000. It describes one of the most common questions I get, which is "Where do we start"...

    General performance problems - where to start:
    If a system is suffering general performance problems, then it is not always easy to decide what to do first, or where to start. Then you may be tempted to collect lots of information, for example Profiler traces and databases which may or may not give any results, but which is guaranteed to require a lot of work both to collect, to send, and to analyse.

    Often, over-indexing is one of the main causes of both bad performance problems and blocking. Indexes take time to maintain, and updating an index causes locks, which in turn can contribute to blocking-problems. so in a cases where the problem can't be located to a specific action, one place to start is to do some index tuning.

    In a case like that, I often begin by asking for just two files:

    • Excel spreadsheet containing Table Information
    • A NAV backup or fob file which contains the customer's objects, but not their data

    Each of these files don't take long to prepare, and are normally small enough to send by email.

    You collect the Excel spreadsheet like this:

    1. In NAV, go to File -> Database -> Information, and click on Tables.
    2. Copy this into an Excel spreadhseet.

    When you get the spreadsheet, sort it by "No. of Records", or by "Size (KB)". The difference is, that "Size (KB)" includes the size of the indexes on each table. So either way of sorting gives you a good idea of which tables are being updated the most, and which tables have lots of both records and indexes.

    Then load the NAV objects that you also received, and for the top 5 - 10 tob tables in the spreadsheet, take a look at the tables in NAV, and check the number of keys and SIFT-indexes. Some times you will immediately see that these tables have had a lot of keys added, and you should try to see if you can reduce the number of keys, using the key properties MaintainSQLIndex, MaintainSIFTIndex and SIFTLevelsToMaintain (remember the SIFTLevelsToMaintain-property don't exist any longer in NAV 5 SP1 where it was removed as part of re-designing the SIFT system).

    Which indexes to disable the SQL-maintenance of:
    Only if you know how the system is being used, can you tell whether a certain index is being used or not. But, for example, a key like "IC Partner Code" in table 17 is only ever useful if the customer is using Intercompany Posting, and the "Additional-Currency Amount"-sumindex field on the same table is only needed if Additional currency is being used. So some times, a few obvious changes can be done. But to really do some index tuning, you need to know more than the objects alone can tell you.

    You can also take the opposite approach, and disable maintenance of all indexes and SIFT (except for the clustered index). This will show you which ones are really needed when certain processes begin to run very slowly, and you will then have to (quickly) re-enable those. This is of course a risky approach, but it can be quicker than analyzing each individual key.

    In deciding which indexes may be disabled, the following tools may help as well:
    If running on SQL Server 2005 or later, run the query "Index Usage" (follow this link)

    The Key Information tool on the SQL Server Resource kit can help you deciding the cost of indexes per table, and help deciding the usefulness of SIFT indexes.
    With Navision Developers Tool (NDT), you can run "Where Used" on a key, to see in which object it is being used. If you use this method, then make sure to select everything in the "Where Used"-options. And keep in mind, that even with everyhthing selected, the NDT cannot see if users are manually specifying certain keys on the forms and reports they are running.

    Lars Lohndorf-Larsen (Lohndorf)
    Escalation Engineer

  • Microsoft Dynamics NAV Team Blog

    Microsoft Dynamics NAV 5.0 SP1 and SQL Server 2000

    • 0 Comments

    Microsoft Dynamics NAV 5.0 SP1 introduces a new way to handle SIFT. Instead of maintaining totals in separate tables, Dynamics NAV 5.0 SP1 uses a SQL feature called indexed views. Indexed views will automatically be maintained by the SQL Server.

     

    With SQL Server 2000, updating an indexed view can be a time consuming process as the SQL Server 2000 might decide to include a clustered index scan as part of its query plan to update the view. If your Microsoft Dynamics NAV implementation includes tables with many records and is based on Microsoft SQL Server 2000 there is a potential risk for experiencing bad performance.

     

    Microsoft Dynamics NAV 5.0 SP1 is optimized for Microsoft SQL Server 2005 and you should consider to upgrade to SQL Server 2005 if you are implementing Microsoft Dynamics NAV 5.0 SP1.

     

    The problem described here, only applies to SQL Server 2000. SQL Server 2005 handles the updates of indexed views much more efficiently.

     

     

    Lars Lohndorf-Larsen (Lohndorf)
    Escalation Engineer

Page 1 of 1 (7 items)