Microsoft Dynamics NAV Sustained Engineering Team Blog : 2005 performance

Cursor Types

Lohndorf — Fri, 20 Feb 2009 11:29:00 GMT

One of the changes in Microsoft Dynamics NAV version 5, was to change from primarily making use of Fast-Forward cursor types to Dynamic cursors. The same change was implemented in version 4 from build 26410, which is Version 4, SP3 Update6(940718) + KB950920.

The change of cursor type can also mean a change in behaviour. With Dynamic cursors, SQL Server more often optimises for the ORDER BY - part of a SQL query than is the case with Fast Forward cursors. This is because a result set based on a dynamic cursor has to include new rows. IF SQL Server were to choose an index that fits the WHERE clause then it would have to sort all rows according to the ORDER BY before returning the first row to the client and that by definition is a STATIC result-set.

Take this query as an example:

SELECT * FROM "Demo Database NAV (6-0)"."dbo"."CRONUS International Ltd_$Item Ledger Entry"

WITH (READUNCOMMITTED)

undefined (("Document No_"=

ORDER BY "Item No_","Posting Date","Entry No_"

With Fast-Forward cursors, in this example SQL Server is likely to try to optimise for the WHERE clause, which is "Document No.". But with the ORDER BY clause specifying a different sorting, SQL Server may then chose a clustered index scan instead.

With Dynamic cursors, SQL Server is more likely to optimise for the ORDER BY clause, which is a valid and existing index. So in this exampe SQL server would chose an index scan on this index. You can see this by running the query from SQL Server Management Studio like this:

declare @p1 int set @p1=-1

--declare @p3 int set @p3=16+4096+8192 -- Fast Forward

declare @p3 int set @p3=2+4096+8192 -- Dynamic

declare @p4 int set @p4=1

declare @p5 int set @p5=49

--declare @p5 int set @p5=15 – FAST 15

exec sp_cursoropen @p1 output,N'

SELECT * FROM "Demo Database NAV (6-0)"."dbo"."CRONUS International Ltd_$Item Ledger Entry"

WITH (READUNCOMMITTED)

WHERE (("Document No_"=@P1))

ORDER BY "Item No_","Posting Date","Entry No_"

,@p3

output,@p4 output,@p5 output,N'@P1 varchar(20)',

'START'

select @p1, @p3, @p4, @p5

exec sp_cursorclose @p1

You can enable / disable the 2nd and 3rd line to switch between Fast-Forward or Dynamic cursor, and see the result in the query plan that gets generated.

How does this affect NAV?
The change in behaviour can mean that certain queries that ran without problems may be slow, after a customer upgrades the platform to the builds mentioned above. In cases

that we have seen, the problem has been limited to one or very few specific queries, typically on filtered forms, that were slow after upgrading the platform. Use the query from here:

Simple query to check the recent performance history II - now including Query Plan information

to help identifying which queries to trouleshoot. Note that the query shows the Cursor Type in the rightmost column. Then look at whether SQL Server has an index to match the ORDER BY clause.

Also, be extra careful using the SQLIndex-property on keys. By setting this property, the ORDER BY-clause may not match a SQL index anymore, and a Dynamic cursor will have

to scan the table.

Why this change?
Internal performance tests show that overall, Dynamic cursors give better performance and fewer blocks. So while the change may cause unforeseen changes in behaviour when a customer upgrades, overall we have seen better performance with Dynamic cursors.

Lars Lohndorf-Larsen

Escalation Engineer

These postings are provided "AS IS" with no warranties and confer no rights. You assume all risk for your use.

Simple query to check the recent performance history II - now including Query Plan information

Lohndorf — Tue, 14 Oct 2008 12:01:00 GMT

One of the queries I use the most, is the pplan-cache query from this post:

Simple query to check the recent performance history

The good thing about the query is that it shows information that could otherwise require a lot of work (collecting and analysing traces files). And the query does not need any advance work or setup. It can just be run. So it's a very easy way to receive information from a system, which is often very useful as a first step in troubleshooting performance. For more details about the result of the query, refer to the post linked above.

Below is a slightly enhanced version of the query. Since the query is based on the cache of compiled query plans, it is not a big step to extend it to also include the query plan itself, and even extract certain information from the plan if you know what you are looking for.

So this query does the same as the original one, but with the following additions:

New column query_plan is included. It shows the query plan as xml which may be difficult to read, but it contains the full plan. Note: Some times, for no apparent reason, the query plan can't be retrieved, so it may not show the query plan on all lines.
cursor_type, just as an example of how to retrieve data from the query plan. If you find other things in the plans that may be interesting, then use the syntax to retrieve this further information.

Here is the updated query

SELECT TOP 100

SUBSTRING(st.text, (qs.statement_start_offset/2) + 1,

((CASE statement_end_offset

WHEN -1 THEN DATALENGTH(st.text)

ELSE qs.statement_end_offset END

- qs.statement_start_offset)/2) + 1) as statement_text,

execution_count,

case

when execution_count = 0 then null

else total_logical_reads/execution_count

end as avg_logical_reads,

last_logical_reads,

min_logical_reads,

max_logical_reads,

plan_handle,

ph.query_plan,

-- Query Plan Information

case when

ph.query_plan.exist('declare namespace ns="http://schemas.microsoft.com/sqlserver/2004/07/showplan";(/ns:ShowPlanXML/ns:BatchSequence/ns:Batch/ns:Statements/ns:StmtCursor/ns:CursorPlan/@CursorRequestedType)[1]') = 0

then '' else

ph.query_plan.value('declare namespace ns="http://schemas.microsoft.com/sqlserver/2004/07/showplan";(/ns:ShowPlanXML/ns:BatchSequence/ns:Batch/ns:Statements/ns:StmtCursor/ns:CursorPlan/@CursorRequestedType)[1]','nvarchar (max)')

end as cursor_type

FROM sys.dm_exec_query_stats as qs

CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) as st

CROSS APPLY sys.dm_exec_query_plan(qs.plan_handle) as ph

ORDER BY max_logical_reads DESC

What I would really like, is to receive feedback on what parts of the query plans are useful. Then extend the query even further to include as much useful information as possible. For exdample, in some cases the query plan contains missing index-information. The lines below can be copied into the query above to include this information. Any feedback on whether this is useful or not, and whether other information from the query plans can be useful is really very welcome. You can add comments about this below.

-- Missing Indexes

,case when ph.query_plan.exist('declare namespace ns="http://schemas.microsoft.com/sqlserver/2004/07/showplan";(/ns:ShowPlanXML/ns:BatchSequence/ns:Batch/ns:Statements/ns:StmtSimple/ns:QueryPlan/ns:MissingIndexes/ns:MissingIndexGroup)[1]') = 0

then ''

else ph.query_plan.value('declare namespace ns="http://schemas.microsoft.com/sqlserver/2004/07/showplan";(/ns:ShowPlanXML/ns:BatchSequence/ns:Batch/ns:Statements/ns:StmtSimple/ns:QueryPlan/ns:MissingIndexes/ns:MissingIndexGroup/@Impact)[1]','nvarchar (max)')

end as missing_index_impact,

case when ph.query_plan.exist('declare namespace ns="http://schemas.microsoft.com/sqlserver/2004/07/showplan";(/ns:ShowPlanXML/ns:BatchSequence/ns:Batch/ns:Statements/ns:StmtSimple/ns:QueryPlan/ns:MissingIndexes/ns:MissingIndexGroup/ns:MissingIndex/@Table)[1]') = 0

then ''

end as missing_index_table,

then ''

end as missing_index_field

Lars Lohndorf-Larsen

Escalation Engineer

These postings are provided "AS IS" with no warranties and confer no rights. You assume all risk for your use.

Session Monitor for SQL Server 2005

Lohndorf — Thu, 11 Sep 2008 18:07:00 GMT

Session Monitor is not a new tool, but it did stop working with SQL Server 2005. So I think that a lot of people stopped using it, which is a shame. The things that were broken now work again, and I would recommend anyone with any kind of blocking problems to implement this tool. With this post I hope to get some people to use it again, and to provide a new tool to anyone who did not know it exists.

Purpose of the tool

The purpose of the Session Monitor tool is to show a live picture of any current blocks in the Microsoft Dynamics NAV database. And to show it in a familiar environment, i.e. a NAV form. It is an extension of the information you get when going to File -> Database -> Information, and drill down on Sessions.

What the tool does:

Shows live information about who is blocked by whom
Shows which user is at the top of a blocking chain
Shows if a user is involved in blocking, i.e. either being blocked or is blocking someone. So you can filter out all sessions that are not involved in blocking
Shows the wait resource, i.e. on which table the block is

This is how the Session Monitor shows a blocking situation:

As default it refreshes every second.

Some of the useful features of Session Monitor are:

Filter on "Involved in Blocking". Then, as soon as there is a block it will show up. This is especially useful when you have more sessions that can fit on the screen, which means that there might otherwise be a block outside of the form.
Wait Resource tells you on which table the block is, which helps you get some idea about in which area of the application it is.
Originating blocker ID shows the head-blocker. This is useful when a user is blocked by someone who is blocked by someone else, etc.
Wait Time shows how long the block has been there.

How to get it to work

As mentioned, the tool stopped working in SQL Server 2005. But there are corrections for this now. First, download the tool, which is part of the "Performance Troubleshooting Guide", available for download here (PartnerSource login required):

Microsoft Navision SQL Resource Kit

The original tool contains a SQL query (Session Monitor (SQL Server).sql) which is the part that doesn't work for SQL Server 2005. So don't run this query. Run the query from this KB article instead (PartnerSource login required here too):

KB 933042 - Error message when you use the Session Monitor feature in Microsoft Dynamics NAV: "Invalid length parameter passed to the substring function"

Note!: Run the query on the NAV database - not on Master, or any other database you have on SQL Server.

In NAV, import the object "Session Monitor (SQL Server).fob" from the Performance Troubleshooting guide.

Make one change - otherwise in some cases it will fail with an INSERT error:

Design codeunit 150011 "Session Monitor Mgt. (SQL Srv)", and replace this line:

RefreshUserActivity()
SELECTLATESTVERSION;
IF Session.FIND('-') THEN
REPEAT
SessionTmp := Session;
// SessionTmp.INSERT; Remove this line and replace with:

IF SessionTmp.INSERT THEN ; // New line
UNTIL Session.NEXT = 0;

Those are the changes needed to get it to run on SQL Server 2005,

Lars Lohndorf-Larsen

Escalation Engineer

These postings are provided "AS IS" with no warranties and confer no rights. You assume all risk for your use.

Modern NAV/SQL troubleshooting II

Lohndorf — Wed, 11 Jun 2008 18:21:00 GMT

Please refer to this post about what I mean with "modern troubleshooting".

This post describes methods that work on any version of SQL Server, including SQL2000. It describes one of the most common questions I get, which is "Where do we start"...

General performance problems - where to start:
If a system is suffering general performance problems, then it is not always easy to decide what to do first, or where to start. Then you may be tempted to collect lots of information, for example Profiler traces and databases which may or may not give any results, but which is guaranteed to require a lot of work both to collect, to send, and to analyse.

Often, over-indexing is one of the main causes of both bad performance problems and blocking. Indexes take time to maintain, and updating an index causes locks, which in turn can contribute to blocking-problems. so in a cases where the problem can't be located to a specific action, one place to start is to do some index tuning.

In a case like that, I often begin by asking for just two files:
Excel spreadsheet containing Table Information
A NAV backup or fob file which contains the customer's objects, but not their data

Each of these files don't take long to prepare, and are normally small enough to send by email.

You collect the Excel spreadsheet like this:
In NAV, go to File -> Database -> Information, and click on Tables.
Copy this into an Excel spreadhseet.

When you get the spreadsheet, sort it by "No. of Records", or by "Size (KB)". The difference is, that "Size (KB)" includes the size of the indexes on each table. So either way of sorting gives you a good idea of which tables are being updated the most, and which tables have lots of both records and indexes.

Then load the NAV objects that you also received, and for the top 5 - 10 tob tables in the spreadsheet, take a look at the tables in NAV, and check the number of keys and SIFT-indexes. Some times you will immediately see that these tables have had a lot of keys added, and you should try to see if you can reduce the number of keys, using the key properties MaintainSQLIndex, MaintainSIFTIndex and SIFTLevelsToMaintain (remember the SIFTLevelsToMaintain-property don't exist any longer in NAV 5 SP1 where it was removed as part of re-designing the SIFT system).

Which indexes to disable the SQL-maintenance of:
Only if you know how the system is being used, can you tell whether a certain index is being used or not. But, for example, a key like "IC Partner Code" in table 17 is only ever useful if the customer is using Intercompany Posting, and the "Additional-Currency Amount"-sumindex field on the same table is only needed if Additional currency is being used. So some times, a few obvious changes can be done. But to really do some index tuning, you need to know more than the objects alone can tell you.

You can also take the opposite approach, and disable maintenance of all indexes and SIFT (except for the clustered index). This will show you which ones are really needed when certain processes begin to run very slowly, and you will then have to (quickly) re-enable those. This is of course a risky approach, but it can be quicker than analyzing each individual key.

In deciding which indexes may be disabled, the following tools may help as well:
If running on SQL Server 2005 or later, run the query "Index Usage" (follow this link)

The Key Information tool on the SQL Server Resource kit can help you deciding the cost of indexes per table, and help deciding the usefulness of SIFT indexes.
With Navision Developers Tool (NDT), you can run "Where Used" on a key, to see in which object it is being used. If you use this method, then make sure to select everything in the "Where Used"-options. And keep in mind, that even with everyhthing selected, the NDT cannot see if users are manually specifying certain keys on the forms and reports they are running.

Lars Lohndorf-Larsen

Escalation Engineer

These postings are provided "AS IS" with no warranties and confer no rights. You assume all risk for your use.

Microsoft Dynamics NAV 5.0 SP1 and SQL Server 2000

Lohndorf — Tue, 03 Jun 2008 14:22:00 GMT

Microsoft Dynamics NAV 5.0 SP1 introduces a new way to handle SIFT. Instead of maintaining totals in separate tables, Dynamics NAV 5.0 SP1 uses a SQL feature called indexed views. Indexed views will automatically be maintained by the SQL Server.

With SQL Server 2000, updating an indexed view can be a time consuming process as the SQL Server 2000 might decide to include a clustered index scan as part of its query plan to update the view. If your Microsoft Dynamics NAV implementation includes tables with many records and is based on Microsoft SQL Server 2000 there is a potential risk for experiencing bad performance.

Microsoft Dynamics NAV 5.0 SP1 is optimized for Microsoft SQL Server 2005 and you should consider to upgrade to SQL Server 2005 if you are implementing Microsoft Dynamics NAV 5.0 SP1.

The problem described here, only applies to SQL Server 2000. SQL Server 2005 handles the updates of indexed views much more efficiently.

Lars Lohndorf-Larsen

Escalation Engineer

These postings are provided "AS IS" with no warranties and confer no rights. You assume all risk for your use.

Changes to Microsoft Dynamics NAV 5.0 SP1 with Microsoft SQL Server

martinni — Tue, 13 May 2008 11:14:00 GMT

I just want to point you to the whitepaper describing the changes done for the Microsoft Dynamics NAV 5.0 SP1 SQL Option:

With the release of Microsoft Dynamics NAV™ 5.0 SP1, major changes have been made to Microsoft Dynamics NAV™ with Microsoft SQL Server. This document outlines these changes and shows how these changes can help you improve the Microsoft Dynamics NAV customer experience. This document will only cover changes to Microsoft Dynamics NAV with SQL Server.

Martin Nielander
Program Manager

These postings are provided "AS IS" with no warranties and confer no rights. You assume all risk for your use.

Modern NAV/SQL troubleshooting

Lohndorf — Mon, 12 May 2008 13:35:00 GMT

This post is the first in a planned series to describe various "modern" methods for troubleshooting performance problems with Microsoft Dynamics NAV on SQL Server. The idea is also to make the best out of information that already exists, either in other places on this blog or anywhere else.

Modern Troubleshooting:
The idea about "Modern Troubleshooting" is:

To take as much of the hard work out of troubleshooting as possible.
To make the system itself (SQL Server and NAV) help you finding out what is wrong with it.
To collect as little and as specific data as possible, to avoid having to read through for example gigabytes of trace files.
To do the troubleshooting remotely as much as possible, rather than spending days on an individual system on site.

I want to make it clear that with "Troubleshooting", I mean "Trouble". And the methods I describe here will be quick-fixes for when something has gone wrong already. I don't mean to reduce the need for thorough performance- or scalability reviews, which can still require lots of work and understanding of the customers business. So I definitely don't want the "Modern Troubleshooting" to replace specialist consulting and ongoing (especially pro-active) performance tuning. Of course it is always better to avoid Trouble in the first place.

Identifying Trouble-queries:
One of the most useful queries I have used for performance troubleshooting recently, is the "TOP 30"- query described here:
Simple query to check the recent performance history

It returns the queries that are currently in SQL Server's plan cache, sorted with the queries causing most reads at the top. Every time SQL Server generates a query plan, then this plan is stored in this cache. And every time SQL Server (re)uses this plan it will also update some statistics about the plan, like execution_count, number of reads, etc. It is this information that the "TOP 30" query gives you.

SQL Server's plan cache is changing all the time, depending on what queries SQL Server runs, so you may get different results depending on what time of the day you run the query. The plan cache is also reset when SQL Server restarts, or you can reset the cache by running DBCC FREEPROCCACHE.

The result of the "TOP 30"-query is easy to copy into an Excel spreadsheet for further analysis, or to send to someone else. So it is simple data to collect, as long as the customer is on SQL2005 or later (Being based on Dynamic Management Views (DMW) which was introduced in SQL Server 2005, it not work for SQL Server 2000).

What to look at:
After receiving the result of the query - preferably in an Excel spreadsheet - these are the things I look at:
Being sorted by "max_logical_reads", you have the "worst" query at the top. But also look at execution_count. If a query ran just once, it might have been a batch job, or something else that is not really causing any problems. Of course, queries that have an execution_count in the 100s or 1.000s may be more relevant to look at. Also see if the queries (stetement_text) look similar, or if many of them look to be in the same area (same tables).

The column diff_quota shows max_logical_reads divided with min_logical_reads. If this number is high, it means that the query plan is inconsistent. This can be either because of inconsistent use of NAV (for example users applying different filters on the same table). Or, because a query plan is good for some queries but bad for others. In this case, some times you can affect the way that SQL Server creates query plans, either by adding RECOMPILE hints, plan guides or index hints. Or by upgrading to a newer version of NAV client (for example see the post "SQL Pre-processing in Microsoft Dynamics NAV 5.0 SP1" for how NAV 5 SP1 will cause different query plans).

If diff_quota is low, it means that the query plan is just consistently bad, which means that it is more likely that the query itself is bad. And, you will have to look for reasons why that query consistently causes the number of reads that it does. The "TOP 30"-query can't really help finding out why a query is causing many reads. But at least it can some times identify which queries to investigate first, which can be a very time consuming task otherwise (collecting and analysing profiler traces, etc).

You can also look at max_elapsed_time, but keep in mind that when a query takes a long time to run because it is being blocked, then the real problem is somewhere else (in the blocking query). So, if a query has a high max_elapsed_time, then see if the query contains a lock (WITH UPDLOCK). If it does, then you are most likely looking at a blocking problem which requires a wider look, and which often cannot be solved by the query you see.

So the "TOP 30"-query is always a good place to start. Some times it won't help you solving any problems, but then you will know that without spending much time. Other times, the query tells you right away which queries are causing problems, which can save you a lot of time. So I would always run this query first. And only if it doesn't help, then begin to look at collecting further information.

Lars Lohndorf-Larsen

Escalation Engineer

These postings are provided "AS IS" with no warranties and confer no rights. You assume all risk for your use.

SQLIndex property

Lohndorf — Mon, 21 Apr 2008 09:49:00 GMT

In some situations, using the SQLIndex property on a key in Microsoft Dynamics NAV can harm performance. In this blog I

describe what to be careful about, and why the use of this property has been removed in the NAV 5 SP1 application.

The property is still available and it is still a valuable tool in performance tuning. But from SP1, it is not used anymore

in the standard application. The document "Changes in NAV 5 0 SP1.doc" on the SP1 product CD lists the 72 tables where the

SQLIndex has been removed (Change A222).

Background:
The key-property SQLIndex was introduced in NAV version 4 SP1. The idea of the property is to make it possible to change

the index on the SQL Server while maintaining the same application (same sorting) in NAV.

The main use of the property is to make the SQL index more selective. In the NAV application there are many keys that begin

with an option which is not very selective, for example the primary key "Document Type","No." on the Sales Header table.

"Document Type" - having only 6 possible options - is not very selective, and SQL Server might choose not to use it. If the

index was changed to be the other way around ("No.","Document Type"), it would be much more selective and more effecient

for SQL Server to use in SELECT statements.

The other benefit of this property is to enable "covering indexes", so that you can have a few indexes to cover for most

searches, and then disable the maintenance of other indexes. For example an index on the "Sales Header" table beginning

with "No." can be used effeciently with many different filters, reducing the need to have one key for every possible exact filter.

Problems:
When you have a query which includes an "ORDER BY"-clause, SQL Server has to return the reords in the order specified by

that clause. If SQL Server doesn't have a matching index, it has to retrieve data using a different index and then do some

internal sorting to return data in the correct order. If there are no good indexes, then SQL Server may choose to

use the clustered index which can be bad enough. But when the query also has an index hint, then SQL Server is forced to use the

index specified by the hint, and this can lead to large amounts of reads.

Example:
In a recent support case, the customer had generally bad performance. In this case, the "SELECT TOP 30"-query from the post "Simple query to check the recent performance history" showed that out of the top 30 "worst" queries, 26 were similar to this one

SELECT TOP 501 * FROM "Reservation Entry" WITH (UPDLOCK, INDEX("$1")) WHERE (("Source ID"=@P1)) AND (("Source Ref_

No_"=@P2)) AND (("Source Type"=@P3)) AND (("Source Subtype"=@P4)) AND (("Source Batch Name"=@P5)) AND (("Source Prod_ Order

Line"=@P6)) AND (("Reservation Status"=@P7)) ORDER BY "Source ID","Source Ref_ No_","Source Type","Source Subtype","Source

Batch Name","Source Prod_ Order Line","Reservation Status","Shipment Date","Expected Receipt Date","Entry No_","Positive"

The query itself looks good enough: WHERE-clause and "ORDER BY"-clause match each other, and there were no immediate

reasons why this query should cause more reads than the number of records in the table. But in the standard application, the SQLIndex property for this key was:

"Source ID","Entry No.",Positive

The idea with this is SQLIndex is to have an index which can cover more situations, and in that way reduce the number of indexes that

need to be maintained on SQLServer. And the SQLindex is fine for the SELECT-part of the query. The problem is, that the

index can't be used for the "ORDER BY"-part of the query.

So what happens, is:

SQL Server may have planned to use the clustered index to read all data and then do some internal sorting. But in this

case, the Index Hint forces SQL Server away from that plan. The result is that SQL Server is forced into doing a very

difficult task while being restricted by the index speicifed by the index hint.

In this case we designed the "Reservation Entry"-table and removed the SQLIndex property from the key, and performance went

up.

Conclusion:
When you see a query which causes many reads, even if SQL Server has a good index, then also consider if the index is

good for the "ORDER BY"-part of the query. The "ORDER BY"-part of the query depends on the key in NAV. But if the SQLIndex

property has been set for this key, then by definition, the "ORDER BY" and the SQL index will not be matching.

You should still consider the use of the SQLIndex property as part of tuning performance of a system. But just be aware that it can also cause problems as described here.

Lars Lohndorf-Larsen

Escalation Engineer

These postings are provided "AS IS" with no warranties and confer no rights. You assume all risk for your use.

Bulk Inserts in Microsoft Dynamics NAV 5.0 SP1

Lohndorf — Mon, 14 Apr 2008 13:18:00 GMT

Bulk Inserts is a new feature in NAV 5.0 SP1 which is designed to improve performance when inserting multiple records by delaying the physical inserts on the SQL Server.

Before this change, inserts happen in the order that the C/AL code is running. With Bulk Insert, the NAV-client delays the actual inserts until the last possible moment in the transaction. This means that tables get locked later, so it reduces the amount of time that a table is locked.

"The last possible moment", means that the inserts will take place just before COMMIT. Or, if you use the return value of the INSERT command (IF Rec.INSERT THEN;). NAV also has to do the inserts if you run any MODIFY, DELETE or FIND methods on the table. So to take full advantage of this feature, you have to consider this when designing a NAV process.

Note: This feature has nothing to do with the Transact SQL command "BULK INSERT". The Bulk Insert all happens on the client. It is not using any special SQL Server features.

The Pseudo-code below illustrates how you will see the effect of Bulk Insert in a SQL Profiler trace. Without Bulk Insert, the SQL updates may look like this:

INSERT INTO "NAV5"."dbo"."CRONUS International Ltd_$G_L Entry"

SELECT TOP 1 FROM "NAV5"."dbo"."CRONUS International Ltd_$Cust_ Ledger Entry"

SELECT TOP 1 FROM "NAV5"."dbo"."CRONUS International Ltd_$G_L REgister"

INSERT INTO "NAV5"."dbo"."CRONUS International Ltd_$G_L Entry"

SELECT TOP 1 FROM "NAV5"."dbo"."CRONUS International Ltd_$Cust_ Ledger Entry"

SELECT TOP 1 FROM "NAV5"."dbo"."CRONUS International Ltd_$G_L REgister"

INSERT INTO "NAV5"."dbo"."CRONUS International Ltd_$G_L Entry"

SELECT TOP 1 FROM "NAV5"."dbo"."CRONUS International Ltd_$Cust_ Ledger Entry"

SELECT TOP 1 FROM "NAV5"."dbo"."CRONUS International Ltd_$G_L REgister"

INSERT INTO "NAV5"."dbo"."CRONUS International Ltd_$G_L Entry"

With Bulk Insert, the same C/AL code would look lilke this:

SELECT TOP 1 FROM "NAV5"."dbo"."CRONUS International Ltd_$Cust_ Ledger Entry"

SELECT TOP 1 FROM "NAV5"."dbo"."CRONUS International Ltd_$G_L REgister"

SELECT TOP 1 FROM "NAV5"."dbo"."CRONUS International Ltd_$Cust_ Ledger Entry"

SELECT TOP 1 FROM "NAV5"."dbo"."CRONUS International Ltd_$G_L REgister"

SELECT TOP 1 FROM "NAV5"."dbo"."CRONUS International Ltd_$Cust_ Ledger Entry"

SELECT TOP 1 FROM "NAV5"."dbo"."CRONUS International Ltd_$G_L REgister"

INSERT INTO "NAV5"."dbo"."CRONUS International Ltd_$G_L Entry"

So, you will see exactly the same SQL updates. But with Bulk Insert, the inserts will be accumulated and run at the end of the process.

Lars Lohndorf-Larsen,

Escalation Engineer

These postings are provided "AS IS" with no warranties and confer no rights. You assume all risk for your use.

My experiences with Microsoft Dynamics NAV 5.0 SP1 …

rmiller — Fri, 04 Apr 2008 14:55:00 GMT

I spent last week performing a Microsoft Dynamics NAV 5.0 Update 1 to Microsoft Dynamics NAV 5.0 SP1 database conversion for a customer. The process went very smoothly and the customer was very excited about the increase in performance we were able to achieve.

Just as an illustration of our success, the customer ships hundreds of packages a day. This process has been very painful in the past. When the conversion was completed and a little tuning was done, we were able to achieve the following improvements …

· Create a Shipment --> Reduced by 27%

· Create a Pick --> Reduced by 15%

· Register a Pick --> Reduced by 54%

· Post a Shipment --> Reduced by 53%

Now, I mentioned a little tuning was necessary so let me elaborate on that. SP1 seems to highlight poor filtering used for the FlowFields. The tuning that I had to do amounted to determining which CALCSUMS were not performing well. I did this by using the Client Monitor in the Performance Toolkit. I was then able to examine the filters that were applied and determine what a more appropriate key would be. I added the additional key to the table along with the appropriate SumIndexField(s). This made all the difference for this customer.

This customer was also plagued by a large number of locking/blocking issues. After applying SP1, with the Bulk Inserts and Indexed Views, the locking/blocking issues became almost non-existent.

Some other pieces of information that might be helpful:

· Log file size – during the conversion, the log file did NOT grow at all.

· Database Size – when the conversion was done, this particular customer was able to get back approximately 10 GB of free space in the database out of a 65 GB database.

· Conversion Time – it took between 40 -60 seconds per GB depending on what hardware was used.

· Indexed Views – do NOT add any additional indexes to the Indexed Views. When you design the table that the Indexed View is associated with, the view will be dropped and recreated and so will any additional indexes that you might have added.

Robert Miller

Escalation Engineer

These postings are provided "AS IS" with no warranties and confer no rights. You assume all risk for your use.

SQL Pre-processing in Microsoft Dynamics NAV 5.0 SP1

Lohndorf — Thu, 03 Apr 2008 09:39:00 GMT

One of the major changes in Microsoft Dynamics NAV version 5 SP1 (in relation to performance on SQL Server), is a new way to send queries to SQL Server. In previous versions of NAV, we some times saw SQL Server 2005 optimizing query plans for extreme parameter-values, which - when re-used from cache for queries with other parameter-values - could cause long response time. A behaviour which is described in more details in KB 935395 on PartnerSource (login required). Some of the updates for NAV version 4 introduced new features to give better control of the query plans that SQL Server makes, such as index hints and Recompile-option.

SP1 for NAV version 5 has restructured the way that queries are sent to SQL Server, with the aim that SQL Server will now make query plans that are optimized for average parameter-values rather than extreme parameter-values. It is also a method which lets SQL Server make the plan, without forcing it in a certain direction with index hints or recompile-option.

Before NAV SP1, a typical query could look like this:

declare @p1 int

set @p1=180150033

declare @p3 int

set @p3=2

declare @p4 int

set @p4=1

declare @p5 int

set @p5=0

exec sp_cursoropen @p1 output,N'SELECT * FROM "Demo Database NAV (5-0)"."dbo"."CRONUS International Ltd_$G_L Entry" WHERE (("G_L Account No_"=@P1)) AND "G_L Account No_"=@P2 AND "Posting Date"=@P3 AND "Entry No_">=@P4 ORDER BY "G_L Account No_","Posting Date","Entry No_" ',@p3 output,@p4 output,@p5 output,N'@P1 varchar(20),@P2 varchar(20),@P3 datetime,@P4 int','1105','','1753-01-01 00:00:00:000',0

select @p1, @p3, @p4, @p5

Notice the parameter-values in bold ('1105','','1753-01-01 00:00:00:000'). SQL Server will make a query plan based on running the query with these parameter-values. It will then cache this plan, and use the same plan for other queries which are identical, but have different parameter-values.

In SP1, the query above will look like this:

declare @p1 int

set @p1=1073741861

declare @p5 int

set @p5=12290

declare @p6 int

set @p6=8193

exec sp_cursorprepare @p1 output,N'@P1 varchar(20),@P2 varchar(20)',N'SELECT * FROM "W1500SP1RTM"."dbo"."CRONUS International Ltd_$G_L Entry" WHERE (("G_L Account No_"=@P1)) AND "G_L Account No_">@P2 ORDER BY "G_L Account No_","Posting Date","Entry No_" ',1,@p5 output,@p6 output

select @p1, @p5, @p6

And then another query:

declare @p2 int

set @p2=180150031

declare @p3 int

set @p3=2

declare @p4 int

set @p4=1

declare @p5 int

set @p5=2

exec sp_cursorexecute 1073741861,@p2 output,@p3 output,@p4 output,@p5 output,'1110',''

select @p2, @p3, @p4, @p5

So before, we had one query. In SP1 we have two! So what's the benefit of that?

If you look at the first query from SP1, notice that it is a sp_cursorprepare statement, and not sp_cursoropen. So the actual query is not run at this point. More importantly, the first query does not contain the parameter-values. This is the query for which SQL Server makes the query plan. Not having the parameter-values, SQL Server makes the plan based on its statistics about the data in the table. Only after this, NAV then executes the statement in the second query (sp_cursorexecute).

This method guarantees that SQL Server's query plan will not be affected by the parameter-values. It means that some times, SQL Server is prevented from making the optimum query plan for a certain set of parameter-values. But remember that the query plan will be re-used for other parameter-values. So at the expense of having a few highly optimized queries, the method will give well optimized queries with better consistensy.

Another cost of this method, is of course that now NAV sends 2 queries instead of 1, requiring an extra roundtrip to SQL Server. But this only happens the first time the query is run. If the same query is run again, NAV will only run the second query (sp_cursorexecute).

One side effect of this is, that tracing a query in SQL Profiler becomes different. With SP1 you will see a lot of queries like the second one above, which does not show what NAV is actually doing. Take a look at the second query again:

exec sp_cursorexecute 1073741861,@p2 output,@p3 output,@p4 output,@p5 output,'1110',''

With SP1 you will see a lot of queries like this, and then wonder what the actual query is. To find out, you need to use the cursor ID, and then find the original sp_cursorprepare statement, which will contain this line:

set @p1=1073741861

and then the actual query.

In summary, this method is designed to give persistently, good overall performance, and to avoid sudden drops in performance that could be the result of cached query plans on SQL Server.

Lars Lohndorf-Larsen

Escalation Engineer

These postings are provided "AS IS" with no warranties and confer no rights. You assume all risk for your use.

Simple query to check the recent blocking history

Lohndorf — Fri, 28 Mar 2008 10:42:00 GMT

SQL Server 2005 keeps (some) history of the number and time that blocking occurred on indexes. This blog contains a query that queries this information to help identifying on which tables blocking happens most. The query shows accumulated numbers (no. of blocks, wait time, etc) since SQL Server was last restarted.

The query does have some known in-accuracies. Also, the result needs to be interpreted: It may not point directly at which indexes need changing or disabling. See below for details about this.

The query uses Dynamic Management Views, which means it will only work from SQL Server 2005 and later.

Make sure to run the query in the NAV database, otherwise you won't see the table names.

So, here it is:

--Use [NAV-Database]

select

db_name(database_id) DB,

object_name(object_id) Obj,

--row_lock_count, page_lock_count,

row_lock_count + page_lock_count No_Of_Locks,

--row_lock_wait_count, page_lock_wait_count,

row_lock_wait_count + page_lock_wait_count No_Of_Blocks,

--row_lock_wait_in_ms, page_lock_wait_in_ms,

row_lock_wait_in_ms + page_lock_wait_in_ms Block_Wait_Time,

index_id

from sys.dm_db_index_operational_stats(NULL,NULL,NULL,NULL)

order by Block_Wait_Time desc

--order by No_Of_Blocks desc

Inaccuracies:
No_Of_Blocks is recorded accurately, but the Block_wait_time is not. SQL Server only records Block_wait_time when the block is a clear pagelock or rowlock. It will not record wait_time in case of a rangelock, which is common in NAV. Also, Block_Wait_Time only gets recorded when a transaction completes. So if a transaction is aborted after the block (for example by Lock-Timeout), then Block_Wait_Time for that transaction will not be counted. This means that the real Block_Wait_Time is likely to be higher, and it may be distributed on different tables / indexes than the query shows. Anyway, I hope the query is still accurate enough to give a good idea about where blocking occurs.

How to interpret the result:
The query shows blocking per index. But you should not put too much importance in the individual index that shows blocking. Instead, look if there may be other indexes in that table which are not used. Remember, an update on Index X may require update on all other indexes in the table. So look at the whole table when deciding if the table is over-indexed. For this, use the query "Index Usage", which shows you the usage of each index in a table.

Blocking will happen on the first table in a process which is blocked. Maybe the process begins processing small tables, but the real blocking happens because processing of other tables takes a long time. In that case, the query will show the blocking on the first table, but not show blocking on the later tables, which may be where the real problem is.

I will be more than happy to receive any feedback on experiences with this query, and suggestions for how to improve it!

Lars Lohndorf-Larsen

Escalation Engineer

These postings are provided "AS IS" with no warranties and confer no rights. You assume all risk for your use.

Index Usage Query

Lohndorf — Tue, 26 Feb 2008 12:29:00 GMT

This is a follow up from an earlier blog "Finding Index usage". In that blog, I described a very simple way to list how indexes are being used. In this blog, the query is much extended so that it now shows your Navision keys, listed by either number of updates, or by their cost divided by their usage, and it shows when an index was last used for reading. The idea is to show a list of indexes that are being maintained, but never or rarely being used.

The query uses SQL Server Dynamic Managament Views (DMW), which means it will only work for for SQL Server 2005 and later.

Feel free to add comments to this blog about how useful (or not) this query is. And about any problems you may find, and suggestions to improve it. All comments will be welcome!

To use it, copy the query below into SQL Server Management Studio. Remember to set the database to your Microsoft Dynamics NAV database (not Master or any other database). Then run it. Depending on the size of your database, it may take a few minutes to run it. First time you run it, I would recommend that you do it when the SQL Server is not otherwise busy, until you konw how long it takes:

-- use NavisionDB

IF OBJECT_ID ('z_IUQ_Temp_Index_Keys', 'U') IS NOT NULL

DROP TABLE z_IUQ_Temp_Index_Keys;

IF OBJECT_ID ('zIUQ_Temp_Index_Usage', 'U') IS NOT NULL

DROP TABLE zIUQ_Temp_Index_Usage

-- Generate list of indexes with key list

create table z_IUQ_Temp_Index_Keys(

[F_Obj_ID] [int] NOT NULL,

[F_Obj_Name] [nvarchar] (128) NULL,

[F_Ind_ID] [int] NOT NULL,

[Index_Column_ID] [int] NOT NULL,

[Index_Key] [nvarchar] (128) NULL,

[Index_Key_List] [nvarchar] (MAX) NULL,

CONSTRAINT [z_IUQ_TempPK] PRIMARY KEY(

[F_Obj_ID],

[F_Ind_ID],

[Index_Column_ID]

)

Insert into z_IUQ_Temp_Index_Keys

select

object_id,

object_name(object_id),

index_id,

Index_Column_ID,

index_col(object_name(object_id),index_id,Index_Column_ID),

from sys.index_columns

-- populate key string

declare IndexCursor cursor FOR

select F_Obj_ID, F_Ind_ID from z_IUQ_Temp_Index_Keys

FOR UPDATE of Index_Key_List

declare @ObjID int

declare @IndID int

DECLARE @KeyString VARCHAR(MAX)

set @KeyString = NULL

open IndexCursor

set nocount on

fetch next from IndexCursor into @ObjID, @IndID

while @@fetch_status = 0 begin

SET @KeyString = ''

SELECT @KeyString = COALESCE(@KeyString, '') + Index_Key + ', '

FROM z_IUQ_Temp_Index_Keys

where F_Obj_ID = @ObjID and F_Ind_ID = @IndID

ORDER BY F_Ind_ID, Index_Column_ID

SET @KeyString = LEFT(@KeyString,LEN(@KeyString) - 2)

update z_IUQ_Temp_Index_Keys

set Index_Key_List = @KeyString

where current of IndexCursor

fetch next from IndexCursor into @ObjID, @IndID

end;

close IndexCursor;

deallocate IndexCursor;

-- Generate list of Index usage

create table zIUQ_Temp_Index_Usage(

[F_Table_Name] [nvarchar](128) NOT NULL,

[F_Ind_ID] [int] NOT NULL,

[F_Index_Name] [nvarchar](128) NULL,

[No_Of_Updates] [int] NULL,

[User_Reads] [int] NULL,

[Last_Used_For_Reads] [datetime] NULL,

[Index_Type] [nvarchar](56) NOT NULL,

[last_user_seek] [datetime] NULL,

[last_user_scan] [datetime] NULL,

[last_user_lookup] [datetime] NULL,

[Index_Keys] [nvarchar] (255) NULL

)

insert into zIUQ_Temp_Index_Usage

select

object_name(US.object_id) Table_Name,

US.index_id Index_ID,

SI.name Index_Name,

US.user_updates No_Of_Updates,

US.user_seeks + US.user_scans + US.user_lookups User_Reads,

case

when (ISNULL(US.last_user_seek,'00:00:00.000') >= ISNULL(US.last_user_scan,'00:00:00.000')) and (ISNULL(US.last_user_seek,'00:00:00.000') >= ISNULL(US.last_user_lookup,'00:00:00.000')) then US.last_user_seek

when (ISNULL(US.last_user_scan,'00:00:00.000') >= ISNULL(US.last_user_seek,'00:00:00.000')) and (ISNULL(US.last_user_scan,'00:00:00.000') >= ISNULL(US.last_user_lookup,'00:00:00.000')) then US.last_user_scan

else US.last_user_lookup

end as Last_Used_For_Reads,

SI.type_desc Index_Type,

US.last_user_seek,

US.last_user_scan,

US.last_user_lookup,

from sys.dm_db_index_usage_stats US, sys.indexes SI where SI.object_id = US.object_id and SI.index_id = US.index_id

order by No_Of_Updates desc

-- Select and join the two tables.

select

TIU.F_Table_Name Table_Name,

--TIU.F_Ind_ID Index_ID,

--TIU.F_Index_Name Index_Name,

TIK.Index_Key_List,

TIU.No_Of_Updates,

TIU.User_Reads,

case

when TIU.User_Reads = 0 then TIU.No_Of_Updates

else TIU.No_Of_Updates / TIU.User_Reads

end as Cost_Benefit,

TIU.Last_Used_For_Reads,

TIU.Index_Type

from zIUQ_Temp_Index_Usage TIU, z_IUQ_Temp_Index_Keys TIK where TIK.F_Obj_Name = TIU.F_Table_Name and TIK.F_Ind_ID = TIU.F_Ind_ID and TIK.Index_Column_ID = 1

and TIU.F_Table_Name not in ('zIUQ_Temp_Index_Usage','z_IUQ_Temp_Index_Keys')

order by No_Of_Updates desc

--order by Cost_Benefit desc

The query will show you one line for each index in the SQL Database. It shows you the table name, and a list of fields in the index. Note that any non-clustered index also contain the clustered index. For example on SQL Server, the key "Document No." in the "Cus. Ledger Entry table" is "Document No.","Entry No.". Also note that the indexes shown by SQL Server is not always shown in the same order as you have defined them in NAV.

The column "No_Of_Updates" basically shows you the cost of this index, since every update requires a lock as well as a write to the database. The next column, "User_Reads", shows you how often this index has been used, either from the UI, or by C/AL code. Compare these two, and you have way to compare the cost against the benefits of each index, as shown in the column "Cost_Benefit", which is simply "No_Of_Updates" / "User_Reads". The column "Last_Used_For_Reads" shows you when an index was actually used for reading.

The query sorts the indexes by "No_Of_Updates", with the most updated (most costly) index first. At the last line of the query you can change the sorting to "order by Cost_Benefit desc", and you are likely to see a different picture.

Finally, the query shows you whether each index is clustered or non-clustered.

The query will create two new tables called "z_IUQ_Temp_Index_Keys" and "zIUQ_Temp_Index_Usage". Although highly unlikely, if you already have tables with these names in your database, then the query will overwrite those without warnings. These tables collect index usage statistics, so if you need to run the query again, for example because you lost the results, or wat to run it with a different sorting, you don't have to run the whole query. Just run the last part of the query - from the section "-- Select and join the two tables.", and it will run much faster. Only after you change indexes, or want an updated view of index usage, you need to run the whole query again.

The data shown by the query is reset every time SQL Server restarts. So if you have recently restarted SQL Server, then the query may not show you the most precise picture of how the indexes are being used over time. Also consider that some indexes may only ever be used for example at end of the month / end of fiscal year, etc. So just because the query shows that a certain index was not used since SQL Server was last restarted, then this index may still be required for specific jobs.

Lars Lohndorf-Larsen

Escalation Engineer

These postings are provided "AS IS" with no warranties and confer no rights. You assume all risk for your use.

How to detect locking order for a NAV process

Lohndorf — Wed, 20 Feb 2008 14:49:00 GMT

The nature of a deadlock is that two processes lock resources in different orders. Deadlocks can in theory be eliminated by ensuring that all processes always lock resources in the same order. This document describes how to determine the locking order of a process in Microsoft Dynamics NAV.

Note, that I mention locking of "resources". For most of the time, this means placing locks on a table, but it could as well mean placing locks on different parts of the same table, or of different indexes in the same, or two different tables. But then we are into micro-tuning, so for this post, when I say "Resource", I mean "Table".

What is a deadlock:
A deadlock will return an error message which specifically says that your activity was deadlocked. The deadlock error-message will read:

"Your activity was deadlocked with another user modifying the [xyz] table.
Start again."

A blocking chain is a situation where a client hangs (becomes unresponsive) until a resource becomes available. In common language you could describe such a situation as a deadlock situation. But technically, this is not a deadlock, since the situation will be resolved eventually. In other words, to any user, an infinite block may look like a deadlock. But if you have deadlocks, then by definition, your users will have deadlock-error messages like the one shown above. One of the first steps in troubleshooting any kind of locking or blocking is to identify exactly what type of problem you have. So always make sure to first find out whether you are dealing with blocking chains, deadlocks, or other issues, since selecting the best troubleshooting methods depend on what exact issue you are looking at.

This post shows how to determine the locking order of a process in NAV. Knowing the locking order of various processes can help identifying potential deadlocks. It can also help in cases where you know that two or more different processes are causing deadlocks, by showing the locking order of each of these processes.

Pre-requisites:
The tool described here is part of the Performance Troubleshooting Guide, which is available on PartnerSource (partner login required). The tools described here require a NAV Partner license.

The SQL Server Performance Troubleshooting Guide, includes an object called "Client Monitor.fob". To begin, import this into the database which contain the processes that you are troubleshooting. It does not have to be a live database - a stand-alone copy of a live database is all you need. In this scenario you are examining locking orders of processes which can be done as well in your office, on a copy or test-version of a database, as in a production environment.

To use "Client Monitor.fob", follow these steps, after importing it into the database:

1. Start Client Monitor (Tools -> Client monitor -> Start). Before starting it, go to the Options tab and de-select "Include Object table activity", and select all the options under "Advanced".
2. Run the processes for which you want to detect the locking order.
3. Stop "Client Monitor".
4. Run form 150025 "Transactions" which was imported in "Client MOnitor.fob". This will take a while, while it collects the information that was collected by the Client Monitor. When it opens, it will show you one line for each transaction.
5. For each of the lines shown in this form, click on Transaction -> "Locking order" to see the locking order of each transaction.

This shows you not just the locking order of each transaction, but you will also see the C/AL code that places the lock. If you compare this information with knowledge of what tables and processes are involved in the deadlocks you are troubleshooting, it can help you decide where locking orders need to be changed.

What locks a record?
NAV will automatically place a lock as soon as any update command is used (INSERT, MODIFY, DELETE), even if the C/AL command LOCKTABLE is not used. On SQL Server, Navision will only lock the record (and potentially adjacent records - see this post).

The C/AL command LOCKTABLE in itself does not lock anything. It only puts NAV into locking-mode. Navision will then lock any records that it accesses. This is why the places in C/AL code that place locks are typically not LOCKTABLE-commands, but for example a FIND('-')-command that comes after a LOCKTABLE command.

What to do next:
Once you have determined that two different processes have different locking orders, you must decide on a locking order. There are no complete guides to which order should be used, but if you look for the C/AL command RECORDLEVELLOCKING in codeunit 80, you can see that this codeunit locks a number of tables in a certain order. At least for tables that are included in this order, it makes sense to stick to this order. For tables which are not included here, you must make your own decisions. Once you have decided on a locking order, you can use one of the following tactics to change it:

Avoid locking:
Some times it is not necessary for a process to access certain tables. For example, the tool may show that table 5765 – “Warehouse Request” is being locked. If a customer is not using warehousing, then the code that locks this table can some times be remarked.

Lock sooner or later:
One of the most common ways to change locking order is to move a lock up so that it happens sooner in the process. For example, lets say you have this line of code:
SalesOrder.INSERT;
And you know that later on in the code, you will be locking one of your own tables. To change the locking order of these two tables, add:
YourOwnTale.LOCKTABLE;
Before the line where you lock the Sales Order table.

Master lock (semaphore):
You can decide that all processes must lock a certain record before they lock anything else. In this case, only 1 process will run at the time. This will effectively eliminate any deadlocking, but it will also reduce concurrency, and can lead to blocking chains instead.

As you can see, two of the tactics above (lock sooner, and Master lock) may reduce the risk of deadlocks. But on the other hand they can also increase blocks, because (more) resource(s) get locked sooner. So always make a total assessment of the problem that deadlocks cause. Some times it may be better to have a weekly deadlock, but optimistic locking, than begin to rearrange locks and cause more blocks.

Lars Lohndorf-Larsen

Escalation Engineer

These postings are provided "AS IS" with no warranties and confer no rights. You assume all risk for your use.

Record-level locking in the SQL Server option for Microsoft Dynamics NAV

Lohndorf — Wed, 06 Feb 2008 11:16:00 GMT

If you are designing a system for high throughput and want to minimize blocking, it's important to know in advance exactly what kind of locking behaviour to expect.

On SQL Server, Microsoft Dynamics NAV uses record-level locking, as opposed to table-level locking on the native database server. In reality, SQL Server can lock a little bit more than the individual records you want to lock: It can also lock the next and the previous records. This happens when SQL Server applies a range-lock.

This post describes how and when a range-lock can occur. It applies to a normal lock, regardless of the "Always Rowlock"-setting in NAV, and lock escalation where SQL Server can escalate a record-level lock to a table-lock - these are seperate issues.

Here are some examples:

Table 7 "Standard Text", contains the following records:
Code Description
MD    Monthly Depreciation
SC    Shipping Charge
SUC Sale under Contract
TE     Travel Expenses

Table 7 is used here, because it is the simplest possible table.

The following C/AL code in NAV will lock just one record (SC - Shipping Charge):

OnRun()
StandardText.LOCKTABLE;
StandardText.GET('SC');

IF NOT CONFIRM('Continue?') THEN
ERROR('Stopped.');

You can confirm this by running it from one client, then open another client and try to update the records in the table.

Now, consider this C/AL code:

StandardText.LOCKTABLE;
StandardText.SETFILTER(Code,'S*');
StandardText.FINDSET;

You might expect that it will lock the two records that fall within this filter (S*): SC and SUC. But this is where SQL Server locks a bit more: It will also lock the records around this filter (MD and TE). This is because as soon as the lock covers a range and not just one individual record, SQL Server has to protect not only the locked records, but also the range itself. In this case, the range covers 2 records, and SQL Server prevents anyone else from inserting new records in this range. It is SQL Server's way to guarantee that the range stays at these 2 records.

Depending on your C/AL code, some times SQL Server protects just the beginning, and some times just the end of the range, and some times both the beginning and the end. So when you lock a range, you should assume that that both the record just before, and just after, will also get locked.

Here is an example from a more real situation. Imagine you have a process to update sales order 2002 (posting / releasing or any other process). You may have C/AL code to lock the sales lines, like this:

SalesLine.SETRANGE("Document Type",SalesLine."Document Type"::Order);
SalesLine.SETRANGE("Document No.",'2002');
SalesLine.LOCKTABLE;
SalesLine.FINDSET;

In this case, also the lines for order 2003 will get locked! So even when users are working on their own documents, they can still end up blocking each other.

When this is causing problems, what methods can be used?

The simple answer is, to make sure that users don't work on records that are just next to each others. One way would be to insert "ghost" records between each normal record. Another way, is to change the sorting. In the example above, if you had added this line at the beginning:
SalesLine.SETCURRENTKEY("Document Type","Sell-to Customer No.");

Then order 2003 will not get locked, because with this sorting, it is not the next record.Or you can design a solution that works in more random areas of a table. Using Number Series means that active documents are often in the same area of the table. If you use random numbers, or GUID as primary keys, then the active documents would be spread through the whole table, and the risk of users updating records next to each other, would be smaller.

These are just some methods to consider, but the main aim would be to spread activity on a contented table to avoid hot-spots in that table.

Open-ended ranges

Locking the last record is used in a number of places in the NAV application, including the posting routines, which normally lock the last record in the entry table. When the record you are locking is the last one in the table, then the "next record" is anywhere between the last record and infinity. So in this case, the range that is locked can become much larger.

If we go back to the Standard Text table and run this C/AL code:
StandardText.LOCKTABLE;
StandardText.FINDLAST;

Then the locked range will include any possible record after the last one (TE - Travel Expenses), so it will prevent from inserting any new records that would come after TE.

If designing an application from scratch, then you may want to take this behaviour into consideration. If changing an existing application, then in some cases it will require major re-design. So always also consider other ways to improve performance. There may be easier ways to resolve blocking problems.

Lars Lohndorf-Larsen

Escalation Engineer

These postings are provided "AS IS" with no warranties and confer no rights. You assume all risk for your use.