How Mailbox quota limit messages can impact server performance

dgoldman — Sat, 01 Apr 2006 21:48:00 GMT

I wanted to talk a little bit in this blog about Mailbox Quota Limits “Mailbox over the limit”, and how they can affect Exchange Server performance.

Overview for Mailbox quota

When you first install Microsoft Exchange Server, users have unlimited mailbox quotas by default. This means that users could possibly fill up the hard disk on the server. The purpose of mailbox quota limits are to control an Exchange Organizations mailbox sizes. Quota limits help to maintain overall database sizes if applied correctly. These quota limits settings can be set a few different ways (system policies, global settings, mailbox level, etc). I will get in to policies and how to apply them later in this blog.

The mailbox quota settings that can be enforced are as follows:

Issue warning: When a user's mailbox reaches a specified size, they are notified that they are over their quota. In Microsoft Outlook 2002 and later, users can run the Mailbox Cleanup Wizard (available from the Tools menu) to help clean out their mailboxes.

Prohibit send: When a user's mailbox reaches a specified size, that user no longer can send mail.

Prohibit send and receive: When a user's mailbox reaches a specified size, that user no longer can send or receive mail. Other users who send mail to that user are notified that their message could not be delivered.

When Exchange encounters a users or users that are over the mailbox quota limit, the information store will create a message (originating from the system attendant). This message is formatted using a string that is retrieved from the MDBSZ.DLL. This string is formatted with predefined values, and the message is delivered via a system delivery path directly into the user’s inbox as a “quota message”. Just so you understand up front, modifying the MDBSZ.DLL is *NOT SUPPORTED* and if you do so, you are doing it at your own risk!

On a brighter note, MCS (Microsoft Consulting Services) will contract to update MDBSZ.DLL with a new string, but that has to be maintained constantly, as each new hot fix and service pack (and version) of Exchange will update MDBSZ.DLL.

Below is an example of a generated quote message to an Exchange user:

Your mailbox has exceeded one or more size limits set by your administrator.

Your mailbox size is 53897 KB.

Mailbox size limits:

You will receive a warning when your mailbox reaches 5000 KB.

You cannot send mail when your mailbox reaches 10000 KB.

You cannot send or receive mail when your mailbox reaches 15000 KB.

You may not be able to send or receive new mail until you reduce your mailbox size.

To make more space available, delete any items that you are no longer using or move them to your personal folder file (.pst).

Items in all of your mailbox folders including the Deleted Items and Sent Items folders count against your size limit.

You must empty the Deleted Items folder after deleting items or the space will not be freed.

See client Help for more information.

In addition to the quota message you may see the following warning logged in the Event Viewer:

Event ID           : 1025
Source             : MSExchangeIS Private
Type                 : Warning
Category           : General
Description       : An error occurred.
Function name or description of problem: EcDoDeliverMessage
Error: 0x4d9

As you can see below I ran the Microsoft Exchange Server Error Code Look-up to resolve the 0x4d9 diagnostic error code.

C:\WINXP\system32>err 0x4d9

# for hex 0x4d9 / decimal 1241 - ecQuotaExceeded - ec.h

For more information on the Microsoft Exchange Server Error Code Look-up, please see http://www.microsoft.com/downloads/details.aspx?familyid=be596899-7bb8-4208-b7fc-09e02a13696c&displaylang=en

NOTE: The purpose for the 1025 event id is when an user or users have exceeded their mailbox storage limit and the storage warning message is *unable* to be delivered to the user by the system attendant.

In situations where both a storage limit warning message is delivered to the user and a 1025 event is logged, the user may have a rule set in his or her mailbox, which is applied on the storage limit warning message. Because of the storage quota limit, the rule cannot be applied, and event 1025 is logged.

The Mailbox Quota Process – Finding users that are over the quota limit

How does Exchange keep track of users that are over the limit? Microsoft Exchange servers retain a cache of exchange mailboxes that are over their mailbox quota limits. This is the only way an Exchange server can keep a running total of the mailboxes that have exceeded the limits.

Exchange has two ways to check for mailboxes that are over their limits:

1. During the routine maintenance scheduled tasks.

2. Per message delivery.

During the routine maintenance scheduled tasks

During maintenance tasks Exchange will use this cache as a reference point to quickly determine if any mailboxes have been marked as “over the quota limit”. If Exchange does not find a particular mailbox in the cache, Exchange will assume that the mailbox is still under the quota limit and will defer quota limit checks until the actual message delivery.

Per Message Delivery

This is self explanatory.

The Mailbox Quota Process – Finding users that are under the quota limit

Exchange has one check that it will perform that will remove a mailbox from the quota limit cache. Also Exchange will never expire any items from this cache.

How can Message Quota limits cause performance issues?

This is an interesting topic and I had a lot of fun working and debugging this one. I should note that not every customer will run in to this problem.

At the time Exchange finds a user or users that are over the quota limits, it generates a system message or multiple messages as previously stated in this blog. This system message contains certain properties, one important property to mention here is the message class property. Each quota message will have a message class of "IPM.Note.StorageQuotaWarning".

When this message is delivered to the mailbox (inserted in to the proper mailbox table in the database) the outlook client is notified that they have a new email. When the client previews this message via the preview pane or double clicks on the email, the email message has to be downloaded to the client. One special thing here that should be noted is that when an Outlook client opens a message with the message class of "IPM.Note.StorageQuotaWarning", the client will make a connection the organizational forms library so it can resolve the form for this message class.

Depending on the total number of policies set and how many mailbox stores are being scanned for users over their quota, a lot of system messages can be generated. The generation of these messages *is not* a problem, as this is normal activity for an Exchange server.

Extremely large exchange organizations may run in to this problem when they only have one or two polices set via schedule to do this quota check. The real problem comes in to play when several thousand users try to open this quota message at the same time. The message can be opened via preview pane or by double clicking on the message to open it. This will cause all of the Outlook users to make a connection to the organizational forms library (on their public folder server) to resolve this form. This can cause severe performance problems for an Exchange server (overwhelming network traffic, RPC thread depletion, etc), and can also manifest itself in the form of the dreaded RPC dialog popup box for Oulook clients.

One way to reduce the the amount of connections to the public folder store and organizational forms library on the exchange server, is to create enough system policies to cover all of the users that you want to check. Once the policies have been created, stagger the times that they run to perform the quota checks. This way the message generation will happen at different times, thus cutting down on the load of traffic to the exchange server when trying to resolve the form for the message class of "IPM.Note.StorageQuotaWarning".

Generating custom Mailbox limit (Quota Messages) for Exchange -(Message Service Tool)

Jason Nelson from the Exchange Dev wrote a tool that allows you to generate custom mailbox limit quota messages.

For more information on this tool: click here!!

You can download the service from http://workspaces.gotdotnet.com/quotamsgsvc

Overview of System Policies

A policy is a collection of configuration settings that is applied to one or more Exchange Server objects of the same class. An administrator can define a set of policies that controls the configuration settings across a large number of servers (objects of the same class). The advantage of this control is that after the policies are defined and implemented, changing the configuration of these servers requires only editing the policy and applying the changes one time in the Exchange System Manager.

The smallest possible collection of configuration settings that can be defined in a single policy is the set of properties on a single property page. Thus, in a mailbox store database policy, an administrator can potentially apply different policies for "limits" versus "database" maintenance schedule.

In the active directory the policies sit under: CN=services,cn=microsoft exchange,cn=org,cn=administrative groups,cn=first administrative group,cn=policies

Creating the System Policies Container for the Administrative Group

Before you can create a policy, you need to create a System Policies container for a corresponding administrative group. To create a System Policies container:

1. Start Exchange System Manager.

2. Expand your Administrative Groups container, and then click the administrative group for which you want to create a system policy. If no administrative groups are visible, refer to the following Microsoft Knowledge Base article for additional information:
239651 XADM: Displaying Groups Within Exchange System Manager

3. Right-click the Administrative Groups container, click New , and then click System Policy Container . A System Policies container is created under the administrative group.

Creating Policies

Server Policy

Server policies allow you to enable logging, message tracking, and log file maintenance on multiple servers in your administrative group. To create a new server policy:

1. Right-click the System Policies container, click New , and then click Server Policy .

2. In New Policy , check the boxes of the tabs you want to use in the policy, and then click OK . The policy properties appear.

3. On the General tab, type the name for this server policy.

4. Click the General (Policy) tab. This is where you select the items that you want to apply to this policy. For a description of the tabs, see the "Policy Template Details" section. The Details tab shows the creation and modification dates, and allows you to enter administrative notes.

5. Click OK to save the policy.

Public Store Policy

Use public store policies to apply administrative tasks such as maintenance, replication intervals, storage limits, and index updates to multiple public stores. To create a public store policy:

1. Right-click the System Policies container, click New , and then click Public Store Policy .

2. In New Policy , check the boxes of the tabs you want to use in the policy, and then click OK . The policy properties appear.

3. On the General tab, type a name for the policy.

4. Select and configure the tabs that you want to configure for this policy. For a description of the tabs, see the "Policy Template Details" section. The Details tab shows the creation and modification dates, and allows you to enter administrative notes.

5. Click OK to save the policy.

Mailbox Store Policy

Use mailbox store policies to apply administrative tasks such as storage limits, indexing updates, maintenance intervals, and mappings to public stores, and offline address lists to multiple mailbox stores. To create a mailbox store policy:

1. Right-click the System Policies container, click New , and then click Mailbox Store Policy .

2. In New Policy , check the boxes of the tabs you want to use in the policy, and then click OK . The policy properties appear.

3. On the General tab, type a name for the policy.

5. Click OK to save the policy.

Policy Template Details

This section describes the tabs for each policy type.

Server Policy

On the General (Policy) tab, set the following options:

Use Enable subject logging and display to log all message subject fields.
Use Enable message tracking to log all mail activity performed by all Exchange components.
Use Remove log files to remove all log files older than the value set in Remove files that are older than (days) .

Public Store Policy

On the General (Policy) tab, set the following options:

Use Clients support S/MIME signatures if your mail clients are using the S/MIME standard.
Use Display inbound Internet messages in fixed font to display incoming variable-width message fonts in a fixed font.

On the Database (Policy) tab, set the following options:

Select a value in Run maintenance during this time , or click Customize to build a custom schedule graphically.

On the Replication (Policy) tab, set the following options:

1. Select a value in Replicate public folder changes , or click Customize to build a custom schedule graphically.

2. Type a value in Replication interval for always (minutes) to limit replication frequency.

3. Type a value in Replication message size limit (KB) to limit the messages replicated.

On the Limits (Policy) tab, set the following options:

Use Delete after (days) to set the maximum number of days that items can remain in the store.
Use Don't permanently delete items until the store has been backed up to preserve items until the store is backed up.
Use Issue warning at (KB) to send a warning when the storage space used reaches the size you specify.

Select a value in Send over-limit messages , or click Customize to build a custom schedule graphically.

Use Age limit for all folders in this store (days) , and then type a value.

Mailbox Store Policy

1. Right-click the newly created System Policies folder under the administrative group, point to New , and then click Mailbox store policy .

2. Select the check boxes for the elements that you want this policy to configure.

3. The example in this article assumes that you select all of the following check boxes:

General
Database
Limits
Full-Text Indexing

4. On the General tab, type the name for the new system policy.
Microsoft recommends that you use a name that describes the policy, for example, "System Policy for Power Users."

5. Click the General (Policy) tab, and then configure a default public store for all of the mailbox stores that are associated with this policy.
If you define more than one public folder store, you can change this association. However, because Outlook client computers can view only the first public folder tree that is linked to the default public folder store, it is not necessary for you to change the association.

6. If you have configured alternative Offline address lists, associate these address lists with your mailbox stores.
It is not necessary to associate these address lists with your mailbox stores, unless you are running either a hosted environment or virtual organizations within Exchange 2000.

7. To archive messages sent or received by mailboxes on this store, click Archive all messages sent or received by mailboxes on this store.
The archive setting is useful for auditing; however, you may need to specify in your users' organizational terms and conditions that you are archiving messages.

8. Click Browse , click the user or the folder that you want to archive, and then click OK .

9. If all of your clients that connect to all mailbox stores support S/MIME digital signing, click Clients support S/MIME signatures.

10. Click Display plain text messages in a fixed-sized font.
Use this setting if your users report that messages from external domains are not displayed properly.

11. Click the Database (Policy) tab, click a time when you want online maintenance to run or click Customize .
If you select a maintenance interval for the mailbox stores that are governed by this policy, this time determines when online compaction runs.

12. If you click Customize , either click a day in the left column, and then click a time on the top row or click and hold the mouse button as you sweep across the time slots to configure the update interval.

NOTE : To configure the schedule to be displayed in hour slots or in 15-minute slots, click the appropriate option under Detail View.

13. Click OK .

14. Click the Limits(Policy) tab, and then select one or all of the following check boxes under Storage limits :

Issue warning at : Select this check box to warn the user that their mailbox has exceeded its limit. However, normal functioning of the mailbox continues.
Prohibit send at : Select this check box to send a warning message that states that the user will not be able to send any more messages until they delete or archive their old mail. However, users can still receive messages.

15. NOTE : Type a value (in kilobytes [KB]) for the size of the mailbox, for example, 100,000 KB for normal users or 200,000 KB for the power users.

Prohibit send and receive at : Select this check box to send a warning message that states that the user has exceeded mailbox limits. If the user receives this message, the user cannot receive any messages.

16. NOTE : Type a value (in kilobytes [KB]) for the size of the mailbox, for example, 100,000 KB for normal users or 200,000 KB for the power users.

14. Either click the time that you want the warning messages to be generated in the Warning message interval box or click Customize .

15. If you click Customize , either click a day in the left column, and then click a time on the top row or click and hold the mouse button as you sweep across the time slots to configure warning settings.

NOTE : To configure the schedule to be displayed in hour slots or in 15-minute slots, click the appropriate option under Detail View .

16. After you configure the warning message interval times, click OK .

17. Set deletion item retention times for messages and for deleted mailboxes under Deletion settings .
Microsoft recommends that you set the deletion setting to 7 days for messages and 30 days for mailboxes.

18. If you want maximum control of your database sizes, configure the Keep deleted items for (days) setting to 0 . Otherwise, Microsoft recommends that you a configure this setting to 7 days .

19. Click the Full-Text Indexing(Policy) tab, and then either click a time that you want to run an incremental update of the full-text index in the Update Interval box or click Customize .

20. If you click Customize , either click a day in the left column, and then click a time on the top row or click and hold the mouse button as you sweep across the time slots to configure warning settings.

NOTE : To configure the schedule to be displayed in hour slots or in 15-minute slots, click the appropriate option under Detail View .

21. Configure the Rebuild interval setting.
If you perform a rebuild operation, the search catalog is completely indexed. Microsoft recommends that you perform a rebuild operation once a week at midnight on Saturday or Sunday.

22. Click OK .

How to Apply a Mailbox Policy to a Mailbox Store

To apply the mailbox policy that you just created to one or more mailbox stores:

1. Right-click the system policy that you just created, and then click Add Mailbox Store .

2. Double-click each mailbox store that you want this policy to control, and then click OK .

3. When you receive a message that prompts you to confirm that you want to add the items to the policy, click Yes .
The mailbox stores are now displayed in the right pane of Exchange System Manager if you click System Policies . Click the policy that you created.

How to Apply a Policy Immediately

To apply a policy immediately, right-click the policy in the left pane in Exchange System Manager, and then click Apply Now . Alternatively, the policy is applied the next time the Recipient Update Service runs.

How to Confirm That You Configured the System Policies Correctly

To confirm that the system policy has been assigned:

1. Open the properties of one of the mailbox stores to which you applied the policy. All the settings that can be configured by using the policy are shaded.

2. Open the properties of a mailbox store on which you applied the system policy, and then click the Policies tab. A list of all of the policies that apply to that particular mailbox store are displayed.

3. Click the System Policies object in Exchange System Manager. A list of policies and the time that these policies were last applied is displayed.

The End......

Dgoldman's WebLog : Exchange Information Store

How Mailbox quota limit messages can impact server performance