After reading Steve’s analogy on software development vs. building bridges, I figured it would be easier to convey some of my own thoughts in a post vs. comments.

 

Being raised around engineering and architectural practices and now working in the software industry, there are some very clear differences in the disciplines. I to get picked on by my father who is a licensed Engineer which I have a lot of respect for. I think one of the most important differences between software engineers and Engineers is this question:

 

When was the last time a Software Engineer had to carry Errors and Omissions insurance?

 

I think this conveys the liability factor state certified Professional Engineers (P.E.) carry. This insurance covers the P.E. in the event a design fails resulting in a costly mistake or worse, the loss of life. This is the critical factor differentiating these two disciplines. To date, I do not recall a situation where we have allowed commercial software to make a decision about the well being of human without the ability of human intervention. So using the bridge analogy, is there someone always standing by to intervene if a bridge is in the process of collapsing and put a halt to it?

 

Well, someone might say, what about AED (Automated External Defibrillator) for example? Yes, that is true it is making decisions through software on the treatment to administer to the heart, but consider the ranges of input and the decision tree the device has to make. Also, humans are standing by to remove the device should it malfunction. Overall the programming of such a device would be considered trivial in comparison to a commercial software application. I may be wrong…

 

Enough on that point, I think we get it. I want to dig in a bit on the differences Steve points out, he is right, software itself has very little tolerance for mistakes, but in some cases it can still run with those mistakes. One wrong choice can bring down the entire application, but is that really the fault of the software development process? Consider the framework in which an application has to work within… the hardware platform, the operating system, etc. All of these elements are what leads to a single point of failure in software. If we could build systems smarter, then this would not be an issue in conjunction with reducing the complexity. Engineering is by its nature the application of known technology, materials, and knowledge to solve a problem. Very rarely is a total new material or technique developed. Researchers develop and define the expected behavior of a widget, but it’s the Engineer that determines how the widget can be applied everyday.

 

The most compelling aspect of this debate is the sheer number of decisions that must be made in software and design specifications. Sometimes software is asked to do more than it was designed to do. The random nature of data exposes design deficiencies in software. The analogy is attempting to place an aircraft carriers on a two lane bridge. First off the Engineer will tell you that I didn’t design the bridge for that load. The engineer designed the bridge with a given criteria in mind (max GVW of 10000 lb. x 5 vehicles). But that doesn’t prevent someone from driving a few tanks over it. The same is true in software. Yes, you can attach that 5gb file to your email message, but that doesn’t mean the software was designed to handle a file that large. As in bridge design, software is designed with certain criteria in mind, but the difference is software is expected to handle all possible criteria.

 

I am really at odds in regards to the comment regarding reinventing the wheel every time. If I recall correctly, only 30% of the code changes from one product revision to another. Even then the fundamental elements (algorithms, data types, etc.) are still reused in code, its how they are organized. The same with building a building. Engineers simply reorganize beams, rivets, etc. Software development bridges the gap between what is considered “pure research” and the application of the research (engineering). In many cases the architect or developer has to fill both shoes. But once a model of a widget is in place (research phase), engineering principals come into play in how we approach the refinement, testing, and documenting of the widget.

 

In summary, we should honor the roots of engineering and apply tried and true principals where and when it is applicable in our development and support processes.

As Naresh pointed out in his history of content conversion blog, Exchange 2000 swapped the roles of our primary transport from X.400 to SMTP. As such we also more closely integrated in native MIME formatted message into Exchange store by introducing the .STM file.

 

We are unfortunately bound by the fact that our richest client (Outlook) still requires messages to be presented in a MAPI format. Up until Exchange 2000, message conversion took place immediately upon receipt of the Information Store (Exch5.5), or when received by the IMC (Exch4.0 and Exch5.0). With Exchange 2000 we introduced a delayed conversion model whereby full conversion of the message would only take place when it was absolutely needed. This introduced several phases of conversion.

 

The first phase is when a message is first received in native SMTP/MIME format and routed into the Information Store, then entire message is referenced by pointer to the message in the STM file. At this point, the message is lives in the SMTP temp tables while the delivery of the message is being handled.

 

In the next phase we parse enough details from the SMTP header message, such as Sender, Date, Receiver, etc. to render the presence of a message in a folder. (think about the information you see when looking at a list of messages in Folder in Outlook) At this stage we also take a look over the MIME data to determine if we have any attachments and if any of the attachments are of type ms-tnef. If a TNEF type is encountered, then all aspects of the message are immediately converted into MAPI format. Such as when a message has arrived with the same organization between two Outlook Clients. Otherwise, we determine if any attachments exists to accurately display the “paper clip” icon beside the message in Outlook. At this point the message is delivered into the correct Inbox folder based upon the recipient.

 

When the an Outlook client attempts to open the message, we simply convert the appropriate bodypart of the MIME message on the fly from MIME to RTF and present it to the client. In addition we reparse the MIME data to retrieve the details of the attachment from the MIME bodypart header and promote those into the EDB file. When the client closes the message, the message body still remains in the STM file and the attachment details remain in the EDB. If the client attempts to forward the message, all aspects of the message are converted from MIME to MAPI format and the representation of the message in the STM file is removed if no one else is using it.

 

If the client attempts to open an attachment, all aspects of the attachment are promoted to the EDB file before being presented to the client. Note that during this time the original MIME message still resides in the STM file, including the attachment information to preserve the original MIME until complete conversion has been performed. The promotion is necessary to maintain the integrity of the original MIME data.

 

Finally if the client modifies an attachment or message body, the entire message is promoted to the EDB and references to the original MIME stream are removed from the STM file. A forward operation is an operation that would also force the complete conversion of the message.

 

Such is the life of an inbound message in Exchange…

 

This is an issue that I ended up spending 6 hours debugging, only because the answer could not be found in any documentation. I know as soon as I post this someone will go post a comment saying "duh, its documented right here <link>"

The scenario is a customer was attempting to give permissions to modify certain Active Directory attributes of Exchange users (e.g. Hidden from Address Book) to a subset of administrators. The permissions where correctly set on the attributes and they could be modified using other tools, however in Active Directory User and Computers, the check boxes corresponding to those attributes where greyed out.

Now, let me preface this and say that I am not an expert when it comes to migration, a guru of what permissions to use in certain cases, etc. I am just stating the facts...

The extension dll (maildsmx.dll) used to manage Exchange users employes logic to only muck with users that are truely "Exchange" users. This includes users on Exchange 2000/2003 and Exchange 5.5. Exchange 2000/2003 users are easily identified, but Exchange 5.5 require an extra step in validating if they should be administered as an Exchange user. This is where the problem occurs, to determine if they are valid 5.5 recipients, a look up is performed against the Active Directory Connector's Connection Agreements. The configuration of the connection agreements are stored under CN=Active Directory Connections,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=Domain,DC=com, which non-administrator accounts do not have access to by default.

In order to make this work, the group in question had to be given Read Only rights to this portion of the AD to determine if the user being administered is being handled by one of the ADC connection agreements.

Learning Essentials

 

Ever wonder how we internally develop our people? How our top people continue to develop? As one might expect we do REQUIRE the technical staff of Product Support Services (PSS) to maintain their MCSE/MCSD certification. However, this is largely accomplished on the individual's own time. The program new hires are put through depends largely on the major product they will be supporting. In Exchange, approximately 3 weeks of compressed training covering topics on the Operating System, Active Directory, and Exchange are covered. In addition further training is provided by the “specialty team” the individual will be a member of such as Administration, Client, etc. In the majority of cases, this training stems directly from the content and whitepapers that has been published and freely consumable by customers.

 

Readiness and Sustained Education is the program internally of providing training related to new products and features such as Service Pack releases, feature additions, depth knowledge in a product, etc. Various teams work together to determine where knowledge is lacking based upon analysis of the cases that are coming in. For example, Storage Area Network concepts. Not directly related to Exchange, but engineers are required to have some conceptual understand of the devices. Further training is also provided in the Exchange group in a triage format where engineers take turns covering topics they have strength in such as performance monitor analysis, cluster recovery, mail flow troubleshooting, etc.

 

So what about Escalation Engineers? It isn’t what you would typically think… years of technical training? Not even close. It all about architecture and fundamentals. In a recent internal meeting, a question was posed to a panel of EE on how do they stay abreast of technology? The answer was simple, we don’t. We rely upon the fundamental understanding of how things work. What I am referring to is how the Operating System Platform works, specifically the interaction of a process running on the operating system.

 

Regardless if you are running the basic command line application or a sophisticated server based application as Exchange, the rules are the same.  If you want to synchronize threads within the process you use one of half-dozen synchronization mechanism (e.g. critical section, mutex, timed event, etc.). If you want to transmit something over TCP/IP from a usermode application you have to use Winsock or some wrapper for it. If you are in kernel mode, then you have to submit requests via IRP (I/O request packets) to the tcpip driver. The process of allocating memory and interfacing with the network all require adherence to the rules set forth by the operating system.

 

So for example say we introduce a new log file replication service. Here would be my immediate questions to attempt to answer:

 

Is a push or a pull scenario?

Who initiates the replication?

How frequent is the replication?

What is the rule used to govern when something should be replicated?

What communication mechanism does it use? RPC? Straight Sockets?

How does it know about it target replication partner?

 

These questions are applicable to any replication scenario; these are the rules of replication so-to- speak… So where is the new information here you might ask? We’ll this may not be new, but when it comes down to applying these concepts, people get caught up in the details and forget to understand the big picture.

 

Lastly, the major key attribute in EEs is their attitude. There is an inherent drive to learn, to develop continuously, and most importantly accept responsibility for resolving a customer issue.

 

So what do I recommend as essential reading? It would be a course on the platform, programming, and debugging. Regardless of the application, it will adhere to using the facilities provide by the operating system. Understanding how the platform works provides the essentials to be able to look at and reverse engineer any application on the fly.

 

• Inside Windows 2000, Third Ed. (ISBN 0735610215)
• Programming Server-Side Applications for Microsoft Windows 2000 (ISBN 0735607532)
• Programming Windows, Fifth Edition (ISBN 157231995X)
• Windows NT Device Driver Development (ISBN 1578700582)
• The C++ Programming Language (Special 3rd Edition) (ISBN 0201700735)
• Algorithms in C++ (ISBN 0201510596)
• Debugging Applications (ISBN 0735608865)
• The help file with Windows Debuggers download (http://www.microsoft.com/whdc/devtools/debugging/default.mspx)
• Transaction Processing : Concepts and Techniques (Morgan Kaufmann Series in Data Management Systems) (ISBN 1558601902)

 

 

In my role I interact with a lot of developers and it is continuously amazing how naive an individual can be regarding the interactions their own code has with other components. I'll shorten this example to the critical points... In one instance I recall conversing with a developer regarding a performance issue in their code that was causing delays responding to requests. In this code path we were looking for configuration information to make a decision on on how to handle a particular request. The lookup of the configuration information was resulting to a call to the Active Directory. In this customer's case they had a stale DNS record that was periodically impacting the lookups to GC/DCs. Because the developer that wrote the client code wasn't aware of the process of the lookup, which was written by another team, they didn't realize a dependency was added to their code path for outbound RPC, network connectivity, and inevitability DNS.

This situation took over two weeks to nail down. Why did it take two weeks? Several factors, issue timing, customer scheduling, data transfer...

1. It wasn't clear from the customer's explanation of the problem if the performance issue was server or client side. Needed to take a network monitor trace between the client and server. Looking at the trace, the requests where clearly making to the server and the server was taking a long time to respond.
2. Hmm, so the problem is server side, let take a look at perfmon - yep request taking a while to respond to the client. Guess need to take some user mode dumps of the process.
3. Found the client request thread... hmm.. its waiting on a DS Lookup. Looking backwards in the stack we find where the lookup is made. When I simply discussed it with the dev, they were very apprehensive to believe it is occurring until I showed them the call stack. The dev just discovered the dependency... opps. They were never aware of it because it always just "worked".
4. Looking at a second netmon taken server side, we could see the server attempting the communication to the directory and timing out, which eventually led us to the stale DNS record.

This is just one example of how important it is to be aware of who is calling your code and what components you call and the interaction of each. You may have the most benign piece of code that does a quota calculation doing a user lookup that isn't itself critical, however someone could easily pickup and use it in a critical code path. Now that benign piece of code all of a sudden becomes critical since it is now called anytime some saves data.

This might sound silly and most people may respond with a big "Duh, I know that", but it still happens more often than not.

I would highly recommend when looking at design elements, look at them from END to END, not just your piece of the puzzle. Document the critical code paths and know the components used at every stage in those critical code paths. Customer don't call stating that "Hey that configuration caching thingy is behaving poorly".

You need to know your Ins and Outs... What are the paths where I accept data and process it and what are the paths where I communicate externally. How can I know how far a request has been down a critical code path and where did the request have problems?

This leds to my next post... supportability in your code.

I have seen this problem several times in the last few weeks and figured it was worth a quick blog entry.

 

Here is what happens:

1. A Unix host with a running daemon sends some sort of email message a User reporting on the status of processing a request.
2. The recipient of the message is on an Exchange Server.
3. The recipient is removed from the Exchange organization.
4. The automated mailing hits the Exchange Server and is NDR’ed back to the sender (e.g. root@host).
5. The Unix receives the NDR, but root is aliased or .forwards to a group or individual on Exchange contains the original recipient in Step 1
6. Exchange NDRs the message back to the Unix host, which in turn the NDR is sent back to individual or group and Exchange NDRs the message back to Unix
7. For each NDR, the original message is encapsulated in the NDR

 

The reason this loop persist is each message receives a new Internet Message ID, thus it is considered a new message. With changes introduced in Exchange 2000 Post Service Pack 3 ( >= 6.0.6521) and in Exchange 2003 we set a maximum nested limit of 250 attachments. This limit can be adjust using a registry key entry outlined below. Eventually the nested depth of NDRs reaches 250 and Exchange begins NDR’ing the message due to Attachment limit being exceeded. At this point the message is ~3mb in size. This loop continues indefinitely as each NDR received by the Unix host is forwarded to the recipient which in turn is NDRed back to Unix.

 

For each message generated by this daemon running on Unix, a seperate looping message situation is created. So assuming over a 24 hour period 1 message is generated every 15 minutes, 96 messages are floating around the system each 3mb in size. By the end of the day 300mb of messages are continously trying to be processed. The eventually leds to the very poor performance from the server and growing STM files.

 

This problem can be prevented by aliasing/.forwarding the Unix contacts to a Distribution List on Exchange that is configured not to NDR messages. This way individuals can come and go from the DL and any delivery failures do not go back to the host.

 

To adjust the limits on number of attachments per message:

 

1. Start Registry Editor.

 

2. Locate and then click the following registry key:

 

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS\Parame

                tersSystem\InternetContent

 

3. On the "Edit" menu, click "Add Value".

 

4. In the "Value Name" box, type "LimitBodyParts" (without the

   quotation marks).

 

5. In the "Data Type" list, click "REG_DWORD", and then click "OK".

 

6. In the "DWORD Editor" dialog box, click "Decimal". In the "Data"

   box, type the number of attachments that you want to set the limit to,

   and then click "OK".

Note:

When LimitBodyParts is set to a value of less than 250 or not present, a default value of 250 is used. The value can be set as high as 4294967295 (0xFFFFFFFF).

 

To prevent the original message from being attached to a NDR, see the following article:

 

http://support.microsoft.com/default.aspx?scid=kb;en-us;308303

 

 

 

[EDIT: 9/21/04 - Added Note regarding acceptable values of LimitBodyParts regkey] 

Like most people, I work on a collection of work-in-progress documents. Over time I started loosing track of where the latest version of a particular document was left... my desktop, tabletpc, laptop, or on my home machine.

Then I remembered a relatively unused feature of Exchange+Outlook which is “Free Docs”. If you didn't know already, you can directly drag a document into a folder in your mailbox. Here you can leave the document while you edit and develop it over time. Coupling this with Outlook Local Cache Mode, you can keep multiple clients in sync with the latest version of a document regardless of where you are or which client you decide to use.

 

 

Overview

IS Maintenance is the term applied to a series of operations performed by the Information Store to ensure logical consistency in the databases. Generally speaking the operations ensure items no longer needed by the system are removed from the database. If a custom schedule has not been defined for the database in Exchange System Manager, a default scheduled period of 12-5AM local time will be used.

 

Overall, there are 10 tasks performed by the Information Store during this period:

1. Purge Indices (public and mailbox stores)
2. Tombstone Maintenance (public and mailbox stores)
3. Dumpster Cleanup (public and mailbox stores)
4. Public Folder Expiry (public stores)
5. Age Folder Tombstone (public stores)
6. Folder Conflict Aging (public stores)
7. Update Server Versions (public stores)
8. Cleanup Secure Folders (public stores)
9. Cleanup Deleted Mailboxes (mailbox stores)
10. Check Messages Table (public and mailbox stores)

 

If the maintenance window ends before one of the ten tasks above is completed, the last one running is recorded. The last task will be allowed to run until completion, which could exceed the maintenance window. During the next maintenance window the information store determines the last task in progress and restarts the process, essentially picking up where it left off on the next task.

 

The progress of the tasks above can be monitored by adjusting the Event Logging Level to minimum for the Exchange Information Store Mailbox/Public. When set to minimum, the following events are logged:

 

Event Type:   Informational

Event Source: MSExchangeIS Public / Private

Event ID:     1208

User:         N/A

Description:

Starting the IS Maintenance tasks.

%n%nFor more information, click http://www.microsoft.com/contentredirect.asp.

 

Event Type:   Informational

Event Source: MSExchangeIS Public / Private

Event ID:     1210

User:         N/A

Description:

The IS Maintenance task <subtask name> completed.

%n%nFor more information, click http://www.microsoft.com/contentredirect.asp.

 

Event Type:   Informational

Event Source: MSExchangeIS Public / Private

Event ID:     1209

User:         N/A

Description:

The IS Maintenance tasks completed.

%n%nFor more information, click http://www.microsoft.com/contentredirect.asp.

 

 

 

I want to delineate the 11th task performed, Online Defragmentation, which is not performed by the Information Store but by the underlying database engine (ESE).  At the completion of the above list of tasks, the Store process initiates Online Defragmentation by calling into (ESE) to inform it to start the defragmentation process. It is only after at least 1 of the 10 tasks above have been successfully completed will this process start.

 

The progress of online defragmentation can be monitored by reviewing the application log without adjusting the Event Logging levels.

 

When the Information Store initiates ESE online defragmentation for a database, the following is logged:

Event Type:   Informational

Event Source: ESE

Event ID:     700

User:         N/A

Description:

%1 (%2) %3Online defragmentation is beginning a full pass on database '%4'.

%n%nFor more information, click http://www.microsoft.com/contentredirect.asp.

 

If a full pass of online defragmentation completes within the maintenance window the following is logged indicating it has completed.

Event Type:   Informational

Event Source: ESE

Event ID:     701

User:         N/A

Description:

%1 (%2) %3Online defragmentation has completed a full pass on database '%4'.

%n%nFor more information, click http://www.microsoft.com/contentredirect.asp.

 

If a pass of the online defragmentation was interrupted due to window expired or backup initiated, the following event is logged:

Event Type:   Informational

Event Source: ESE

Event ID:     704

User:         N/A

Description:

%1 (%2) %3Online defragmentation of database '%4' was interrupted and terminated. The next time online defragmentation is started on this database, it will resume from the point of interruption.

%n%nFor more information, click http://www.microsoft.com/contentredirect.asp.

 

If the previous online defragmentation cycle was previously interrupted before it completed a full pass of the database the following is logged instead of the event above:

Event Type:   Informational

Event Source: ESE

Event ID:     702

User:         N/A

Description:

%1 (%2) %3Online defragmentation is resuming its pass on database '%4'.

%n%nFor more information, click http://www.microsoft.com/contentredirect.asp.

 

If a resumed pass of the database completes within the maintenance window the following is logged:

Event Type:   Informational

Event Source: ESE

Event ID:     703

User:         N/A

Description:

%1 (%2) %3Online defragmentation has completed the resumed pass on database '%4'.%n%nFor more information, click http://www.microsoft.com/contentredirect.asp.

 

Things to keep in mind…

An important note to make about IS Maintenance and online defragmentation is the large number of changes made to the database.  To maintain recoverability of the database, each of the changes made by these tasks are also transacted in the transaction logs. Depending upon the number of changes made in the database during these tasks, a large number of log files can be generated during the maintenance windows.

 

Backups schedules should also be monitored so they do not overlap with the IS Maintenance schedule. Backup will not preempt the 10 initial tasks performed by the Information Store, therefore they will run simultaneously. However, when initiating a backup of a database, the online defragmentation process is suspended until the backup is completed.

 

Below is a list of every action during Online Maintenance and the possible configuration operations to control their behavior.

 

Purge Indices (public and mailbox stores)

One of the benefits of the Extensible Storage Engine (ESE or aka JET) is its ability to dynamically create indexes when information is resorted or requested in a different sort order. Unchecked, the Information Store database could end up with thousands of indexes on a table.

 

The Information Store handles this situation by keeping track of indexes by assigning them an expiry time. Information about the index is added to an internal table referred to as the indexing aging table. During online maintenance the index aging table is walked and each entry’s value of ptagIndexDeleteTime is compared to the current time. If the index has expired, the index is removed. This process runs every 24 hours during the Information Store maintenance cycle and removes entries older than 8 days for Exchange 5.5 and 40 days for Exchange 2000/2003 by default.

 

Default Inverval:               Every 24 hours

Default Criteria:                 Remove Indexes older than 8 days on Exchange 5.5

and older than 40 days on Exchange 2000/2003.

Configurable Settings:

 

DC=<Domain>

CN=Configuration

              CN=Services

                     CN=Microsoft Exchange

                           CN=<ExchOrgName>

                                  CN=Administrative Groups

                                         CN=<site>

                                                CN=Servers

                                                       CN=<servername>

                                                              CN=InformationStore

                                                                     CN=<Storage Group>

                                                                           CN=<Database>

 

Object Class:              Database Object

Attribute Name:            msExchAgingKeepTime

Description:               The amount of time in milliseconds seconds to keep unused indexes and views in the database. This value is applies to Exchange 2000/2003 and supersedes the registry values below.

 

 

 

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS\ParametersSystem

 

Value Name:                Aging Keep Time

Radix:                     REG_DWORD

Description:               The amount of time in milliseconds seconds to keep unused indexes and views in the database. This value is honored by both Exchange 5.5 and 2000/2003.

 

Value Name:                Aging Clean Interval

Radix:                     REG_DWORD

Description:               The interval (in milliseconds) at which the store attempts to purge unused indices. This value is honored by both Exchange 5.5 and 2000/2003.

 

For Exchange 5.5:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\ParametersPublic

- or –

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\ParametersPrivate

 

For Exchange 2000/2003:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\<ServerName>\Public-<Public Folder GUID>

-or-

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\<ServerName>\Private-<Public Folder GUID>

 

Value Name:                Reset Views

Radix:                     REG_DWORD

Description:               Setting this value to 1 will cause the Store to remove all indexes regardless of age on the next cycle of IS maintenance for the corresponding database. The value is then set back to 0. By setting this value, clients may experience initial delays rebuilding previously cached views.

 

Tombstone Maintenance (public and mailbox stores)

Each folder maintains a list of deleted messages for a folder in ptagMidsetDeleted on the folder object itself. When a message is deleted from a folder, an entry is made in ptagMidsetTombstones. If this folder is replicated, this gives an indication of the messages delete operations that need to be propagated. When the propagation occurs, the entries are moved from ptagMidsetTombstones and merged into ptagMidsetDeleted. If a folder is not replicated, it is necessary to clean up this situation through the Information Store maintenance cycle. This process runs every 24 hours and is not configurable.

 

Dumpster Cleanup (public and mailbox stores)

When a message is deleted from a client a simple flag is set to indicate (ptagMsgDeleted) the message is deleted. By default a client will not display messages that have this flag set. In order to view these messages you use the dumpster functionality in the client. If the dumpster is not turned on, messages that have been deleted will be removed from the folder when this process runs during the maintenance cycle. If dumpster functionality has been enabled, only messages that have been deleted and have exceeded the retention period will be removed during this maintenance routine.

 

This process also checks for folders that have exceeded the retention period and will remove them as well.

 

Configurable Settings:

 

For Exchange 5.5:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\ParametersPublic

- or –

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\ParametersPrivate

 

For Exchange 2000/2003:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\<ServerName>\Public-<Public Folder GUID>

-or-

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\<ServerName>\Private-<Public Folder GUID>

 

Value Name:                Deletion Thread Period

Radix:                     REG_DWORD

Description:               The interval (in seconds) at which the store attempts to purge unused indices. This value is honored by both Exchange 5.5 and 2000/2003.

 

Public Folder Expiry (public stores)

This is a maintenance process of traversing through all the messages in Public Folders and removing the ones that have exceeded the expiry time.

 

Configurable Settings:

 

For Exchange 5.5:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\ParametersPublic

 

For Exchange 2000/2003:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\<ServerName>\Public-<Public Folder GUID>

 

Value Name:                Replication Expiry

Radix:                     REG_DWORD

Description:               The interval (in milliseconds) at which the store removes items that have exceeded the retention period.

 

Age Folder Tombstone (public stores)

This maintenance process removes deleted public folders that have exceeded the tombstone lifetime (180days default). When a Public Folder is deleted, it is tombstoned for replication purposes. This allows for items that are deleted to be tracked during the replication process. This removes a maximum of 500 folders per a 24 hour period.

 

Configurable Settings:

 

For Exchange 5.5:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\ParametersPublic

 

For Exchange 2000/2003:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\<ServerName>\Public-<Public Folder GUID>

 

Value Name:                Replication Folder Tombstone Age Limit

Radix:                     REG_DWORD

Description:               The interval (in days) to keep tombstones.

 

Folder Conflict Aging (public stores)

When two users have edited and modified the same message in a public folder, the resulting saved messages are defined as being in conflict.  This can happen when a simultaneous save occurs on the same public folder server, or if the message is edited on two servers that contain replicas of the folder.  If no action is taken by the owners of the Public Folder within Conflict Age Limit, this process takes the appropriate action based upon the property PR_RESOLVE_METHOD on the folder itself. The resolution method is defined in the Exchange System Manager on the Public Folder object.

 

Configurable Settings:

 

For Exchange 5.5:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\ParametersPublic

 

For Exchange 2000/2003:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\<ServerName>\Public-<Public Folder GUID>

 

Value Name:                Replication Folder Conflict Age Limit

Radix:                     REG_DWORD

Description:               The interval (in days) items in conflict are allowed to exist. 

 

 

Update Server Versions (public stores)

This process simply updates the version information as necessary for any public folder databases containing a replica of the system configuration folder. No configurable settings are available.

 

Cleanup Secure Folders (public stores)

This process is only valid for Exchange 5.5. This process attempts to age out folders that have been secured in other sites and the site is no longer present in the directory. Each folder is examined and for each secure folder where it’s home site is not visible in the directory a timer is set to remove the folder. The default is 7 days.

 

Configurable Settings:

 

For Exchange 5.5:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\ParametersPublic

 

Value Name:                Secure Folder Aging Task Frequency

Radix:                     REG_DWORD

Description:               The interval (in days)

 

Site Folder Check (public stores)

Every 1 hour a public database will check to ensure no duplicate site folders exists within an Administrative Group and removes them. No configurable settings are available for this process.

 

Cleanup Deleted Mailboxes (mailbox stores)

This process cycles through mailboxes that currently do not have Directory Service objects. After a default of 30 days, the mailboxes are deleted. This process runs every 24 hours and no interval adjustments are available.

 

Configurable Settings:

 

DC=<Domain>

       CN=Configuration

              CN=Services

                     CN=Microsoft Exchange

                           CN=<ExchOrgName>

                                  CN=Administrative Groups

                                         CN=<site>

                                                CN=Servers

                                                       CN=<servername>

                                                              CN=InformationStore

                                                                     CN=<Storage Group>

                                                                           CN=<Database>

Object Class:              Database Object

Attribute Name:            msExchMailboxRententionPeriod

Description:               The amount of time in seconds to keep retain mailboxes without a corresponding Directory Object.

 

 

 

Check Messages Table

This process traverses the message table for a given database and looks for messages that currently do have a reference count of 0 indicating no folder currently has a reference to the message. Messages encountered are deleted. A maximum of 50000 messages will be deleted in a given cycle. This process runs every 24 hours and no interval adjustment is available.

 

Online Defragmentation

If at least one of the maintenance task was able to complete within the maintenance schedule, the Information Store requests that ESE begin an online defragmentation cycle and complete a single full pass of the database. This process will run for at least 15 minutes at the end of the maintenance window.

 

The intention for online defragmentation is to free up pages in the database by compacting records onto the fewest number of pages possible, thus reducing the amount of I/O necessary. The ESE database engine does this by walking the database metadata (information in the database that describes tables in the database) and for each table, visit each page in the table and attempt to move records onto logically order pages. If you have an understanding of BTrees, we start at the furthest page to the right and begin compressing records to the left most page. This does not necessarily mean the pages are in order, but the movement is from a logical perspective.

 

It is possible this process does not complete before the time indicated by the Information Store. In such a case the ESE engine stores the last table successfully processed. At the next Online Defragmentation request, ESE picks up on the next table in the metadata will complete the work until all tables have been processed or the maintenance window ends. If Online Defragmentation is able to process all tables before the maintenance window ends, it will restart at the first table listed in the metadata and continue processing until all tables are processed or the maintenance window ends.

 

It is import to point out however, if IS maintenance is set to always and one of the above tasks completes (note some only run once every 24 hours), then Online Defragmentation could be invoked.

 

Configurable Settings:

 

For Exchange 5.5:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\ParametersPublic

 

For Exchange 2000/2003:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\<ServerName>\Public-<Public Folder GUID>

-or-

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\<ServerName>\Private-<Public Folder GUID>

 

Value Name:                OLD Minimum RunTime

Radix:                     REG_DWORD

Description:               Amount of time to run after completing at least 1 of IS Maintenance tasks (in minutes)

                           Default value of 15 minutes

 

Value Name:                OLD Completion Time

Radix:                     REG_DWORD

Description:               The amount of time (in seconds) beyond the maintenance window Online Defragmentation will be allowed to run.
Default value of 1 hour (3600sec)

 

EDITS:
9/9/04 - Adjusted units for Aging Keep Time from milliseconds to seconds.

9/28/04 - Clarified Online Defragmentation process

9/30/05 - Changed "Deletion Thread Period" milliseconds to seconds