Microsoft Records Management Team Blog

Announcing the DoD 5015.2 Resource Kit for SharePoint Server 2007

Ethan Gur-esh — Sat, 09 Feb 2008 12:05:00 GMT

[Cross-posted from the SharePoint Team Blog.]

Back in May of last year, we announced that MOSS 2007 had passed the U.S. government's DoD 5015.2 certification. Now, all of the work that was done to achieve that important milestone has been packaged and released as a resource kit.

The DoD 5015.2 Resource Kit is intended only for customers, who are required to run their records management system in a DoD 5015.2 Chapter 2 certified state. In case you're wondering, we currently have no plans to provide support for DoD 5015.2 Chapter 4 (classified records).

However, the Resource Kit is not intended for customers, who would like to enhance the records management functionality of MOSS 2007 with particular 5015.2 oriented features but are not required to run their system in a certified configuration. Alternatively, sample code and documentation will be available shortly via the MSDN Dev Center for MOSS 2007 for the most frequently requested features such as:

Generating a Unique ID
Display search results in Grid View
Referencing and Linking - Using a secondary table to maintain the relationship
Referencing and Linking - Using the asset picker
Referencing and Linking - Using a custom field to display the relationship
Create Document Library from a list item
Put multiple items on Hold using SPQuery
Custom expiration formula based on metadata (2 parts)
Custom expiration formula based on metadata (cont.)
Creating a custom router & extracting date received (2 parts)
Creating a custom router & extracting date received (cont)
Prevent record modification but allow metadata modification in event handler
Prevent record modification but allow metadata modification by overriding upload page
Programmatically defaulting/inheriting a field value
Using ProcessBatchData to batch update list items
Using Metadata tags to route documents into document libraries
Create a record category programmatically (2 parts)
Create a record category programmatically (cont)
Routing content to a record category based on a metadata value

Officially, the DoD 5015.2 Resource Kit for SharePoint Server 2007 is now available to customers through our partner and Microsoft Consulting Service (MCS) channels. The reason for requiring partner/MCS involvement is that DoD implementations are not as simple as a "double-click installation." They require deep understanding of records management practices as well as business process and policy requirements. In order to successfully implement a DoD 5015.2 certified solution based on MOSS 2007, customers should work with their respective Microsoft account managers to engage a partner that has been trained on the DoD 5015.2 Resource Kit. Partners may download the Resource Kit's source code for free via MS Connect's DoD 5015.2 Resource Kit Source Code download site.

Also on the download site is a DoD 5015.2 Resource Kit Installer that partners as well as customers can download only for evaluation purposes and not for production use. This additional download provides a way for evaluators to learn about the DoD 5015.2 oriented records management capabilities provided by the Resource Kit while deciding which features, if not all, will be suitable for addressing their organization's requirements.

For the latest answers to frequently asked questions about the DoD 5015.2 Resource Kit for SharePoint Server 2007, go to http://www.microsoft.com/sharepoint/capabilities/ecm/dod5015.mspx.

We're taking a break...

Ethan Gur-esh — Wed, 28 Mar 2007 07:28:00 GMT

We are taking a break from posting on this site because all of the topics that we had wanted to blog about have now been posted. Moreover, Records Management (RM) is a feature area that is part of SharePoint Server’s larger set of Enterprise Content Management capabilities, which also include Document Management (DM) and Web Content Management (WCM). It’s very likely that we will merge this blog with the ECM Team Blog. If you have RM related topics that you’d like to see us blog about, please leave a comment here.

In the meantime, you can participate in community Q&A and discussion in the SharePoint ECM forum at http://forums.microsoft.com/MSDN/ShowForum.aspx?ForumID=1213&SiteID=1.

E-mail Records Management Part 4: Quota Management and Sending E-mail to the Records Center

Ethan Gur-esh — Fri, 09 Feb 2007 00:21:16 GMT

Greetings, everyone! In this post, we are going to conclude our discussion of e-mail records management by talking about some of the other policies that can apply to Microsoft Exchange 2007’s managed folders. If you haven’t read the previous posts on e-mail management, I suggest that you start from the beginning.

In this post, we’ll start by talking about mailbox quota management. In the “old days,” quota management was all about enforcing one size quota on an entire mailbox (e.g. all users have 1GB of mail storage). With managed folders, an Exchange administrator can set a quota on each folder. When a particular managed folder is over its quota, Outlook will notify the user directly in the client. There is no need to send an “over quota” e-mail, and, once again, this is an example of the Office user interface educating users about corporate policy.

For instance, suppose that a managed folder, “Non-Work Related,” has a 25 MB quota. When the Non-Work Related folder is over its quota in a user’s mailbox, it is flagged in red, and the policy statement alerts the user:

Note: Red circles were added for clarity.

In this example, the IT-related goal is to minimize the amount of corporate resources dedicated to non-work related e-mail while still giving end users a lot of control over their own inboxes.

While this approach is no panacea, it is a way of encouraging good user behavior. For instance, I know many of us (myself included) take the following approach to managing our mailbox quotas: Sort the inbox by mail size and delete the largest items without regard to corporate retention policy. Folder quotas are a more precise tool to help IT departments manage storage costs while reducing the impact on corporate retention policy.

Now I’d like to switch gears and describe a policy related to managed folders than many readers of this blog have been waiting for: sending e-mail to a records repository. Any managed folder can be configured such that all e-mail sent to that folder will be journaled off to a records repository, such as a SharePoint Records Center.

What does this mean? E-mail messages and their attachments are sent off to SharePoint along with an extract of their metadata (e.g. From, To, CC, BCC, Subject). They are then passed through the same records routing framework that documents from SharePoint go through when they are sent to a Records Center site. By configuring the Records Routing list, records managers can route e-mail messages to particular document libraries based upon a label associated with each managed folder. For example, all e-mail dragged to an “Agreements with Partners” managed folder might be routed to a “Contracts” document library. Once in this library, the e-mail would pick up the same Contract metadata schema and expiration policy as Microsoft Word contracts stored in that library. This allows for the management of e-mail records alongside documents and other types of records using a submittal method that end users already use every day (i.e. dragging and dropping to folders).

One obvious question about this process is, “What about metadata on the e-mail record?” On the one hand, e-mail has a larger amount of system-generated metadata than documents do (To, From, CC, BCC, Subject). However, unlike SharePoint, Exchange and Outlook don’t have a way to collect custom metadata on an item (e.g. “Date this contract was signed.”). By taking the e-mail out of Exchange and into SharePoint, we expose the ability to collect a rich amount of metadata about the items. Not to mention, we also have other advanced records management functionality: complex disposition, workflows, and holds are just a few examples.

How does the user enter that metadata? To decrease the tax on the end user during the submission process, we don’t collect the metadata immediately when the user drags the e-mail to a managed folder. Instead, SharePoint queues the submitted messages in a holding zone. After a specified period of time, SharePoint sends one e-mail to the submitter asking him to fill out the missing metadata on all pending items that he owns. We then provide spreadsheet-like UI for bulk entering metadata:

Organizations can choose how often they want to collect the metadata, and it’s really dependent on the individual business process. Highly regulated organizations with lots of records may send out the e-mail reminders often. Other companies may ask their employees to spend a few minutes on Friday updating the metadata for the e-mail records submitted that week.

Getting users to enter metadata is always a challenge in any records management project, especially e-mail. We’ve designed this process to minimize the impact on the user’s everyday work and to make the bulk entering of metadata as easy as possible.

Of course, if there is no need to collect extra metadata, the email message is sent to the correct document library without any further user intervention.

So that concludes our 4-part tour of e-mail records management. As always, keep the questions and comments coming. We only hit the tip of the iceberg with regards to Exchange 2007’s compliance features. I encourage you to check out a couple Exchange blog posts (here and here) for more information. Their blog is targeted to the Exchange administrator, but there is still good information for records managers in there.

Thanks,

Adam Harmetz

Program Manager

E-mail Records Management, Part 3: E-mail Retention

Ethan Gur-esh — Thu, 18 Jan 2007 03:56:09 GMT

In my previous post, I described how organizations can define a set of e-mail classifications (i.e. managed folders) and how end users can use those folders to classify content. In this post, I’ll describe one of the main uses of this classification system: e-mail retention policies.

I mentioned in the last post that one of the first steps of an e-mail management plan is for a records manager to create a list of all the types of e-mail content within an organization. During this process, a records manager should also define the retention periods for each class of e-mail.

For instance, a records manager may decide that “Research and Development Design” e-mails should be kept for three years and then deleted. She may also define a broader and more generic class of content called “Long Term Business Need,” which might have the policy that e-mail must be kept for five years. Generally, the policies for e-mail will closely match the retention rules for managing documents in Microsoft Office SharePoint Server 2007. For example, both product specification documents and e-mail messages in the category “Research and Development Design” would be kept for three years and then deleted.

As you might expect, these e-mail retention rules are made concrete when a managed folder is assigned a retention policy for the mail inside it. This is done on Microsoft Exchange Server 2007 and the rules are enforced by Exchange, not Outlook. This allows an IT department to centralize the management and enforcement of the rules (e.g. deletion of the e-mail) without having to configure an application on each end user desktop.

The types of retention rules that can be defined are fairly broad. The event that triggers the expiration of an e-mail is based upon an auto-captured piece of metadata, such as the date the e-mail was sent. Exchange provides both move and delete as possible expiration actions (more on that below).

E-mail retention rules can also vary depending on the type of content in the folder. For instance, voice-mail messages might only be kept for 60 days, while e-mail message might be kept for a year. (As an aside, this allows policy decisions to be made based upon the medium and not just on the content itself. There was a really interesting comment thread that talked about whether that’s appropriate).

In addition, a records manager can also specify policies on non-managed folders, such as the Inbox and Sent Items folder. There is also a “default policy” that will be applied to all user-created folders. Generally, these policies should have a retention period shorter than the periods on the managed folders. This will encourage users to classify e-mail that they want to keep by moving it out of their Inbox and into a long term managed folder.

As it’s been described so far, the feature has only addressed the pure compliance scenario: non-important e-mail will be deleted; important e-mail will be kept for as long as there is a business justification for it. However, if we stopped there, we’d have introduced a major issue with our approach. This is the “So what did you do with my e-mail?” problem. Any e-mail records management solution should be sensitive to end users’ need to know what’s happening to their e-mail messages. We’ve done a couple things to address this.

Sharp-eyed readers of my previous post will notice one of the primary things we’ve done to make people comfortable with e-mail retention. We’ve provided records managers with a way to communicate the corporate policy on a folder. Within Outlook 2007, every folder displays a policy statement provided by the records manager:

Just like the Information Policy Bar in Word, Excel, and PowerPoint, this policy statement supports the communication of corporate policy directly within the Office applications. This is valuable irrespective of our policy enforcement features. Instead of having to visit a hard-to-find corporate intranet site or watch a video, users can learn about corporate compliance directly within the application they work with everyday.

Customers can also use a “recycle bin” approach as a backstop to prevent the accidental deletion of important e-mail. Before deleting an e-mail, Exchange can move it to a “Cleanup Review folder,” where it will sit for short amount of time (generally thirty days). With this approach, the user can visit her Cleanup Review folder, see what’s about to be deleted, and – if appropriate – move a particularly important e-mail to a managed folder with a longer term retention policy.

As you can see, our approach to managing e-mail is similar to our approach to managing documents in the Office SharePoint Server collaborative spaces. All files are classified based upon their content, and then an expiration policy is applied unique to the classification. In Office SharePoint Server document management, we focus more on user-collected metadata. With Exchange, we only have auto-populated metadata to work with. But it’s the same general concept for both.

In the next post, we’ll conclude our discussion of managed folders and talk about a couple of other types of policies that can apply to e-mail. Until then, keep the blog comments and questions coming!

Thanks,

Adam Harmetz

Program Manager

E-mail Records Management, Part 2

Ethan Gur-esh — Sat, 06 Jan 2007 00:21:45 GMT

In the previous post, we covered the basics of what’s involved in e-mail records management and why it’s important. Now it’s time to get into some of the details of our approach to the e-mail records management problem.

In this post, I’m going to discuss the e-mail classification challenge. I’ll show how Exchange 2007 allows records managers to specify a corporate classification structure for e-mail and how that structure can be customized for individual departments or users. I’ll also discuss how information workers can view that structure and use it to classify their e-mail according to corporate policies.

The first step in e-mail records management is not a technological problem at all. Instead, a records manager needs to examine her organization and create a list of all the different types of communications in it. This work is very similar to the file plan building process for Microsoft Office SharePoint Server 2007 described in our documentation. For example, records managers could define e-mail types like Research and Development, Human Resources, and Non-Work Related.

This “file plan for e-mail” will be very specific to each organization. Depending on the business and e-mail retention needs of an organization, it may have a very broad set of e-mail types (e.g. “Long Term General Business” and “Non-work related”) or have a very specific set of types (e.g. “Human Resources – China” and “Accounting – Sarbanes Oxley Compliance”). The scopes of the classifications often depend on the types of policies that need to be applied to e-mail (more on that in later posts).

Once they are defined, each e-mail classification is mapped to a managed folder in Exchange 2007, which is similar to a content type in SharePoint. Managed folders appear to users simply as folders in their mailbox. They are a way to expose the corporate e-mail classifications to end users. Since the folder concept is generally familiar to all information workers who use e-mail, the corporate taxonomy is now modeled in a form that users can easily understand.

To be concrete, here’s an example of e-mail classifications that a user might see in Outlook after managed folders have been deployed:

Managed folders can be customized to meet the needs of each user. This can be done in two ways. First, although there might be dozens of managed folders defined within an organization, they can be configured so that each user only sees the managed folders appropriate to him or her. For instance, a product developer in the R&D department won’t see the Human Resources managed folder. This can either be configured ahead of time by the records manager based upon group membership, or an organization can deploy an interface that allows users to specify which managed folders they want to see.

Another way that managed folders can be customized is that users can create their own folder structure within a managed folder. For example, if a records manager has defined “Research and Development” as a type of e-mail content and has created and deployed a Research and Development managed folder to all users in the R&D arm of her organization, individual users can create their own folder structure within the Research and Development folder. So a researcher working on Project Vienna and Project Fiji could create subfolders within the Research and Development managed folder to separate the mail for each of those projects:

In this way, users can create a folder structure that reflects how they perform their everyday work while still conforming to the e-mail classifications required by records managers.

How do users classify content into managed folders? As we’ve already seen, the managed folders appear just like any other folders in the user’s mailbox. They can use any familiar method to move items between folders. For example, users can simply drag and drop e-mail between folders, use copy/paste, or right click on a mail item and select “Move to Folder.” Users can also set up rules to automatically move or copy items when they arrive. For instance, all e-mail from a certain distribution list or person can automatically be copied to a particular managed folder:

Finally, the new Outlook 2007 ribbon makes it easy to classify e-mail using a simple drop down menu.

In this way, users have an entire corporate classification system exposed within their Outlook environment. The experience is customized to each individual user, who can submit and classify e-mail in any way that he or she feels comfortable. We felt very strongly that, in order to allow customers to successfully manage e-mail, we needed to provide users with a variety of ways of accomplishing their tasks using tools that they are already familiar with.

Of course, classifying all this content doesn’t completely solve the e-mail management problem. Organizations have to take action based upon the classifications. There are a variety of out of the box policies that can be configured on a managed folder. For example, each managed folder can have a unique expiration policy or size quota. And, of course, e-mail submitted to managed folders can be sent off to a SharePoint Records Center where it is retained along with other types of content. We’ll be talking about all of these options in the next few posts.

Thanks for reading,

Adam Harmetz

Program Manager

AIIM Records Management Survey

Ethan Gur-esh — Tue, 02 Jan 2007 23:55:00 GMT

Happy New Year, everyone!

We’ll return to our talk on email records management later this week, but I wanted to point you to a recent AIIM survey on records management and compliance. Over 800 organizations answered questions about their records management practices, and there are some pretty detailed results. Overall, I think it shows the growth opportunity for the entire industry, as well as the need for an easy to use, comprehensive strategy and solution.

John Mancini is the president of AIMM and coordinator for the survey. You can access the results and look at his comments via his blog.

Thanks!

Adam Harmetz

Program Manager

Email Records Management, Part 1

Ethan Gur-esh — Wed, 20 Dec 2006 00:11:00 GMT

Hi everyone! Now that Vista, Exchange, and the Office System are available to businesses, we thought we’d switch gears on this blog. So far, the blog has focused on the records management capabilities in Microsoft Office SharePoint Server 2007. However, around the office, we always talk about compliance and records management as spanning the entire information lifecycle and not just one particular product or service. When we were researching customer needs for the 2007 release, one of the themes we constantly heard was that there is a need to manage all types of content – no matter where it lives – and a need to integrate various products together for a complete end-to-end experience.

E-mail is a key type of content in organizations. Therefore, the next couple posts will focus on managing the information contained in corporate e-mail. We’ll introduce some of the compliance and records management features in Microsoft Exchange 2007 and Microsoft Outlook 2007. Of course, we’ll be sure to mention the tight integration with MOSS 2007 as well.

At first glance, e-mail is a pretty strange place to talk about records management. When most people think of managing corporate records, they think of archiving signed contracts, financial statements, and the like. They don’t think of trying to manage ad hoc communications like e-mail.

Increasingly, though, e-mail is where a lot of an organization’s important information lives. A recent AIIM survey showed that over 70% of information workers spend a fifth of their time or more on e-mail related tasks. Certainly here at Microsoft we live in an e-mail centric world. On average, over 3 million e-mail messages are sent internally within Microsoft every day!

For many organizations, what used to be communicated in written memos or other easily archivable formats is now being sent as e-mail. Since e-mail is where business is being conducted, important information that needs to be retained is stored there as well.

What’s more, the need to manage e-mail has been emphasized in recent headlines. For instance, this year marked a landmark case in electronic records retention: Perelman vs. Morgan Stanley. In this case, Morgan Stanley was heavily punished by the judge for consistently failing to produce e-mail records during the discovery process. On several occasions, employees at Morgan Stanley found tape backups of e-mail records related to the case even after the company signed statements stating that they had turned over all relevant records. Morgan Stanley had no consistent process in place for managing the flow of information in e-mail, and this could end up costing the firm hundreds of millions of dollars.

Flashy court cases aren’t the only wake up call, though. The oft-mentioned changes to Rule 26 of the Federal Rules of Civil Procedure finally codify the requirement to manage electronic information. Specifically, these new rules focus on the eDiscovery process and they make it clear that companies must have a policy for managing all types of electronic information, regardless of where it is stored. If you want to learn more, our partner Iron Mountain has an excellent whitepaper on the new rules.

And yet, in a recent ARMA survey, 43% of respondents have no plan for managing e-mail records. So we have a situation where a lot of information is being generated, there is a new and urgent need to manage that information flow, and most companies don’t have a plan yet.

It was with this customer problem in mind that we developed a solution to the e-mail management problem. As we describe the solution in the next few blog posts, we’ll be talking about things like:

· How end users can classify e-mail messages according to a corporate schema

· How e-mail can be retained and then ultimately deleted based on its classification

· How storage quotas can be enforced on individual folders (e.g. this folder is limited to 200MB)

· How important e-mail can be sent to a SharePoint Records Center – directly from the Outlook client

· How Exchange and Outlook search can be used to search for e-mail during the eDiscovery process

To a certain degree, managing e-mail is very different from managing other corporate records. The sheer volume of information requires broad strokes and less attention to each individual item. Process and rules becomes very important.

However, even though there are some differences, records are records because of their content, not their format. So we’ve made sure that the e-mail management features we’ll describe in the next few posts align with the “big picture” we outlined in the early posts on this blog. Themes like “low tax on the end user” and “low deployment costs” will run through our e-mail experience as well.

Thanks for reading and stay tuned!

Adam Harmetz

Program Manager

Compliance Features in the 2007 Microsoft Office System

Ethan Gur-esh — Tue, 05 Dec 2006 00:49:00 GMT

Hi everyone,

I wanted to point you to an excellent whitepaper that discusses compliance across the entire 2007 Microsoft Office System. Rather than talk about specific certifications Microsoft is pursuing (like DoD 5015.2), the paper explains the other side of compliance. It provides an overview of the compliance space and showcases the tools that Microsoft provides to help customers comply with regulations like Sarbanes-Oxley. Certainly many of the features on this blog are mentioned in the paper, but there are plenty of other products and features in there as well (Excel Services, Microsoft Office Forms Server 2007, Microsoft Exchange Server 2007, and a lot more)

Here’s a summary of the paper:

“Compliance Features in the 2007 Microsoft Office System” showcases compliance-related features and extensibility opportunities within the 2007 Microsoft Office system, and demonstrates how the Office system can help you meet the demands of regulatory compliance. Out-of-the-box, the 2007 Office system provides many of the fundamental components required to support compliance regulations, such as auditing, records management, and data security. However, some degree of development and customization is necessary to tailor regulation-compliant solutions for particular organizations and environments. This paper provides examples of extending the platform to build custom compliance solutions for the financial services, healthcare, and accounting fields. The target audience includes developers, technical architects, and technical decision makers interested in delivering business solutions that leverage and extend the compliance-related feature set of the 2007 Microsoft Office system.

Joanna Bichsel, the author of the paper, also has a blog post about it.

Happy Reading!

Adam Harmetz

Program Manager

The 2007 Office System ships!

Ethan Gur-esh — Sat, 18 Nov 2006 05:49:03 GMT

We want to make sure everyone has heard that we’ve completed work on the 2007 release of the Microsoft Office System, including all of the Records Management features we’ve been talking about on this blog! (Below are a few links to press articles about the product launch.) The product will be available to enterprise customers on November 30, 2007.

This represents a huge milestone for our team, and we’re looking forward to the conversations that will ensue here once the community starts using these products – especially about where the Records Management team should focus our efforts for the next version of the Office system, which we’ll be starting work on soon. :-)

Thanks for reading!

- Ethan Gur-esh, Program Manager.

Some press coverage:

DoD 5015.2 Certification for Office SharePoint Server 2007

Ethan Gur-esh — Wed, 08 Nov 2006 09:57:00 GMT

Hi everyone! My name is Adam Harmetz and I work on the Records Management team here in Redmond. I met many of you at the ARMA conference a couple weeks ago. I had a great time there demoing Office SharePoint Server 2007 and getting feedback from everyone.

One of the questions that we often heard at the Microsoft ARMA booth was, “Is Office SharePoint Server 2007 going to be certified against the DoD 5015.2 standard?” I wanted to take some time today to elaborate on our certification plans.

First off, for those of you who aren’t familiar with the standard, DoD 5015.2 is a records management certification managed by the Joint Interoperability Test Command of the United States Department of Defense. You can visit their website for more information about the criteria and testing procedures. Generally, though, there are two different types of DoD 5015.2 certifications: Chapter 2 and Chapter 4. Chapter 2 contains the mandatory criteria that are required by all records management applications used by United States government agencies. Chapter 4 is specific just to applications that process classified records.

Chapter 2 is the more widely used part of the DoD 5015.2 standard and Microsoft is pursuing Chapter 2 certification only.

Even outside the US Government, Chapter 2 is looked upon as a very comprehensive “fitness test” for a records management application. As such, many companies and organizations will only consider purchasing DoD 5015.2 Chapter 2 certified records management applications. By requiring DoD 5015.2 certification, they can ensure that all the products they consider can be extended and configured to meet even the most serious of records management needs.

The DoD 5015.2 standard is widely used outside the United States as well. Many non-US companies require DoD 5015.2 certification and the DoD 5015.2 criteria have had a significant impact on many of the non-US standards (e.g. MOREQ). In a previous post, Ethan has made a few comments about why we haven’t announced other certification initiatives.

Microsoft is working on a Records Center Add-on Pack for Office SharePoint Server which will provide the additional functionality to meet the DoD 5015.2 criteria. The add-on pack will be freely available to all customers with Office SharePoint Server 2007, so anyone can take advantage of the new features.

Microsoft is slotted to take the DoD 5015.2 certification test on May 14, 2007 and we plan to release the add-on pack shortly after the certification process is complete.

We are looking forward to talking about what functionality we are adding with the add-on pack in later posts. Since the pack is built entirely on top of our existing Records Center, it’s a great showcase for how the Office SharePoint Server 2007 Records Center can be extended to meet specific customer needs.

Thanks for reading!

Adam Harmetz

Program Manager

The Business Side -- Is there a separate "Compliance / RM edition" of Office SharePoint Server 2007?

Ethan Gur-esh — Fri, 03 Nov 2006 21:59:46 GMT

There are many challenges in getting an RM department and an IT department to work together: cultural issues, procedural issues, and more. But one of the first challenges for working with IT is simply acquiring RM software -- in many organizations, hardware & software purchases, including systems used for RM, are accounted for in the IT department's budget. And the metrics that IT often uses to make purchasing decisions usually don't favor RM considerations. (This may be changing thanks to the increased importance of compliance for many organizations, but it's likely to remain a challenge for Records Managers for the near future.)

Fortunately, organizations that choose Office SharePoint Server 2007 won’t have to deal with this disconnect. :-)

We've stated on this blog from the beginning that for a records management program to succeed in an organization, all of the organization's content management systems need to have appropriate RM capabilities. For that reason, all of the Records Management capabilities described in this blog are available in Office SharePoint Server 2007. There will NOT be separate RM or “compliance” editions of Office SharePoint Server specific to records management.

This can have a major impact on how records managers and IT departments justify investments in RM software. IT departments have several reasons to deploy the Office SharePoint server 2007 in addition to its Records Management abilities. (Here’s a link to the "Top 10 benefits of Office SharePoint Server". Only item #3 talks about Records Management.) So within your organization the RM group won’t be the only people asking IT to deploy this system. And from the IT department’s perspective, the “Return on Investment” for RM within Office SharePoint Server 2007 is much more palatable – you aren’t requiring them to deploy an RM application in addition to their Content Management systems; and the business benefits include more than reduced legal & regulatory risk.

So not only can the 2007 release help you be more successful at implementing records management programs in your organizations, but you'll also have an easier time getting IT to acquire & deploy this system in the first place.

Thanks for reading!

- Ethan Gur-esh, Program Manager.

Records Management team Halloween costume

Ethan Gur-esh — Wed, 01 Nov 2006 02:18:00 GMT

This might seem a bit off-topic, but we just had to share some photos of our team's Halloween costume this year.

Every year at Microsoft, many employees and teams dress up for Halloween and welcome trick-or-treaters to the Microsoft campus. This year, the Office "Enterprise Content Management" team (the group within Office product team that includes the Records Management team) decided that our team costume should reflect our recent experience at the ARMA Conference, particularly the warm reception that I personally received resulting from our blogging efforts. (See the "People read our blog!" section of this post for more details.)

As such, we thought it would be appropriate to share some photos of the team costume with all of you.

Here is a picture of two members of our testing team (JD Klaka and Mike Dalton), showing their "Ethan / RecMan blog fan" t-shirts:

Here are several members of the team surprising me by with the team costume. I'm on the far left, and those who saw me at the ARMA conference will note that I'm not usually that shade of red ;-) :

Finally, here is a photo of several members of the Office Document Lifecycle team in costume:

(Pictured, Left to right: Lera Pakhomova, Jason Cahill, Mike Dalton, Ethan Gur-esh, JD Klaka, Eilene Hao, Adam Harmetz, Zach Rosenfield, Jon Rosenberg.)

Thanks for reading, and Happy Halloween to all those who celebrate it!
- Ethan Gur-esh, Program Manager.

ARMA Conference Podcasts

Ethan Gur-esh — Tue, 31 Oct 2006 01:42:31 GMT

As part of the outreach activities for ARMA 2006, several conference speakers & participants recorded podcasts about topics of interest for Records Managers. (Kudos to the ARMA conference organizers for this effort – this is a really innovative way to help inform people who couldn’t attend. :-) )

Tina Torres, Microsoft’s Corporate Records Manager, recorded a podcast about what Microsoft is seeing and doing in the RM space with the 2007 releases.

Tina’s podcast can be found here.

The rest of the ARMA podcasts can be found here.

- Ethan Gur-esh, Program Manager.

ARMA International Conference 2006

Ethan Gur-esh — Sat, 28 Oct 2006 01:43:00 GMT

What a week! Most of the Records Management team is now back in Redmond from the ARMA International Conference in San Antonio, Texas, and I want to talk a bit about the event for the benefit of everyone who wasn’t able to attend.

First and foremost, a few acknowledgements:

I would like to thank ARMA International on behalf of Microsoft for hosting a great and well-executed conference, and allowing us to participate as technology sponsor. It was a real pleasure for us to be involved, both as a company and as the individuals who got to spend a few days in San Antonio, TX.
I personally would like to thank all of the Microsoft people, both from the product teams, the sales & marketing groups, and our Legal and Corporate Affairs group for participating at the show. It was a major time commitment from some busy people, and it was really appreciated. :-)

With those thanks covered, on to a summary of the event…

Microsoft booth at the ARMA Expo

Despite a sub-optimal location for the Microsoft booth on the tradeshow floor (the best floor spots are usually the ones near the food/drinks ;-) ), our booth saw non-stop traffic from the beginning to the very end. Attendance was so strong that most of our people worked double-shifts to help us talk to everyone. We even ran out of Office SharePoint Server & Exchange 2007 Beta DVDs… and we’d brought a lot of them to the show. Here are the links to download the Betas for those who missed out:

Office SharePoint Server 2007 Beta: http://www.microsoft.com/office/preview/
Exchange Server 2007 Beta: http://www.microsoft.com/exchange/preview/

It was great to see so many people interested in what we were doing, asking really thoughtful questions, and responding so positively to the 2007 release and our demonstrations. The conceptual model of “collaborative spaces” vs. “records spaces” that we first outlined on this blog resonated with many people, including practitioners, other vendors, and even members of the ARMA leadership.

I’m glad our product people and our corporate practitioners were there – I think that the attendees really appreciated our being able to have conversations from both of those perspectives.

And it was clear that there’s been a real “word-of-mouth effect” for Microsoft’s Records Management efforts in our 2007 releases – we had many attendees come by saying that “I heard from others that you guys are doing big things in RM now and that I have to come see your demo.” So thanks to everyone who is spreading the word!

Our “Technology Spotlight Session” on Tuesday, October 24

In addition to our booth at the ARMA expo, Tina Torres and I also presented a “Technology Spotlight Session” on Tuesday, October 24 about the 2007 release and our experiences using it internally at Microsoft. (Here are the links to the presentation, which has been split into 3 PPT files: Part 1, Part 2, Part 3)

We were blown away by the attendance for this session! The ballroom where we gave the presentation was very full, and it’s quite a large room. Judging from the feedback we got during and after the session, it was really well-received. (On several occasions we were told that “it’s clear you guys really understand this space!”.) “Telling the story” from both the practitioner and the product perspective really struck a chord with the conference attendees.

People read our blog!

One other takeaway from the event was that this blog has been a valuable resource for the Records Management community. We’ve had raw “page view” statistics about the blog for a while (it actually ranks in the top 6% of all Microsoft blogs by audience size), but this event was one of the first times we could gauge how well it was serving Records Managers. It was really gratifying to hear so many people who came to the booth saying “It’s great to meet you. We read the blog regularly!” When we started this blog we weren’t really sure if it would be a good way to reach the Records Management community… but it’s great to know that it has.

And while we’re on that topic… probably the most surprising takeaway from the conference was the amount of personal recognition that I received. (The running joke among the Microsoft participants after the show was that next year we should give away “Ethan bobble-head dolls” instead of Hostess Twinkies. :-) )

I should firstly say that I’m very flattered and grateful. At the same time I’d like to clarify that there are a lot of people on the Microsoft Records Management team that deserve the credit that was being given to me. It’s been a real pleasure working on the team building these products for the last few years, and especially fun to be the “public face” for the team now that we’re so close to releasing the products. But there are many people on the team who’ve worked over the past few years to both understand the Records Management space and to build the products… I take every piece of kind personal feedback as a “team” compliment, and will relay those to them accordingly.

Also, if people now think we understand Records Management, a lot of people deserve credit for making that happen. First and foremost, the RM department in Microsoft’s Legal & Corporate Affairs spent many hours helping to educate us early on and give us feedback, as did many other customers who participated in Microsoft Office’s early adopter programs. (I can’t mention specific names of those organizations, but they know who they are. :-) ) So I wanted to thank them one more time for all of their participation in this effort.

We want your feedback!

Finally, I want to ask all of you who were at the conference for your feedback. This was our first time at ARMA as a vendor, and we’d love to hear your thoughts on what went well with our booth & session, where we can do better next year, etc. A few attendees shared some comments with us during the event -- please keep your feedback coming!

Thanks for reading, and thanks again to ARMA and its membership for a really great conference & week!

- Ethan Gur-esh, Program Manager.

Iron Mountain and Microsoft Collaboration

Ethan Gur-esh — Wed, 25 Oct 2006 19:15:17 GMT

As I “teased” in my last post, this week at the ARMA Conference Microsoft and Iron Mountain unveiled a new collaboration to enable both companies to better help organizations deal with their records management challenges. (The press release can be found here.)

This is a new type of collaboration for both organizations because it’s not limited just to technology – we’ll be working together on several fronts:

We’ll be working with Iron Mountain Consulting services to help them plan and execute records management programs for their clients that can be implemented on the 2007 releases of Microsoft Office & Exchange.
Iron Mountain will be making their hosted digital archive offerings available as “Records Centers” for organizations that want to use SharePoint for collaboration while using a hosted offering for their long-term records storage. (Remember when we said a while back that the interfaces for Records Center were open for 3^rd parties to use? ;-) )

We’re really excited about this collaboration with Iron Mountain, and think it’s a great thing for our customers for several reasons. Firstly, it’s really validating to have one of the biggest service providers in the industry supporting our approach for Records Management.

Secondly, many of the challenges of successfully implementing a records management program are on the “business side” rather than the “technology side” – determining appropriate retention schedules, managing the required cultural & process changes within your organization, etc. Iron Mountain’s consulting services can offer some of the expertise needed to address these challenges. By helping their consultants deeply understand the Records Management capabilities of Microsoft’s 2007 releases, records managers that want to use Exchange & Office 2007 for their solution will have an easier time leveraging that expertise for their organizations.

Finally, making Iron Mountain’s hosted offerings more easily available to organizations using Microsoft Office & Exchange 2007 gives our customers more flexibility in terms of where/how they store their records, while still being able to take advantage of the new records management capabilities of our products.

Thanks for reading!

- Ethan Gur-esh, Program Manager.

Hello from ARMA / Hey, where did you go?

Ethan Gur-esh — Sun, 22 Oct 2006 00:38:41 GMT

I owe everyone an apology… Getting ready for the ARMA International conference this week (I’m writing this post in the Expo hall here in San Antonio) and working through the last few weeks of getting the 2007 Office system ready to release didn’t leave me with enough time to keep this blog as “fresh” as we’d all like.

Please don’t take this to mean that we’re done introducing our Records Management capabilities in Exchange & Office 2007– far from it. In the next few posts we’ll start talking about e-mail records management (a huge problem space for so many organizations today), our efforts around getting Office SharePoint Server 2007 certified for the Department of Defense’s 5015.2 standard, and much more.

And this coming week alone there will be several items on this blog, including the announcement of a new collaboration with a major records management solution provider that we’ll be unveiling at ARMA.

So please stay tuned!

- Ethan Gur-esh, Program Manager.

Physical Records Management in the 2007 Office system

Ethan Gur-esh — Thu, 05 Oct 2006 11:00:14 GMT

So far in this blog, we’ve talked directly about electronic records – the files created in document authoring applications, e-mail systems, etc.

But few records managers out there (if any) have the luxury of dealing solely with electronic records… for most organizations records will continue to come in both electronic and physical form for a long time. And even the “physical” category is fairly broad – depending on your organizations needs, you may need to preserve printed documents as records (especially if they’ve been hand-signed) or physical objects. (For example, Microsoft’s hardware team that produces our keyboards & mice need to retain our device prototypes as records!)

Some aspects of a records management program may be different for electronic & physical records -- for example, the storage of physical records is a very different challenge than electronic storage. (Especially from a cost-per-record standpoint; physical record storage costs don’t follow Moore’s law ;-) ). But from the underlying business/legal/regulatory requirements for records retention are independent of record format – whether it’s a Word document, an e-mail, or a piece of paper, it’s the content of the record that determines its retention schedule.

For that reason, it would be ideal for most organizations to have a records management application that can support both their physical and electronic RM needs – so that they don’t need to configure, manage, and maintain file plans in two separate systems. Fortunately, the 2007 release of Office SharePoint Server supports the management of both physical and electronic records.

Physical Records – just another type of “item”

The reason we haven’t labeled this post as a new “records management feature” is that in the 2007 release, we don’t think of physical records as being separate from the concepts we’ve already described. Rather, we took great care in making sure that as we created records management features like Information Management Policies, Expiration, Barcodes, etc., that all of those features can enable physical RM.

The core concept behind physical RM in the 2007 release is that SharePoint supports the management of not only “Documents” (electronic files, like the kinds authored by the Microsoft Office client applications), but also generic “items” (objects that don’t necessarily have an electronic file).

“Items” can be customized using Content Types to add metadata schema appropriate to representing physical records – for example information about the location of the physical object represented by the item, a description to enable searching, identifiers to relate sucessive "parts" of a collection of records, etc. Items can also optionally include attachments (electronic files relevant to the item) -- e.g. a scanned image of a physical document, for online browsing.

Workflows can be written to target non-document “items” – for physical records this would enable organizations to define appropriate human-centric workflows for transferring records, sending them to a custodian temporarily for review, etc.

And information management policies can also be applied to “items”, just like for electronic records. The barcodes automatically assigned by the barcode policy can be printed out on sticky labels & affixed to the physical records to facilitate retrieving the corresponding “item” using a barcode scanning device. Expiration policies can be configured to trigger a workflow as the disposition action for physical records, and that workflow can drive the appropriate process for the physical object.

So in short, the tools to enable an organization to manage physical records alongside their electronic records are available in the 2007 release of the Office system. In fact, Microsoft’s internal records management department (our first and best customer) is in the process of migrating their physical records management program onto Office SharePoint Server 2007!

Where partners add value

While it’s true that the 2007 release enables physical records management, it’s also fair to say that many organizations will want additional solutions/customizations beyond what we include “in the box”. As mentioned above (and in some earlier comments on the blog), there are several capabilities particular to managing physical records, including:

Imaging solutions that integrate with scanner/copier/printer hardware to automate the ingestion & capture of physical records.
Space/warehouse management solutions to handle allocating shelf space in physical storage sites.
Integration with off-site record storage providers.

We’re absolutely aware of the value of these types of additional solutions for physical records management -- and we’re actively working with our community of software, hardware, and service providers to help them extend their solutions onto the 2007 Office system. While I won’t name any individual partners/solutions in this post, stay tuned for more details on that subject soon on this blog.

Thanks for reading!

- Ethan Gur-esh, Program Manager.

Records Management Feature -- Hold Orders

Ethan Gur-esh — Wed, 27 Sep 2006 10:17:43 GMT

In some of our first posts, we talked about the high costs of litigation & discovery. And our customer research wasn’t the only motivation for addressing this particular problem– it turns out that Microsoft’s Legal & Corporate Affairs (LCA) department occasionally has to deal with litigation… and they were more than happy to help us understand the costs & pain of the whole process. :)

So in this post, we’re going to discuss the hold order capabilities we’ve built into the 2007 release to help organizations deal with these challenges.

To explain the feature, we’ll walk through the general process of a hold order, and how that process is enabled in the 2007 release. (The process I’m describing here is intentionally generalized – your organization’s process may have additional steps, but as a basic outline it’s applicable to most.)

Step 1: Create a new hold order

In order to start putting records “on hold”, you need to define the basic parameters of the hold order. From a business perspective, this probably includes identifying what types of records are likely to be relevant to the hold order, verifying that a hold order is required, etc. From a RM application perspective, this is about creating a new “hold order” in the system, so that records can be tagged as responsive to that hold order.

(One of the early learnings from Microsoft’s LCA group: you often don’t have the luxury of having only one hold order in effect at a time… so being able to know which records are relevant to which hold is a critical capability for hold order management.)

In the Records Center, hold orders are managed in the “Holds” list -- this list, which is automatically included in the Records Center site, is used to track the set of hold orders currently in effect.

The "Holds" list in the Records Center

Each entry in this list represents a hold order, and for each hold order you can provide metadata: a name, a description, and the person who is responsible for managing the hold order. In addition to this user-defined metadata, the Holds list also tracks some additional information about each hold order automatically:

· The status of the hold: If the hold order is “Active”, then it represents a matter that is still ongoing – records tagged as being relevant to this hold must be retained as long as the hold is active, and new records can also be added to this hold. If the hold is no longer active, records tagged as relevant may be able to resume their normal disposition schedules, and no new items can be put on that hold.

· The list of items currently on that hold: Each hold order includes a list of the items currently marked as being relevant to that hold, to facilitate the triaging all those items to ensure relevance, producing them to external parties, etc.

And in addition to storing this metadata, the entries in the hold list also provide the “actions” for dealing with those hold orders (see image below) --the first of which is finding relevant records to put on the hold.

Viewing the properties of an entry in the "Holds" list

Step 2: Putting relevant records “on hold”

The point of a hold order is to ensure that items relevant to that hold order are identified and preserved accordingly. The first challenge is simply finding and “holding” those items -- without requiring a lot of manual effort from your legal team.

Thanks to the integrated technology architecture of the 2007 Office system, we’ve got a really powerful tool to address this problem: search. The hold order feature includes the capability to specify a search query to identify relevant items (and verify that the query is returning the right set of records), and then automatically put all the items matching that query on that particular hold, as shown in the image below.

Search-based interface for adding records to a hold

This “search and add to hold” capability is a vital aspect of hold order management – it allows organizations to automatically make a first-pass at identifying relevant items and quickly ensuring their preservation. And after that initial preservation of potentially relevant records, we’ve also got the capabilities for your legal team to later add or remove individual items from the hold order as necessary, shown in the image below.

Removing an individual item from a hold order

What does it mean for an item to be “on hold”?

Each item in the Records Center can be tagged as relevant to on or more hold orders. (For example, in the below screenshot the record has been tagged for both the “Regulatory investigation #1245-07” and the “Contoso Litigation EU” hold orders.)

Records on hold will not be dispositioned as long as any of the holds applied to it remain active. The records’ expiration policy will be suspended, and custom solutions will also be prevented from inadvertently deleting the held record.

Viewing the properties for an item that's been added to multiple hold orders

Step 3: Releasing the hold order

The last step in hold order management is dealing with the day when the underlying matter requiring the hold order has been resolved, and all of the records subject to that hold may be able to resume their normal disposition schedules.

Releasing a hold order is also a quick and easy process. When the underlying matter is resolved, a hold order can be released just by using the “Release hold” verb for the appropriate entry in the “Holds” list. This action will automatically update the hold status on all items to which that hold order had been applied, including resuming normal expiration if appropriate.

Is the hold order process audited?

Another important aspect of our hold order features are how they take advantage of our auditing capabilities. All of the stages of the hold order process – creating new hold orders, putting records “on hold” (manually or using our search-based capabilities), releasing a hold, etc. all generate an audit trail. So if your hold order processes are ever called into question (and it happens – trust us), that audit information can be used to demonstrate that your litigation response processes were carried out in an appropriate and timely manner.

Thanks for reading!
- Ethan Gur-esh, Program Manager.

Records Center Q&A

Ethan Gur-esh — Tue, 19 Sep 2006 23:54:00 GMT

Thanks to everyone who posted questions about the Records Center or sent them in via the “contact” form. (It’s great to receive such detailed questions – it shows how many people are really interested in the records management capabilities of the next release!)

So before moving on to our next topic (hold orders in the Records Center), this post will try to answer most of the questions we’ve received so far.

What if I don’t want to have separate collaboration and records center sites… can my end-users work directly against the records center?

As we’ve said in various posts to date, we believe that the model of separate collaboration sites vs. record sites provides a significant usability/ adoptability advantage for most organizations implementing a records management program. That said certain organizations may want to institute a program where their end-users are required to (a) directly understand their company’s file plan, (b) file records directly into the file plan, and (c) treat all of the content they work on as records.

Fortunately, the Records Center site in the 2007 release can be configured to support this model as well – thanks to the improvements made in the security model of Windows SharePoint Services. In the 2007 release, users can be granted very specific/restricted rights to objects in the Records Center site – for example, the ability to add new records, but not the ability to modify any existing record (and even without the ability to view existing records if desired). And these rights can be granted selectively across individual document libraries within the Records Center site itself – so you could control which series in the file plan specific users would be able to see / file records into.

So from a capability standpoint, the 2007 release can certainly be used to enable a records management program of this type. But the underlying cultural challenges to getting such a program successfully adopted are still significant, and we’d strongly advise organizations considering this approach to think about how to mitigate those issues before proceeding.

Do the Content Type names used in the collaboration site need to be the same as those in the “Records Center”?

No. Thanks to the record routing list, you are free to have completely different names for Content Types in the Records Center vs. the collaboration sites that declare records. You can use the routing list to define how Content Types in those collaboration sites correspond to those in the Records Center.

This allows you to name Content Types in the collaboration sites in terms that are meaningful to your end-users (e.g. “Quarterly Filing”), while keeping names in the Records Center relevant for records managers (e.g. “SEC-regulated financial report”).

How do non-SharePoint repositories declare records using the Records Center “Web Service”?

The Records Center provides a public Web Service to allow any system to be able to declare records just like SharePoint sites can. (It also provides a public e-mail service --- but we’ll get back to that when we talk about the e-mail records management capabilities in the 2007 release of Microsoft Exchange in our next few posts.) But this raises the question – how does a non-SharePoint site, which doesn’t have a notion of Content Types, declare records?

The answer is that the definition of the Web Service requires only that when a record is declared, that declaration includes a textual property specifying the classification the sender proposes for that record. It’s then up to the Record Center to determine how to route the record, based on that proposed classification (using the information in the routing list).

When a record is declared from a SharePoint site, the name of the Content Type in that site is used as the proposed classification. However, non-SharePoint systems are free to specify the proposed classification for the record as they see fit. And to make it easier for non-SharePoint systems to declare records matching the record series in the Records Center, the Web Service includes additional methods to let those other systems query the Records Center to learn what the current entries in the routing list are.

For anyone interested in the specifics of this Web Service and how it’s used to declare records, we’ve included a code sample demonstrating it in our “Enterprise Content Management Starter Kit”.

Is the Records Center site (or any of the other features introduced to this point) available only for Microsoft Office documents?

Certainly not. We know that the requirements for records management are not specific to a particular file format, and that many records aren’t created or kept as Microsoft Office file formats like .doc, .xls, .ppt, etc. So we made a conscious effort in this release to ensure that our records management features work well for any type of file, including Office files, CAD, PDF, images, you name it.

The only features we’ve talked about so far that are specific to Office files (and require the 2007 release of the Office desktop applications) are:

· Document Information Panel: Without the 2007 Office desktop applications, users will be required to fill in document metadata using the SharePoint browser-based experience, rather than in the context of the authoring application.

· Policy Statement: Only Office files opened in the Office 2007 applications will prominently display to end-users the policy statement for those files.

· Client-side functionality of Labels and barcode policies: You can configure label and barcode policies for any type of item, and Office SharePoint Server 2007 will automatically generate labels & barcodes for those items. However, only Office files opened in the Office 2007 applications will prompt users to include those labels or barcodes in print-outs of those documents.

All of the other features discussed in this blog, including the Records Center site, treat non-Office files on an equal footing to Office files.

Thanks for reading, and please keep the questions coming!

- Ethan Gur-esh, Program Manager.

Records Management Feature -- "Records Center" sites (Part I)

Ethan Gur-esh — Tue, 12 Sep 2006 02:59:00 GMT

Now that we’ve covered how records are declared from collaboration SharePoint sites, in this post we’re going to start exploring the “Records Center” site template provided in Microsoft Office SharePoint Server 2007 for collecting & managing records in an organization.

What is a “Records Center” site?

A “Records Center” site can be thought of as a “Records Management Application” in the traditional sense. While other SharePoint sites are generally optimized to maximize end-users productivity, Records Centers are primarily for use by Records Managers. The structure of the site can be organized to match your file plan (rather than how end-users might want to organize content), the policies for content in the site can be configured to match your retention schedules, and the metadata for that content can be set up to capture whatever information you need for the long-term management of those records. (It really is your site. :) )

Because it’s based on the same technology foundation as the other sites in Office SharePoint Server 2007, all of the features we’ve discussed in this blog are available in Records Center sites. And the Records Center also has some additional capabilities to make it a great Records Management Application.

What are the additional/unique capabilities of a Records Center site?

There are several capabilities unique to Records Center sites in the 2007 release. These are:

Automatically receiving/routing records declared from other sources: As mentioned in the previous post, declaring records from SharePoint sites is a lightweight process because Records Centers are able to determine how the Content Type of a declared record translates to an appropriate record series in the file plan, and then file the record into the appropriate location. This is the capability that we’re going to talk about in this posting.
Hold orders: Of course, one of the primary drivers of Records Management in an organization is to ensure that those records are available in the event of an inquiry or litigation. The Records Center includes a powerful hold order system to locate records relevant to particular event requiring a hold order, suspending disposition of those records for the duration of the event, and for resuming normal disposition once those events have ended. (More about this in our next few posts.)
Separate access controls: In the 2007 release SharePoint has improved greatly its ability to control security settings at a very granular level --including security for individual documents or list items-- and to grant users only specific permissions (for example, the ability to add items but not to edit them). This is an especially valuable capability for a Records Center because it gives you the flexibility to specify whether users can access any sections of the Records Center, whether they can view or add items, independent of the permissions those users have on collaborative sites (where they probably have permission to add, edit, and delete work-in-progress items).

How does a “Records Center” site receive records declared from SharePoint?

As mentioned in our last post, the burden on end-users when declaring records is minimal – the Records Center site does the work of inferring where that record belongs in the file plan, based on its Content Type. This inference is accomplished in the Records Center using a special list called the “Record Routing” list. This list stores a the set of rules that specify how the Content Types of records in collaborative spaces map to record series in the Record Center’s file plan. It is the tool by which a Records Manager can define these mappings once, rather than require end-users to specify for each record at declaration time where in the file plan it belongs.

A (simple) example Records Center site, showing the "Records Routing" list

Each rule can specify one or more Content Type names that identify records belonging to a particular series, a description, and the location where records with a Content Type matching that rule will be filed. This ability to have multiple names is a subtle but critical capability – it allows a Records Manager to specify that even if different regions or departments have different Content Type names for their respective collaborative spaces, those names all represent the same type of record in the file plan and should be filed together.

Additionally, one routing rule can be marked as the “default” – this is the location to which records will be routed whose Content Type name doesn’t match any other rule.

The following image shows the user experience for configuring a routing rule for a “Supplier Contracts” record series. The rule specifies that any declared record whose Content Type is “Supplier Contracts”, “Agreement”, or “Contract” should be filed in the “Supplier Contracts” document library in the Records Center. (It also shows a field that I haven’t mentioned yet… more on that in a later post.)

Configuring a routing entry for a Records Center site

So the first step for the Records Center in receiving a declared record is to match the name of the Content Type of the declared record to the appropriate rule in the routing list (which may be the “default” rule if necessary). Once the correct location has been determined, the Records Center will check to see if the metadata attached to the record before it was declared matches the metadata required for that location in the Records Center. Any metadata that matches will be used to populate the corresponding fields in the Records Center.

If the existing metadata is enough to fill in all required Record Center fields, then the record will be routed to the specified location, and the declaration process is complete. If additional metadata is required, then the record will be temporary kept in a special “holding zone” to which the user will be directed to fill in that metadata before the record is routed to the specified location.

In addition to storing the record itself, the Records Center will also store all of the record’s metadata and audit history from the collaborative space as separate files in the specified location. So in the event of any dispute over the authenticity/accuracy of that record, even long after it was declared, the original information about the record from its collaborative life is still available.

Hopefully this post helps explain how the 2007 release of the Office SharePoint system helps greatly reduce some of the pain generally associated with end-user record declaration & classification.

In our next posts we’ll continue looking at the capabilities of Records Center sites, including how they can be configured to allow end-users to manually file records if desired, how the routing of declared records can be extended to perform functions like de-duplication and routing based on metadata other than Content Types, and of course hold orders.

Thanks for reading!

- Ethan Gur-esh, Program Manager.

2006 ARMA International Conference

Ethan Gur-esh — Thu, 31 Aug 2006 20:33:56 GMT

Just a quick note to let you all know that Microsoft will be the technology sponsor of the 2006 ARMA International conference in San Antonio, TX, for both the main conference and the Electronic RM preconference. Several members of the Records Management team -- myself included -- will be in attendance.

During the conference we’ll have a “Technology Spotlight Session” (Tuesday, October 24 from 8:30AM to 9:45AM), as well as a booth space (#616) at the conference Exposition.

For those of you who are planning to attend, we look forward to seeing you there and continuing the conversations we’ve had on this blog “face-to-face” -- our experiences this year at the MER, AIIM and ARMA Houston conferences have been terrific and we look forward to another opportunity to engage with the records management community.

For those who won’t be there, we’ll make sure to post links to any conference material as soon as they’re available online.

- Ethan Gur-esh
Program Manager.

Records Management Feature -- Declaring Records in SharePoint Sites

Ethan Gur-esh — Thu, 24 Aug 2006 04:33:00 GMT

Finally! After much patience on the part of this community, we can now talk about the next big area of our records management capabilities in the 2007 release – record declaration, classification, and the “Records Center” site.

But before moving forward, it’s worth spending a few minutes recapping what we’ve talked about so far, and how it fits into the “big picture” that we introduced in some of our earliest posts.

A quick recap

In our posts about “RM in the Information Age”, we proposed a framework that categorized DM/RM systems into “collaborative spaces” and “records spaces” as being appropriate for many organizations.

The first set of features we introduced, Content Types and the Document Information Panel, are especially relevant when dealing with “collaborative spaces” –they help ensure that employees will be classifying content and filling in appropriate metadata for it from the earliest stages of its lifecycle – even before that information becomes a record.

Information Management Policies also play a significant role in collaboration spaces, since they give records managers and compliance officers a way to introduce appropriate policies and controls into active content without adding additional burden on employees – such as expiring unimportant content to reduce unnecessary retention and auditing user activities on content where appropriate. (Of course, these capabilities are also vital for enforcing disposition schedules in a records space, but more on that in the next few posts.)

Now that we’ve laid out that toolkit for records managers trying to cope with collaborative spaces, let’s talk about what happens when content in those spaces reach a point where it needs to be declared as a record.

When does content become a record?

Before discussing any technology, the first question for record declaration is “when does content become a record”? For some organizations and types of information, content must be considered a record from the moment of its creation. (Think of a bill of materials created for a customer delivery, for example.)

However many types of information should not be considered records until they reach a certain lifecycle stage. For example, think of a professional services firm publishing a research report: early drafts of the reports may not need to be retained as records, but the final version delivered to its client is “evidence of a business transaction” and must be preserved accordingly.

Ensuring that records falling into the latter category are retained and declared is especially challenging – given that the employees who are collaborating on that content and performing the business transactions aren’t always highly motivated to actively retain those records.

Fortunately, the 2007 release of the Office system provides the ability to reduce the employee burden on record declaration and classification to a minimum. Content types provide the way for content to be classified, and workflows provide the way to automate business processes like content publishing and finalization… now let’s talk about SharePoint’s record declaration features.

Connecting collaborative sites to a “Records Center”

A key component of our original framework for “collaboration” and “records” spaces is a mechanism by which records can flow from collaboration spaces to a records space.

To address this need, we’ve defined an “open” set of interfaces for how collaborative sites can communicate with records sites, and enabled all of our products in the Office 2007 system to use those interfaces. (For the more technology inclined: the interfaces are defined using Web Services and e-mail protocols. And when we say “open” we mean that any Records Management application, not only the 2007 release of Office SharePoint Server, can be enabled to work with the declaration capabilities discussed below.)

So, a Windows SharePoint Services collaboration site can be connected to a Records Center site for use as its “records space”. (And when we start talking about e-mail records management, we’ll see that Exchange 2007 can also use a Records Center site for that purpose.)

The key concept of this interface, which you’ll learn more about in our next posting where we explore the inner working of a Records Center site, is that it provides a very simple way for collaborative spaces to interface with records spaces, without the collaborative space needing to know the inner workings (or complete file plan) of the records space. This is the main reason why record declaration from Windows SharePoint Services or Office SharePoint Server 2007 is a minimal end-user burden.

What happens when a record is declared from a SharePoint site?

Now that we’ve talked through the concepts, let’s look at what it means for a record to be declared from a SharePoint site.

Record declaration in SharePoint can be part of an automated process (for example, a workflow) or a manually initiated process via a “Send To” command (see image below).

Caption: Menu for a document in a SharePoint team site, showing the “Send to” command for the Records Center. (Note: The name displayed for the Records Center is customizable. The term “Northwind Records Center” is just an example.)

Whether it occurs as part of an automated process or manually, when a record is declared, the latest version of the record, all of its metadata, its audit history, and its content type information are sent to the Records Center site to which the SharePoint site is connected. The user doesn’t need to specify which Records Center site to use, nor do they need to specify a record series at declaration time – this information is inferred from the content type by the Records Center. (We’ll explore exactly how this inference is made in our next post.)

Once the Records Center has determined the appropriate record series, it will either accept the submission as is or, if additional metadata is required for the record in the Records Center beyond what was provided while the record was “active,” it can prompt the user to provide that information.

But in either case, the burden on the user is minimal. In fact, if the record is declared automatically via a workflow and no additional metadata for the record is required, then no end-user interaction is needed at all.

What happens to content in a SharePoint site after it is declared as a record?

The last question to address in this post is what happens after active content in a SharePoint site is declared as a record. For physical records, once a record is declared (i.e. sent to a records storage location) then the original owners of the record no longer have easy access to it –the physical object is out of their hands.

However, with digital records that need not be the case: once the active content has been declared as a record, an “official” record copy can be retained in the Records Center site, but content owners can still have a reference/working copy in their collaborative spaces (and thus have the same level of access to the content even after it has been declared for an appropriate period of time).

This is the approach we’ve enabled in SharePoint 2007. When a record is declared (and a copy sent to the Records Center site), we leave a working copy of the record in the SharePoint collaboration site, which can be expired appropriately using expiration policies. Workflows or other automated solutions can certainly choose not to leave a working copy in the original location, but by default the working copy is left in place – to minimize the end-user disruption of working with content that has been declared.

Hopefully this post starts to answer some of the big questions that we haven’t addressed thus far though it probably raises other questions. In our next post, we’ll explore in more depth the “Records Center” site provided by Office SharePoint Server 2007, including how it stores records declared from collaborative spaces like SharePoint sites and Exchange 2007 e-mail servers.

Thanks for reading!

- Ethan Gur-esh, Program Manager.

Our plans for documenting records management features

Ethan Gur-esh — Thu, 17 Aug 2006 20:13:00 GMT

Hello, everyone. My name is Rob Silver. I’m a technical writer working on content supporting the records management features and scenarios that Tina, Ethan, and Jason have been discussing with you in this blog. As I’ve learned about the needs of records managers and the IT teams that work with them, and about the great records management feature-set built into Microsoft Office SharePoint Server, I’ve become aware of how important effective, comprehensive documentation can be to the success of a records management deployment based on Office SharePoint Server.

I’d like to use this posting to start a conversation with you records managers, information analysts, IT pros, and legal specialists who are considering using Office SharePoint Server to implement your records management solution. I’ll point you to the Web site where we regularly post our IT content set, including my records management planning content, and hopefully get some of your feedback on it. And I’ll tell you about our content plans (both the content directly related to records management and the wider content set that will enable your IT teams to deploy and maintain Office SharePoint Server successfully).

The heart of our content plans for IT professionals, application administrators, information analysts, and solutions designers (including, of course, records managers) is a set of online books that guide you through the "lifecycle" of planning, deploying, customizing, operating, and troubleshooting your solution based on Office SharePoint Server. You can see this content taking shape at the Microsoft Office SharePoint Server Tech Center. Notice the set of content links in the middle of the page under "SharePoint Server 2007 Technical Library (Beta)." This is beta content, so take it with a grain of salt, but there’s a lot there already and it will continue to grow as we approach the product release (and it will continue to grow beyond then as we get your feedback).

For records management, I focused on first delivering you planning and overview content. I did this because, first, I wanted to make it clear what we mean by records management. I thought this was important both to enterprises just getting started in records management and to actively working records managers who may want to make sure that we mean the same thing as they do by "records management." Secondly, in our model of managing a successful deployment, effective planning is a prerequisite to an effective deployment. Planners need content to help them build a design specification document that then drives deployment. Our planning content is designed to support that activity.

So let’s take a look at some of the planning content of interest to records managers. On the Microsoft Office SharePoint Server Tech Center home page, click on Planning and Architecture. This will take you to the TOC of our planning content. There are two sections that will be particularly interesting to you: of course, Plan records management, but also Plan document management.

Most of the topics in Plan document management are relevant to you. Our program managers and product developers designed a set of core features (such as auditing, workflows, policies, and the other features that Jason and Ethan have been writing about) that apply both to active document management and to records management. So to understand, for example, "content types," you learn about them in the core document management content and then learn how to apply them to records in the records management content. Also, learning about both our active document management and our records management features will get you thinking about some of the benefits of integrating these features across the active document and records management wings of your enterprise.

Now let’s look at the contents in Plan records management. Here’s what I’ve written so far:

These links are live! Please read the topics and give me your feedback:

Are the topics relevant to you?
Do they go into enough detail?
Do they give you enough guidance in making decisions?
Did you have questions, as you read the content, that weren’t answered?

Here are a couple other topics I plan on writing soon:

Planning physical records retention
Planning email integration

What other planning topics related to records management would you like to see?

In this blog posting, I focused on our planning content for records management solutions based on Office SharePoint Server. I’ll post again in a while and talk about how our deployment and operations content is designed to support you.

I’d like to close by modifying something that Jason Cahill wrote in a previous posting on this blog: "The most important step that you can take to lower the cost of running your organization, whether you’re wearing your IT hat, your legal hat, or your records management hat, is to have consistent business policies and practices, backed up with good software and technology and documentation to help manage the volume of your enterprise’s content."

I hope that, based on your feedback, we can give you the great content you need to help you in planning, deploying, running (and occasionally troubleshooting) your records management solution based on Office SharePoint Server.

Thanks for reading,

Rob Silver, Office SharePoint Server Documentation Team

Records Management Feature(s) – Barcode and Labeling Policies

Ethan Gur-esh — Thu, 10 Aug 2006 08:54:00 GMT

Before moving on to our next topic of record declaration & classification (which I know that many of you are anxiously waiting to hear about), this week’s posting discusses the last two features in the Information Management Policy framework of the 2007 release of the Office system – labels & barcodes.

Both of these features target a problem of particular interest to records managers – namely, how to manage physical records or documents in a manner consistent with their electronic counterparts? This is of course, requirement #1 for using a Records Management Application to handle physical records – while the application stores a digital object that represents the record, containing metadata about the record (like where the physical record is being stored), the appropriate disposition schedule for the record, etc., users need a barcode or labeling system to identify which physical records correspond to which digital objects.

So in the 2007 release of the Office system, we’ve included features for labeling & barcodes to automatically assign identifiers to content.

What do you mean by labeling & barcodes?

When we talked to customers about how they assigned identifiers to objects, we heard two main approaches:

Metadata-based labels: Some preferred to use an identification scheme that communicated metadata about the objects in the identifiers themselves. For example, some of our legal customers assigned identifiers to records that were based on the number of the client account, the jurisdiction of the matter in question, the year the matter was initiated, and the number of the folder that contained the record, like: Northwind—Massachusets—2006—001 .
Machine-readable barcodes: Other customers preferred to use machine-readable identifiers like barcodes. While these didn’t communicate any information about the record directly, they could be read by scanner devices to automatically look up the record information from the application.

So in the 2007 release, we’ve provided features that can use either approach.

Label Policies:

Label policies are the mechanism for assigning metadata-based identifiers. Label policies specify how to assemble the metadata about an item or document into an identifier, and the label is automatically filled in as the metadata is entered. And the label can of course be searched for to locate items on the server.

Here is a screenshot of the settings page for configuring a Label policy:

Barcode policies:

Barcode policies (as you may expect) are the mechanism for assigning machine-readable identifiers. Once a barcode policy is applied to a Content Type, barcodes will automatically be assigned to items as they are created or uploaded to the system, and items can be searched for using their barcode value.

Here is a screenshot of an item that has been assigned a barcode via a policy:

The barcode format is totally configurable – Office SharePoint Server 2007 includes a component for generating barcodes that conform to the “Code 39” barcode symbology (formally known as ANSI/AIM BC1-1995), but components that provide barcodes in other formats can be plugged in as well.

Barcodes and labels in the Office 2007 client applications

And while everything mentioned above applies to managing any type of item or document in Office SharePoint Server, label & barcode policies have some additional capabilities when applied to Word, Excel, and PowerPoint documents.

Policies on these types of documents can specify that the label/barcode should be included in print-outs of these documents, and the Office 2007 applications will then provide users with an opportunity to add the label or barcode automatically when they print those documents out.

Here's a screenshot of a user printing out a document with a barcode policy:

In this way, document print-outs can be made to include the appropriate labels & barcodes to ensure that they can be easily correlated with the digital documents in Office SharePoint Server.

Thanks for reading!
- Ethan Gur-esh, Program Manager.

Records Management Feature: Auditing

Ethan Gur-esh — Fri, 28 Jul 2006 11:40:00 GMT

I hope that the last few posts on various aspects of the "business side" of records management have been of value to everyone. In the next few posts, we're going to get back to introducing you to the Records Management features of Office SharePoint Server 2007. In this post, we're going to examine a critical capability for both records management and regulatory compliance -- Auditing.

For many organizations, especially those in regulated industries like the life sciences, records management requires more than just the long-term management of record "content"... they are also required to retain information about the lifecycle of those records, such as who contributed to the creation of each record, who approved or signed off on it, who viewed it before it was published, etc. And these requirements don't apply solely to regulated companies: "audit trails" about certain types of records may also be valuable for organizations if a record’s authenticity is ever challenged.

For these reasons, we've made the 2007 release of the Microsoft Office system an auditable system of record -- audit policies can be configured for documents and items in Office SharePoint Server 2007 to specify which events will be audited for each Content Type, via the Information Management Policy capabilities mentioned in our earlier posts.

As you can see in the image above, audit policies can be configured to automatically record user actions that affects the lifecycle of document & record content, such as when items are edited, viewed, versioned, published, and deleted. Additionally, custom solutions built on top of the Office SharePoint Server 2007 can also add relevant entries to the audit log, such as when an approval workflow is completed.

Office SharePoint Server 2007 can also be configured to automatically audit "site level" events that may be relevant for regulatory compliance, including searches queries made anywhere in the site, changes made to security settings, and changes made to the metadata schemas of document libraries and Content Types.

And while users of collaborative spaces can be given full rights to active content, the audit log is tightly restricted. Only administrators (or users who are granted sufficient privileges) are able to view the audit history, using Microsoft Office Excel-based reports. And no user can selectively edit or delete individual audit entries.

Auditing of the records management program

In addition to allowing records managers & administrators to specify how user actions on content will be audited, Office SharePoint Server 2007 also automatically audits the enforcement of the organization's records management program: events such as the disposition of content, the creation or modification of Information Management Policies, applying & managing holds, are always audited. So in the event that your records management processes are themselves ever disputed, there will be a reliable audit trail for those processes as well.

Thanks for reading,
- Ethan Gur-esh, Program Manager.