Ronalus

Content Publishing Folder Splitter

In short ...

• CPsplitter splits the exported content publishing files into sub folders and updates their manifests.
• CPsplitter can be downloaded from a link at the bottom of this post.

Enhanced overview

When exporting content from SharePoint using the Content Migration API - the exported content can be packaged and compressed in CAB files.

To faster Content Migration EXPORT - you can choose to avoid content compression.

This choice has another side effect: the content is exported directly to your chosen target location, and you avoid the use of the system chosen intermediate storage folder.

 So you happily run your faster process and export a couple of thousands of files to your desired folder, until you suddenly notice that the operation is gracefully  s l o w i n g   d o w n  as more and more files are added to this folder . . .

 This is where CPsplitter comes into the picture: it waits for the manifest to be released from the EXPORT process, and moves the manifest, along with all its files, to a sub folder, letting the EXPORT process keep running at full speed ahead.

"The manifest?" you may ask, "What is that?"

·         Well, every exported piece of content is listed in an XML file that is named 'manifest.xml', which is later used in the IMPORT operation.

·         There can be multiple manifest files for a single EXPORT, and you can roughly control the number of files listed in each manifest by setting the 'File Max Size' in the EXPORT application.

"The EXPORT application?" you ask ...

·         Well, there are multiple tools and means to export content from SharePoint 2007, each is best suited for certain scenarios. Some are part of the product, like STSADM and the Central Administration Content Deployment process. Others are highly customized applications that were developed by our partners, and there are even a couple of freely downloadable samples (like the SharePoint Content Deployment Wizard from CodePlex, the Content Publishing tool that accompanies my Content Migration API in Action" post and others) that you can use for some scenarios.

SO ... assuming you would like to give it a try - here is a simple sample walkthrough:

1.       Prepare

1.1.    Make sure you can install and run the applications on the SharePoint server (in a TEST environment!)

1.2.    Select the SharePoint 2007 site you want to export

1.3.    Download the Content Publishing application and unzip

1.4.    Download the CPsplitter application (see link by the end of the article) and unzip

2.       Run the Content Publishing application and in the EXPORT pane ...

2.1.    Set 'Export file location' to be c:\kuku

2.2.    Unselect 'File Compression'

2.3.    Type you site URL in the 'Site URL'

2.4.    Set 'File Max Size' to be 1 (meaning: 1 MB)

2.5.    Click EXPORT to get it going and acknowledge you wish to run it J

3.       While the EXPORT is running ...

3.1.    Look at c:\kuku and see the files getting in

3.2.    Run CPsplitter

3.3.    Click 'Demo' to set the properties

3.4.    Click 'Split Files into Folders'

3.5.    Monitor CPsplitter logging and c:\kuku

So far so good for export …

But now you want to IMPORT, and the manifests are not in the root folder, and the files are scattered all over …

No worries – you’ve got CPsplitter!

Run CPsplitter, make sure MANIFEST.XSLT is in place and the TARGET FOLDER is correct and click UPDATE MANIFESTS.

 “What will it do?” you ask …

·         Here is what it will (hopefully) do: every manifest, from every sub folder of the TARGET FOLDER, will be updated to refer to the files in the sub folder, and will be copied to the root TARGET FOLDER, which is used for the IMPORT of Content Publishing.  

Content Migration Package Explorer

Now, that you have exported content from SharePoint, and just before you import it into another farm, you may want to take a look at the exported package.

If you do - you may want to use the CMP Explorer, which is attached by the end of this article.

Alternatively, you could use any of your favorite XML editors :)

Background

Content migration EXPORT produces either a single CMP file or a folder with multiple XML files.

To specify the desired output type - use the FILECOMPRESSION EXPORT settings.

To extract files from a CMP file - copy it and rename its extension to CAB.

 

How To?

Once you have the files in a folder, you are ready to use the CMP Explorer.

Simply type the folder name and click to inspect some of the content of the XML files.

More information

Content migration output XML files are described in details in the WSS SDK, and their schemas are at C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\XML\Deployment*.xsd .

Here are excerpts from the WSS SDK:

• Manifest.xml is the primary file used by content migration.
• Requirements.xml contains information that is used as a preliminary check before any import takes place.
• ExportSettings.xml is used to verify the logic of the export and ensure that what is expected in the export is included in the package.
• RootObjectMap.xml defines the top-level object to import.
• SystemData.xml contains all the default objects that are installed on a server that is extended with Windows SharePoint Services. This information is used primarily when the export/import operation is retaining GUIDs between the source and destination locations. This file contains the schema version. If this does not match the version at the destination, the import fails.
• UserGroup.xml contains all the user and group information from the source Web site.
• LookupListMap.xml maintains a simple lookup list that records SharePoint list item (list item to list item) references.
• ViewFormsList.xml maintains a list of Web Parts and tracks whether each is a view or form.

Disclaimer
This posting is provided "AS IS" with no warranties. Use of included utilities are subject to the terms specified here.

Have a great CMPing,
Ronalus

A little GUID picker

While nowhere near the SharePoint Explorer, the below attached little GUID picker turned out to be quite useful for a quick peek at SharePoint 2007 lists' / views' GUIDs.

No need to deploy on the server. Using SharePoint's web services it can run remotely.

Content Migration API in Action

Content deployment works well when a clear connection between the source farm and the destination farm is always available. However, a reliable, functional connection between a source authoring farm and a destination production farm is not always available. For example, geographic barriers, firewalls that prevent access, or network outages are all situations in which using a network connection to transport an export package to a destination production farm is not reliable. In these situations, Microsoft recommends that you work with the Microsoft Office SharePoint Server 2007 object model and the Microsoft Windows SharePoint Services 3.0 Content Migration API to programmatically complete the export and import steps of content migration and find an alternative way to transport the export package to the destination production farm and then run the custom import code.

Above is from the article How to: Customize Deployment for Disconnected Scenarios , which also provides a complete code sample for using the content migration API.

Starting from this code sample, I walked through the API and got to a little application, which can be downloaded from the link below. Do note that as the application uses the SharePoint object model, it should run on the SharePoint 2007 server. Make sure to thoroughly test in test environment, as it is still work in progress...

When running the application, take a look at the session log on the right side of the screen. It lists the properties used in the deployment and may help you develop your own tailor made tools.

Tip: for incremental deployment you need 4 settings:

1. ExportChangeToken field should contain the token from a previous export
2. ExportMethod should be ExportChanges
3. ExportObjectType should be either site or web
4. Import RetainChangeIdentity should be selected

To make things easy, the little application (AKA CP) places the ExportChangeToken for you in the correct field upon an Export completion. You can also save the token, along with a descriptive statement, in an XML file, for future use.

[Update: Oct. 2, 2007] As CP takes its first steps, new ideas emerge. Two of them were already implemented:

1. A batch version (for you, Sharon)
2. An MSBuild task (to comply with your great development, Neeraj)

Both CPbatch and CPtask are also in the below downloadable ZIP.

To run them - use a settings XML file, which can be generated by "Save Preferences" in CP (the interactive version).

For other samples see Migrate a Web site from one location to another  and the WSS SDK.

Cool usage of STSADM for content migration can be found in Jackie's blog: SharePoint Content Migration Object Model and Content Migration Packages .

Disclaimer
This posting is provided "AS IS" with no warranties. Use of included utilities are subject to the terms specified here.

Enjoy content migration,

Ronalus

stsadmWin 2007 is here

One of the most frequently used utilities in SharePoint (2007 as well as 2003) is STSADM.

Although highly improved in 2007, it maintains backward compatible with 2003 syntax.

As STSADM is a command line utility, and the command-line is somewhat limited compared to Windows GUI, especially when it comes to non-English usage, there is a room for GUIs like stsadmWin (STSADM for Windows).

How stsadmWin works?

stsadmWin reads a list of commands from an XML file (namely: operations.xml), and displays a combo with the operations, denoting mandatory and optional parameters, and linking to the utility’s help.

The above download includes an XML file with SharePoint 2003 operations.

An XML file for MOSS 2007 RTM is attached by the end of this post (in stsadmWin.zip ).

Help information

stsadmWin displays help for the selected operation in the 2003 version.

This feature is currently not available in the 2007 version.

To get more information on STSADM 2007 see:

 * Stsadm command-line tool (Office SharePoint Server)

 * Stsadm command-line tool (Windows SharePoint Services)

 * Complete reference of all STSADM operations (with parameters) in MOSS 2007

 * Run STSADM /?   -or-   STSADM –help <OPERATION>

To get a list of stsadm commands grouped by function - see Ian Morrish's WSS Demo .
The list is updated to SP1 and even indicates if a command is unique  to stsadm or if there is a Central Admin/SSP equivalent.

stsadmWin extensibility

Unlike the 2003 version, STSADM in 2007 is extensible. Meaning: you may get newer commands (operations) available as you add components to your environment.

To generate an XML file that suites your environment – you can use the GetOperations utility (also attached by the end of this post - in stsadmWin.zip).

Note: it is advisable to backup existing XML before generating an updated version …

A note on the generated XML:

 * A couple of operations may be generated with wrong syntax.

 * The ones I currently know about are: osearch and profilechangelog.

 * An updated syntax will be available in operationsUpdate.xml, which is referenced below.

 * You will only need this update when you re-generate operations.xml. In this case - you have to update operations.xml with the content of operationsUpdate.xml .

 * The operations.xml files that are referenced below - already include the correct syntax.