Copying a SharePoint 2010 Site Collection from one Farm to Another

On a number of occasions, I’ve wanted to migrate a SharePoint 2010 Site Collection from a Production environment back over to a Testing or Development environment with, and the results have met with mixed success. The most common issue, was that the Managed Metadata term sets didn’t “hook up” correctly.

This problem is pretty much to be expected, given every SharePoint Farm has a separate Managed Metadata Service. And even if two Managed Metadata Services were “seeded” with the same term sets initially, they would never be “internally” equivalent. Why? Because, within every Managed Metadata Service, every term set, and within every term set, every term has a unique identifier assigned to it, as well as its textual description, to ensure uniqueness.

So therefore when you restore a site collection from one farm to another, and an item has a reference to a managed metadata field with string description of “Blue Plastic”, internally, that refers to some unique identifiers for both the term-set and the individual term, that was established within the Managed Metadata Service of the Farm where that term was first created. On a different farm where a term exists at exactly the same point of an equivalent looking term set hierarchy called “Blue Plastic”, the relationship simply won’t hook up and the Managed Metadata fields on the lists and libraries of the restored site collection become unusable.

There’s a number of different “approaches” documented as to how to resolve these, but the one I’ve used recently and seems to work, involves backing up the entire Managed Metadata store and restoring it over the existing Managed Metadata store on a Destination Farm prior to restoring a Site Collection. I’ve documented the approach here, in the hope it might help others, and to solicit feedback from others to help refine the technique I’ve used, in case there’s some improvements that can be made.

Of course this only works well if you always consider your “Production” Managed Metadata Service to be the “Single Source of Truth”, but I think in most organisations that would tend to be the case.

High Level Steps Involved in the Copy

In the following steps, I’ve described the Production SharePoint Farm from which the Site Collection we want to copy originated as the Source Farm or Site Collection, and the Test or Development Farm to which the Site Collection is to be restored as the Destination Farm or Site Collection.

The high level steps involved in the copy of the Site Collection from the Source Farm to the Destination Farm are shown below:

Extract the current Source Farm SharePoint Solutions. As the current Source Farm may be running a different version of SharePoint solutions to that which the Destination Farm is, we need to extract the current SharePoint Solutions from the SharePoint solution store on the Source Farm.
Update the SharePoint Solutions on the Destination Farm, or add them if they don’t already exist. This involves transferring the backed up solution files, and either updating existing solutions on the Destination Farm, or adding the ones that don’t exist.
Backup the Managed Metadata Service from the Source Farm. This backup is then used to ensure all of the managed metadata values at the Destination Farm match those at the source environment, once the target Site Collection is restored.
Restore the Managed Metadata Service to the Destination Farm from the backup just taken.
Backup the required Site Collection from the source environment. This backup contains all of the appropriate content from your production environment and potentially references to the Managed Metadata Service.
Restore the required Site Collection to the destination environment. This will restore all content form the Source Site Collection environment into the Destination Site Collection.
Make any final adjustments to the Site Collection “Post Restore” on the Destination Farm.

Detailed Steps Involved in the Copy

Extract the Current SharePoint Solutions from the Source

The following PowerShell Script can be used to extract the SharePoint Solutions:

[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint")

$solutions = [Microsoft.SharePoint.Administration.SPFarm]::Local.Solutions;
foreach ($solution in $solutions)
{
   $solution.SolutionFile.SaveAs("c:\solution\" + $solution.Name);
}

This code used reflection to create an SPFarm object from the Microsoft.SharePoint.Administration namespace. It then gets the Local farm instance, and retrieves a reference to the Solutions collection. After that it iterates through the solutions in Farm Solution Store and for each one saves them locally in a directory c:\solution. Running the PowerShell script doesn’t generate any output except when the assembly is successfully loaded. However when you examine the directory c:\solution, you should see a .wsp file that corresponds to each of the SharePoint Solutions in the Solution store.

Update or Add the SharePoint Solutions to the Destination

To update or add the solutions to the Destination Farm, firstly physically copy all of the .wsp files to the destination SharePoint Server. Then examine the list of existing solutions on the Destination Farm. To do this, navigate to Central Administration | System Settings | Manage Farm Solutions.

After determining which Solutions already exist, and which don’t, run the PowerShell Integrated Scripting Environment (ISE) as an Administrator.

Key things to note about this:

You must run the PowerShell ISE as an Administrator
You must always remember to run the PowerShell ISE and not the PowerShell ISE (x86) which is the 32-bit version. As SharePoint 2010 is a 64-bit application, you must run the 64-bit version of the PowerShell ISE to ensure you can execute the appropriate SharePoint PowerShell libraries.

Prior to executing any SharePoint related PowerShell commands, you need to add the SharePoint powershell “snapin”. To do this, we execute the following command:

add-pssnapin microsoft.sharepoint.powershell

After this, you have access to all of the SharePoint related Powershell cmdlets. The ones of interest to us relate to updating existing and adding new solutions.

Update-SPSolution: Upgrades a previously deployed SharePoint Solution
Add-SPSolution: Uploads a SharePoint solution package that has previously not been deployed.

You can get further documentation about these commands with Powershell by using the get-help PowerShell cmdlet, as shown below.

One of the most useful forms of the get-help PowerShell cmdlet is to use the –examples switch which instead of providing the detailed explanation and syntax of the command, provides specific examples of its use.

So assuming you copied the solutions from the source environment to a folder on the destination SharePoint Central Administration server E:\solution, and that a potentially older version of the first solution shown above already existed on destination server xxx.wsp, the command required to update it would be:

Update-SPSolution -Identity xxx.wsp -LiteralPath E:\solution\xxx.wsp -GACDeployment

For solutions that did not exist at all in the destination environment, the command would look as follows:

Add-SPSolution -LiteralPath E:\solution\xxx.wsp

Once all of the solutions from the Source Farm have either been updated or added to the Destination Farm, this step is completed.

Backup the Managed Metadata Service from the Source

This is one of the more complex parts of the process, but is crucial to the successful subsequent restore of the required Site Collection, as it ensures that the Managed Metadata Links operate correctly when restored.

The script to backup the Managed Metadata service on the Source SharePoint environment is deceptively simple.

However the two constant values highlighted above will vary for each SharePoint installation, and you need to determine them. They are:

$mmsAppId: Service Application ID for the Managed Metadata Service on the Source SharePoint Environment
$mmsproxy: ID for the Managed Metadata Service proxy

To determine the Service Application ID for the Managed Metadata Service, open Central Administration, and navigate to Manage Service Applications | Managed Metadata Service | Manage. From the page you arrive at, examine the URL.

The ID at the end of the URL, passed as the QueryString parameter tsid is the required ID to be used for the variable $mmsAppId in the PowerShell script. To determine the ID for the Managed Metadata Service Proxy, return to the Manage Service Applications page, select Managed Metadata Service Proxy by selecting the line on the page, but don’t click on it.

Then click on the Properties button on the ribbon. Once this modal dialog has been shown, right mouse click on the dialog, and select the browser properties of that window. When this is shown, select the value associated with the QueryString variable ID from the URL of this properties window as shown below. It is the value associated with the QueryString parameter appid. And this value can be assigned to the PowerShell Script variable $mmsproxy.

After determining these two variable values, you can run the script shown above. It takes a very short amount of time to run, and if it runs successfully doesn’t generate any output. In my experience, the backup file created is usually of a very modest size.

Restore the Managed Metadata Service to the Destination

On the Destination Farm, the restore process is essentially the reverse of the backup process. The script to perform the restore is once again remarkably simple, and looks very similar to the backup script.

The two PowerShell variables $mmsAppId and $mmsproxy have the same meanings as in the backup script, but need to be determined for the destination SharePoint Farm as obviously, they will be different. However you use exactly the same techniques described above.

There are a couple of other key requirement for the import to succeed:

The account under which the Managed Metadata Service Application Pool runs, must have bulkadmin permissions on the SQL Server instance supporting the Destination SharePoint Farm, and
The backup file to be restored must be on the same server as the database. In our case, and for our purposes, this was most often the case when restoring to single-server SharePoint Test and Development Farms.

Backup the required Content Site Collection from the Source Farm

I won’t document this step extensively, as this simply involves a “standard” SharePoint 2010 Site Collection Backup which is an administrative command most if not all SharePoint Administrators and Developers would be quite familiar with.

Restore the required Site Collection to the Destination Farm

The restore script at the destination is relatively simple, as you can see below. It actually involves only one PowerShell command, Restore-SPSite, but I have used a number of separate PowerShell variables just to make the particular values more obvious.

A number of variables are used in the Script that can be replaced to suit your Destination Farm:

$SiteCollURL: The Site Collection Name into which the Site Collection Backup will be restored at the destination SharePoint Farm
$backupPath: The Literal Path on the server where the script will run where the Site Collection Backup can be found
$DatabaseServer: The Name of the SQL Server Instance supporting SharePoint
$ContentDatabaseName: The Name of the Underlying SharePoint Content Database Name into which the Site Collection will be restored

The -Force option overwrites the current content, and -GradualDelete option causes existing content to be deleted over time by Timer Jobs. The -Verbose option causes a slightly higher degree of output, but that still doesn’t generate much.

Make any final adjustments to the Destination Environment

There maybe issues that need to be addressed once you’ve restored the Site Collection to your Destination Farm. The most common issues include:

Site Collection Owners will likely be invalid, and need to be set to Active Directory users local to the AD the Destination Farm is located in.
The inappropriate or unavoidable use of absolute URLs in the content of the Source Farm Site Collection. Many of these will become obvious quickly however, particularly if the two Farms are network isolated. If they’re not network isolated, this can present a few issues, because you may be “inadvertently” still accessing production content or data from your Test or Development Farm, which wouldn’t be very good.

I hope these steps prove useful to you when you’re trying to copy a Site Collection from one SharePoint 2010 Farm to another, and I’d welcome your feedback on the approach I’ve proposed, and any suggested improvements.

References

The following post in particular proved invaluable in determining an approach for the backup and restore of the Managed Metadata service:

http://www.chaitumadala.com/2011/10/sharepoint-2010-how-to-backuprestore.html – This article provides a terrific overview of the approach to performing a backup and restore of the Managed Metadata service using PowerShell commands. Particularly informative are the great tips on how to determine the Service Application and Service Application Proxy Identifiers easily.