Copy content types Across Farms (SharePoint)

Reddit It | Digg This | Add to your: del.icio.us, Google, Yahoo

Content Types in SharePoint can be very useful, and can be created by manually by users or automatically by various products or procedures. In one case, content types may be created automatically during a migration of content from CMS to SharePoint MOSS 2007 publishing server. Regardless of where they came from, they can be copied from one site to another, even moved across farms.

Under most circumstances, Gary Lapointe's stsadm extensions can be used to help automate the process. He has operations for copying content types between sites in the same farm, for propogating content types and for exporting content types from a site. Strangely there is no import content types, which would have enabled cross-farm content type copying.

To copy content types across farms, you'll need to do the gl-exportcontenttypes and then create a SharePoint feature that adds them to your site collection, site, ... whatever. If you don't want to install Gary Lapointe's stsadm extensions, you'll need to use his source code to write a console application that does the same thing. If you have the console application, you can drop it onto your source server (no install needed) and do the export.

For the export, I used the following flags:
url = http://hostname
contentTypeGroup = "Custom Content Types"
contentTypeName = null
excludeParentFields = false
includeFieldDefinitions = true
includeListBindings = false
listName = null
removeEncodedSpaces = false
featureSafe = true
outputFile = "ctypes.xml"

Once I had my output (ctypes.xml), I had to remove any ... tag pairs. For some reason they were included in some of the field definitions (site columns) associated with the exported content types. I don't know what the ultimate effect of this is, but it didn't matter to me in this case, and it worked without it.

The structure of the ctypes.xml file should look like this (values removed):
<elements xmlns="http://schemas.microsoft.com/sharepoint/">
<field id=" Name="" sourceid=" StaticName="" group=" RowOrdinal="" type=" DisplayName="" readonly=" Sealed="" colname=" PITarget="" piattribute=" /">
[...]
<contenttype id=" Name="" group=" Version="">
<folder targetname=" /">
<fieldrefs>
<fieldref id=" Name="">
<fieldref id=" Name="">
</fieldrefs></contenttype></elements>

Create a 12 hive feature directory:
C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\FEATURES\MyCtypes

Copy the ctypes.xml file into the directory.

Add feature.xml with content like this (with a new Id GUID, and change Scope if desired, remember Scope="Site" means SiteCollection):
<?xml version="1.0" encoding="utf-8"?>
<feature id="031D991D-2559-4512-BE86-024E9CA035B3"
Title="My Imported content types"
Description="Content types imported from wherever"
Version="1.0.0.0"
Scope="Site"
ImageUrl="Health.GIF"
Hidden="FALSE"
DefaultResourceFile="core"
xmlns="http://schemas.microsoft.com/sharepoint/">
<elementmanifests>
<elementmanifest location="ctypes.xml">
</elementmanifests>
</feature>

Install the feature this way:
"C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\BIN\stsadm.exe" -o installfeature -name MyCtypes

(if your xml file doesn't comply to proper schema, this installfeature will fail)

Now go activeate the feature. In this case, go to "Site Collection Features" and click activate for the new feature. This will add the content types to the site collection.

Uninstall this way:
"C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\BIN\stsadm.exe" -o uninstallfeature -name MyCtypes

This uninstall will remove the content types from the site collection. I suspect this will fail if the content types are in use by the site.

Porting YetAnotherForum.NET to SharePoint

Reddit It | Digg This | Add to your: del.icio.us, Google, Yahoo

Idea

YetAnotherForum.NET (YAF.NET) on SharePoint, sounds like a great idea!
(The main focus of this article is partitioning data between YAF.NET forums based on SharePoint site.)

It sounds like YAF.NET might already handle authentication of users arriving through Active Directory (ie: auto register), but I don't want just a YAF.NET install next to my SharePoint system. My needs are to have YAF.NET be a feature that can be deployed across hundreds (thousands?) of SharePoint sites without intermingling the content (Topics/Posts) between sites.

It seemed like it might be a weekend effort, but with family activities, other personal projects and the size of the effort, I've decided that I just won't have the time to port YetAnotherForum.NET to SharePoint 2007 (WSSv3). I will share my findings though.

Approach #1

Inversion of Control, decoupling the data access mechanism from the application so a new data store (ie: SharePoint) could be injected. The approach would be to take all the DB access methods in YAF.NET, and make them use an interface, and put the existing DB access code into a class to satisfy that data interface. The phase 1 result would be a code refactored YAF.NET that works exactly the same as it did.

The second phase of the port would be to create another implementation of the data access interface that utilizes SharePoint lists instead of the SQL database. Plug in that new interface, and the YAF.NET should work fine without knowing that its running on SharePoint. It should be possible to satisfy all the database needs of YAF.NET with SharePoint lists and CAML for access. The only need that might not work is YAF.NET's free-text search capabilities.

Data will need to be compartmentalized into the SharePoint sites that YAF.NET was enabled for. This will help application scaling and content ownership. There are some practical data limits to SharePoint's core list capabilities, but utilizing separate YAF.NET lists per SharePoint site avoid those issues. The YAF.NET SharePoint data access would have to utilize the lists for the current site to make this work, would be pretty easy once the rest was in place.

Example YAF.NET loading from SharePoint site context:
http://hostname/sites/TestSite/_layouts/YAF/default.aspx

The SharePoint site at "/sites/TestSite" would have the normal YAF.NET data sources:

[...]

The entire YAF.NET for SharePoint system would be packaged as a SharePoint site collection or web scoped feature in a SharePoint solution file. That means that once the solution has been added to your SharePoint system, site administrators could go to Site Features (or Site Collection Features) and enable that feature. The initialization of the feature would run, creating the proper lists and default data.

This approach sounds like the best SharePoint solution, but without in-depth knowledge of YAF.NET, it feels like 3-4 weeks worth of development and testing.

Approach #2

A compromise solution to porting YAF.NET to SharePoint would be to partition the data for SharePoint sites within the current YAF.NET SQL database. By adding a new "Context" column to all existing tables, then slightly modifying all the SQL in YAF.NET, the system could function within the context of the SharePoint site it was accessed from. This approach simulates different data stores for each SharePoint site, but does not change the native source YAF.NET already uses, so the approach is less complex.

This approach sounds like a reasonable SharePoint solution, and avoids some of the questions about SharePoint list data scalability issues by continuing to use native SQL Server. Because this approach is generally many simple modifications, it feels more like 1-3 weeks worth of development and testing.

I started some work down this approach to flush out the main issues. Some of these steps are required for either approach...

Live in SharePoint Layouts
YAF.NET would need to function in the common SharePoint "layouts" directory. This is where common SharePoint functions/systems need to live, and they are given the opportunity to be "aware" of the site context that they are invoked from. This part was quick and easy.

Misc:
In \web.config, I enabled session state, and switched trust level to full. Neither is a great idea for SharePoint, but worked for my development purposes...
pages enablesessionstate="true" ...
trust level="Full" originUrl=""

For this url to work:
http://hostname/sites/TestSite/_layouts/YAF/default.aspx

...YAF.NET needs to be copied to:
C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\LAYOUTS\YAF
(I put the YAF.NET binaries into the webroot/bin for testing)

Config.cs needs to know about site context:

        /// 

        /// Determine the site context from the http request.
        /// /
        /// /sites/TestSite
        /// 
        static public string UrlContextPath
        {
            get
            {
                string scriptUrl = HttpContext.Current.Request.ServerVariables["SCRIPT_NAME"];
                string rawUrl = HttpContext.Current.Request.RawUrl;
                int pos = rawUrl.ToUpper().IndexOf(scriptUrl.ToUpper());
                if (pos <>
                {
                    return "/";
                }
                return "/" + rawUrl.Substring(0, pos);
            }
        }

UrlBuilder.cs needs to build urls with site context:

        public string BuildUrl(string url)
        {
            return string.Format("{0}{1}?{2}",
                Config.UrlContextPath.Substring(1),
                HttpContext.Current.Request.ServerVariables["SCRIPT_NAME"],
                url
                );

            //return string.Format("{0}?{1}",HttpContext.Current.Request.ServerVariables["SCRIPT_NAME"],url);
        }

Partition Data by Site Context

This is the BIGGEST porting task, requiring changes to all 31 database tables, 1 view, 162 stored procedures, and all corresponding code invocations of SQL and stored procedures.

Add the Context field to all database tables. For example:

IF NOT EXISTS (SELECT 1
          FROM   sysobjects
          WHERE  id = Object_id(N'yaf_AccessMask')
          AND Objectproperty(id,N'IsUserTable') = 1)
CREATE TABLE dbo.yaf_AccessMask (
[AccessMaskID] INT IDENTITY NOT NULL,
[Context]      NVARCHAR(255) NOT NULL,
[BoardID]      INT NOT NULL,
[Name]         NVARCHAR(50) NOT NULL,
[Flags]        INT NOT NULL CONSTRAINT DF_yaf_AccessMask_Flags DEFAULT (0))
GO

IF NOT EXISTS (SELECT 1
          FROM   syscolumns
          WHERE  id = Object_id('yaf_AccessMask')
          AND name = 'Context')
BEGIN
ALTER TABLE dbo.yaf_AccessMask
ADD Context NVARCHAR(255) NOT NULL DEFAULT '/'
END
GO

Modify all SQL and Stored Procedures to use the new Context field. For example:

CREATE PROCEDURE [dbo].[yaf_accessmask_delete](
                @AccessMaskID INT,
                @Context NVARCHAR(255) = '/')
AS
    BEGIN
        DECLARE  @flag INT
        SET @flag = 1
        IF EXISTS (SELECT 1
                   FROM   yaf_ForumAccess
                   WHERE  AccessMaskID = @AccessMaskID)
            OR EXISTS (SELECT 1
                       FROM   yaf_UserForum
                       WHERE  AccessMaskID = @AccessMaskID)
        SET @flag = 0
        ELSE
        DELETE FROM yaf_AccessMask
        WHERE       AccessMaskID = @AccessMaskID
        AND         Context = @Context
        SELECT @flag
    END
GO

Modify all invocations to SQL and Stored Procedures to use the new Context field. For example:

        static public bool accessmask_delete( object accessMaskID )
        {
            using ( SqlCommand cmd = new SqlCommand( "yaf_accessmask_delete" ) )
            {
                cmd.CommandType = CommandType.StoredProcedure;
                cmd.Parameters.AddWithValue( "@AccessMaskID", accessMaskID );
                cmd.Parameters.AddWithValue("@Context", Config.UrlContextPath);
                return (int)ExecuteScalar(cmd) != 0;
            }
        }

With a few of these changes in effect, YAF.NET continued to function properly, which helped prove the viability of the approach, but also proved that it would take a non-trivial amount of time to complete. I did not continue the effort due to my lack of availability.

Additional Issues:

Web Part - These approaches are targeted at making YAF.NET work as a companion to SharePoint, not fully integrate it. An expectation of any SharePoint admin would be how to drop a dynamic view of the forum onto a SharePoint Web Part Page. Luckily YAF.NET is constructed with web user controls, which translate pretty easily into web parts. Look at SmartPart WebPart (or other sample code) for how to load a normal web user control in a Web Part container.

Isolation - Some elements of YAF.NET need to be shared rather than isolated between SharePoint web sites. Data such as Users, Permissions and Access Masks are some examples of data that might need to be shared across all SharePoint sites (especially users!)

Overlapping domains - Some elements of YAF.NET need to pull data from SharePoint rather than handled by YAF.NET. For instance, Users and Permissions should come from SharePoint, otherwise SharePoint admins will need to setup users/permissions TWICE (once for SharePoint sites, and another time for YAF.NET).

Default data - Some data is provisioned during the one-time YAF.NET install. Some amount of data provisioning may need to occur once per SharePoint site. During setup/initialization. For instance, the AccessMask is setup with some default permissions. If data in YAFF.NET is merely partitioned between SharePoint sites, that one-time data initialization will only benefit one SharePoint site.