Updating content types using SharePoint Web Services (SPServices)

When you deploy an updated content type which is created from code and have new fields, your changes won’t automatically be pushed to children that inherit from your content type. This is only the case when you have created your content type using code. In the sandbox solution, I’m using the SharePoint Web Services to add new fields to content types that inherit from the updated content type.

Why the web services? With the push from Microsoft to go more and more client side, I found it a challenge to not do something with simply a farm solution or a server-side script. During the development of the web part, I’ve found 2 annoyances by using the SPServices library and jQuery/SharePoint:

  1. SPServices says every method name is unique. This is not true. There’s a method “UpdateContentType” for both the Lists.asmx and the Webs.asmx. Due to this overlooked bug, the call will always be directed to the Webs.asmx web service. I’ve modified the SPServices js file to comment out the WSops.UpdateContentType method for Webs.
  2. jQuery will always put your generated attributes to lower case. The web services are case-sensitive as you can see from the following screenshot (taken from STSSOAP.DLL, decompiled with ILSpy)
    caseSensitive

The web part itself is pretty straight forward. You’ll get a list of content types with update links.

content type updater webpart

When using firebug or another web dev tool, you can track all the calls that are made to the Webs.asmx & Lists.asmx

content type updater console window

I’ve deployed a custom content type to my site, created a list with that content type and then made an update to the site content type. As you can see there’s a column missing from the list content type:

site content type

list content type

After I click on “Update” next to “CustomContentType”, the script will detect a difference between these two and update accordingly:

soap call new fields

And the updated list content type:

updated list content type

All in all I’m pretty happy with the result. You can do a lot with just the web services from SharePoint, but they take some time to get used to. Also the documentation isn’t always great as for instance they mention to use <FieldRefs> while in the source code of the web service itself there’s only a parse of the <Fields> tag.

For the full source, or if you want to download and test the web part itself, you can go to CodePlex.

High CPU usage due to _vti_bin/cellstorage.svc/CellStorageService

Recently I faced a cpu hang by a w3wp process of one of the application pools. The process stayed on roughly 50% of cpu usage, which caused performance issues for the other processes running on the WFEs. Recycling the application pool helped at first, but after a few moments the process started spiking again, slowing everything down.

First thing to do is check the ULS logs: nothing special there. It was also pretty hard to pinpoint one specific line as a cause to the high cpu usage.

On to more specific analysis then: Process Explorer. The tool is great for checking your process threads and checking the callstack that is causing any interference.

Below is an example of how the stack window looks:

procmon

The method that was hanging during the issue was GetMetadatainternalinterfacefromplublic. Not much more info was available from the stack window. There was no custom code involved during the performance decrease, so further investigation was needed. I noted down the thread id and made a dump of the running process using task manager:

create dump file

Once the dump was created, I installed DebugDiag v1.2. Analysis of the dump was pretty straight forward using the performance analyzers:

debugdiagperfmon

The callstack of the thread with entry point mscorwks!GetMetaDataInternalInterfaceFromPublic was:

System.Data.SqlClient.TdsParserStateObject.WritePacket(Byte) 
System.Data.SqlClient.TdsParser.WriteByteArray(Byte[], Int32, Int32, System.Data.SqlClient.TdsParserStateObject) 
System.Data.SqlClient.TdsParser.WriteValue(System.Object, System.Data.SqlClient.MetaType, Byte, Int32, Int32, Int32, System.Data.SqlClient.TdsParserStateObject) 
System.Data.SqlClient.TdsParser.TdsExecuteRPC(System.Data.SqlClient._SqlRPC[], Int32, Boolean, System.Data.Sql.SqlNotificationRequest, System.Data.SqlClient.TdsParserStateObject, Boolean) 
System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(System.Data.CommandBehavior, System.Data.SqlClient.RunBehavior, Boolean, Boolean) 
System.Data.SqlClient.SqlCommand.RunExecuteReader(System.Data.CommandBehavior, System.Data.SqlClient.RunBehavior, Boolean, System.String, System.Data.Common.DbAsyncResult) 
System.Data.SqlClient.SqlCommand.RunExecuteReader(System.Data.CommandBehavior, System.Data.SqlClient.RunBehavior, Boolean, System.String) 
System.Data.SqlClient.SqlCommand.ExecuteReader(System.Data.CommandBehavior, System.String) 
System.Data.SqlClient.SqlCommand.ExecuteReader(System.Data.CommandBehavior) 
Microsoft.SharePoint.Utilities.SqlSession.ExecuteReader(System.Data.SqlClient.SqlCommand, System.Data.CommandBehavior, Microsoft.SharePoint.Utilities.SqlQueryData, Boolean) 
Microsoft.SharePoint.SPSqlClient.ExecuteQueryInternal(Boolean) 
Microsoft.SharePoint.SPSqlClient.ExecuteQuery(Boolean) 
DomainNeutralILStubClass.IL_STUB(System.String, System.String, System.Object, Int32, System.Object, Microsoft.SharePoint.Library.PutFileOpt, System.String, System.String, Int32, Int32, System.Object, System.Object, System.Object, System.String, Byte, Int64, System.String, System.String, System.String, Int32, System.String, Int32, Int32, Int32, System.Guid, UInt32 ByRef, System.String ByRef, System.String ByRef, Byte ByRef, Int32 ByRef) 
Microsoft.SharePoint.Library.SPRequest.PutFile(System.String, System.String, System.Object, Int32, System.Object, Microsoft.SharePoint.Library.PutFileOpt, System.String, System.String, Int32, Int32, System.Object, System.Object, System.Object, System.String, Byte, Int64, System.String, System.String, System.String, Int32, System.String, Int32, Int32, Int32, System.Guid, UInt32 ByRef, System.String ByRef, System.String ByRef, Byte ByRef, Int32 ByRef) 
Microsoft.SharePoint.SPFile.SaveBinaryStreamInternal(System.IO.Stream, System.String, Boolean, Boolean, Boolean, Boolean, Boolean, System.String, Microsoft.SharePoint.SPUser, System.DateTime, System.Object, Microsoft.SharePoint.SPFileFragmentPartition, Microsoft.SharePoint.SPFileFragmentId, Microsoft.SharePoint.SPFileFragmentPartition[], System.IO.Stream, System.String, Boolean, SPLockType, System.String, System.TimeSpan, Boolean, Boolean, Boolean, Boolean, System.Guid, Microsoft.SharePoint.SPVirusCheckStatus ByRef, System.String ByRef, System.String ByRef, Boolean ByRef) 
Microsoft.SharePoint.Library.SPServerStorageBridge 
Microsoft.SharePoint.SPSecurity 
Microsoft.SharePoint.Utilities.SecurityContext.RunAsProcess(CodeToRunElevated) 
Microsoft.SharePoint.SPSecurity.RunWithElevatedPrivileges(System.Threading.WaitCallback, System.Object) 
Microsoft.SharePoint.SPSecurity.RunWithElevatedPrivileges(CodeToRunElevated) 
Microsoft.SharePoint.Library.SPServerStorageBridge.Microsoft.SharePoint.Library.IServerStorageBridge.PutStreams(Microsoft.SharePoint.Library.IStreamBridge, Microsoft.SharePoint.Library.IStreamBridge, System.String, Boolean, UInt64, System.String, System.String, System.String, Int32, System.String, Int32, Boolean, Boolean, Boolean, System.String ByRef, Boolean ByRef, System.String ByRef, UInt32 ByRef) 
Microsoft.SharePoint.Utilities.SandboxServerBigBridge.CallServerStorageBridge_PutStreams(Microsoft.SharePoint.Utilities.SandboxMessageType, Microsoft.SharePoint.Utilities.SandboxCommunicator) 
Microsoft.SharePoint.Utilities.SandboxServerBigBridge.EventLoopFor(Microsoft.SharePoint.Utilities.SandboxMessageType, Microsoft.SharePoint.Utilities.SandboxCommunicator) 
Microsoft.SharePoint.Utilities.SPUtility.PerformSandboxOperation(Microsoft.SharePoint.Utilities.SandboxCommunicator, Microsoft.SharePoint.Utilities.ExecuteCellStorageBinaryRequestParameters) 
Microsoft.SharePoint.Utilities.SPUtility.ExecuteCellStorageBinaryRequest(Microsoft.SharePoint.SPFile, Boolean, System.IO.Stream, Boolean, System.Guid ByRef, System.String, Boolean, System.String, Boolean, System.String, System.String, System.String, Int64, System.String, Int64, Boolean, System.String ByRef, Boolean ByRef, Int32 ByRef, System.String ByRef, Boolean ByRef, Boolean ByRef, Int32 ByRef) 
Microsoft.SharePoint.SoapServer.CellStoragesImpl 
Microsoft.SharePoint.SoapServer.CellStoragesImpl.FProcessSubRequests(System.Xml.XmlReader, System.Xml.XmlWriter, Microsoft.SharePoint.SPFile, System.Collections.Generic.Dictionary`2, ResourceQuota) 
Microsoft.SharePoint.SoapServer.CellStoragesImpl.FProcessRequest(Microsoft.SharePoint.SPWeb, System.Xml.XmlReader, System.Xml.XmlWriter, System.Collections.Generic.Dictionary`2, ResourceQuota) 
Microsoft.SharePoint.SoapServer.CellStoragesImpl 
System.ServiceModel.Channels.BodyWriter.WriteBodyContents(System.Xml.XmlDictionaryWriter) 
System.ServiceModel.Channels.Message.OnWriteMessage(System.Xml.XmlDictionaryWriter) 
System.ServiceModel.Channels.MtomMessageEncoder.WriteMessage(System.ServiceModel.Channels.Message, System.IO.Stream, System.String, System.String, System.String, Boolean) 
System.ServiceModel.Channels.HttpOutput.WriteStreamedMessage(System.TimeSpan) 
System.ServiceModel.Channels.HttpOutput.Send(System.TimeSpan) 
System.ServiceModel.Channels.HttpRequestContext.OnReply(System.ServiceModel.Channels.Message, System.TimeSpan) 
System.ServiceModel.Channels.RequestContextBase.Reply(System.ServiceModel.Channels.Message, System.TimeSpan) 
System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.Reply(System.ServiceModel.Dispatcher.MessageRpc ByRef) 
System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessageCleanup(System.ServiceModel.Dispatcher.MessageRpc ByRef) 
System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(System.ServiceModel.Dispatcher.MessageRpc ByRef) 
System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage4(System.ServiceModel.Dispatcher.MessageRpc ByRef) 
System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean) 
System.ServiceModel.Dispatcher.ChannelHandler.DispatchAndReleasePump(System.ServiceModel.Channels.RequestContext, Boolean, System.ServiceModel.OperationContext) 
System.ServiceModel.Dispatcher.ChannelHandler.HandleRequest(System.ServiceModel.Channels.RequestContext, System.ServiceModel.OperationContext) 
System.ServiceModel.Dispatcher.ChannelHandler.AsyncMessagePump(System.IAsyncResult) 
System.ServiceModel.Diagnostics.Utility 
System.ServiceModel.AsyncResult.Complete(Boolean) 
System.ServiceModel.Channels.InputQueue`1 
System.ServiceModel.Channels.InputQueue`1[[System.__Canon, mscorlib]].EnqueueAndDispatch(Item, Boolean) 
System.ServiceModel.Channels.InputQueue`1[[System.__Canon, mscorlib]].EnqueueAndDispatch(System.__Canon, System.ServiceModel.Channels.ItemDequeuedCallback, Boolean) 
System.ServiceModel.Channels.InputQueueChannel`1[[System.__Canon, mscorlib]].EnqueueAndDispatch(System.__Canon, System.ServiceModel.Channels.ItemDequeuedCallback, Boolean) 
System.ServiceModel.Channels.SingletonChannelAcceptor`3[[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib]].Enqueue(System.__Canon, System.ServiceModel.Channels.ItemDequeuedCallback, Boolean) 
System.ServiceModel.Channels.SingletonChannelAcceptor`3[[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib]].Enqueue(System.__Canon, System.ServiceModel.Channels.ItemDequeuedCallback) 
System.ServiceModel.Channels.HttpChannelListener.HttpContextReceived(System.ServiceModel.Channels.HttpRequestContext, System.ServiceModel.Channels.ItemDequeuedCallback) 
System.ServiceModel.Activation.HostedHttpTransportManager.HttpContextReceived(System.ServiceModel.Activation.HostedHttpRequestAsyncResult) 
System.ServiceModel.Activation.HostedHttpRequestAsyncResult.BeginRequest() 
System.ServiceModel.Activation.HostedHttpRequestAsyncResult.OnBeginRequest(System.Object) 
System.ServiceModel.PartialTrustHelpers.PartialTrustInvoke(System.Threading.ContextCallback, System.Object) 
System.ServiceModel.Activation.HostedHttpRequestAsyncResult.OnBeginRequestWithFlow(System.Object) 
System.ServiceModel.Channels.IOThreadScheduler 
System.ServiceModel.Channels.IOThreadScheduler 
System.ServiceModel.Channels.IOThreadScheduler 
System.ServiceModel.Channels.IOThreadScheduler 
System.ServiceModel.Channels.IOThreadScheduler 
System.ServiceModel.Diagnostics.Utility 
System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32, UInt32, System.Threading.NativeOverlapped*) 

User Time   01:37:04.765 
Kernel time   00:00:05.553

Notice the User Time of over 1h30! Seems that a SQL statement was staying open. When taking the creation time of the thread, I was able to check the ULS logs with more info. There were calls to a certain OneNote file and the _vti_bin/cellstorage.svc/cellstorageservice.svc was called. There were also CsiSrvExe.exe processing starting at that point.

OneNote uses the Core Storage Infrastructure service to send user edits to the SharePoint server. This server stores the master copy of the notebook. The Core Storage Infrastructure service processes file writes by storing them in a hot table and then periodically flushing those writes to the actual file. Anyway, there was something seriously going wrong with this web service.

Using the command to list the requests:

C:\Windows\System32\inetsrv\appcmd.exe list request

I was able to determine an open connection that never was closed:

REQUEST "a90000018000473e" (url:POST /_vti_bin/cellstorage.svc/CellStorageService, time:1874982 msec, client:0.0.0.0, stage:ExecuteRequestHandler, module:ManagedPipelineHandler)

Checking the web.config of the web services, everything seemed fine and it should force close after a timeout. I searched for other experiences with the CellStorageService and found the following discussion:

http://social.technet.microsoft.com/Forums/en-US/d4b10c36-2f9e-45af-aab8-9a234ccb9ce8/sharepoint-workspace-error-cellstoragesvc?forum=sharepointgeneralprevious

The post by Doc WattsMan led me to the solution: switching from Kerberos to NTLM and back, the service seemed to be “resetted” and everything runs fine since then. I’m still trying to figure out which update caused the trouble, but in the meantime you can try the solution for yourself.

Create list views across webs using PowerShell or code

Sometimes you’ll want to have one basic list, which is accessible on all your subsites. One way to do this is using the content query web part. This however, blocks the functionalities a standard list view offers. So instead of the content query web part, we’ll use the basic XsltListViewWebPart. An important note: the standard ListViewWebPart class cannot work across your different webs inside your site collection. You have to use the Xslt one.

First, get an instance of the list you want to reference in your subsite:

$webWithList = Get-SPWeb http://weburl
$list = $webWithList.Lists["ListName"]

Then, get the web part manager of the page of your subsite where you want to insert the list view:

$targetWeb = Get-SPWeb http://weburl/subweb
$wpm = $targetWeb.GetLimitedWebPartManager("default.aspx", [System.Web.UI.WebControls.WebParts.PersonalizationScope]::Shared);

Now that we’ve got both required objects, it’s time to create the XsltListViewWebPart:

$ListViewWebPart = New-Object Microsoft.SharePoint.WebPartPages.XsltListViewWebPart
$ListViewWebPart.ListId = $list.id
$ListViewWebpart.Title = $list.title
$ListViewWebPart.WebId = $list.parentweb.id

It’s very important the WebId is set or otherwise you’ll get a nasty error like this:

“List does not exist. The page you selected contains a list that does not exist. It may have been deleted by another user.”

Last but not least, add the web part to your page:

$wpm.AddWebPart($ListViewWebPart, "Left", 0)

In one flow the script looks like this:

$webWithList = Get-SPWeb http://weburl
$list = $webWithList.Lists["ListName"]
$targetWeb = Get-SPWeb http://weburl/subweb
$wpm = $targetWeb.GetLimitedWebPartManager("default.aspx", [System.Web.UI.WebControls.WebParts.PersonalizationScope]::Shared);
$ListViewWebPart = New-Object Microsoft.SharePoint.WebPartPages.XsltListViewWebPart
$ListViewWebPart.ListId = $list.id
$ListViewWebpart.Title = $list.title
$ListViewWebPart.WebId = $list.parentweb.id
$wpm.AddWebPart($ListViewWebPart, "Left", 0)

If you would like for instance feature stapling, you can use the following code block to your web feature activation:

public override void FeatureActivated(SPFeatureReceiverProperties properties)
{
    var web = properties.Feature.Parent as SPWeb;
    using (var rootWeb = web.Site.OpenWeb())
    {
        var list = rootWeb.Lists[Constants.ListName];
        string file = web.RootFolder.WelcomePage;
        var webPartManager = web.GetLimitedWebPartManager(file, PersonalizationScope.Shared);
        var webPart = new XsltListViewWebPart
        {
            ListId = list.ID,
            Title = list.Title,
            WebId = list.ParentWeb.ID
        };
        webPartManager.AddWebPart(webPart, "Left", 0);
    }
}

Check if an item has been previously approved and approve again

Sometimes, you’ll have to check in an event receiver if an item has been previously approved and then do an action.

The following extension will return a boolean wether or not the item has been approved in the past or not.

public static bool HasBeenPreviouslyApproved(this SPListItem item)
{
    bool hasBeenPreviouslyApproved = false;
    if (item.Versions.Count > 1)
    {
        var prevPublishedVer = item.Versions.Cast<SPListItemVersion>().Where(v => v.Level == SPFileLevel.Published).LastOrDefault();
        if (prevPublishedVer != null)
        {
            hasBeenPreviouslyApproved = true;
        }
    }
    return hasBeenPreviouslyApproved;
}

As an example you can put the state back to approved if a modification was made. By default this always goes back to pending when an item gets edited, but in some cases you want to keep the state to approved.

if (properties.ListItem.HasBeenPreviouslyApproved())
{
    using (var web = properties.OpenWeb())
    {
        this.EventFiringEnabled = false;
        properties.ListItem.ModerationInformation.Status = SPModerationStatusType.Approved;
        properties.ListItem.SystemUpdate(false);
        this.EventFiringEnabled = true;
    }
}

Please note that when you use this code in an event receiver, it only works in the ItemUpdated event.

As a result, an announcement which was once approved, will always be approved, until a content editor decides to change the approval status back to pending or decline.

announcement approved

Manage your Forms Based Authentication Membership Provider using PowerShell

In SharePoint you can configure a FbaMembershipProvider so that you can store and retrieve membership data from SQL Server. Unlike AD, you don’t have a real management tool for managing FBA users. In cases like this, where you want to help people out when they locked their account or they need a password reset, you can always go to PowerShell.

The first step is of course launching PowerShell. Then you have to “prep” your session so that you can connect to the SQL database. When you’ve already configured your Web Application’s web.config file, you’re good to go. All you need to do is load in that web.config file so the connectionstrings are available.

Do this by the following statement:

[System.AppDomain]::CurrentDomain.SetData("APP_CONFIG_FILE", "C:\inetpub\wwwroot\wss\VirtualDirectories\<iisDir>\web.config")

When you’ve loaded your config file, it’s time to initialize the System.Web assembly.

[void][System.Reflection.Assembly]::LoadWithPartialName("System.Web")

When the assembly gets loaded, the config path is used to read data for the membership providers.

You can list all available membership providers:

[System.Web.Security.Membership]::Providers

FbaMembershipProvider

We’re gonna need the second one. The collection let’s us index on the “Name” property so we’ll go ahead and select the second one:

$provider = [System.Web.Security.Membership]::Providers["FbaMembershipProvider"]

Now that we have the $provider object, we can use all the useful methods it offers:

ChangePassword
ChangePasswordQuestionAndAnswer
CreateUser
DeleteUser
FindUsersByEmail
FindUsersByName
GeneratePassword
GetAllUsers
GetHashCode
GetNumberOfUsersOnline
GetPassword
GetUser
GetUserNameByEmail
ResetPassword
UnlockUser
UpdateUser
ValidateUser

Update SharePoint Timer Job’s progress bar – the easy way

Every time I develop a timer job, I always add a progress bar. Well, if the job runs long enough that is. It adds that bit more refinement to the whole project and it’s pleasant to see your job crunching through the numbers.

For this example I’ve set up a standard timer job project:

timer job project structure

  • A Web Application scoped feature which installs the timer job
  • A static class ProgressUpdater which will handle the timer job’s progress updating
  • A class service which will do something
  • A class TimerJob which is our core job definition

I’ve chosen to add a class service to demonstrate the update process from outside the JobDefinition scope.

For the timer job I have the following code:

using Microsoft.SharePoint.Administration;
using System;

namespace SharePointTimerJob
{
    public class TimerJob : SPJobDefinition
    {
        private const string JOBNAME = "Custom Timer Job";

        public TimerJob()
            : base()
        {
        }

        public TimerJob(SPWebApplication webApplication)

            : base(JOBNAME, webApplication, null, SPJobLockType.Job)
        {
            this.Title = JOBNAME;
        }

        public TimerJob(string jobName, SPService service, SPServer server, SPJobLockType targetType)

            : base(jobName, service, server, targetType)
        {
        }

        public static string JobName
        {
            get { return JOBNAME; }
        }

        public override void Execute(Guid contentDbId)
        {
            ProgressUpdater.Job = this;

            var service = new Service();
            service.Process();
        }
    }
}

As you can see it’s pretty straight forward. One note here is the ProgressUpdater.Job = this; line. This sets the Job property of the ProgressUpdater class to the executing instance.

The next piece of code is the ProgressUpdater:

namespace SharePointTimerJob
{
    public static class ProgressUpdater
    {
        public static TimerJob Job
        {
            get;
            set;
        }

        public static void Update(int percentage)
        {
            if (Job == null)
            {
                // throw or return
                return;
            }

            if (percentage >= 0 && percentage <= 100)
                Job.UpdateProgress(percentage);
        }
    }
}

And the code of the service to trigger an Update call:

using System.Threading;

namespace SharePointTimerJob
{
    public class Service
    {
        public void Process()
        {
            int[] collection = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 };
            int count = 0;
            foreach (var item in collection)
            {
                Thread.Sleep(5000);
                count++;
                ProgressUpdater.Update((count * 100) / collection.Length);
            }
        }
    }
}

All this does is sleep the thread and update afterwards, but you get the idea.

When you run the timer job you’ll see that the progress updates nicely in ticks of 10% because of the collection, which is 10 items:

progress 10

progress 40

SharePoint 2010: PowerShell to Clear the Timer Job Cache

Bram Nuyts:

Great script when you need to clear the timer job cache

Originally posted on SharePoint Tips by Nick Hobbs:

I have been clearing the Timer Job Cache manually when needed for years. A few months back I looked for a script to do this automatically and ended up writing one since it can be a slow and laborious task to perform on a multi-server SharePoint farm, and I have now finally got around to sharing it here.

The Timer Job Cache may need to be cleared for a number of reasons, however the main ones I have come across are mainly in development environments when redeploying the same or updated solutions repeatedly, where updated assemblies are not used by timer jobs, and also if for example a small development VM runs out of disk space (frugal disk allocation) because of database transaction logs or data files filling up then the timer jobs stop working. I am sure there are several other reasons you might want or need to clear the…

View original 1,194 more words