Tuesday, 20 December 2011

Deploying Databases with Object-Level Permissions

Cross-posted from Jason Lee's Blog

As I mentioned in my last post, we’re currently creating some guidance on deploying enterprise-scale applications. As we go along, I plan to blog about a few of the things that I find particularly tricky to figure out.
This time I want to look at database deployment. When you build a web application project in Visual Studio 2010, the Web Publishing Pipeline features allow you to hook into the IIS Web Deployment Tool (commonly known as “Web Deploy”) to package and optionally deploy your web application. As part of this process you can also deploy local databases to a target server environment. This is all nice and easy to configure through the project property pages in Visual Studio 2010, as shown below.

I don’t want to describe this process in any detail, you can find that elsewhere on the web (for example here).

Deploying databases in this way has advantages and disadvantages. On the plus side:
  • It’s easy.
  • It’s UI-driven.
  • It figures out most of the settings for you.
On the downside:
  • There’s no support for differential updates. In other words, the destination database is destroyed and recreated every time you deploy, so you’ll lose any data.
  • Some of the default database deployment settings are unsuitable for many real-world scenarios – this is the issue I want to focus on here.
In many cases, you’ll want to avoid the Web Deploy approach altogether and use VSDBCMD.exe to deploy and update your databases, but that’s a conversation for another day. In this post I want to focus on how you can change some of the default behaviours for database deployment using Web Deploy. In particular, Web Deploy omits object-level permissions by default. This causes problems if your database (a) contains stored procedures and (b) grants execute permissions on the stored procedures to database roles.

Suppose you’re using the Visual Studio 2010/Web Deploy approach to deploy an ASP.NET membership database from a local development machine to a destination server environment. (NB you’d typically only deploy a membership database if you’ve modified the schema, otherwise it’s easier just to run ASPNET_REGSQL.exe and create the database from scratch on the destination server). You opt for a full deployment, including schema and data from the source database. On the source database, you can see that various database roles are granted execute permissions on stored procedures:

However, when the database is recreated on the destination database server, these permissions are missing:

This can be mystifying at first—essentially, you add users to the built-in database roles such as aspnet_Membership_BasicAccess and aspnet_Membership_FullAccess, but membership of these roles has no effect. They’re basically just empty roles that aren’t mapped to any permissions.

The problem is that stored procedures are “objects” in database terms (don’t ask me, I’m not a DBA), and by default Web Deploy does not include object-level permissions when it scripts the database. To change this behaviour, you need to modify the project file for your web application project.

1. In the Solution Explorer window, right-click your web application project node, and then click Unload Project.

2. Right-click the project node again, and click Edit [project file].

3. Locate the PropertyGroup element that corresponds to your build configuration (for example Release|AnyCPU).

<PropertyGroup
    Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">

3. Within this element, locate the PreSource element and add a Permissions=”True” attribute, as shown below. The Permissions attribute indicates that the database script should include all permissions, including object-level permissions, which are defined in the source database.

<PublishDatabaseSettings>
  <Objects>
    <ObjectGroup Name="ApplicationServices-Deployment" Order="1">
      <Destination Path="[Destination Database Connection String]" />
      <Object Type="dbFullSql">
        <PreSource Path="[Source Database Connection String]"
                   ScriptSchema="True"
                   ScriptData="True"
                   Permissions="True"
                   CopyAllFullTextCatalogs="False"
                   DriDefaults="True" />
        <Source Path="[Where to save a copy of the script]"
                Transacted="True" />
      </Object>
    </ObjectGroup>
  </Objects>
</PublishDatabaseSettings>

4. Save and close the project file.

There’s a whole host of settings you can add to the PreSource element to configure how your database is deployed. For example, if you want to deploy a database that already exists on the destination server, you need to add a ScriptDropsFirst=”True” attribute to the PreSource element – otherwise Web Deploy will complain that you’re trying to create objects that already exist. The full list of properties that you can set as PreSource attibutes can be tricky to track down unless you know how the database deployment process works:
  • Web Deploy uses the dbFullSql provider to deploy databases (the link includes some properties you can use as PreSource attributes).
  • Under the covers, the dbFullSql provider uses SQL Server Management Objects (SMO) to generate database scripts. The ScriptingOptions Properties page describes some SMO properties you can specify as PreSource attributes.

Alternatively, to get a full list of properties, you can run the following Web Deploy command:
msdeploy.exe –verb:sync –source:dbFullSql /?

There’s much more to database deployment than I can cover in a quick blog post, and we’ll cover these kinds of issues in much more detail when we publish to MSDN. For now though I hope this helps to shed some light on the intricacies of database deployment.

Deploying Web Packages as a Non-Administrator User

Cross-posted from Jason Lee's Blog

Regular readers (all six of you ;-)) will have noticed that I haven’t posted about SharePoint for a while. For the last couple of months I’ve been working with the Developer Guidance team at Microsoft to write some MSDN content on enterprise-scale web deployment and application lifecycle management. I’ll let you know when the content is available, and I don’t plan to duplicate it here. What I want to do is just to draw attention to a couple of areas that I found particularly tricky to figure out.

The first area involves the IIS Web Deployment Tool (commonly known as “Web Deploy”), and a gotcha around deploying web packages as a non-administrator user. For brevity I’ll have to assume that you’re broadly familiar with:


One of the big advantages of Web Deploy 2.0, on IIS 7 or later, is that non-administrator users can deploy web packages to specific IIS web sites. This is generally useful in two scenarios:

  • Hosted environments, where tenants need control over specific sites but do not have server-level administrator privileges.
  • Enterprise environments, where members of a development team may need to deploy specific sites but do not typically have server-level administrator privileges.

If you want to enable non-administrator users to deploy web packages, you need to configure the Web Deploy Handler on the target IIS web server. The other deployment approaches (the remote agent and the temp agent) don’t allow users who aren’t server administrators to deploy packages. I’ll assume that you’ve configured the Web Deployment Handler to allow a non-administrator user (FABRIKAM\User) to deploy content to a specific IIS website, as described here.

By default, the Web Deploy Handler exposes an HTTPS endpoint at the following address:

https://[server name]:8172/MSDeploy.axd

For example:

https://TESTWEB1:8172/MSDeploy.axd

However, when a non-administrator user deploys a web package to the Web Deploy Handler, they need to add the IIS website name to the endpoint address as a query string:

https://[server name]:8172/MSDeploy.axd?site=[site name]

For example:

https://TESTWEB1:8172/MSDeploy.axd?site=DemoSite

Why the difference? In a word, authorization. Your non-administrator user doesn’t have server-level access to IIS, they only have access to specific IIS websites. If they attempt to connect to the server-level endpoint, Web Deploy will an ERROR_USER_UNAUTHORIZED error. The event log on the destination server will show an IISWMSVC_AUTHORIZATION_SERVER_NOT_ALLOWED error like this:


So you’ve got to use the site query string. Now for the gotcha.

Due to an open bug in the current version of Web Deploy (2.1), you can’t specify a query string in the endpoint address if you use the .deploy.cmd file generated by Visual Studio to deploy your web package. In other words, this won’t work:

DemoProject.deploy.cmd /Y /M:https://TESTWEB1/MSDeploy.axd?site=DemoSite /U:FABRIKAM\User /P:Pa$$w0rd A/:Basic -allowUntrusted

I’ve seen some fairly bizarre “workarounds” for this—for example, drop the query string and use an administrator account—this works, but it kind of defeats the object when the whole point of the exercise was to use a non-administrator user to deploy the web package. What you need to do is to use Web Deploy (MSDeploy.exe) directly rather than running the .deploy.cmd file.

All the .deploy.cmd file contains is a bunch of parameterized Web Deploy commands. This is put together by the build process to take some of the work out of the deployment. For example, you don’t need to specify the location of the web package, the Web Deploy providers to use for the source and destination, the Web Deploy verb, or the location of the .SetParameters.xml file, because the .deploy.cmd file knows this already. However, there’s nothing to stop you using the raw Web Deploy commands directly. The easiest way to do this is to look at the output when you run the .deploy.cmd file – you’ll see the actual MSDeploy.exe commands written to the console window. You should see something like this (ignore the line breaks):

msdeploy.exe
-source:package='…\DemoProject.zip'
-dest:auto,
computerName='https://TESTWEB1:8172/MSDeploy.axd?site=DemoSite',
userName='FABRIKAM\User',
password='Pa$$w0rd',
authtype='Basic'
-verb:sync
-setParamFile:"…\DemoProject.SetParameters.xml"
-allowUntrusted

Run this command directly from the command line, using your non-administrator user credentials, and the deployment should succeed.

Packaging and deploying web applications is a fairly broad and complex topic, and I’ve had to gloss over many of the details in this blog post. The content we’re developing for MSDN will cover these kinds of issues in much more detail, and I’ll link to the content as soon as it’s available.

Thanks to Tom Dykstra at Microsoft for helping me troubleshoot the issue and pointing out the bug.

Tuesday, 8 November 2011

Installing Windows Intune Endpoint Protection over existing anti-malware software

How does Windows Intune install Window Intune Endpoint Protection when there is existing anti-malware software on the client computer?

Over the past year Content Master has been involved in training many Microsoft employees, Partners, and IT Pros on the technical ins and outs of Windows Intune. A question we have been asked many times is exactly how the Windows Intune installation process behaves when installed on computers with existing anti-malware software. It’s important to recognise that when the Windows Intune client is installed the overriding principle is to ensure that all client computers are protected from malware. This means that if you install Windows Intune on a computer with no existing malware protection the Windows Intune Endpoint Protection (WIEP) agents will be installed. This is exactly what you would expect, but things get a little more complicated when an existing anti-malware solution is in place; so that is the focus of this blog.

Before you deploy the Windows Intune client, we strongly recommend that you create default Windows Intune policies to set a baseline for deploying updates, software, and especially WIEP. By determining the WIEP settings you require and configuring the appropriate policy settings before deploying the client to any computer, you can take control of the deployment process and save time and effort later on.

Now let’s examine the options available in the policy setting of the Windows Intune Agent Settings policy and see how they change the installation behaviour. Then we will then look at the behaviour when installing the Windows Intune client.

WIEP Settings in the Windows Intune Agent Settings policy

The Windows Intune Agent Settings policy includes settings that control the behaviour of Endpoint Protection, updates and network bandwidth.

Figure 1 shows the Endpoint Protection setting that controls the installation behaviour of WIEP:

WI Agent Settings EP

Figure 1: Endpoint Protection Settings

Yes enables Windows Intune Endpoint Protection on client computers. If Windows Intune Endpoint Protection is not already installed on a client computer, it will be installed.

No disables Windows Intune Endpoint Protection on client computers. If Windows Intune Endpoint Protection is not already installed on a client computer, it will be installed only if another endpoint protection application is not already installed on the computer. When Windows Intune Endpoint Protection is installed in this case, it is installed and configured as disabled.

Only on computers that are unprotected when Endpoint Protection is installed enables Windows Intune Endpoint Protection on client computers that do not already have another endpoint protection application installed. If another endpoint protection application is already installed on a client computer, this policy setting causes Endpoint Protection to not be installed on that computer. This is the default and recommended option.

You can find more information on the Endpoint Protection Policy Settings and other Windows Intune Agent Policy Settings in Windows Intune Help.

Installation Behaviour:

The default behaviour of the WIEP installation is to ensure that the client computer is protected. If you create a Windows Intune Agent Settings policy and configure the Endpoint Protection policy settings, you can control the WIEP installation behaviour. This is described in the flowchart below:

New Flowchart

Figure 2: WIEP Installation Behaviour Flowchart

There are five paths through this flowchart:

Path 1 (1-2-3): This path describes the simplest situation; where there is no endpoint protection software installed on the client computer the Windows Intune client installation will install and enable WIEP.

Path 2 (1-2-4-5): This path describes an environment where Microsoft Security Essentials (MSE) or Forefront Endpoint Protection (FEP) is installed on the client computer. In this case, the existing endpoint protection software is automatically upgraded to WIEP.

Path 3 (1-2-4-6-7): This path describes an environment where MSE or FEP is not installed on the client computer, and there is no Endpoint Protection policy defined in the Windows Intune Agent Settings policy. In this case, WIEP is not installed and the client computer continues to use the existing endpoint protection software.

Path 4 (1-2-4-6-8-9): This path describes an environment where MSE or FEP is not installed on the client computer, there is an Endpoint Protection policy defined in the Windows Intune Agent Settings policy, and the existing endpoint protection software is recognised.[1] In this case, the Windows Intune client installation removes the existing endpoint protection software and installs WIEP.

Path 5 (1-2-4-6-8-10): This path describes an environment where MSE or FEP is not installed on the client computer, there is an Endpoint Protection policy defined in the Windows Intune Agent Settings policy, and the existing endpoint protection software is not recognised. In this case, the Windows Intune client installation installs WIEP in a disabled state and the client computer continues to use the existing anti-malware software.

Windows Intune will always install the Windows Intune Endpoint Protection agent, even in Paths 3 and 4 where the Windows Intune Endpoint Protection engine is not installed. This is illustrated in figures 3 and 4 below:

WIEP Agent installed

Figure 3: Windows Intune Endpoint Protection Agent installed

WIEP installed

Figure 4: Windows Intune Endpoint Protection Agent and Windows Intune Endpoint Protection engine installed

Recommendations

The most important thing is to determine whether you want to use Windows Intune to manage your endpoint protection, with all the benefits that brings. Windows Intune Endpoint Protection helps enhance the security of computers in your organization by providing real-time protection against potential threats; keeping malware definitions up-to-date; and automatically running scans that you can control with policies. All of this is integrated into the Windows Intune administrator console to provide a centralised point of management, along with customisable reports that help you to keep track of malware issues and their resolution.

If you decide to use WIEP, you should plan and implement the appropriate Endpoint Protection policy settings in the Windows Intune Agent Settings policy before you deploy the Windows Intune client software. In this way, you can control the WIEP installation as the client joins the Windows Intune account.

As always, it is wise to experiment with different methods of installing WIEP in a test lab, so that you can determine which method will work best in your organisation. For example, you may want to verify that the Windows Intune client installation can successfully remove existing anti-malware software from your client computers. You may also want to verify that WIEP can co-exist with your existing endpoint protection application if it is not one of the recognised packages.

Conclusion

Windows Intune includes a fully featured and cloud enabled endpoint protection solution based on the Forefront Endpoint Protection 2010 product. If you plan for the deployment of WIEP and configure the appropriate Windows Intune Agent Settings policy before you install the Windows Intune client on computers throughout your organisation you will have more control of the endpoint protection status as soon as they are managed by Windows Intune.


[1] A list of anti-malware software that Windows Intune recognises can be found at: http://onlinehelp.microsoft.com/windowsintune/hh127706.aspx

Blogger: User Profile: Rose Malcolm

Thursday, 22 September 2011

Building Windows Phone Apps: A Developer’s Guide

For anyone looking for a great book on developing Windows Phone 7 applications, take a look here.

“This book combines useful, technical information about the platform, written by those who’ve been there and done it, with some stories about the people and the apps they’d written. The book will act as a handy reference that gives you a ‘puddy up’ into developing on the platform and, importantly, helps you to avoid making the same mistakes others have made.”

The best news is that the digital download is completely free, although you can also get a hard copy for a mere £3.86!

Monday, 29 August 2011

Working with the Documents Tab on the SharePoint Ribbon

Cross-posted from Jason Lee's Blog

This week I've been taking a look at using ribbon controls with the SharePoint JavaScript client object model to drive some custom functionality. Ribbon customizations for SharePoint 2010 are fairly well documented. However, when you work with contextual tab groups—and the Documents tab in particular—there are a few nuances and idiosyncrasies that it's worth being aware of up front.

In this case, I want to add a ribbon button that enables the user to perform some additional actions when they select a file in a document library. There are countless scenarios in which you might want to do this – for example, you might add a "Request a copy of this document in large print/audio format/Welsh" control to the ribbon and use the document metadata to prepopulate an InfoPath form. To start with, however, I want to keep it simple:

  • When the user selects a document in a document library, display a button on the ribbon.
  • When the user clicks the button, display some information about the selected document as a client-side notification.

The logical place to put this button is on the Documents tab. This is part of the Library Tools contextual tab group – it's contextual because it's only displayed when the context is relevant, i.e. when the user browses to a document library. The Documents tab is selected automatically when the user selects one or more documents in the library's list view web part:










Let's take it one bit at a time for now, and I'll provide a full code listing at the bottom. Firstly, like all declarative ribbon customizations, we start with a CustomAction feature element:

<CustomAction Id="Jason.SP.GSD"
              Location="CommandUI.Ribbon"
              Sequence="11"
              RegistrationType="List"
              RegistrationId="101">


The key point of note here is that if you plan to add controls to a contextual tab, you must use the RegistrationType and RegistrationId attributes to target your ribbon customizations to an appropriate list type. If you're deploying controls to a standard ribbon tab, you can get away with omitting these attributes. It didn't initially occur to me that it should be any different in this case – I'm adding controls to the Documents tab, the Documents tab only shows up when I'm looking at a document library, I shouldn't have to worry about scope, right? But no – if you don't set these attributes, your controls simply won't show up on the tab. In this case, a RegistrationType of "List" and a RegistrationId of "101" scopes our ribbon customization to the document library base type.


Next, we define the controls we want to add to the ribbon. This process is identical regardless of whether you're adding to a contextual tab or a regular tab. In this case, we want to add a new group named "Jason's Actions" to the Documents tab. Within this group, we want to create a single button labelled "Get Selection Details". To accomplish this we need to create two CommandUIDefinition elements – one to define the maximum size of my group element, and one to define the group itself. Creating tabs, groups, and controls has been covered comprehensively elsewhere, so I don't want to go into too much detail – if you're looking for more information in this area, Chris O'Brien's blog post series is an excellent place to start. 

In this case, we'll use the absolute minimum markup required to add a new button in its own group - a Group element to define the group and the controls within it, and a MaxSize element that defines how the group should be rendered on the ribbon. You can specify many more elements if you want – for example you can add a Scale element to specify how your group should render at different sizes, and you can define your own GroupTemplate to specify precisely how controls within your group should be arranged. However, each group must have a matching MaxSize element – otherwise it won't appear on the tab. The easiest approach to creating ribbon controls is to pick out existing controls that resemble what you're looking for and take a look at how they're defined. Let's say we want our group and button to look like the Share & Track group shown here – a large, simple layout with a single control:







To replicate this group, the first step is to take a look at the group definition. Ribbon controls are defined in the 14\TEMPLATE\GLOBAL\XML\CMDUI.XML file. To find specific elements in this file, unless you know the ID of the element you're looking for, it's best to start with the top-level elements and narrow down your search – start by finding the right tab group (Id="Ribbon.LibraryContextualGroup"), then locate the correct tab (Id="Ribbon.Document"), then identify the group you're looking for. In this case, the group we want to borrow from has an ID of "Ribbon.Documents.Share":

<Group Id="Ribbon.Documents.Share"
       Sequence="40"
       Command="ShareGroup"
       Description=""
       Title="$Resources:core,cui_GrpShare;"
       Image32by32Popup=".../formatmap32x32.png" 
       Image32by32PopupTop="-128" 
       Image32by32PopupLeft="-64"
       Template="Ribbon.Templates.Flexible2">

By examining the definition of this group, we can figure out the properties we need:

  • The Share & Track group has a Template attribute of Ribbon.Templates.Flexible2. This identifies the group template that gets applied to the group (also defined in CMDUI.XML if you want to take a closer look). We'll use this value to apply the same layout to our own group.
  • The Share & Track group has a Sequence attribute of 40. We'll use a value of 41 to place our group immediately to the right of the Share & Track group.
Next, we can take a look at how controls are defined within the group. For example, the following markup defines the E-mail a Link button you saw in the previous image:

<Button Id="Ribbon.Documents.Share.EmailItemLink"
        Sequence="10"
        Command="EmailLink"
        Image16by16=".../formatmap16x16.png" 
        Image16by16Top="-16" 
        Image16by16Left="-88"
        Image32by32=".../formatmap32x32.png" 
        Image32by32Top="-128" 
        Image32by32Left="-448"
        LabelText="$Resources:core,cui_ButEmailLink;"
        ToolTipTitle="$Resources:core,cui_ButEmailLink;"
        ToolTipDescription="...,cui_STT_ButEmailLinkDocument;"
        TemplateAlias="o1"
/>

In this case, the TemplateAlias attribute is the value that interests us. Every group template contains one or more placeholders, represented by ControlRef elements, in which you can place your controls. In this case, the E-mail a link button specifies that it should be added to the o1 placeholder in the Flexible2 group template. If we use the same value in our own button, we should get the same result.


Finally, we can also take a look at the matching MaxSize element for the Share & Track group. Remember that these elements are always paired – a Group element always has a corresponding MaxSize element defined within the same tab. Within each MaxSize element, the GroupId attribute identifies the corresponding group:

<MaxSize Id="Ribbon.Documents.Scaling.Share.MaxSize"
         Sequence="40"
         GroupId="Ribbon.Documents.Share"
         Size="LargeLarge" 
/>

In this case, all we're interested in is the Size attribute. A group template can define multiple layouts, and this attribute identifies the specific layout in the Flexible2 template that we want to use – in this case, the LargeLarge layout. 


We can now use all this information we've collected to define our group and button:

<CommandUIExtension>
  <CommandUIDefinitions>
    <CommandUIDefinition Location="Ribbon.Documents.Scaling._children"> 
      <MaxSize Id="Jason.SP.GSD.JasonsActions.MaxSize"
        Sequence="11"
        GroupId="Jason.SP.GSD.JasonsActions"
        Size="LargeLarge" /> 
    </CommandUIDefinition>
    <CommandUIDefinition Location="Ribbon.Documents.Groups._children">
      <Group Id="Jason.SP.GSD.JasonsActions"
        Sequence="41"
        Title="Jason's Actions"
        Description="Contains custom document actions"
        Template="Ribbon.Templates.Flexible2"> 
          <Controls Id="Jason.SP.GSD.JasonsActions.Controls">
            <Button Id="Jason.SP.GSD.JasonsActions.GetButton"
              Sequence="1"
              Image32by32=".../ThumbsUp.PNG"
              LabelText="Get Selection Details"
              Description="Gets the details of the selected document"
              TemplateAlias="o1"
              Command="Jason.SP.GSD.GetCmd" />
          </Controls>
        </Group>
      </CommandUIDefinition>
    </CommandUIDefinitions>


There are a few additional points worth mentioning at this stage:

  • You need a CommandUIDefinition element for each block of XML you want to add to the ribbon.
  • When setting the Location attribute, imagine you're slotting the XML directly into the CMDUI.XML file. Look up the ID of the parent element you want to add to, and append "._children" to get your Location value. For example, we want to add our group to the Groups element with an ID of "Ribbon.Documents.Groups", so our Location attribute is "Ribbon.Document.Groups._children".
Note that the button has a Command attribute value of "JL.GetSelectionDetails". This ties the button to a CommandUIHandler element in which we can define the JavaScript that should run when the user clicks the button, as shown below:

    <CommandUIHandlers>
      <CommandUIHandler  
        Command="Jason.SP.GSD.GetCmd"
        EnabledScript="javascript:
          SP.ListOperation.Selection.getSelectedItems().length == 1;"
        CommandAction="javascript: 
          var selectedItems = 
            SP.ListOperation.Selection.getSelectedItems();
          var item = selectedItems[0];
          var itemID = item['id'];
          if (item['fsObjType'] == 0) {
            SP.UI.Notify.addNotification(String.format(
              'Document selected: ID={0}', itemID));
          }
          else {
            SP.UI.Notify.addNotification(String.format(
              'Folder selected: ID={0}', itemID));
          }" 
      /> 
    </CommandUIHandlers>
  </CommandUIExtension>
</CustomAction>


The first point of interest is the EnabledScript attribute. When you add a control to the Documents tab it is disabled by default – you must use this attribute to specify the conditions under which the control should be enabled. The EnabledScript attribute should specify (or call) a JavaScript function that returns a Boolean value – true to enable the control, false to disable it. In this case, we want the button to be enabled when the user has selected a single document in the document library list view. The JavaScript client-side object model for SharePoint includes a class named SP.ListOperation.Selection for just this kind of eventuality. We can use the getSelectedItems method to return a collection of the items selected in the list view, then check that the length of the collection is equal to 1.


Note: In this example I've added all my JavaScript logic directly to the CommandUIHandler element. As your JavaScript logic grows larger and more complex, a better option would be to deploy a standalone JavaScript file. Yaroslav Pentarskyy describes this approach in this blog post.


Next, the CommandAction attribute specifies the JavaScript function we want to call when our button is clicked. The getSelectedItems method returns a Dictionary of key-value pairs. The value of each dictionary entry is an object with two attributes – id and fsObjType. The id attribute represents the integer ID of the list item, while the fsObjType attribute represents the type of list item object – 0 for a document or a list item, 1 for a folder. While this doesn't give us a great deal of information about the selected item, the integer ID gives us enough information to submit a query for additional document metadata, should we so wish. In this case, as a proof of concept, we simply display a notification containing the document ID when the user clicks our button.
Here's the button in its default disabled state:








When we select a document, the button is enabled:








When we click the button, a notification displays the integer ID of the selected document:












And that concludes today's task. Next time I plan to cover how to extend this to do something useful with the selected document. The contents of the feature element are shown below in their entirety for reference.


<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
  <CustomAction Id="Jason.SP.GSD" 
                Location="CommandUI.Ribbon" 
                Sequence="11" 
                RegistrationType="List" 
                RegistrationId="101">
    <CommandUIExtension>
      <CommandUIDefinitions>
        <CommandUIDefinition Location="Ribbon.Documents.Scaling._children">
          <MaxSize Id=" Jason.SP.GSD.JasonsActions.MaxSize" 
                   Sequence="11" 
                   GroupId="Jason.SP.GSD.JasonsActions" 
                   Size="LargeLarge" />
        </CommandUIDefinition>
        <CommandUIDefinition Location="Ribbon.Documents.Groups._children">
          <Group Id=" Jason.SP.GSD.JasonsActions" 
                 Sequence="41" 
                 Title="Jason's Actions" 
                 Description="Contains custom document actions" 
                 Template="Ribbon.Templates.Flexible2">
            <Controls Id="Jason.SP.GSD.JasonsActions.Controls">
              <Button Id=" Jason.SP.GSD.JasonsActions.GetButton" 
                      Sequence="1" 
                      Image32by32="/SiteCollectionImages/RibbonIcons/ThumbsUp.PNG" 
                      LabelText="Get Selection Details" 
                      Description="Gets the details of the selected document" 
                      TemplateAlias="o1" 
                      Command=" Jason.SP.GSD.GetCmd" />
            </Controls>
          </Group>
        </CommandUIDefinition>
      </CommandUIDefinitions>
      <CommandUIHandlers>
        <CommandUIHandler Command="Jason.SP.GSD.GetCmd" 
                          EnabledScript="javascript:
                            SP.ListOperation.Selection.getSelectedItems().length == 1;" 
                          CommandAction="javascript:
                            var selectedItems = 
                              SP.ListOperation.Selection.getSelectedItems();
                            var item = selectedItems[0];
                            var itemID = item['id'];
                            if (item['fsObjType'] == 0) {
                              SP.UI.Notify.addNotification(String.format(
                                'Document selected: ID={0}', itemID));
                            }
                            else {
                              SP.UI.Notify.addNotification(String.format(
                                'Folder selected: ID={0}', itemID));
                            } 
        "/>
      </CommandUIHandlers>
    </CommandUIExtension>
  </CustomAction>
</Elements>

Friday, 26 August 2011

Kinect SDK: Gesture Recognition Pt III

Introduction

In my previous blog post I discussed the development of a robust and extensible PostureRecognizer class. The class is used to recognize the start of a gesture. As a reminder, my high-level approach to the gesture recognition process is as follows:

  • Detect whether the user is moving or stationary.
  • Detect the start of a gesture (a posture).
  • Capture the gesture.
  • Detect the end of a gesture (a posture).
  • Identify the gesture.

This blog post will focus on capturing the gesture that occurs once the starting posture has been identified.

Implementation

The UI is unchanged from my previous post, with the exception of another Canvas being added to draw the captured gesture on.

The StreamManager constructor creates an instance of the PostureRecognizer class and registers an event handler for the PostureDetected event. It then registers the path where the gesture data will be saved, via a stream. An instance of the GestureRecognizer class is then created, and the name of the canvas to draw the captured gesture on is passed to the DrawGesture method.

        public StreamManager(Canvas canvas)
        {
            this.PostureRecognizer = new PostureRecognizer();
            this.PostureRecognizer.PostureDetected += 
               new Action<Posture>(OnPostureDetected);
            
            this.gestureCanvas = canvas;
            this.path = Path.Combine(Environment.CurrentDirectory, @"data\gestures.dat");
            this.stream = File.Open(this.path, FileMode.OpenOrCreate);
            this.gestureDetector = new GestureRecognizer(this.stream);
            this.gestureDetector.DrawGesture(this.gestureCanvas, Colors.Red);
        }

The StartGestureCapture and EndGestureCapture methods are shown below. The StartGestureCapture method invokes the StartGestureCapture method in the GestureRecognizer class, and if a gesture is already being captured, stops the capture. The EndGestureCapture method invokes the EndGestureCapture method in the GestureRecognizer class, and invokes the SaveGesture method in the GestureRecognizer class, provided that a gesture is being captured. 
        private void StartGestureCapture()
        {
            this.gestureDetector.GestureName = this.GestureName;
            if (this.gestureDetector.IsRecordingGesture)
            {
                this.gestureDetector.EndGestureCapture();
                return;
            }
            this.gestureDetector.StartGestureCapture();
        }
        private void EndGestureCapture()
        {
            if (this.gestureDetector.IsRecordingGesture)
            {
                this.gestureDetector.EndGestureCapture();
                this.gestureDetector.SaveGesture(stream);
            }
        }

The GetSkeletonStream method is as in the previous post, and invokes the TrackPostures method in the PostureRecognizer class. If a posture is identified, the RaisePostureDetected method is invoked which sets the CurrentPosture property to the identified posture, and invokes the PostureDetected event. The handler for the event is the OnPostureDetected method in the StreamManager class and was registered in the StreamManager constructor.
        public event Action<Posture> PostureDetected;
        private void RaisePostureDetected(Posture posture)
        {
            if (this.currentPosture != posture)
            {
                this.CurrentPosture = posture;
            }
            if (this.PostureDetected != null)
            {
                this.PostureDetected(posture);
            }
        }

The OnPostureDetected method is shown below. If a gesture is not being captured, it invokes the StartGestureCapture method, and stores the identified posture in the startPosture variable. If a gesture is being captured and the posture is different to the posture stored in the startPosture variable, the EndGestureCapture method is invoked. The detected Joints are then enumerated, and the Add method in the GestureRecognizer class is invoked if the gesture involves the right hand.
        private void OnPostureDetected(Posture posture)
        {
            if (this.gestureDetector.IsRecordingGesture == true)
            {
                if ((posture != Posture.None) &&
                    (posture != startPosture))
                {
                    this.EndGestureCapture();
                }
            }
            else
            {
                this.StartGestureCapture();
                this.startPosture = posture;
            }
            foreach (Joint joint in this.skeleton.Joints)
            {
                if (joint.Position.W < 0.9f || 
                    joint.TrackingState != JointTrackingState.Tracked)
                {
                    continue;
                }
                if (joint.ID == JointID.HandRight)
                {
                    this.gestureDetector.Add(joint.Position, 
                                             this.KinectRuntime.SkeletonEngine);
                }
            }
        }

The GestureData class is used to model the gesture data. It contains the a Position and the Time the Position was captured.
    public class GestureData
    {
        public Vector Position { get; set; }
        public DateTime Time { get; set; }
    }

The GestureRecognizer class is shown below. Gestures are stored in a collection named gestures, of type List<GestureData>. An instance of the TemplateRecognizer class is also created, which will use template matching to perform gesture identification. The bulk of the work in this class is performed in the Add method. It stores the passed in Vector as a GestureData object, and adds that to the gestures collection. It then draws the position as a small ellipse on the Canvas contained in the displayCanvas variable. Then, if there is more data in the gestures collection than the value of WindowSize, data is removed from both the start of the collection and the displayCanvas. The position data is then added to an instance of the Gesture class. The StartGestureCapture, EndGestureCapture, and SaveGesture methods largely invoke methods in the TemplateRecognizer class.
    public class GestureRecognizer
    {
        private Gesture gesture;
        private string gestureName; 
        private readonly List<GestureData> gestures = new List<GestureData>();
        private readonly TemplateRecognizer.TemplateRecognizer templateRecognizer;
        private readonly int windowSize;
        private DateTime lastGestureDate = DateTime.Now;
        private Canvas displayCanvas;
        private Colour displayColour;
        protected List<GestureData> Gestures
        {
            get { return this.gestures; }
        }
        public string GestureName
        {
            get { return this.gestureName; }
            set { this.gestureName = value; }
        }
        public bool IsRecordingGesture
        {
            get { return this.gesture != null; }
        }
        public TemplateRecognizer.TemplateRecognizer TemplateRecognizer
        {
            get { return this.templateRecognizer; }
        }
        public int WindowSize
        {
            get { return this.windowSize; }
        }
        public GestureRecognizer(Stream stream, int windowSize = 100)
        {
            this.templateRecognizer = new TemplateRecognizer.TemplateRecognizer(stream);
            this.windowSize = windowSize;
        }
        public GestureRecognizer(string gestureName, Stream stream, int windowSize = 100)
        {
            this.gestureName = gestureName;
            this.templateRecognizer = new TemplateRecognizer.TemplateRecognizer(stream);
            this.windowSize = windowSize;
        }
        public void Add(Vector position, SkeletonEngine engine)
        {
            GestureData newGesture = new GestureData
            {
                Position = position,
                Time = DateTime.Now
            };
            this.gestures.Add(newGesture);
            if (this.displayCanvas != null)
            {
                Ellipse ellipse = new Ellipse
                {
                    HorizontalAlignment = HorizontalAlignment.Left,
                    VerticalAlignment = VerticalAlignment.Top,
                    Height = 8,
                    Width = 8,
                    StrokeThickness = 8,
                    Stroke = new SolidColorBrush(this.displayColour),
                    StrokeLineJoin = PenLineJoin.Round
                };
                float x, y;
                engine.SkeletonToDepthImage(position, out x, out y);
                x = (float)(x * this.displayCanvas.ActualWidth);
                y = (float)(y * this.displayCanvas.ActualHeight);
                Canvas.SetLeft(ellipse, x - ellipse.Width / 2);
                Canvas.SetTop(ellipse, y - ellipse.Height / 2);
                this.displayCanvas.Children.Add(ellipse);
            }
            if (this.gestures.Count > this.WindowSize)
            {
                GestureData gestureToRemove = this.gestures[0];
                if (this.displayCanvas != null)
                {
                    this.displayCanvas.Children.RemoveAt(0);
                }
                this.gestures.Remove(gestureToRemove);
            }
            if (this.gesture != null)
            {
                Vector2 vector = new Vector2
                {
                    X = position.X,
                    Y = position.Y
                };
                this.gesture.Points.Add(vector);
            }
        }
        public void DrawGesture(Canvas canvas, Color colour)
        {
            this.displayCanvas = canvas;
            this.displayColour = colour;
        }
        public void StartGestureCapture()
        {
            this.gesture = new Gesture(this.WindowSize);
        }
        public void EndGestureCapture()
        {
            this.templateRecognizer.Add(gesture);
            this.gesture = null;
        }
        public void SaveGesture(Stream stream)
        {
            this.templateRecognizer.Save(stream);
        }
    }

The Gesture class stores all the positions of the captured gesture in a collection called points, of type List<Vector>.
    [Serializable]
    public class Gesture
    {
        private List<Vector> points;
        private readonly int samplesCount;
        public List<Vector> Points
        {
            get { return this.points; }
            set { this.points = value; }
        }
        public Gesture(int samplesCount)
        {
            this.samplesCount = samplesCount;
            this.points = new List<Vector>();
        }
    }

The TemplateRecognizer class will perform the bulk of the processing required for gesture identification, which will be covered in a future blog post. At the moment the class simply handles serialization and deserialization of the gesture data, by using a BinaryFormatter. The constructor deserializes the gesture data if it exists, while the Save method serializes the gesture data to a file.
    public class TemplateRecognizer
    {
        private readonly List<Gesture> gestures;
        public TemplateRecognizer(Stream stream)
        {
            if (stream == null || stream.Length == 0)
            {
                this.gestures = new List<Gesture>();
                return;
            }
            BinaryFormatter formatter = new BinaryFormatter();
            this.gestures = (List<Gesture>)formatter.Deserialize(stream);
        }
        public void Add(Gesture gesture)
        {
            this.gestures.Add(gesture);
        }
        public void Save(Stream stream)
        {
            BinaryFormatter formatter = new BinaryFormatter();
            formatter.Serialize(stream, this.gestures);
        }
    }

The application is shown below. It determines whether the user is moving or stationary, and recognizes the start of a gesture – in this case, a posture. Once a posture is identified (currently hard coded to RightHello) the gesture points are stored in a collection and drawn on the canvas. When a different posture is identified the gesture capture stops.

gesture3<

Conclusion


The Kinect for Windows SDK beta from Microsoft Research is a starter kit for application developers. It allows access to the Kinect sensor, and experimentation with its features. My gesture recognition process now determines whether the user is moving or stationary, recognizes a posture and then captures the gesture that follows, before saving it to a file. The next step of the process will be to scale the gesture data to a reference, before saving it to a file.