Working with the Documents Tab on the SharePoint Ribbon

Cross-posted from Jason Lee's Blog

This week I've been taking a look at using ribbon controls with the SharePoint JavaScript client object model to drive some custom functionality. Ribbon customizations for SharePoint 2010 are fairly well documented. However, when you work with contextual tab groups—and the Documents tab in particular—there are a few nuances and idiosyncrasies that it's worth being aware of up front.

In this case, I want to add a ribbon button that enables the user to perform some additional actions when they select a file in a document library. There are countless scenarios in which you might want to do this – for example, you might add a "Request a copy of this document in large print/audio format/Welsh" control to the ribbon and use the document metadata to prepopulate an InfoPath form. To start with, however, I want to keep it simple:

• When the user selects a document in a document library, display a button on the ribbon.
• When the user clicks the button, display some information about the selected document as a client-side notification.

The logical place to put this button is on the Documents tab. This is part of the Library Tools contextual tab group – it's contextual because it's only displayed when the context is relevant, i.e. when the user browses to a document library. The Documents tab is selected automatically when the user selects one or more documents in the library's list view web part:

Let's take it one bit at a time for now, and I'll provide a full code listing at the bottom. Firstly, like all declarative ribbon customizations, we start with a CustomAction feature element:

<CustomAction Id="Jason.SP.GSD"
              Location="CommandUI.Ribbon"
              Sequence="11"
              RegistrationType="List"
              RegistrationId="101">

The key point of note here is that if you plan to add controls to a contextual tab, you must use the RegistrationType and RegistrationId attributes to target your ribbon customizations to an appropriate list type. If you're deploying controls to a standard ribbon tab, you can get away with omitting these attributes. It didn't initially occur to me that it should be any different in this case – I'm adding controls to the Documents tab, the Documents tab only shows up when I'm looking at a document library, I shouldn't have to worry about scope, right? But no – if you don't set these attributes, your controls simply won't show up on the tab. In this case, a RegistrationType of "List" and a RegistrationId of "101" scopes our ribbon customization to the document library base type.

Next, we define the controls we want to add to the ribbon. This process is identical regardless of whether you're adding to a contextual tab or a regular tab. In this case, we want to add a new group named "Jason's Actions" to the Documents tab. Within this group, we want to create a single button labelled "Get Selection Details". To accomplish this we need to create two CommandUIDefinition elements – one to define the maximum size of my group element, and one to define the group itself. Creating tabs, groups, and controls has been covered comprehensively elsewhere, so I don't want to go into too much detail – if you're looking for more information in this area, Chris O'Brien's blog post series is an excellent place to start. 

In this case, we'll use the absolute minimum markup required to add a new button in its own group - a Group element to define the group and the controls within it, and a MaxSize element that defines how the group should be rendered on the ribbon. You can specify many more elements if you want – for example you can add a Scale element to specify how your group should render at different sizes, and you can define your own GroupTemplate to specify precisely how controls within your group should be arranged. However, each group must have a matching MaxSize element – otherwise it won't appear on the tab. The easiest approach to creating ribbon controls is to pick out existing controls that resemble what you're looking for and take a look at how they're defined. Let's say we want our group and button to look like the Share & Track group shown here – a large, simple layout with a single control:

To replicate this group, the first step is to take a look at the group definition. Ribbon controls are defined in the 14\TEMPLATE\GLOBAL\XML\CMDUI.XML file. To find specific elements in this file, unless you know the ID of the element you're looking for, it's best to start with the top-level elements and narrow down your search – start by finding the right tab group (Id="Ribbon.LibraryContextualGroup"), then locate the correct tab (Id="Ribbon.Document"), then identify the group you're looking for. In this case, the group we want to borrow from has an ID of "Ribbon.Documents.Share":

<Group Id="Ribbon.Documents.Share"
       Sequence="40"
       Command="ShareGroup"
       Description=""
       Title="$Resources:core,cui_GrpShare;"
       Image32by32Popup=".../formatmap32x32.png" 
       Image32by32PopupTop="-128" 
       Image32by32PopupLeft="-64"
       Template="Ribbon.Templates.Flexible2">

By examining the definition of this group, we can figure out the properties we need:

• The Share & Track group has a Template attribute of Ribbon.Templates.Flexible2. This identifies the group template that gets applied to the group (also defined in CMDUI.XML if you want to take a closer look). We'll use this value to apply the same layout to our own group.
• The Share & Track group has a Sequence attribute of 40. We'll use a value of 41 to place our group immediately to the right of the Share & Track group.

Next, we can take a look at how controls are defined within the group. For example, the following markup defines the E-mail a Link button you saw in the previous image:

<Button Id="Ribbon.Documents.Share.EmailItemLink"
        Sequence="10"
        Command="EmailLink"
        Image16by16=".../formatmap16x16.png" 
        Image16by16Top="-16" 
        Image16by16Left="-88"
        Image32by32=".../formatmap32x32.png" 
        Image32by32Top="-128" 
        Image32by32Left="-448"
        LabelText="$Resources:core,cui_ButEmailLink;"
        ToolTipTitle="$Resources:core,cui_ButEmailLink;"
        ToolTipDescription="...,cui_STT_ButEmailLinkDocument;"
        TemplateAlias="o1"
/>

In this case, the TemplateAlias attribute is the value that interests us. Every group template contains one or more placeholders, represented by ControlRef elements, in which you can place your controls. In this case, the E-mail a link button specifies that it should be added to the o1 placeholder in the Flexible2 group template. If we use the same value in our own button, we should get the same result.

Finally, we can also take a look at the matching MaxSize element for the Share & Track group. Remember that these elements are always paired – a Group element always has a corresponding MaxSize element defined within the same tab. Within each MaxSize element, the GroupId attribute identifies the corresponding group:

<MaxSize Id="Ribbon.Documents.Scaling.Share.MaxSize"
         Sequence="40"
         GroupId="Ribbon.Documents.Share"
         Size="LargeLarge" 
/>

In this case, all we're interested in is the Size attribute. A group template can define multiple layouts, and this attribute identifies the specific layout in the Flexible2 template that we want to use – in this case, the LargeLarge layout. 

We can now use all this information we've collected to define our group and button:

<CommandUIExtension>
  <CommandUIDefinitions>
    <CommandUIDefinition Location="Ribbon.Documents.Scaling._children"> 
      <MaxSize Id="Jason.SP.GSD.JasonsActions.MaxSize"
        Sequence="11"
        GroupId="Jason.SP.GSD.JasonsActions"
        Size="LargeLarge" /> 
    </CommandUIDefinition>
    <CommandUIDefinition Location="Ribbon.Documents.Groups._children">
      <Group Id="Jason.SP.GSD.JasonsActions"
        Sequence="41"
        Title="Jason's Actions"
        Description="Contains custom document actions"
        Template="Ribbon.Templates.Flexible2"> 
          <Controls Id="Jason.SP.GSD.JasonsActions.Controls">
            <Button Id="Jason.SP.GSD.JasonsActions.GetButton"
              Sequence="1"
              Image32by32=".../ThumbsUp.PNG"
              LabelText="Get Selection Details"
              Description="Gets the details of the selected document"
              TemplateAlias="o1"
              Command="Jason.SP.GSD.GetCmd" />
          </Controls>
        </Group>
      </CommandUIDefinition>
    </CommandUIDefinitions>

There are a few additional points worth mentioning at this stage:

• You need a CommandUIDefinition element for each block of XML you want to add to the ribbon.
• When setting the Location attribute, imagine you're slotting the XML directly into the CMDUI.XML file. Look up the ID of the parent element you want to add to, and append "._children" to get your Location value. For example, we want to add our group to the Groups element with an ID of "Ribbon.Documents.Groups", so our Location attribute is "Ribbon.Document.Groups._children".

Note that the button has a Command attribute value of "JL.GetSelectionDetails". This ties the button to a CommandUIHandler element in which we can define the JavaScript that should run when the user clicks the button, as shown below:

    <CommandUIHandlers>
      <CommandUIHandler  
        Command="Jason.SP.GSD.GetCmd"
        EnabledScript="javascript:
          SP.ListOperation.Selection.getSelectedItems().length == 1;"
        CommandAction="javascript: 
          var selectedItems = 
            SP.ListOperation.Selection.getSelectedItems();
          var item = selectedItems[0];
          var itemID = item['id'];
          if (item['fsObjType'] == 0) {
            SP.UI.Notify.addNotification(String.format(
              'Document selected: ID={0}', itemID));
          }
          else {
            SP.UI.Notify.addNotification(String.format(
              'Folder selected: ID={0}', itemID));
          }" 
      /> 
    </CommandUIHandlers>
  </CommandUIExtension>
</CustomAction>

The first point of interest is the EnabledScript attribute. When you add a control to the Documents tab it is disabled by default – you must use this attribute to specify the conditions under which the control should be enabled. The EnabledScript attribute should specify (or call) a JavaScript function that returns a Boolean value – true to enable the control, false to disable it. In this case, we want the button to be enabled when the user has selected a single document in the document library list view. The JavaScript client-side object model for SharePoint includes a class named SP.ListOperation.Selection for just this kind of eventuality. We can use the getSelectedItems method to return a collection of the items selected in the list view, then check that the length of the collection is equal to 1.

Note: In this example I've added all my JavaScript logic directly to the CommandUIHandler element. As your JavaScript logic grows larger and more complex, a better option would be to deploy a standalone JavaScript file. Yaroslav Pentarskyy describes this approach in this blog post.

Next, the CommandAction attribute specifies the JavaScript function we want to call when our button is clicked. The getSelectedItems method returns a Dictionary of key-value pairs. The value of each dictionary entry is an object with two attributes – id and fsObjType. The id attribute represents the integer ID of the list item, while the fsObjType attribute represents the type of list item object – 0 for a document or a list item, 1 for a folder. While this doesn't give us a great deal of information about the selected item, the integer ID gives us enough information to submit a query for additional document metadata, should we so wish. In this case, as a proof of concept, we simply display a notification containing the document ID when the user clicks our button.

Here's the button in its default disabled state:

When we select a document, the button is enabled:

When we click the button, a notification displays the integer ID of the selected document:

And that concludes today's task. Next time I plan to cover how to extend this to do something useful with the selected document. The contents of the feature element are shown below in their entirety for reference.

<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
  <CustomAction Id="Jason.SP.GSD" 
                Location="CommandUI.Ribbon" 
                Sequence="11" 
                RegistrationType="List" 
                RegistrationId="101">
    <CommandUIExtension>
      <CommandUIDefinitions>
        <CommandUIDefinition Location="Ribbon.Documents.Scaling._children">
          <MaxSize Id=" Jason.SP.GSD.JasonsActions.MaxSize" 
                   Sequence="11" 
                   GroupId="Jason.SP.GSD.JasonsActions" 
                   Size="LargeLarge" />
        </CommandUIDefinition>
        <CommandUIDefinition Location="Ribbon.Documents.Groups._children">
          <Group Id=" Jason.SP.GSD.JasonsActions" 
                 Sequence="41" 
                 Title="Jason's Actions" 
                 Description="Contains custom document actions" 
                 Template="Ribbon.Templates.Flexible2">
            <Controls Id="Jason.SP.GSD.JasonsActions.Controls">
              <Button Id=" Jason.SP.GSD.JasonsActions.GetButton" 
                      Sequence="1" 
                      Image32by32="/SiteCollectionImages/RibbonIcons/ThumbsUp.PNG" 
                      LabelText="Get Selection Details" 
                      Description="Gets the details of the selected document" 
                      TemplateAlias="o1" 
                      Command=" Jason.SP.GSD.GetCmd" />
            </Controls>
          </Group>
        </CommandUIDefinition>
      </CommandUIDefinitions>
      <CommandUIHandlers>
        <CommandUIHandler Command="Jason.SP.GSD.GetCmd" 
                          EnabledScript="javascript:
                            SP.ListOperation.Selection.getSelectedItems().length == 1;" 
                          CommandAction="javascript:
                            var selectedItems = 
                              SP.ListOperation.Selection.getSelectedItems();
                            var item = selectedItems[0];
                            var itemID = item['id'];
                            if (item['fsObjType'] == 0) {
                              SP.UI.Notify.addNotification(String.format(
                                'Document selected: ID={0}', itemID));
                            }
                            else {
                              SP.UI.Notify.addNotification(String.format(
                                'Folder selected: ID={0}', itemID));
                            } 
        "/>
      </CommandUIHandlers>
    </CommandUIExtension>
  </CustomAction>
</Elements>

Kinect SDK: Gesture Recognition Pt III

Introduction

In my previous blog post I discussed the development of a robust and extensible PostureRecognizer class. The class is used to recognize the start of a gesture. As a reminder, my high-level approach to the gesture recognition process is as follows:

• Detect whether the user is moving or stationary.
• Detect the start of a gesture (a posture).
• Capture the gesture.
• Detect the end of a gesture (a posture).
• Identify the gesture.

This blog post will focus on capturing the gesture that occurs once the starting posture has been identified.

Implementation

The UI is unchanged from my previous post, with the exception of another Canvas being added to draw the captured gesture on.

The StreamManager constructor creates an instance of the PostureRecognizer class and registers an event handler for the PostureDetected event. It then registers the path where the gesture data will be saved, via a stream. An instance of the GestureRecognizer class is then created, and the name of the canvas to draw the captured gesture on is passed to the DrawGesture method.

        public StreamManager(Canvas canvas)
        {
            this.PostureRecognizer = new PostureRecognizer();
            this.PostureRecognizer.PostureDetected += 
               new Action<Posture>(OnPostureDetected);
            
            this.gestureCanvas = canvas;
            this.path = Path.Combine(Environment.CurrentDirectory, @"data\gestures.dat");
            this.stream = File.Open(this.path, FileMode.OpenOrCreate);
            this.gestureDetector = new GestureRecognizer(this.stream);
            this.gestureDetector.DrawGesture(this.gestureCanvas, Colors.Red);
        }

The StartGestureCapture and EndGestureCapture methods are shown below. The StartGestureCapture method invokes the StartGestureCapture method in the GestureRecognizer class, and if a gesture is already being captured, stops the capture. The EndGestureCapture method invokes the EndGestureCapture method in the GestureRecognizer class, and invokes the SaveGesture method in the GestureRecognizer class, provided that a gesture is being captured. 

        private void StartGestureCapture()
        {
            this.gestureDetector.GestureName = this.GestureName;
            if (this.gestureDetector.IsRecordingGesture)
            {
                this.gestureDetector.EndGestureCapture();
                return;
            }
            this.gestureDetector.StartGestureCapture();
        }
        private void EndGestureCapture()
        {
            if (this.gestureDetector.IsRecordingGesture)
            {
                this.gestureDetector.EndGestureCapture();
                this.gestureDetector.SaveGesture(stream);
            }
        }

The GetSkeletonStream method is as in the previous post, and invokes the TrackPostures method in the PostureRecognizer class. If a posture is identified, the RaisePostureDetected method is invoked which sets the CurrentPosture property to the identified posture, and invokes the PostureDetected event. The handler for the event is the OnPostureDetected method in the StreamManager class and was registered in the StreamManager constructor.

        public event Action<Posture> PostureDetected;
        private void RaisePostureDetected(Posture posture)
        {
            if (this.currentPosture != posture)
            {
                this.CurrentPosture = posture;
            }
            if (this.PostureDetected != null)
            {
                this.PostureDetected(posture);
            }
        }

The OnPostureDetected method is shown below. If a gesture is not being captured, it invokes the StartGestureCapture method, and stores the identified posture in the startPosture variable. If a gesture is being captured and the posture is different to the posture stored in the startPosture variable, the EndGestureCapture method is invoked. The detected Joints are then enumerated, and the Add method in the GestureRecognizer class is invoked if the gesture involves the right hand.

        private void OnPostureDetected(Posture posture)
        {
            if (this.gestureDetector.IsRecordingGesture == true)
            {
                if ((posture != Posture.None) &&
                    (posture != startPosture))
                {
                    this.EndGestureCapture();
                }
            }
            else
            {
                this.StartGestureCapture();
                this.startPosture = posture;
            }
            foreach (Joint joint in this.skeleton.Joints)
            {
                if (joint.Position.W < 0.9f || 
                    joint.TrackingState != JointTrackingState.Tracked)
                {
                    continue;
                }
                if (joint.ID == JointID.HandRight)
                {
                    this.gestureDetector.Add(joint.Position, 
                                             this.KinectRuntime.SkeletonEngine);
                }
            }
        }

The GestureData class is used to model the gesture data. It contains the a Position and the Time the Position was captured.

    public class GestureData
    {
        public Vector Position { get; set; }
        public DateTime Time { get; set; }
    }

The GestureRecognizer class is shown below. Gestures are stored in a collection named gestures, of type List<GestureData>. An instance of the TemplateRecognizer class is also created, which will use template matching to perform gesture identification. The bulk of the work in this class is performed in the Add method. It stores the passed in Vector as a GestureData object, and adds that to the gestures collection. It then draws the position as a small ellipse on the Canvas contained in the displayCanvas variable. Then, if there is more data in the gestures collection than the value of WindowSize, data is removed from both the start of the collection and the displayCanvas. The position data is then added to an instance of the Gesture class. The StartGestureCapture, EndGestureCapture, and SaveGesture methods largely invoke methods in the TemplateRecognizer class.

    public class GestureRecognizer
    {
        private Gesture gesture;
        private string gestureName; 
        private readonly List<GestureData> gestures = new List<GestureData>();
        private readonly TemplateRecognizer.TemplateRecognizer templateRecognizer;
        private readonly int windowSize;
        private DateTime lastGestureDate = DateTime.Now;
        private Canvas displayCanvas;
        private Colour displayColour;
        protected List<GestureData> Gestures
        {
            get { return this.gestures; }
        }
        public string GestureName
        {
            get { return this.gestureName; }
            set { this.gestureName = value; }
        }
        public bool IsRecordingGesture
        {
            get { return this.gesture != null; }
        }
        public TemplateRecognizer.TemplateRecognizer TemplateRecognizer
        {
            get { return this.templateRecognizer; }
        }
        public int WindowSize
        {
            get { return this.windowSize; }
        }
        public GestureRecognizer(Stream stream, int windowSize = 100)
        {
            this.templateRecognizer = new TemplateRecognizer.TemplateRecognizer(stream);
            this.windowSize = windowSize;
        }
        public GestureRecognizer(string gestureName, Stream stream, int windowSize = 100)
        {
            this.gestureName = gestureName;
            this.templateRecognizer = new TemplateRecognizer.TemplateRecognizer(stream);
            this.windowSize = windowSize;
        }
        public void Add(Vector position, SkeletonEngine engine)
        {
            GestureData newGesture = new GestureData
            {
                Position = position,
                Time = DateTime.Now
            };
            this.gestures.Add(newGesture);
            if (this.displayCanvas != null)
            {
                Ellipse ellipse = new Ellipse
                {
                    HorizontalAlignment = HorizontalAlignment.Left,
                    VerticalAlignment = VerticalAlignment.Top,
                    Height = 8,
                    Width = 8,
                    StrokeThickness = 8,
                    Stroke = new SolidColorBrush(this.displayColour),
                    StrokeLineJoin = PenLineJoin.Round
                };
                float x, y;
                engine.SkeletonToDepthImage(position, out x, out y);
                x = (float)(x * this.displayCanvas.ActualWidth);
                y = (float)(y * this.displayCanvas.ActualHeight);
                Canvas.SetLeft(ellipse, x - ellipse.Width / 2);
                Canvas.SetTop(ellipse, y - ellipse.Height / 2);
                this.displayCanvas.Children.Add(ellipse);
            }
            if (this.gestures.Count > this.WindowSize)
            {
                GestureData gestureToRemove = this.gestures[0];
                if (this.displayCanvas != null)
                {
                    this.displayCanvas.Children.RemoveAt(0);
                }
                this.gestures.Remove(gestureToRemove);
            }
            if (this.gesture != null)
            {
                Vector2 vector = new Vector2
                {
                    X = position.X,
                    Y = position.Y
                };
                this.gesture.Points.Add(vector);
            }
        }
        public void DrawGesture(Canvas canvas, Color colour)
        {
            this.displayCanvas = canvas;
            this.displayColour = colour;
        }
        public void StartGestureCapture()
        {
            this.gesture = new Gesture(this.WindowSize);
        }
        public void EndGestureCapture()
        {
            this.templateRecognizer.Add(gesture);
            this.gesture = null;
        }
        public void SaveGesture(Stream stream)
        {
            this.templateRecognizer.Save(stream);
        }
    }

The Gesture class stores all the positions of the captured gesture in a collection called points, of type List<Vector>.

    [Serializable]
    public class Gesture
    {
        private List<Vector> points;
        private readonly int samplesCount;
        public List<Vector> Points
        {
            get { return this.points; }
            set { this.points = value; }
        }
        public Gesture(int samplesCount)
        {
            this.samplesCount = samplesCount;
            this.points = new List<Vector>();
        }
    }

The TemplateRecognizer class will perform the bulk of the processing required for gesture identification, which will be covered in a future blog post. At the moment the class simply handles serialization and deserialization of the gesture data, by using a BinaryFormatter. The constructor deserializes the gesture data if it exists, while the Save method serializes the gesture data to a file.

    public class TemplateRecognizer
    {
        private readonly List<Gesture> gestures;
        public TemplateRecognizer(Stream stream)
        {
            if (stream == null || stream.Length == 0)
            {
                this.gestures = new List<Gesture>();
                return;
            }
            BinaryFormatter formatter = new BinaryFormatter();
            this.gestures = (List<Gesture>)formatter.Deserialize(stream);
        }
        public void Add(Gesture gesture)
        {
            this.gestures.Add(gesture);
        }
        public void Save(Stream stream)
        {
            BinaryFormatter formatter = new BinaryFormatter();
            formatter.Serialize(stream, this.gestures);
        }
    }

The application is shown below. It determines whether the user is moving or stationary, and recognizes the start of a gesture – in this case, a posture. Once a posture is identified (currently hard coded to RightHello) the gesture points are stored in a collection and drawn on the canvas. When a different posture is identified the gesture capture stops.

<

Conclusion

The Kinect for Windows SDK beta from Microsoft Research is a starter kit for application developers. It allows access to the Kinect sensor, and experimentation with its features. My gesture recognition process now determines whether the user is moving or stationary, recognizes a posture and then captures the gesture that follows, before saving it to a file. The next step of the process will be to scale the gesture data to a reference, before saving it to a file.