Kinect SDK: Camera Tilt

Introduction

A previous post examined the basics of accessing the video and depth streams from the Kinect sensor. This post builds on that, and examines controlling the camera tilt of the Kinect sensor.

Implementation

The application documented here simply get the video stream from the Kinect sensor, calculates the FPS of the received stream, and allows the user to control the camera tilt via two buttons. The first step in implementing this application (after creating a new WPF project) is to include a reference to Microsoft.Research.Kinect. This assembly is in the GAC, and calls unmanaged functions from managed code. To use the NUI API you must import the Microsoft.Research.Kinect.Nui namespace to your application.

I then built a basic UI, using XAML. The code can be seen below.

<Window x:Class="KinectDemo.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        Title="Kinect Demo" ResizeMode="NoResize" SizeToContent="WidthAndHeight"
        Loaded="Window_Loaded" Closed="Window_Closed">
    <Grid>
        <Grid.ColumnDefinitions>
            <ColumnDefinition Width="Auto" />
            <ColumnDefinition Width="200" />
        </Grid.ColumnDefinitions>
        <StackPanel Grid.Column="0">
            <TextBlock Focusable="False"
                            HorizontalAlignment="Center"
                            Text="Video Stream" />
            <Image x:Name="video"
                        Height="300"
                        Margin="10,0,10,0"
                        Width="400" />
        </StackPanel>
        <StackPanel Grid.Column="1">
            <GroupBox Header="Motor Control"
                      Height="100"
                      VerticalAlignment="Top"
                      Width="190">
                <StackPanel HorizontalAlignment="Center" 
                            Margin="10"
                            Orientation="Horizontal">
                    <Button x:Name="motorUp"
                            Click="motorUp_Click"
                            Content="Up"
                            Height="30"
                            Width="70" />
                    <Button x:Name="motorDown"
                            Click="motorDown_Click"
                            Content="Down"
                            Height="30"
                            Margin="10,0,0,0"
                            Width="70" />
                </StackPanel>
            </GroupBox>
            <GroupBox Header="Information"
                      Height="100"
                      VerticalAlignment="Top"
                      Width="190">
                <StackPanel Orientation="Horizontal" Margin="10">
                    <TextBlock Focusable="False" 
                           Text="Frame rate: " />
                    <TextBlock x:Name="fps" 
                           Focusable="False"
                           Text="0 fps"
                           VerticalAlignment="Top"
                           Width="50" />
                </StackPanel>
            </GroupBox>
        </StackPanel>
    </Grid>
</Window>

MainWindow.xaml.cs contains the following class-level declarations. The important declaration here is the Camera object.

        private Runtime nui;
        private Camera cam;
        private int totalFrames = 0;
        private int lastFrames = 0;
        private DateTime lastTime = DateTime.MaxValue;

Then in the Window_Loaded event handler, the instance of the Camera object should be set to the NuiCamera property, after the NUI has been initialized.

        private void Window_Loaded(object sender, RoutedEventArgs e)
        {
            this.nui = new Runtime();
            try
            {
                nui.Initialize(RuntimeOptions.UseColor);
                this.cam = nui.NuiCamera;
            }
            catch (InvalidOperationException)
            {
                MessageBox.Show("Runtime initialization failed. Ensure Kinect is plugged in");
                return;
            }
            try
            {
                nui.VideoStream.Open(ImageStreamType.Video, 2, ImageResolution.Resolution640x480, ImageType.Color);
            }
            catch (InvalidOperationException)
            {
                MessageBox.Show("Failed to open stream. Specify a supported image type/resolution.");
                return;
            }
            this.lastTime = DateTime.Now;
            nui.VideoFrameReady += new EventHandler<ImageFrameReadyEventArgs>(nui_VideoFrameReady);
        }

To adjust the camera tilt, set the ElevationAngle property of the Camera object to a value between –27 (Camera.ElevationMinimum) and +27 (Camera.ElevationMaximum). However, the tilt mechanism in the sensor array is not designed for frequent use. Code should not make calls to the tilt device more than 15 times in any two-minute window. Changing the tilt too often will result in an error message.

The motorUp_Click event handler simply attempts to increase the camera ElevationAngle by 5. This is wrapped up in a try-catch block in case it’s not possible to adjust the camera tilt.

        private void motorUp_Click(object sender, RoutedEventArgs e)
        {
            try
            {
                this.cam.ElevationAngle += 5;
            }
            catch (InvalidOperationException)
            {
                MessageBox.Show("Failed to move Kinect motor.");
            }
            catch (ArgumentOutOfRangeException)
            {
                MessageBox.Show("Elevation angle is out of range.");
            }
        }

The motorDown_Click event handler simply attempts to decrease the camera ElevationAngle by 5. This is wrapped up in a try-catch block in case it’s not possible to adjust the camera tilt.

        private void motorDown_Click(object sender, RoutedEventArgs e)
        {
            try
            {
                this.cam.ElevationAngle -= 5;
            }
            catch (InvalidOperationException)
            {
                MessageBox.Show("Failed to move Kinect motor.");
            }
            catch (ArgumentOutOfRangeException)
            {
                MessageBox.Show("Elevation angle is out of range.");
            }
        }

The remainder of the code is as in this previous post.

The application looks as follows.

Conclusion

The Kinect for Windows SDK beta from Microsoft Research is a starter kit for application developers. It enables access to the Kinect sensor, and experimentation with its features. Controlling camera tilt, via the NUI API, is a trivial task.