We've Moved!

After many happy years on Blogger, we're consolidating the Content Master technology blog into our coporate Web site. Come and join us at http://www.contentmaster.com/technology-blog/!

Accessing Image Pixel Data in a C++/CX Windows Store App

As part of a Windows Store app I recently helped to develop, there was a requirement to perform imaging operations that manipulated the pixel values of the image. Rather than have an Image control display a BitmapImage, we displayed a WriteableBitmap instead, allowing us to manipulate an image at the pixel level. However, the task of doing this is not as straightforward as it first appeared. In particular, many of the posts concerning this on the forums were sub-optimal.

The pixel data in a WriteableBitmap is stored in the PixelBuffer property. However, this property is read-only. In managed languages, you can use the AsStream extension method to access the underlying buffer as a stream. However, to access the pixel data from a C++ app requires you to query the PixelBuffer for the IBufferByteAccess type, and then access its Buffer.

Originally we managed the lifetime of our COM variables manually, via calls to AddRef and Release. However, we then discovered the ComPtr class, which creates a smart pointer type that represents the interface specified by the template parameter. The advantage of using a ComPtr is that it automatically maintains a reference count for the underlying interface pointer and releases the interface when the reference count goes to zero, making calls to AddRef and Release redundant. The ComPtr class also provides the As method, which can be thought of as a shortcut for calling QueryInterface to retrieve a pointer to a supported interface on an object.

The free function that we developed to access pixel data is shown below. It retrieves the pixel data from the provided IBuffer object in the WriteableBitmap, and returns a pointer to it.

byte* GetPointerToPixelData(IBuffer^ buffer)

{

    // Cast to Object^, then to its underlying IInspectable interface.

    Object^ obj = buffer;

    ComPtr<IInspectable> insp(reinterpret_cast<IInspectable*>(obj));

    // Query the IBufferByteAccess interface.

    ComPtr<IBufferByteAccess> bufferByteAccess;

    ThrowIfFailed(insp.As(&bufferByteAccess));

    // Retrieve the buffer data.

    byte* pixels = nullptr;

    ThrowIfFailed(bufferByteAccess->Buffer(&pixels));

    return pixels;

}

The code converts an IBuffer object to its underlying COM interface, IBufferByteAccess. This is necessary because an IBuffer object only exposes two properties – Capacity and Length. However, an IBufferByteAccess is used to represent an IBuffer as an array of bytes, with the Buffer method returning the array of bytes.

The ThrowIfFailed function is used to convert a HRESULT failure code to a Windows Runtime exception.

inline void ThrowIfFailed(HRESULT hr)

{

    if (FAILED(hr))

    {

        throw Exception::CreateException(hr);

    }

}

To get the the pixel data from a WriteableBitmap you can invoke GetPointerToPixelData, and pass the PixelBuffer from the WriteableBitmap as a parameter (m_image is a WriteableBitmap).

byte *pPixels = GetPointerToPixelData(m_image->PixelBuffer);

In order to access and modify the pixel data you should iterate through it, performing your desired imaging operation. The code below iterates through the pixel data, changing every pixel value to 0 (black). Note that the pixel data is stored in BGRA format.

for (int y = 0; y < height; y++)

{

    for (int x = 0; x < width; x++)

    {

        pPixels[(x + y * width) * 4]     = 0; // B

        pPixels[(x + y * width) * 4 + 1] = 0; // G

        pPixels[(x + y * width) * 4 + 2] = 0; // R

        pPixels[(x + y * width) * 4 + 3] = 0; // A

    }

}

After the loops are executed, the modified pixel data will appear in the WriteableBitmap after it’s redrawn via a call to Invalidate. Alternatively, in some cases, you can use property change notification to redraw the WriteableBitmap.

Summary

The GetPointerToPixelData free function can be used to robustly and reliably get pixel data from a WriteableBitmap. The function uses the ComPtr class, which creates a smart pointer type that represents the interface specified by the template parameter. The advantage of using a ComPtr is that it automatically maintains a reference count for the underlying interface pointer and releases the interface when the reference count goes to zero.

The Repository Pattern in C++/CX

In a previous blog post I demonstrated how to implement the MVVM pattern in C++/CX, by way of a small sample app that allows the user to view the photos in their Pictures library. In this blog post I will extend the sample app so that access to the underlying data source, in this case the file system, is through a centralized data access layer. The main advantage of this approach is that a repository that accesses a data source, such as the file system, can easily be swapped out for a repository that access a different data source, such as the cloud.

The sample application can be downloaded here.

The Repository Pattern

The repository pattern separates the logic that retrieves and persists data from the business logic that acts upon that data. The repository is responsible for:

• Mediating between the data source and the business layers of the app.
• Querying the data source for data.
• Persisting changes to the data source.

This separation of business logic from the data source has benefits, including:

• Centralizing access to the underlying data source via a data access layer.
• Isolating the data access layer to support unit testing.
• Improving the code’s maintainability by separating business logic from data access logic.

For more information, see The Repository Pattern.

Implementing the Repository Pattern in C++/CX

The sample app implements the repository pattern by defining an abstract base class with pure virtual member functions, to provide a base class from which other repository classes must inherit. A pure virtual member function is a member function that is defined as virtual and is assigned to 0. Such an abstract base class can’t be used to instantiate objects and serves only to act as an interface. Therefore, if a subclass of this abstract class needs to be instantiated, it has to implement each of the virtual functions, which results in it supporting the interface defined by the abstract base class.

The Repository abstract base class defines a number of pure virtual member functions, which must be overridden by member functions with the same signatures in a derived class. The FileSystemRepository class derives from the Repository class, and overrides the pure virtual member functions to query the file system for photos. A shared instance of the FileSystemRepository class is then used by the view model classes to read photos from the file system. The following illustration shows this relationship:

Using this architecture also allows mock implementations of the repository to be created, to support unit testing. A mock implementation could be passed into the view model classes from unit tests, with the mock class simply returning mock photos rather than actual photos from the file system. For more information, see the Hilo project.

In the sample app, the ViewModelLocator class contains a member variable, m_repository, of type Repository, which is instantiated as a shared pointer of type FileSystemRepository in the class constructor. This instance is created as a shared pointer so that there’s only a single instance of the FileSystemRepository class in the app, which is then passed between the required classes.

ViewModelLocator::ViewModelLocator()

{

    m_repository = std::make_shared<FileSystemRepository>();

}

The ViewModelLocator class has properties that retrieve a view model object for each page of the app.

PhotoViewModel^ ViewModelLocator::PhotoVM::get()

{

    return ref new PhotoViewModel(m_repository);

}

The PhotoVM property creates a new instance of the PhotoViewModel class, and passes in the shared pointer instance of the FileSystemRepository class, to the PhotoViewModel constructor. The shared pointer instance of the FileSystemRepository class is then stored in the m_repository member variable of the PhotoViewModel class.

PhotoViewModel::PhotoViewModel(shared_ptr<Repository> repository) : 

    m_repository(repository)

{

}

The Image control in the PhotoView class binds to the Photo property of the PhotoViewModel class. In turn, the Photo property invokes the GetPhotoAsync member function to retrieve the photo for display.

task<FileInformation^> PhotoViewModel::GetPhotoAsync()

{

    return m_repository->GetPhotoAsync(m_path);

}

GetPhotoAsync invokes GetPhotoAsync on the shared pointer instance of the FileSystemRepository class that is stored in the m_repository member variable.
The Repository abstract base class defines two pure virtual member functions, including GetPhotoAsync. The FileSystemRepository class inherits from the Repository class, and provides overridden implementations of the member functions. Here’s the code for the GetPhotoAsync member function.

task<FileInformation^> FileSystemRepository::GetPhotoAsync(String^ path)

{

    auto fileQuery = CreateFileQuery(KnownFolders::PicturesLibrary, path, IndexerOption::UseIndexerWhenAvailable);

    auto fif = ref new FileInformationFactory(fileQuery, ThumbnailMode::PicturesView);

    return create_task(fif->GetFilesAsync(0, 1)).then([](IVectorView<FileInformation^>^ files)

    {

        FileInformation^ file = nullptr;

        auto size = files->Size;

        if (size > 0)

        {

            file = files->GetAt(0);

        }

        return file;

    }, task_continuation_context::use_current());

}

The GetPhotoAsync member function calls an inline function, CreateFileQuery, to create a file query that will be used by an instance of the FileInformationFactory class to query the file system using Advanced Query Syntax. GetFilesAsync is then called on the FileInformationFactory instance, which returns an IVectorView collection of FileInformation objects, representing each file found in the file system that matches the query. However, in this case the IVectorView collection will only contain a maximum of one FileInformation object, as GetFilesAsync will only return one file that matches the query. The value-based continuation, which runs on the main thread, then gets the first file from the IVectorView collection, before returning it to the GetPhotoAsync member function in the PhotoViewModel class, which in turn returns it to the Photo property in that class, for display.

Summary

The repository pattern can be implemented in C++/CX by defining an abstract base class with pure virtual member functions. Subclasses of this class can then implement each of the virtual functions, which results in the subclass supporting the interface defined by the abstract base class. The main advantage of this approach is that a repository that accesses a data source, such as the file system, can easily be swapped out for a repository that access a different data source, such as the cloud.

MVVM in C++/CX

(cross-posted from the MSDN UK Team Blog)

Previously I’ve written about the power of C++/CX for developing Windows Store applications. C++ is the language for power and performance on WinRT, and the combination of C++11 and C++/CX makes C++/CX read a lot like C#, while giving you the benefits of native code. In addition, the ability to develop the UI using XAML enables the designer-developed workflow present in the .NET world.

The MVVM pattern lends itself naturally to XAML application platforms, including C++/CX. This is because it leverages some of the specific capabilities of XAML, such as data binding and commands. While managed code developers are well-versed in the MVVM pattern, C++ developers typically are not. Therefore, the purpose of this article is to demonstrate how to implement MVVM in C++/CX. However, before I get into the details of implementing MVVM in C++/CX, I’ll first provide a quick overview of MVVM.

The MVVM Pattern

The Model-View-ViewModel pattern can be used on all XAML platforms. Its intent is to provide a clean separation of concerns between the user interface controls and their logic. There are three core components in the MVVM pattern - the model, the view, and the view model:

• The view is responsible for defining the structure, layout, and appearance of what the user sees on the screen. Ideally, the view is defined purely with XAML, with a limited code-behind that does not contain business logic.
• The model in MVVM is an implementation of the application's domain model that includes a data model along with business and validation logic.
• The view model acts as an intermediary between the view and the model, and is responsible for handling the view logic. Typically, the view model interacts with the model by invoking methods in the model classes. The view model then provides data from the model in a form that the view can easily use.

The following illustration shows the relationships between the three components.

The main benefits that MVVM provides include a separation of concerns, enabling the developer-designer workflow, and increasing application testability. For more information about MVVM, see Developing a Windows Phone Application using the MVVM Pattern, Implementing the MVVM Pattern, and Advanced MVVM Scenarios.

Implementing MVVM in C++/CX

In order to demonstrate implementing MVVM in C++/CX I’ve written a small photo viewing sample app, which can be downloaded here. In the app, there are two pages. The first page presents a thumbnail gallery view of photos in the users Pictures library. Clicking on a thumbnail takes the user to the second page which displays the full image.

Each page in the app is implemented as a view, with each view having a corresponding view model class. Both view models share the same model class.

The app needs to connect its views to its view models, and it does this by using a view model locator. The ViewModelLocator class has properties that retrieve a view model object for each page of the app, which the view model object being assigned to the DataContext of the page that represents the view.

<local:PhotoViewerPage

    x:Class="PhotoViewer.MainView"

    ...

    DataContext="{Binding Source={StaticResource ViewModelLocator}, Path=MainVM}">

The ViewModelLocator static resource is declared in App.xaml. The MainVM property, in the ViewModelLocator class, simply returns an instance of the MainViewModel class.

MainViewModel^ ViewModelLocator::MainVM::get()

{

    if (nullptr == m_mainViewModel)

    {

        m_mainViewModel = ref new MainViewModel();

    }

    return m_mainViewModel;

}

Controls in the view can then bind to objects in the view model.

<GridView ...

    ItemsSource="{Binding Photos}"

    ... >

The code above shows a GridView control binding to the Photos property in the instance of the MainViewModel class.

For a view model to participate in data binding with the view, it must have the Windows::UI::Xaml::Data::Bindable attribute to ensure that the type is included in XAML’s generated file.

[Windows::UI::Xaml::Data::Bindable]

[Windows::Foundation::Metadata::WebHostHiddenAttribute]

public ref class MainViewModel sealed : public ViewModelBase

{

    ...

}

In addition, you need to include the header for your view model in the App.xaml.h header file, either indirectly or directly. This ensures that the types necessary to work with XAML are generated at compile time. In the sample app, all view model header files are included in the ViewModelLocator.h file, which is included in App.xaml.h.

View models that need to notify views that a property value has changed must raise the PropertyChanged event. To do this, view model classes need to implement the INotifyPropertyChanged interface. Visual Studio provides an implementation of the INotifyPropertyChanged interface in the BindableBase template class that you can use as a base class for any XAML data source. Therefore, view model classes can inherit the INotifyPropertyChanged implementation by deriving from the BindableBase class. In the sample app, the ViewModelBase class derives from the BindableBase class.

public ref class ViewModelBase : public Common::BindableBase

{

    ...

}

Whenever view models need to tell the UI that a bound property has changed, they call the OnPropertyChanged method that they inherited from the BindableBase class.

ImageSource^ PhotoViewModel::Photo::get()

{

    if (nullptr == m_photo)

    {

        ...

        OnPropertyChanged("Photo");

        ...

    }

    return m_photo;

}

In addition, you can also use data binding for UI controls that cause the app to perform operations. If the control derives from ButtonBase, the control’s Command property can be databound to an ICommand property on the view model. When the control’s command is invoked, the code in the view model is executed. For more information, see the Hilo project.

Summary

C++/CX apps can use XAML for the UI, enabling C++/CX developers to use the MVVM pattern in an app to make the app easier to test, maintain, and evolve. The accompanying sample app enables a user to view a gallery of thumbnails of photos in their Pictures library, and view a selected image full size. The app follows the MVVM pattern of UI controls on the view binding to objects on the view model.