Thursday, January 7, 2010

Silverlight Unit Testing Framework: Asynchronous Testing of Behaviors

Last month, I bogged about Unit Testing ViewModels AND Views using the Silverlight Unit Testing Framework. I wanted to take that post a step further and talk about some more advanced testing scenarios that are possible.

The site itself provides a lot of information about how to get started and what is available with the framework. One thing to keep in mind that is a radical shift from other testing frameworks is that the Silverlight testing framework runs on the UI thread. This means it does not spawn multiple threads for multiple tests and in fact requires the tests to run "one at a time" so they can take advantage of the test surface that is supplied.

This is a bit different than other frameworks but in my opinion, makes a lot of sense when dealing with Silverlight. The framework provides incredible flexibility for configuring and categorizing your tests.

If you are searching for a very comprehensive example of the framework in use, look no further than the Silverlight Toolkit. This comes with all source code and in fact uses the testing framework for its tests. You will find not only advanced scenarios for testing, but thousands upon thousands of tests! (I also am guessing that a new standard for PC performance has been invented by mistake ... let's all compile the entire toolkit and compare how long it takes!)

Tagging Tests

One thing you'll find if you run the toolkit tests is that you can enter a tag to filter tests. For example, type in "Accordion" to only run the 800 or so unit tests for accordion-type controls.

To use tag functionality, simply "tag" your test like this:

public class PartModuleTest 

I've tagged the test to be a MEF-related test. When I wire up the framework, I can filter the tag like this:

UnitTestSettings settings = UnitTestSystem.CreateDefaultSettings();
settings.TagExpression = "MEF";
this.RootVisual = UnitTestSystem.CreateTestPage(settings);  

When I run the tests, only my tests tagged with MEF will run! The toolkit provides an example of a UI that allows you to select the tag, then run the test.

Asynchronous Tests

It is often necessary to test methods that are asynchronous or require event coordination. An example may be a service that must wait on return values, or a user control that must be loaded into the framework before you can test it. The Silverlight Unit Testing Framework provides the Asynchronous tag to facilitate this type of test. This tells the framework not to move onto the next test nor consider the current test method complete until an explicit call to TestComplete is made.

There are several "helper" methods supplied for asynchronous processing that we'll explore in a minute. To use these methods requires inheriting from one of the base test classes such as SilverlightTest which provides the methods as well as the test surface to add controls to.

In PRISM, MEF, and MVVM Part 1 of 3: Unity Glue I explored various options for binding the view model to the view. The 3rd and final method I reviewed was using an attached behavior. I would like to write some unit tests for that behavior (indeed, if I were a test-driven development or TDD purist, I would have written those tests first).

In order to test the behavior, I need to attach it to a FrameworkElement and then validate it has done what I expected it to do. But how do I go about doing that in our unit test environment?

Attached Behaviors

Similar to other controls in other frameworks, Silverlight controls have a distinct life cycle. It varies slightly depending on whether the control has been generated in XAML or through code. There is a great summary table of these events on Dave's Blog. What's important to note is that values and properties are set as soon as you, well, set them, but bindings don't take effect until they are inserted into the visual tree. In XAML, the XAML node becomes part of the tree and fires the Loaded event once it is fully integrated. In code, this happens after the element is added as the child of some other element that is in the tree. This allows Silverlight to parse the hierarchy and propagate dependency properties.

So what we essentially want to do is take our behavior, attach it to an element, and then wait for the Loaded event to fire so we can inspect the element and see that it has been modified accordingly (in this case, we expect that the DataContext property has been set to our view model).

Setting up the Project

The testing framework provides some handy templates for getting started. I add a new project and select the Silverlight Test Project template. I then add references to the projects I'll be testing and the supporting frameworks like PRISM and MEF.

Next, I'll want to build some helper classes to help me test my functionality.

Helper Classes

I like to create a folder called Helper and place my stubs, mocks, and other helper classes there. These may be utilities, like the Exception Expected utility I use, or classes that are used for the testing framework.

First, I'll create a test view model with a simple string and string collection property for testing:

public class TestViewModel 
    public TestViewModel()
        ListOfItems = new List<string>();

    public TestViewModel(List<string> items)
        ListOfItems = items;            

    public string Property { get; set; }

    public List<string> ListOfItems { get; set; }

If my view models have common methods described in a base class or interface, I might use a mocking framework to mock the class instead.

The Test Class

The behavior I created has an affinity to the Unity inversion of control (IoC) container. It could be refactored otherwise, but it made sense for the sake of the demonstration. Therefore, I'll need to have a container for testing, as well as the view model. My test class starts out looking like this (notice I base it on SilverlightTest):

public class ViewModelBehaviorTest : SilverlightTest
    const string TESTPROP = "Test Property";

    IUnityContainer _container;

    TestViewModel _viewModel; 

    public void ClassInitialize()
        _container = new UnityContainer();

        _viewModel = new TestViewModel() { Property = TESTPROP }; 

        ViewModelBehavior.Container = _container; 

I create a reference to the entire test class for the container and the test view model. When the class is initialized (this is one-time setup, before all tests are run) I create a container, a view model, and tell the container that anytime someone asks for the view model, give them the specific instance I created. I also set the container on the type for the view model behavior class, so it knows what to use when resolving the view model.

The Code Behind Test

For my first test, I'll programmatically attach the behavior and test that it works. The view model behavior takes in a string that is the fully qualified type name for the view model, and then uses the unity container to resolve it. Therefore, my test looks like this:

[Description("Test creating an element and attaching in code behind.")]
public void TestAttach()
    TextBlock textBlock = new TextBlock();
    textBlock.SetValue(ViewModelBehavior.ViewModelProperty, typeof(TestViewModel).AssemblyQualifiedName);
    textBlock.Loaded += (o, e) => 
        Assert.IsNotNull(textBlock.DataContext, "The data context was never bound.");
        Assert.AreSame(textBlock.DataContext, _viewModel, "The data context was not bound to the correct view model.");



There's a few things going on here, so let's break them down!

The TestMethod attribute tags this method to be run by the framework. It is decorated with a description, which I can view on the output when the test is run and helps make the test more, ah, descriptive. The first thing I do is create a test block and attach the view model property. Here, I'm taking the test view model and getting the fully qualified name and using that to set the attached property. We want to make sure everything works fine and there are no errors during binding, so this is where the asynchronous pieces come into play.

The Asynchronous tag tells the framework that we're waiting on events, so don't consider this test complete until we explicitly tell the framework it's complete. When the text block fires the Loaded event, we confirm that the data context is not null and that it in fact contains the exact instance of the view model we created in the class initialization. Then we tell the framework the test is complete by calling EnqueueTestComplete, which is provided by the base class.

Finally, if you were to run this without the last line, the test would stall because the text block would never get loaded. We add it as a child of the test surface, and this injects it into the visual tree and fires the loaded event.

The XAML Test

I'm not completely confident with this test because the whole reason for creating a behavior was so I could attach the view model in XAML and not use code behind. Therefore, I should really test attaching this behavior through XAML. So, at the top of the test class we'll create the necessary XAML and wrap it in a UserControl:

const string TESTXAML = 
    "<UserControl " +
        "xmlns=\"\" "  +
        "xmlns:x=\"\" " +
        "xmlns:vm=\"clr-namespace:PRISMMEF.Common.Behavior;assembly=PRISMMEF.Common\">" +
            "<Grid x:Name=\"LayoutRoot\" Background=\"White\" " +
                "vm:ViewModelBehavior.ViewModel=\"PRISMMEF.Tests.Helper.TestViewModel, PRISMMEF.Tests, Version=\">" +
                "<ListBox x:Name=\"ListBox\" ItemsSource=\"{Binding ListOfItems}\"/>" +

If you think the constant is ugly, you can always add an actual XAML file, set it as an embedded resource, then read it in instead. That would give you the full functionality of the editor to tweak the test code. Here, we simply create a control with a grid and a list box. The list box uses the attached behavior and also binds the list.

I want to test the list binding as well, so I add a collection to my test class:

private static readonly List<string> _testCollection = new List<string> { "test1", "test2" };         

In the class initialize method, I'll pass this into the view model's constructor so it is set on the ListOfItems property.

Now, we can create the control from XAML, load it, and test it:

[Description("Test creating from XAML")]
public void TestFromXaml()
    UserControl control = XamlReader.Load(TESTXAML) as UserControl;
    control.Loaded += (o, e) => 
        ListBox listBox = control.FindName("ListBox") as ListBox;

        Assert.IsNotNull(listBox, "ListBox was not found.");
        Assert.IsNotNull(listBox.DataContext, "The data context was never bound.");
        Assert.AreSame(listBox.DataContext, _viewModel, "The data context was not bound to the correct view model.");

        IEnumerable<string> list = listBox.ItemsSource as IEnumerable<string>; 
        List<string> targetList = new List<string>(list); 
        CollectionAssert.AreEquivalent(targetList, _testCollection, "Collection not properly bound."); 



Now we load the control from XAML and wire in the Loaded event to test for the data context and the instance. Then, I take the items from the list box itself and compare them with the original list using CollectionAssert. The AreEquivalent does a set comparison. Then we signal the test is complete.

There's no code for this example because it was very straightforward and I'll likely be posting a more comprehensive example in the future as the result of a talk I'll be giving. Be sure to tune into MSDN's geekSpeak on Wednesday, February 17th, 2010 when I will be the guest to cover exclusively the topic of the Silverlight Unit Testing Framework (the talks are all stored on the site in case you read this after the event).


Jeremy Likness

1 comment:

  1. Jeremy
    I've written a similar unit test to yours for a view I am testing, I have checked that my mockViewModel is populated (and implements INotifyPropertyChanged) and that the binding on the control is correct (I also know that when run for real the everything works), but when I get the value from the ItemsSource, it returns null. I am having similar problems with other bound properties as well.

    public void TestInitialize()

    WorkflowData data = new WorkflowData()
    Progress = new ObservableCollection(new ProgressItem[] {
    new ProgressItem() { Name = "Part 1", State = ProgressItemState.Past },
    new ProgressItem() { Name = "Part 2", State = ProgressItemState.Present },
    new ProgressItem() { Name = "Part 3", State = ProgressItemState.Future }})
    viewModelMock.Model = data;
    view = new CaseProgressView(viewModelMock);

    public void ItemsControlTest()
    Assert.IsTrue(((UserControl)view).DataContext is ICaseProgressViewModel, "Data context has not been set");
    view.Loaded += (s, e) =>
    string bindingPath = "Model.Progress";
    ItemsControl itemsControl = ControlLocationHelper.GetElementByBinding(view, ItemsControl.ItemsSourceProperty, bindingPath);
    Assert.IsNotNull(itemsControl, "Required control cannot be located in the view");
    Assert.IsTrue(itemsControl.DataContext is ICaseProgressViewModel, "Data context has not been set on items control");

    BindingExpression binding = itemsControl.GetBindingExpression(ItemsControl.ItemsSourceProperty);

    var dataItem = binding.DataItem as ICaseProgressViewModel;
    Assert.IsNotNull(dataItem, "Binding incorrect");
    Assert.AreEqual(bindingPath, binding.ParentBinding.Path.Path, "Control Property is bound to wrong ViewModel property");

    ObservableCollection items = itemsControl.ItemsSource as ObservableCollection;
    CollectionAssert.AreEquivalent(viewModelMock.Model.Progress, items, "Bound items not same as expected");
    int expectedCount = viewModelMock.Model.Progress.Count;
    int actualCount = itemsControl.Items.Count;
    Assert.AreEqual(expectedCount, actualCount, "The number of items in the control does not match the number of items in the view model");


    viewModelMock.IsEnabled = true;

    The test fails on the collection assert because ItemsSource is null.

    Any help would be greatly appreciated.