Silverlight UI Automation Testing using Prism 4.0

One popular gripe about Silverlight has been the lack of integrated testing tools. There are several types of tests you may perform against a software project. Unit tests can be performed with the aid of the Silverlight Unit Testing Framework and automated with a third-party tool such as StatLight.

Automation testing involves hosting the actual Silverlight application in a browser and performing a walkthrough based on a script. Typically, this script will follow a "happy path" through the application, but more detailed automation tests check for known error conditions as well. The automation tests simulate entering text, clicking buttons, and scanning results.

Silverlight automation testing is possible, and has been for some time, but it is far easier to do with some helper projects supplied by the Prism team. If you are interested in running actual tests within the Visual Studio IDE, this post will provide step-by-step instructions to get you where you need to be.

Please take the time to go through this step-by-step if you are serious about automation testing. There are lots of small pieces that may seem complex at first, but once you've walked through the steps, it should be fairly straightforward and easy for you to set up new projects and automate the testing to a greater extent than shown here.

Get to Know Your Peers

The automation testing is performed with the help of Automation Peers, a feature that has been around for Silverlight since at least 2.0. Automation peers provide a consistent interface for interacting with controls. They have a very practical use for accessibility (that article shows how long it's been around — so why do so many people ignore it?) While the base controls supplied by Silverlight have their own peers, you'll need to learn how to create your own custom automation peers if you wish to automate the testing of custom controls.

Grab the Latest Prism Drop

Now we need to get Prism. The latest drop as of this writing is Drop 4 but they are releasing fast and furious and should have the full release out by by winter of 2010. (Keep in mind this blog post may quickly become obsolete, as Microsoft is working on a native solution for this).

OK, you've grabbed it and installed it? Great, let's get going. I'm not going to provide a completed project for this because everything you need - all source code and steps - are included in this post.

Create a Simple Project

Let's create a very simple project to get started. What we'll do is create two text boxes and a button. When you enter text in the first box and click the button, it should get updated to the other box. Easy enough, but then we'll automate the tests to ensure the update is happening.

Create a new Silverlight application.

Give it a name UIAutomation and include a solution (check the option to create a directory) UIAutomationSln (or whatever your naming preference is).

Host it in a web application, and make sure the version is Silverlight 4.

Now we'll add a simple set of controls. Set the grid to three columns, then add two text boxes and one button. The XAML looks like this (be sure to key in the Click attribute so it auto-generates the code-behind handler).

   <Grid x:Name="LayoutRoot" Background="White">

        <Grid.ColumnDefinitions>

            <ColumnDefinition/>

            <ColumnDefinition/>

            <ColumnDefinition/>

        </Grid.ColumnDefinitions>

        <TextBox x:Name="txtSource" Grid.Column="0"/>

        <TextBox x:Name="txtTarget" Grid.Column="2"/>

        <Button x:Name="btnSubmit" Content=" GO " Grid.Column="1" Click="btnSubmit_Click"/>

    </Grid>

In the code-behind, handle the click event and move the source text to the target:

using System.Windows;



namespace UIAutomation

{

    public partial class MainPage

    {

        public MainPage()

        {

            InitializeComponent();

        }



        private void btnSubmit_Click(object sender, RoutedEventArgs e)

        {

            txtTarget.Text = txtSource.Text;

        }

    }

}

At this point, you can hit F5 and run it to see it does what we want.

Preparing for Automation

Right now, we need to do one small thing to prepare the controls for automation. Instead of building a custom automation peer, we're going to take advantage of some built-in Silverlight functionality that exposes automation for us. First, we'll simply define some automation-specific identifiers for the controls. Open up the XAML for the main page and add the automation properties you see below:

As you can see, these do not have to be the same as the names of the controls. These identifiers consistently expose the controls to the automation system. Now we're ready to test it!

Add the Prism Acceptance Test Library

We're going to add a library from Prism that will help with the test automation. To do so, you'll need to right-click on the solution, and choose "add ... existing project."

Navigate to the Prism acceptance test library, and select the project file.

Add the Test Project

Now, we'll add a test project. This project is a regular Visual Studio Test Project, not a Silverlight test project. Call the project UIAutomation.Test.

Next, we'll need to add a few references. First, add the reference to the acceptance test library to the newly added test project. Right click on references, choose "add reference" and select the acceptance test library from the "Projects" tab.

Finally, go into the same dialog, but this time add the UI automation references from the .NET tab:

Add your Control Resources

The Prism acceptance test library is designed to be used with projects that share code between Silverlight and WPF. For this reason, a special resource file is used to map between the project types and the automation identifiers for the controls. We'll go ahead and build our own resource dictionary to map the controls. Anything in Silverlight should end with a _Silverlight prefix.

Create a folder called "TestData" in the test project, and add a resource file called "Controls":

In the resource dictionary, fill out the mapping for the control names to the automation identifiers that we added earlier. Notice by convention, we're using the control id, followed by the Silverlight designation, for the key, then putting the automation id as the value.

Because the test class must use this resource file, we want to make sure it gets copied to the output directory. There are two things we'll need to do. I'll cover the second step later. The first step is to make sure the resource is set to "copy always". Simply select the resource file, go into properties, and set the copy attribute:

Add the Application Configuration

The acceptance test library drives from a configuration file that you place in the test project. Right click on the test project and choose "add new item." Select "Application Configuration File" and keep the default, then click "Add." This will add an App.config file to the root of your test project. Open this file up, and paste the following:

<?xml version="1.0" encoding="utf-8" ?>

<configuration>

  <configSections>

    <sectionGroup name="BrowserSupport">

      <section name="Browsers" type="System.Configuration.NameValueSectionHandler, System, Version=1.0.3300.0, Culture=neutral, PublicKeyToken=b77a5c561934e089, Custom=null" />

    </sectionGroup>

  </configSections>



  <appSettings>

   

    <!-- Browser Path and process parameters -->

    <add key="IEPartialPath" value="\\Internet Explorer\\iexplore.exe"/>

    <add key="FirefoxPartialPath" value="\\Mozilla Firefox\firefox.exe"/>

    <add key="SafariPartialPath" value="\\Safari\Safari.exe"/>

    <add key="IEAppProcessName" value="iexplore"/>

    <add key="FirefoxAppProcessName" value="firefox"/>

    <add key="SafariAppProcessName" value="Safari"/>



    <!-- Time to wait for the application to be launched -->

    <add key="ApplicationLoadWaitTime" value="60000"/>



    <!-- Test Data config files -->

    <!--<add key="TestDataInputFile" value=".\TestData\TestDataInput.resx"/>-->

    <add key="ControlIdentifiersFile" value=".\TestData\Controls.resx"/>

  </appSettings>



  <!-- Config section for Cross-Browser support -->

  <BrowserSupport>

    <Browsers>



      <add key ="InternetExplorer" value ="AcceptanceTestLibrary.Common.CrossBrowserSupport.InternetExplorerLauncher" />

      <!--<add key="FireFox" value="AcceptanceTestLibrary.Common.CrossBrowserSupport.FirefoxLauncher" />

      <add key="Safari" value="AcceptanceTestLibrary.Common.CrossBrowserSupport.SafariLauncher" />-->

    </Browsers>

  </BrowserSupport>



</configuration>

Note that we've done the bare minimum to set up an Internet Explorer launch and pointed to the controls resource we created. You can obviously tinker with these settings and include other browsers as part of the test. The Prism example has more in the application settings, such as the path to the application and some other settings, but these are the key ones for the application to work.

Set up a Base Automation Helper

We're not quite ready for the test class. To make it easy to access our automation peers, we can build a base class that exposes the properties we need. This will help us abstract access to the controls for our tests.

In the test project, add a new C# class called MainPageBase.cs. Populate it with the following code:

using System.Windows.Automation;

using AcceptanceTestLibrary.Common;

using AcceptanceTestLibrary.TestEntityBase;



namespace UIAutomation.Test

{

    public static class MainPageBase<TApp>

       where TApp : AppLauncherBase, new()

    {



        public static AutomationElement Window

        {

            get { return PageBase<TApp>.Window; }

            set { PageBase<TApp>.Window = value; }

        }



        public static AutomationElement TextBoxSource

        {

            get { return PageBase<TApp>.FindControlByAutomationId("txtSource"); }

        }



        public static AutomationElement TextBoxTarget

        {

            get { return PageBase<TApp>.FindControlByAutomationId("txtTarget"); }

        }



        public static AutomationElement Button

        {

            get { return PageBase<TApp>.FindControlByAutomationId("btnSubmit"); }

        }

    }

}

Notice we are using some helper classes to find the control. However, what happens is that the context for this application is recognized as Silverlight, so the label we pass (for example, txtTarget) is appended with the Silverlight designation. Ultimately, our control dictionary is accessed with the key txtTarget_Silverlight, which then maps to our automation id of "TargetText" and this is how the automation peer is found. If we had a WPF application that was sharing code, we could simply add a WPF-specific entry and name the automation peer something completely different.

Add the Test Class

OK, now that all of the infrastructure is in place, we can add our test! Go into the automatically generated UnitTest1.cs class. Remember how I told you there were two steps we needed to take in order for the control resource file to be available for testing? This is where we'll make the second step. We're going to add deployment items for the test project as well as the web project. This ensures that both the resources and the test web page are copied to the test sandbox so they are available. Add this code to the top of the class:

namespace UIAutomation.Test

{    

#if DEBUG

    [DeploymentItem(@".\UIAutomation.Test\bin\Debug")]

    [DeploymentItem(@".\UIAutomation.Web", "SL")]

#else

    [DeploymentItem(@".\UIAutomation.Test\bin\Release")]    

    [DeploymentItem(@".\UIAutomation.Web\","SL")]    

#endif

    [TestClass]

    public class UnitTest1

We're instructing the test engine to copy the contents of the test output to the test directory. We also want to create a subdirectory called "SL" and put the output of the web project there. This gives us a path to our test page so we can run the unit tests. We also need to configure the test to use the deployment hints.

Under the main solution, there should be a folder called Solution Items. Double-click on the file Local.testsettings.

Click on the deployment setting, and make sure that Enable Deployment is checked. Once checked, click the Apply button in the lower right corner of the dialog.

Change the class to inherit from the FixtureBase provided by Prism:

[TestClass]    

public class UnitTest1 : FixtureBase<SilverlightAppLauncher>

Set up the using statements to include the namespaces we'll need:

using System.Reflection;

using System.Threading;

using AcceptanceTestLibrary.Common;

using AcceptanceTestLibrary.Common.Silverlight;

using AcceptanceTestLibrary.TestEntityBase;

using AcceptanceTestLibrary.UIAWrapper;

using Microsoft.VisualStudio.TestTools.UnitTesting;

Now, let's add some code to launch the browser and load the Silverlight test page, as well as tear it down when the test is finished. You'll want to tweak the delays to a value that works well for you. Put this at the top of the class:

        private const string APP_PATH = @"\SL\UIAutomationTestPage.html";

        private const string APP_TITLE = "UIAutomation";



        #region Additional test attributes

        // Use TestInitialize to run code before running each test 

        [TestInitialize]

        public void MyTestInitialize()

        {

            var currentOutputPath = (new System.IO.DirectoryInfo(Assembly.GetExecutingAssembly().Location)).Parent.FullName;



            MainPageBase<SilverlightAppLauncher>.Window = LaunchApplication(currentOutputPath + APP_PATH, APP_TITLE)[0];

            Thread.Sleep(5000);

        }



        // Use TestCleanup to run code after each test has run

        [TestCleanup]

        public void MyTestCleanup()

        {

            PageBase<SilverlightAppLauncher>.DisposeWindow();

            SilverlightAppLauncher.UnloadBrowser(APP_TITLE);

        }



        #endregion

We are using the Silverlight application launcher, a helper provided with the Prism project, to launch our test page. Notice that we get the output directory for the test that is running, then append the path to the test page. The test page is underneath the SL subdirectory because that's how we defined it with the deployment item.

Now that we have it launched and ready to tear down, we can write the actual automation test. Here's what we'll do:

1. Simulate typing text into the source text box
2. Confirm that the target text box is blank
3. Simulate clicking the button
4. Confirm that the target text box now has the text we entered into the source text box

Here's how we do it:

[TestMethod]

public void TextBoxSubmission()

{

    const string TESTVALUE = "TestValue";



    // set up the value

    var txtBox = MainPageBase<SilverlightAppLauncher>.TextBoxSource;

    Assert.IsNotNull(txtBox, "Text box is not loaded");

    txtBox.SetValue(TESTVALUE);

    Thread.Sleep(1000);

    Assert.AreEqual(txtBox.GetValue(), TESTVALUE);



    // ensure the text block is empty to start with

    var txtBox2 = MainPageBase<SilverlightAppLauncher>.TextBoxTarget;

    Assert.IsNotNull(txtBox2, "Target text box is not loaded.");

    Assert.IsTrue(string.IsNullOrEmpty(txtBox2.GetValue()), "Target text box is not empty.");



    var btnSubmit = MainPageBase<SilverlightAppLauncher>.Button;

    Assert.IsTrue(btnSubmit.Current.IsEnabled, "Submit Button is not enabled");

    btnSubmit.Click();



    Thread.Sleep(1000);



    var actual = txtBox2.GetValue();



    Assert.AreEqual(TESTVALUE, actual, "Text block was not updated.");



    Thread.Sleep(1000);

}

As you can see, writing the automation tests is relatively straightforward. We don't have "record and playback" but it's easy to grab a control, tell it to do something, and then query the result. Build the project and make sure there are no errors.

Run the Test

Remember what I mentioned earlier about lots of steps? We're there. We've set it up, and should be good to go. If you have issues, go back and check the steps to make sure you didn't miss anything. Again, once you get the hang of it, you'll find it's not that difficult to get up and running and to write some great automation tests. There are good examples in the Prism samples, specifically in the MVVM quick start.

Let's open the test view:

Select the test and click "Debug Selection":

You should eventually see a browser window pop up. The browser may complain about security. If this happens, simply click on the yellow bar and allow the blocked content. Be sure to do this before the launch times out:

You can literally watch the Silverlight application appear in the browser, then see the text entered into the source box, and eventually the button will click and the text should post to the target box. If all goes well, you'll get the familiar green check box:

There you go ... Silverlight UI automation! If you've made it this far, then you have what you need to set this up for your own projects. I've heard of some companies who don't use Silverlight because they are under the impression it doesn't support automated testing. If you're at one of those companies, be sure to go grab your manager, drag them to your cube and show them that it should now be an approved platform for you and your fellow developers!

Using Hints for Generic MEF Exports

It is very common to have base classes and interfaces that use generic types to define standard behaviors across business applications. One challenge with the Managed Extensibility Framework is that it doesn't directly support generic exports and imports: in other words, there is no way to effectively do the following:

...
[ImportMany(AllowRecomposition=true)]
public IService<,>[] GenericService { get; set; }
...

While there is a GenericCatalog you can use, I wanted something a little more flexible and specific to the application.

The Example Service

Let's assume we have an example "service" interface that does something to an instance of a type. We define the interface like this:

public interface IService<T> 
{
   void DoSomething(T instance); 
}

The Locator

The goal is to have a composite application that automatically picks up services that support different types of T and a locator that easily gives us the instance we are looking for. The locator looks like this:

public interface IServiceLocator
{
    IService<T> GetServiceFor<T>();
}

This allows us to easily ask for the service, and do something with it:

var service = serviceLocator.GetServiceFor<MyClass>();
service.DoSomething(myInstance);

The problem with the locator is that we don't have a basic "generic" import for the various services, and we'd have to do a lot of dancing with reflection to parse out types as they became available in order to wire them in. In this case, I felt it was easier to come up with an intermediary class to facilitate finding the service.

Hinting Around

I call it a hint because it hints to the locator where to find the right service. The interface for a service hint looks like this:

public interface IServiceLocatorHint
{
    bool ServicesType<T>();
    IServiceInterface<T> GetServiceFor<T>();
}

As you can see, the hint has two methods. One determines whether or not the hint is capable of producing the service for a given type, and the other returns that service. Now, let's assume in a dynamically loaded module we implement the service contract for MyClass using a class called MyService. It looks like this:

[Export]
public class MyService : IService<MyClass> 
{
   void DoSomething(MyClass instance) 
   {
      Debug.WriteLine(instance.ToString());
   }
}

Notice I am exporting the service as the concrete type. Next, I build a simple hint:

[Export(typeof(IServiceLocatorHint))]
public class MyModuleHints : IServiceLocatorHint
{
    [Import]
    public MyService MyServiceInstance { get; set; }
 
    public bool ServicesType<T>()
    {
        return typeof(T).Equals(typeof(MyClass));
    }

    public IServiceInterface<T> GetServiceFor<T>()
    {
        if (ServicesType<T>())
        {
            return (IServiceInterface<T>) MyServiceInstance;
        }

        throw new NotSupportedException();
    }

}

Putting it all Together

Now that we have the service wired by MEF with all dependencies, and the hint wired as well, we can implement the locator class.

[Export(typeof(IServiceLocator))]
public class ServiceLocator : IServiceLocator
{
    [ImportMany(AllowRecomposition = true)]
    public IServiceLocatorHint[] Hints { get; set; }

    public IServiceInterface<T> GetServiceFor<T>()
    {
        var serviceHint = (from hint in Hints 
                        where hint.ServicesType<T>() 
                        select hint).FirstOrDefault();

        if (serviceHint == null)
        {
            throw new NotSupportedException();
        }

        return serviceHint.GetServiceFor<T>();
    }
}

The class is simple. As modules are loaded, the main list is recomposed to include any new hints that were found. When the user requests a service, it simply finds the hint that satisfies the type, then returns the corresponding service.

Using this flexible locator class is as simple as importing it, then asking for the service:

[Import]
public IServiceLocator Locator { get; set; }

public void ProcessClass(MyClass item)
{
   var service = Locator.GetServiceFor<MyClass>();
   service.DoSomething(item); 
}

If you have multiple services in a module, you can easily build a base class that uses a dictionary to register the instances and streamline the methods that check for support and return the instances. The power of MEF is that new services are easily discovered as plugins and extensions are loaded into the main application, and you can basically build an application around what you don't know yet, rather than having to constrain it based upon what you do know.

Sterling OODB v0.1 Alpha Released!

I'm please to share that I released the alpha version of the Sterling Object-Oriented Database for Silverlight 4.0 and Windows Phone 7.0 applications today.

I dug into more details about the project in this introductory post. There have not been many significant changes to the source code since then, but I did create a set up project that installs the DLLs and configures the registry keys so they appear in the "add reference" dialog on the .NET tab for both Silverlight and Windows Phone 7 applications.

Please take a moment to read and participate in the release discussion that you can find by clicking here. You can grab the installer and the source code at this link.

Our team is really some part time help and me and we all have full time careers so it's tough to give the project the attention and detail it deserves. I learned in the past that sometimes it's more important to "get it going" than to "get it perfect." Instead of holding back months before formalizing the release, I wanted to get it out there so people can use it, comment on it, and provide feedback.

You can help our team in two very important ways. First, if you use the tool, please provide feedback, both positive and constructive, so we know where/when/how the project is being used and how to prioritize future improvements.

Second, if you are a developer and feel you can contribute your skills, please contact me and we can discuss you joining the team. Probably the two biggest needs on the team right now are someone who has the time to build out a more thorough reference application for the Windows Phone 7 version, and someone who is an installation wizard to help improve the install process. The documentation is also rudimentary so technical writers are welcome as well.

Thanks, and I look forward to your feedback!

Silverlight Profiling Part 2: The Easy Way

After posting about profiling Silverlight 4 out of the box, the author of a tool was kind enough to share with me his free .NET profiling tool. It's free, and it does a lot more than Silverlight. The manual is very straightforward but I wanted to make a quick an easy "light tour" to show you how I profiled the X-Fit app using the tool. In less than 4 minutes I'll run two different profiles, a trace and a memory scan. The memory scan lets you see individual classes, number of objects, size, etc, so the video doesn't tap into the full features, but should show how fast and easy it makes profiling.

Sorry, this was quick and dirty over lunch so I didn't encode it to Silverlight - Jing and ScreenToaster came through (link to new window):

Performance Profiling Silverlight 4 Step-by-Step

It turns out you can profile Silverlight 4 applications. Really! But the steps are scattered about the web in bits and pieces, so I wrote this to pull them all together. I'm going to use my reference X-Fit application for this, but you can obviously insert your own. Here we go!

Set up PowerShell

Yeah, we're going to make things a little easier and set up PowerShell. If you're not familiar with PowerShell, don't worry. Here is a decent description of PowerShell, but if you really want to get deep into it, read Wintellectual John Robbin's blog which is full of goodness. Really the main thing you want to do is to enable script support in PowerShell, and get familiar with running scripts.

Set up your Symbol Server

We're going to need a valid symbol server to make things work. If you're not familiar with this, it's basically a place that Windows debug can go to figure out symbols (labels, names, etc) for code as you are debugging, profiling, etc. John Robbins has a fantastic script that you can use to set this up.

1. Get John Robbins' script here (just copy it and save it to a file called "symbolsetup.ps1" and if I have to tell you without the quotes, you're not ready to profile)
2. Execute the script in PowerShell. You must specify the full path and add a flag. I do it like this: & "c:\users\jeremylikness\my documents\powershell\symbolsetup.ps1" -public
3. This should set it all up for you. Beware! The first time you debug, you will have quite a delay as all of the symbols are downloaded - this only happens once.

Prepare your Application

To profile, you must compile the application in release mode. Yes, you read that correctly. The symbol server will provide all of the symbols and details necessary, but you must compile in release mode for this to work.

In our example, we'll pull down the Reference X-Fit Application. Download it, unzip it, and build it in release mode.

While you can debug most applications from a test page, this application uses dynamic modules so it is best to publish it to a site. You can use your local development environment, but it's easier if you just make an application in IIS. I'll leave that to you, but when we profile, the parameter we'll pass to launch it will be some type of URL.

Let's Profile!

First, credits go to the Windows Profiler Team for their detailed post for this. It contains what I'm going to share here, but I was missing some of the setup.

Go to your start menu, navigate to Visual Studio 2010, Tools, and select the command prompt as an Administrator.

1. Navigate to the release directory for the main XAP file. In this case, it is under XFit/Bin/Release for the main XAP in X-Fit. If you have dynamic modules, you can copy the DLL and PDB files to this directory.
2. Open Task Manager and sort by image name. Note the PIDs of the existing iexplore.exe processes.
3. Set up the profiling environment:
   vsperfclrenv /sampleon
4. Launch a new process, passing the page you want to debug:
   "c:\program files (x86)\internet explorer\iexplore.exe" http://localhost/xfit/
5. Note the new IE process (Hint: you can also go into Visual Studio and choose "attach to process" to see the title/type of process to narrow down the correct one that is hosting the Silverlight application — don't actually attach, just use this to get the process identifier)
6. Start profiling the process (in this case, my PID was 4436):
   VSPerfCmd /start:sample /output:xfitdata /attach:4436
7. Now go into the application and perform some actions. I created an account, then looked at my BMI, then updated my account and put in some errors and hovered over some information icons just to get a good sample
8. End the session:
   VSPerfCmd /detach
VSPerfCmd /shutdown
VSPerfClrEnv /off

Examine the Data

Once I shut everything down, a new file was created called xfitdata.vsp. Now the fun part: analyzing the data. In Visual Studio 2010, you can go to File -> Open File and navigate to the VSP file to open it. You'll see a message that it is analyzing the file. This may take several minutes, so be patient. If you have your output window open, you should see messages as the symbols are analyzed:

Probably the easiest way to filter the data is to go to the module view:

Then you can expand and see where the application is spending it's time:

If you double-click on the higher level item, a window will open with the details including (if available) the source code.

There you have it ... step-by-step profiling Silverlight 4.

What YOU Are Interested In

I was very excited to see two of my blog posts featured in the 900th issue of Silverlight Cream which ranked the top articles of 2010 thus far. These were Top 10 Silverlight Myths and Facts to Bust Them (#4) and Simple Dialog Service in Silverlight (#17).

I appreciate your interest and out of curiosity decided to examine my own blog to see what trends look like for the past year. What posts are you looking at the most? And where do you spend most of your time? Here's what I found:

Last 6 Months

1. Model-View-ViewModel (MVVM) Explained
2. MVVM with MEF in Silverlight Video Tutorial
3. Dynamic Module Loading with Silverlight Navigation using PRISM

Last 3 Months

1. Model-View-ViewModel (MVVM) Explained
2. MVVM with MEF in Silverlight Video Tutorial
3. PRISM, MEF, and MVVM Part 1 of 3: Unity Glue

Last Month

1. Model-View-ViewModel (MVVM) Explained
2. Introducing Sterling, the OODB for Silverlight and Windows Phone 7
3. Tips and Tricks for INotifyPropertyChanged

I'm assuming you actually read the posts because you spent an average of 4 - 5 minutes on each of the top ones.

What does this tell me? It shows there is a major interest in dynamic modules, in MEF, in local storage for Silverlight, and of course the Model-View-ViewModel pattern. This is good, because these are topics I consistently write about and speak about at public events. However, one other thing I've found is just as important.

We need more quick starts, easy step-by-step tutorials, and guidance for these patterns!

With the few talks I've done that provided feedback from the audience, the lower ratings are always tied to either "too much content" or "too advanced." I appreciate that feedback and it tells me you want more simple, 1-2-3 style posts and snippets of information. I also know my future talks should focus on the fundamentals and not dive into advanced topics unless I know for a fact the audience is going to have experience coming into it.

I appreciate all of your feedback, both direct and indirect. Can you help out even more with this post? I'd love to hear what blog posts, articles, video tutorials, and talks you would like to see but feel haven't been addressed. Use the comments below to share with me what you'd like to see more of, and how you'd like to see it. It will only help me improve the content that I share here and at users groups.

Thanks!

Introducing Sterling, the Object-Oriented Database for Silverlight and Windows Phone 7

Today, I'm excited to share a project that I've been working on for some time now. The project, called Sterling (for Sterling Silverlight, of course!) is a very light weight approach to handling serialization and deserialization to and from isolated storage. I call it an object-oriented database because it provides LINQ to Object queries over keys and indexes.

I guess the timing works well ... I was recently honored with the 2010 Microsoft Most Valuable Professional (MVP) award for my work with Silverlight. Much of this award is about what we do for the community, and this is one of my first major efforts to put a utility and open source project out there. Lots of hours went into this but I hope there will be great benefits for those of you who find value in using it.

Sterling is not yet released and I don't anticipate we'll have an alpha ready until late July. I work on many projects for my company, so this effort has been primarily late nights or snatches during lunch and breaks and will probably continue to be so. I'm excited that some members of the community have offered to reach out and help push development forward.

First, let me emphasize that Sterling is not intended as a replacement for a robust, transactional database system. There are some commercial and non-commercial solutions that are optimized for performance and massive scale. I wrote Sterling to be a simple serialization engine to avoid having to deal with the goop of writing my own serializers all of the time. I also know that even if you have a simple list of contacts, querying the contact names is going to be a priority over deserializing all of them, so I built a key and index infrastructure to facilitate storing key items in memory so you can bind to lists and combo-boxes without de-serializing the entire object.

Having said all of this, while a lot of more powerful features may not exist, my goal was to hit 80% of the needs with 20% of the footprint. Not only is Sterling incredibly lightweight, but extensible and available on Windows Phone 7.

Here are the key goals I had in mind:

Non-intrusive

I don't like mangling my classes to persist them. I didn't want to force anyone to inherit from a base class or decorate the class with attributes to make it work. I believe I've achieved this goal. Defining a "table" to sterling is as simple as passing a type and a lambda expression that returns the key:

public override List<ITableDefinition> _RegisterTables()
{
 return new List<ITableDefinition>
                       {
                           CreateTableDefinition<Contact,string>(c => c.Email)
                       }
}

I can even do this for sealed classes or third-party classes - whatever your code can see, Sterling can, too. Sterling automatically serializes the same values that the BinaryWriter class supports. If you need something custom, no problem - just define a serializer of your own and register it with Sterling.

Lightweight

I wanted Sterling to fit on the Windows Phone 7 and also to facilitate building projects without bloating them. I purposefully kept it lightweight. Currently the DLL weighs in at only 70 kilobytes, which I believe is very trivial compared to some solutions I've seen.

Flexible and Portable

Again, these are both features that collapse into the previous items. By keeping it flexible, I can accommodate needs I didn't know at design time. There is a very loose interface for logging and for extending serialization. This allows you to do pretty much whatever you like, without having to think about the underpinnings of setting up tables and folders and checking if they exist, etc. Portability means it was easy to build on Windows Phone 7 and should be very easy to bring forward into future versions of Silverlight.

The Reference Application

The easiest way to get to know Sterling is by the reference application. I will warn you that it takes a long time to build because of the time it takes to initially serialize. I decided a good test case would be the USRDA nutrient database. It contains over 500,000 data elements. I had to write some parsers to take the source text-based database files and turn them into data objects that Sterling could deal with, but once the conversion is done, you can see the power of how Sterling operates.

The left column shows a custom logger I made that is spitting out Sterling log information. The data models were straightforward. A food group is a general "category" for foods to fall under. The food description is an actual food item, and contains a list of nutrient data elements (things like calories, protein, vitamin A, etc). These point to a nutrient definition.

public class FoodGroup
{
    public int Id { get; set; }
    public string GroupName { get; set; }
}

public class NutrientDefinition
{
    public int Id { get; set; }

    public string UnitOfMeasure { get; set; }

    public string Tag { get; set; }

    public string Description { get; set; }

    public int SortOrder { get; set; }
}

public struct NutrientDataElement
{
    public int NutrientDefinitionId { get; set; }
    public double AmountPerHundredGrams { get; set; }
}

public class FoodDescription
{
    public FoodDescription()
    {
        Nutrients = new List();
    }

    public int Id { get; set; }

    public int FoodGroupId { get; set; }

    public string Description { get; set; }

    public string Abbreviated { get; set; }

    public string CommonName { get; set; }

    public string Manufacturer { get; set; }

    public string InedibleParts { get; set; }

    public double PctRefuse { get; set; }

    public string ScientificName { get; set; }

    public double NitrogenFactor { get; set; }

    public double ProteinCalories { get; set; }

    public double FatCalories { get; set; }

    public double CarbohydrateCalories { get; set; }

    public List&t;NutrientDataElement> Nutrients { get; set; }
}

Because I have a struct for the nutrient information, I had to provide a custom serializer:

public class FoodSerializer : BaseSerializer  
{
    public override bool CanSerialize(Type targetType)
    {
        return targetType.Equals(typeof (NutrientDataElement));                
    }

    public override void Serialize(object target, BinaryWriter writer)
    {
        var data = (NutrientDataElement)target;
        writer.Write(data.NutrientDefinitionId);
        writer.Write(data.AmountPerHundredGrams);
    }

    public override object Deserialize(Type type, BinaryReader reader)
    {
        return new NutrientDataElement
                    {
                        NutrientDefinitionId = reader.ReadInt32(),
                        AmountPerHundredGrams = reader.ReadDouble()
                    };
    }
}

As you can see, fast and easy to do. Defining the tables took a little bit of thought. I wanted a "covered index" for food groups so I wouldn't have to de-serialize them at all. For the food descriptions, I needed an index on description and food group for fast filtering and sorting. Finally, the nutrient definitions provided an index for unit of measure and sort order (so they sort consistently in each food item).

public class FoodDatabase : BaseDatabaseInstance
{
    public override string Name
    {
        get { return "Type Database"; }
    }

    public const string FOOD_GROUP_NAME = "FoodGroup_GroupName";
    public const string FOOD_DESCRIPTION_DESC_GROUP = "FoodDescription_Description_Group";
    public const string NUTR_DEFINITION_UNITS_DESC = "NutrientDefinition_Units_Description";
    public const string NUTR_DEFINITION_SORT = "NutrientDefinition_Sort";
        
    protected override List<ITableDefinition> _RegisterTables()
    {
        return new List<ITableDefinition>
                    {
                        CreateTableDefinition<FoodGroup, int>(fg => fg.Id)
                            .WithIndex<FoodGroup, string, int>(FOOD_GROUP_NAME, fg => fg.GroupName),
                        CreateTableDefinition<FoodDescription, int>(fd => fd.Id)
                            .WithIndex<FoodDescription, string, int, int>(FOOD_DESCRIPTION_DESC_GROUP,
                                                                            fd =>
                                                                            Tuple.Create(fd.Description, fd.FoodGroupId)),
                        CreateTableDefinition<NutrientDefinition,int>(nd=>nd.Id)
                            .WithIndex<NutrientDefinition,string,string,int>(NUTR_DEFINITION_UNITS_DESC,
                            nd=>Tuple.Create(nd.UnitOfMeasure,nd.Description))
                            .WithIndex<NutrientDefinition,int,int>(NUTR_DEFINITION_SORT,
                            nd=>nd.SortOrder)
                    };
    }
}

The main view model allows you to select groups and enter search terms, displays the food items, and then will show a chart breaking out nutrition information when you click on the food item. The food group list is queried using the index like this:

public IEnumerable<FoodGroup> FoodGroups
{
    get
    {
        return DesignerProperties.IsInDesignTool
                    ? _samples.AsEnumerable()
                    : from fg in
                            SterlingService.Current.Database.Query<FoodGroup, string, int>(
                                FoodDatabase.FOOD_GROUP_NAME)
                        select new FoodGroup {Id = fg.Key, GroupName = fg.Index};
    }
}

Note that I have a default list for design-time, otherwise I run the actual query.

When you hover over a food group, I supply a tool-tip to show the number of food items in that group. I decided to go ahead and store these in a dictionary after querying the first time, but only because I know they won't change. The converter looks like this:

private readonly Dictionary<int,int> _foodCounts = new Dictionary<int,int>();

public object Convert(object value, Type targetType, object parameter, CultureInfo culture)
{
    var count = DesignerProperties.IsInDesignTool ? 500 : 0;
            
    var foodGroup = value as FoodGroup;
    if (foodGroup != null && !DesignerProperties.IsInDesignTool)
    {
        if (_foodCounts.ContainsKey(foodGroup.Id))
        {
            count = _foodCounts[foodGroup.Id];
        }
        else
        {
            count =
                (from index in
                        SterlingService.Current.Database.Query<FoodDescription, string, int, int>(
                            FoodDatabase.FOOD_DESCRIPTION_DESC_GROUP)
                    where index.Index.Item2.Equals(foodGroup.Id)
                    select index).Count();
            _foodCounts.Add(foodGroup.Id,count);
        }
    }

    return string.Format("There are {0} food items in this food group.", count);
}

Here you can see I'm providing a design-time default. If I'm not in the designer, then I go ahead and calculate the amount and save it using the index I created. The index has two values (description and food group key) so I use a Tuple to access the value. None of this requires any de-serialization because I'm only touching the index.

The search view model uses the same index to pull food items. The query is a bit more complex. I'm not allowing you to search all 10,000 food items. You must narrow it to a category or enter at least three characters of a search (the search is a containing search, not a "starts with" or "ends with").

public IEnumerable<FoodDescriptionIndex> SearchResults
{
    get
    {
        if (DesignerProperties.IsInDesignTool)
            return _sampleDescriptions.AsEnumerable();

        if (_currentGroup != null)
        {
            if (string.IsNullOrEmpty(_searchText) || _searchText.Length < 3)
            {
                var query1 = from fg in
                                    SterlingService.Current.Database.Query
                                    <FoodDescription, string, int, int>(
                                        FoodDatabase.FOOD_DESCRIPTION_DESC_GROUP)
                                where
                                    fg.Index.Item2.Equals(_currentGroup.Id)
                                select
                                    new FoodDescriptionIndex {Id = fg.Key, Description = fg.Index.Item1};

                return query1.Count() == 0 ? _noResults.AsEnumerable() : query1;
            }

            // group and search text)
            var query2 = from fg in
                        SterlingService.Current.Database.Query
                        <FoodDescription, string, int, int>(
                            FoodDatabase.FOOD_DESCRIPTION_DESC_GROUP)
                    where
                        fg.Index.Item2.Equals(_currentGroup.Id) &&
                        fg.Index.Item1.ToUpperInvariant().Contains(_searchText.ToUpperInvariant())
                    select
                        new FoodDescriptionIndex {Id = fg.Key, Description = fg.Index.Item1};
                        
            return query2.Count() == 0 ? _noResults.AsEnumerable() : query2;
        }

        if (string.IsNullOrEmpty(_searchText) || _searchText.Length < 3)
            return _noResults.AsEnumerable();

        var query3 = from fg in
                    SterlingService.Current.Database.Query
                    <FoodDescription, string, int, int>(
                        FoodDatabase.FOOD_DESCRIPTION_DESC_GROUP)
                where
                    fg.Index.Item1.ToUpperInvariant().Contains(_searchText.ToUpperInvariant())
                select
                    new FoodDescriptionIndex { Id = fg.Key, Description = fg.Index.Item1 };
        return query3.Count() == 0 ? _noResults.AsEnumerable() : query3; 
    }
}

Notice how I'm using a "covered index" (covered for the items I need, which include the key and description) to query and return a list of types that are bound to the list box. You can see when you run the example this happens very fast. Finally, when you click on the item, I am always deserializing rather than trying to cache a ton of objects.

The food description context class synchronizes the current food item between view models. Notice when it is passed a value, it loads the new item:

public class FoodDescriptionContext : BaseNotify 
{
    public static FoodDescriptionContext Current = new FoodDescriptionContext();

    public FoodDescription CurrentFoodDescription { get; private set; }

    private int _foodDescriptionId; 

    public int FoodDescriptionId
    {
        get { return _foodDescriptionId; }
        set
        {
            _foodDescriptionId = value;
            CurrentFoodDescription = SterlingService.Current.Database.Load<FoodDescription>(value);
            RaisePropertyChanged(()=>CurrentFoodDescription);
            RaisePropertyChanged(()=>FoodDescriptionId);
        }
    }

}

This isn't thread-safe, but does it need to be? The user can only click on one item at a time.

Finally, the whole engine is configured using an application service.

There's a lot more to it but I wanted to get this out there and have people start looking at it to provide me with feedback. While there is not yet a release, you can visit the Sterling Codeplex site to download the code (pre-alpha, so use at your own risk, right?) and build/test/integrate on your own. Let me know what you like and don't like and if you are interested in being a serious beta tester, and we'll see what we can do to release a solid version 1.

(PS, I took a simple list project for the Windows Phone 7 just to prove the engine works there - I simply save the list to the database then bind to the query - but I'm looking for a more comprehensive example there, so volunteers are welcome to work on that as well!)

Customer IApplicationService for Silverlight Applications

In my experience working with Silverlight applications, probably one of the most underused features I've witnessed is the ability to abstract application-level functionality using the IApplicationService and IApplicationLifetimeAware interfaces. This, in turn, results in the over-use (and abuse) of the Startup and Exit events in the Application object. Before you get angry with me shaking a finger, I'll admit I've done this quite a bit myself.

IApplicationService

The IApplicationService interface allows you to define a class that is part of the overall application lifecycle. It provides a mechanism to create a service that lasts the duration of a Silverlight application. It is called once when the application starts with an ApplicationServiceContext, and called again when the application ends.

One of the most useful things you can do with this service class is use it to intelligently parse parameters passed to the Silverlight application. How many times have you found yourself hooking into the application startup and then parsing these out? What's worse, in larger, complex applications, many different portions of the application share responsibility for handling these parameters. It breaks the concept of loose coupling to have a single front-end parsing it out and then stuffing it into an object to dole it out to the consumers that need it.

By implementing IApplicationService, you can create a service for your functionality that handles its own initialization parameters. There is no "central" place that has to understand everything. Silverlight supports using multiple application services, so you can have a logger service that looks at the logging parameter and a file upload service that looks at the file storage location parameter, so on and so forth.

Here's what a service that handles writing to the debugger might look like:

public class LoggerService : IApplicationService
{
    const string TRACE_LEVEL_KEY = "TraceLevel";
            
    public LoggerService()
    {
        _traceLevel = TraceLevel.Warning; // default
    }
   
    private TraceLevel _traceLevel; 

    public ILogger Logger { get; private set; }
        
    public static LoggerService Current { get; private set; }

    public void StartService(ApplicationServiceContext context)
    {
        Current = this;
        if (context.ApplicationInitParams.ContainsKey(TRACE_LEVEL_KEY))
        {
            _traceLevel = (TraceLevel)Enum.Parse(typeof (TraceLevel), context.ApplicationInitParams[TRACE_LEVEL_KEY], true);
        }

        Logger = new CustomLogger(TraceLevel);
        Logger.WriteLine(TraceLevel.Information, "Logger service started.");
    }

    public void StopService()
    {
        Logger.WriteLine(TraceLevel.Information, "Logger service stopped.");
    }
}

This service now allows me to configure the logging level in the web page using initParams, like this:

<object data="data:application/x-silverlight-2," type="application/x-silverlight-2" width="100%" height="100%">
   <param name="source" value="ClientBin/MySilverlightApp.xap"/>
   <param name="onError" value="onSilverlightError" />
   <param name="initParams" value="TraceLevel=Warning" />
   <param name="background" value="white" />
   <param name="minRuntimeVersion" value="4.0.50401.0" />
   <param name="autoUpgrade" value="true" />    
</object>

What's more, if it is in a project by itself, adding the service to a new application is so simple it's hard to believe more people don't take advantage of this. After referencing the project, I won't have to mangle my App.xaml.cs and hook into any events in order to use the new service, parse out parameters, etc. Nope, I just add a simple reference to it in the App.xaml that looks like this:

<Application.ApplicationLifetimeObjects>
    <MySilverlightApp:LoggerService/>
</Application.ApplicationLifetimeObjects>

IApplicationLifetimeAware

Now, if you really want to do some cool things, you can extend the class even further by implementing IApplicationLifetimeAware. This will give you four additional methods: Starting, Started, Exiting, and Exiting.

Use these to perform set up and clean up actions.

For example, in the Sterling Windows Phone 7 Example I wrote (Sterling is an open source object-oriented database for Silverlight 4.0 and Windows Phone 7) I created a database service to make it easy to set up and tear down the database engine. The service looks like this:

public class DatabaseService : IApplicationService, IApplicationLifetimeAware
{
    public static DatabaseService Current { get; private set; }

    public ISterlingDatabaseInstance Database { get; private set; }

    private SterlingEngine _engine;
        
    public void StartService(ApplicationServiceContext context)
    {
            
        Current = this;
        _engine = new SterlingEngine();
    }

    public void StopService()
    {
        _engine = null;
    }

    public void Starting()
    {
        _engine.Activate();
        Database = _engine.SterlingDatabase.RegisterDatabase<PhoneDatabase>();
    }

    public void Started()
    {
        return;
    }

    public void Exiting()
    {
        _engine.Dispose();
    }

    public void Exited()
    {
        return;
    }
}

As you can see, I am managing everything that would normally get hooked into the application startup and exit events. Instead of adding to that code-behind, however, I can abstract these hooks in a re-usable component that is easily dropped into or out of the App.xaml as needed. Just like the previous example, the only thing needed to hook into the database in a Windows Phone 7 application is now just this:

<Application.ApplicationLifetimeObjects>
   <SterlingExample:SterlingService/>
</Application.ApplicationLifetimeObjects>

The Silverlight framework guarantees my service is called to start only once, so I take advantage of that call to assign a static property. That way, anywhere in the application I need to reference the service, I can use ServiceName.Current (for example, SterlingService.Current).

Don't limit yourself to just that, however. One common task in composite Silverlight applications is "bootstrapping" objects. If you are using the Managed Extensibility Framework (MEF), you may need to configure some catalogs and then call the CompositionInitializer. You can do this in an application service as well. In fact, many of my applications have nothing in the App.xaml.cs code behind except a call to InitializeComponent. Why? Because I can set the root visual in a service, like this:

[Import]
public Shell MainShell { get; set; }

public void StartService(ApplicationServiceContext context)
{
    _myCatalog = new AggregateCatalog(new DeploymentCatalog()); 
    var container = new CompositionContainer(_mainCatalog);
    CompositionHost.Initialize(container);              
    CompositionInitializer.SatisfyImports(this);
}

public void Starting()
{
    Application.Current.RootVisual = MainShell;
}

When called, the service sets up the catalogs/containers/etc. and satisfies imports to generate the shell. If the shell has any dependencies, these are also handled by MEF. Then, in the starting method, we wire the shell to the root visual because this is called just before the full visual tree is loaded.

I hope to see this powerful feature used more often to add some quality customer service to Silverlight applications.