One popular gripe about Silverlight has been the lack of integrated testing tools. There are several types of tests you may perform against a software project. Unit tests can be performed with the aid of the Silverlight Unit Testing Framework and automated with a third-party tool such as StatLight.
Automation testing involves hosting the actual Silverlight application in a browser and performing a walkthrough based on a script. Typically, this script will follow a "happy path" through the application, but more detailed automation tests check for known error conditions as well. The automation tests simulate entering text, clicking buttons, and scanning results.
Silverlight automation testing is possible, and has been for some time, but it is far easier to do with some helper projects supplied by the Prism team. If you are interested in running actual tests within the Visual Studio IDE, this post will provide step-by-step instructions to get you where you need to be.
Please take the time to go through this step-by-step if you are serious about automation testing. There are lots of small pieces that may seem complex at first, but once you've walked through the steps, it should be fairly straightforward and easy for you to set up new projects and automate the testing to a greater extent than shown here.
Get to Know Your Peers
The automation testing is performed with the help of Automation Peers, a feature that has been around for Silverlight since at least 2.0. Automation peers provide a consistent interface for interacting with controls. They have a very practical use for accessibility (that article shows how long it's been around — so why do so many people ignore it?) While the base controls supplied by Silverlight have their own peers, you'll need to learn how to create your own custom automation peers if you wish to automate the testing of custom controls.
Grab the Latest Prism Drop
Now we need to get Prism. The latest drop as of this writing is Drop 4 but they are releasing fast and furious and should have the full release out by by winter of 2010. (Keep in mind this blog post may quickly become obsolete, as Microsoft is working on a native solution for this).
OK, you've grabbed it and installed it? Great, let's get going. I'm not going to provide a completed project for this because everything you need - all source code and steps - are included in this post.
Create a Simple Project
Let's create a very simple project to get started. What we'll do is create two text boxes and a button. When you enter text in the first box and click the button, it should get updated to the other box. Easy enough, but then we'll automate the tests to ensure the update is happening.
Create a new Silverlight application.
Give it a name UIAutomation and include a solution (check the option to create a directory) UIAutomationSln (or whatever your naming preference is).
Host it in a web application, and make sure the version is Silverlight 4.
Now we'll add a simple set of controls. Set the grid to three columns, then add two text boxes and one button. The XAML looks like this (be sure to key in the Click attribute so it auto-generates the code-behind handler).
<Grid x:Name="LayoutRoot" Background="White">
<Grid.ColumnDefinitions>
<ColumnDefinition/>
<ColumnDefinition/>
<ColumnDefinition/>
</Grid.ColumnDefinitions>
<TextBox x:Name="txtSource" Grid.Column="0"/>
<TextBox x:Name="txtTarget" Grid.Column="2"/>
<Button x:Name="btnSubmit" Content=" GO " Grid.Column="1" Click="btnSubmit_Click"/>
</Grid>
In the code-behind, handle the click event and move the source text to the target:
using System.Windows;
namespace UIAutomation
{
public partial class MainPage
{
public MainPage()
{
InitializeComponent();
}
private void btnSubmit_Click(object sender, RoutedEventArgs e)
{
txtTarget.Text = txtSource.Text;
}
}
}
At this point, you can hit F5 and run it to see it does what we want.
Preparing for Automation
Right now, we need to do one small thing to prepare the controls for automation. Instead of building a custom automation peer, we're going to take advantage of some built-in Silverlight functionality that exposes automation for us. First, we'll simply define some automation-specific identifiers for the controls. Open up the XAML for the main page and add the automation properties you see below:
As you can see, these do not have to be the same as the names of the controls. These identifiers consistently expose the controls to the automation system. Now we're ready to test it!
Add the Prism Acceptance Test Library
We're going to add a library from Prism that will help with the test automation. To do so, you'll need to right-click on the solution, and choose "add ... existing project."
Navigate to the Prism acceptance test library, and select the project file.
Add the Test Project
Now, we'll add a test project. This project is a regular Visual Studio Test Project, not a Silverlight test project. Call the project UIAutomation.Test.
Next, we'll need to add a few references. First, add the reference to the acceptance test library to the newly added test project. Right click on references, choose "add reference" and select the acceptance test library from the "Projects" tab.
Finally, go into the same dialog, but this time add the UI automation references from the .NET tab:
Add your Control Resources
The Prism acceptance test library is designed to be used with projects that share code between Silverlight and WPF. For this reason, a special resource file is used to map between the project types and the automation identifiers for the controls. We'll go ahead and build our own resource dictionary to map the controls. Anything in Silverlight should end with a _Silverlight prefix.
Create a folder called "TestData" in the test project, and add a resource file called "Controls":
In the resource dictionary, fill out the mapping for the control names to the automation identifiers that we added earlier. Notice by convention, we're using the control id, followed by the Silverlight designation, for the key, then putting the automation id as the value.
Because the test class must use this resource file, we want to make sure it gets copied to the output directory. There are two things we'll need to do. I'll cover the second step later. The first step is to make sure the resource is set to "copy always". Simply select the resource file, go into properties, and set the copy attribute:
Add the Application Configuration
The acceptance test library drives from a configuration file that you place in the test project. Right click on the test project and choose "add new item." Select "Application Configuration File" and keep the default, then click "Add." This will add an App.config file to the root of your test project. Open this file up, and paste the following:
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<configSections>
<sectionGroup name="BrowserSupport">
<section name="Browsers" type="System.Configuration.NameValueSectionHandler, System, Version=1.0.3300.0, Culture=neutral, PublicKeyToken=b77a5c561934e089, Custom=null" />
</sectionGroup>
</configSections>
<appSettings>
<!-- Browser Path and process parameters -->
<add key="IEPartialPath" value="\\Internet Explorer\\iexplore.exe"/>
<add key="FirefoxPartialPath" value="\\Mozilla Firefox\firefox.exe"/>
<add key="SafariPartialPath" value="\\Safari\Safari.exe"/>
<add key="IEAppProcessName" value="iexplore"/>
<add key="FirefoxAppProcessName" value="firefox"/>
<add key="SafariAppProcessName" value="Safari"/>
<!-- Time to wait for the application to be launched -->
<add key="ApplicationLoadWaitTime" value="60000"/>
<!-- Test Data config files -->
<!--<add key="TestDataInputFile" value=".\TestData\TestDataInput.resx"/>-->
<add key="ControlIdentifiersFile" value=".\TestData\Controls.resx"/>
</appSettings>
<!-- Config section for Cross-Browser support -->
<BrowserSupport>
<Browsers>
<add key ="InternetExplorer" value ="AcceptanceTestLibrary.Common.CrossBrowserSupport.InternetExplorerLauncher" />
<!--<add key="FireFox" value="AcceptanceTestLibrary.Common.CrossBrowserSupport.FirefoxLauncher" />
<add key="Safari" value="AcceptanceTestLibrary.Common.CrossBrowserSupport.SafariLauncher" />-->
</Browsers>
</BrowserSupport>
</configuration>
Note that we've done the bare minimum to set up an Internet Explorer launch and pointed to the controls resource we created. You can obviously tinker with these settings and include other browsers as part of the test. The Prism example has more in the application settings, such as the path to the application and some other settings, but these are the key ones for the application to work.
Set up a Base Automation Helper
We're not quite ready for the test class. To make it easy to access our automation peers, we can build a base class that exposes the properties we need. This will help us abstract access to the controls for our tests.
In the test project, add a new C# class called MainPageBase.cs. Populate it with the following code:
using System.Windows.Automation;
using AcceptanceTestLibrary.Common;
using AcceptanceTestLibrary.TestEntityBase;
namespace UIAutomation.Test
{
public static class MainPageBase<TApp>
where TApp : AppLauncherBase, new()
{
public static AutomationElement Window
{
get { return PageBase<TApp>.Window; }
set { PageBase<TApp>.Window = value; }
}
public static AutomationElement TextBoxSource
{
get { return PageBase<TApp>.FindControlByAutomationId("txtSource"); }
}
public static AutomationElement TextBoxTarget
{
get { return PageBase<TApp>.FindControlByAutomationId("txtTarget"); }
}
public static AutomationElement Button
{
get { return PageBase<TApp>.FindControlByAutomationId("btnSubmit"); }
}
}
}
Notice we are using some helper classes to find the control. However, what happens is that the context for this application is recognized as Silverlight, so the label we pass (for example, txtTarget) is appended with the Silverlight designation. Ultimately, our control dictionary is accessed with the key txtTarget_Silverlight, which then maps to our automation id of "TargetText" and this is how the automation peer is found. If we had a WPF application that was sharing code, we could simply add a WPF-specific entry and name the automation peer something completely different.
Add the Test Class
OK, now that all of the infrastructure is in place, we can add our test! Go into the automatically generated UnitTest1.cs class. Remember how I told you there were two steps we needed to take in order for the control resource file to be available for testing? This is where we'll make the second step. We're going to add deployment items for the test project as well as the web project. This ensures that both the resources and the test web page are copied to the test sandbox so they are available. Add this code to the top of the class:
namespace UIAutomation.Test
{
#if DEBUG
[DeploymentItem(@".\UIAutomation.Test\bin\Debug")]
[DeploymentItem(@".\UIAutomation.Web", "SL")]
#else
[DeploymentItem(@".\UIAutomation.Test\bin\Release")]
[DeploymentItem(@".\UIAutomation.Web\","SL")]
#endif
[TestClass]
public class UnitTest1
We're instructing the test engine to copy the contents of the test output to the test directory. We also want to create a subdirectory called "SL" and put the output of the web project there. This gives us a path to our test page so we can run the unit tests. We also need to configure the test to use the deployment hints.
Under the main solution, there should be a folder called Solution Items. Double-click on the file Local.testsettings.
Click on the deployment setting, and make sure that Enable Deployment is checked. Once checked, click the Apply button in the lower right corner of the dialog.
Change the class to inherit from the FixtureBase provided by Prism:
[TestClass] public class UnitTest1 : FixtureBase<SilverlightAppLauncher>
Set up the using statements to include the namespaces we'll need:
using System.Reflection; using System.Threading; using AcceptanceTestLibrary.Common; using AcceptanceTestLibrary.Common.Silverlight; using AcceptanceTestLibrary.TestEntityBase; using AcceptanceTestLibrary.UIAWrapper; using Microsoft.VisualStudio.TestTools.UnitTesting;
Now, let's add some code to launch the browser and load the Silverlight test page, as well as tear it down when the test is finished. You'll want to tweak the delays to a value that works well for you. Put this at the top of the class:
private const string APP_PATH = @"\SL\UIAutomationTestPage.html";
private const string APP_TITLE = "UIAutomation";
#region Additional test attributes
// Use TestInitialize to run code before running each test
[TestInitialize]
public void MyTestInitialize()
{
var currentOutputPath = (new System.IO.DirectoryInfo(Assembly.GetExecutingAssembly().Location)).Parent.FullName;
MainPageBase<SilverlightAppLauncher>.Window = LaunchApplication(currentOutputPath + APP_PATH, APP_TITLE)[0];
Thread.Sleep(5000);
}
// Use TestCleanup to run code after each test has run
[TestCleanup]
public void MyTestCleanup()
{
PageBase<SilverlightAppLauncher>.DisposeWindow();
SilverlightAppLauncher.UnloadBrowser(APP_TITLE);
}
#endregion
We are using the Silverlight application launcher, a helper provided with the Prism project, to launch our test page. Notice that we get the output directory for the test that is running, then append the path to the test page. The test page is underneath the SL subdirectory because that's how we defined it with the deployment item.
Now that we have it launched and ready to tear down, we can write the actual automation test. Here's what we'll do:
- Simulate typing text into the source text box
- Confirm that the target text box is blank
- Simulate clicking the button
- Confirm that the target text box now has the text we entered into the source text box
Here's how we do it:
[TestMethod]
public void TextBoxSubmission()
{
const string TESTVALUE = "TestValue";
// set up the value
var txtBox = MainPageBase<SilverlightAppLauncher>.TextBoxSource;
Assert.IsNotNull(txtBox, "Text box is not loaded");
txtBox.SetValue(TESTVALUE);
Thread.Sleep(1000);
Assert.AreEqual(txtBox.GetValue(), TESTVALUE);
// ensure the text block is empty to start with
var txtBox2 = MainPageBase<SilverlightAppLauncher>.TextBoxTarget;
Assert.IsNotNull(txtBox2, "Target text box is not loaded.");
Assert.IsTrue(string.IsNullOrEmpty(txtBox2.GetValue()), "Target text box is not empty.");
var btnSubmit = MainPageBase<SilverlightAppLauncher>.Button;
Assert.IsTrue(btnSubmit.Current.IsEnabled, "Submit Button is not enabled");
btnSubmit.Click();
Thread.Sleep(1000);
var actual = txtBox2.GetValue();
Assert.AreEqual(TESTVALUE, actual, "Text block was not updated.");
Thread.Sleep(1000);
}
As you can see, writing the automation tests is relatively straightforward. We don't have "record and playback" but it's easy to grab a control, tell it to do something, and then query the result. Build the project and make sure there are no errors.
Run the Test
Remember what I mentioned earlier about lots of steps? We're there. We've set it up, and should be good to go. If you have issues, go back and check the steps to make sure you didn't miss anything. Again, once you get the hang of it, you'll find it's not that difficult to get up and running and to write some great automation tests. There are good examples in the Prism samples, specifically in the MVVM quick start.
Let's open the test view:
Select the test and click "Debug Selection":
You should eventually see a browser window pop up. The browser may complain about security. If this happens, simply click on the yellow bar and allow the blocked content. Be sure to do this before the launch times out:
You can literally watch the Silverlight application appear in the browser, then see the text entered into the source box, and eventually the button will click and the text should post to the target box. If all goes well, you'll get the familiar green check box:
There you go ... Silverlight UI automation! If you've made it this far, then you have what you need to set this up for your own projects. I've heard of some companies who don't use Silverlight because they are under the impression it doesn't support automated testing. If you're at one of those companies, be sure to go grab your manager, drag them to your cube and show them that it should now be an approved platform for you and your fellow developers!
