Friday, March 13, 2015

Async/Await Explained

Last night I had the opportunity to present at the Gwinnett Microsoft .NET User Group. The topic I chose is not a new one, but an important one. Recently in several interviews I've been surprised to learn how little is truly understood about multi-threading in managed code and specifically the use of the async and await keywords. This presentation was designed specifically to walk through the basics behind threading and demonstrate when and how it is useful. The abstract and deck are below, along with a link to the repository with sample code.

Although two new keywords have been lurking in C# for years now, many people still don't fully understand what they do or how they facilitate asynchronous programming. Contrary to what some developers seem to believe, tagging a method as async doesn't make it run in a separate thread and await doesn't pull threads from the thread pool. These contextual keywords cause the compiler to generate complicated code so you don't have to with numerous advantages from improving the responsiveness of your application, simplifying the source, and enabling better scale. Learn exactly how async and await work, what happens under the covers, when you should use them and how to write your own APIs to take advantage of async and await whether it's in Windows Runtime, desktop, or web-based applications. (Full Source and Deck)

Jeremy Likness

Wednesday, March 11, 2015

AngularJS Integration Tests with Mocks and Magic

As a web developer I’m not a huge fan of full end-to-end tests. My opinion is changing with maturing frameworks like protractor but I still think looking for a “button” with an “id” is a fragile test that may have to change often. I am far more interested in what happens when the button is clicked than the button itself, because what starts out as a button might end up as a hyperlink or a block of text or something entirely different.

angularintegration

Data-binding provides a powerful abstraction between presentation logic and the user interface. “Applying a filter” is still presentation logic, but data-binding allows me to expose that as a method on an object and then let the designer worry about the details of what it is bound to. This was an effective way to code and test in the XAML days when we were using “view models” and I find it just as effective on the web.

For that reason, instead of a true “end-to-end” test, I prefer to enhance my AngularJS unit tests with integration tests. A unit test has no dependencies. I should be able to run it and mock dependencies so that it executes regardless of network connectivity, the presence of a web service or the status of a database. On the other hand, integration tests require a little bit of setup and expect things to be there, whether it is an active service or even a hot database on the backend. I test up to the data-binding, but leave the UI untouched.

The challenge with Angular is that the ngMock library makes it really, really easy to completely abstract the HTTP layer. It also makes it easy to test in general, which is why I like to use it even with my integration tests. The problem is that, as far as I know, I can’t opt-out of the $httpBackend. (If I’m wrong and there is a way other than using the end-to-end mock library, let me know!) Don’t get me wrong, it is fantastic for unit tests. To illustrate my point …

The Unit Test

Consider the world’s almost-smallest Angular app that does nothing other than expose an API that calls a service endpoint and returns a value based on whether or not it successfully connected. This is the app:

(function (app) {
 
    app.factory("oDataSvc", ['$q', '$http', function ($q, $http) {
        return {
            checkEndPoint: function () {
                var deferred = $q.defer();
                $http.get("http://services.odata.org/V4/TripPinServiceRW")
                    .then(function () {
                        deferred.resolve(true);
                    }, function () {
                        deferred.resolve(false);
                    });
                return deferred.promise;
            }
        };
    }]);

})(angular.module('test', []));

Now I can write a test. First, I’m going to wire up the latest version of Jasmine and make sure Jasmine is working.

(function() {
 
    var url = "http://services.odata.org/V4/TripPinServiceRW";
    describe("jasmine", function() {
        it("works", function() {
            expect(true).toBe(true);
        });
    });
})();

Now I can set up my unit test. First I want to capture the service and the $httpBackend and verify I’ve handled all requests after each test.

describe("angular unit test", function() {
 
    var oDataService, httpBackend;
    beforeEach(function() {
        module("test");
    });
    beforeEach(inject(function($httpBackend, oDataSvc) {
        httpBackend = $httpBackend;
        oDataService = oDataSvc;
    }));
    afterEach(function() {
        httpBackend.verifyNoOutstandingExpectation();
        httpBackend.verifyNoOutstandingRequest();
    });
});

Then I can make sure I was able to retrieve the service:

it("is registered with the module.", function () {
    expect(oDataService).not.toBeNull();
});

Now comes the fun part. I can set up my test but it will hang on the promise until I set up expectations for the backend and then flush it. I can do this all synchronously and test how my service deals with hypothetical response codes. Here’s the example that I use to set up “success”:

describe("checkEndPoint", function() {
 
    it("should return true upon successful connection",
        function () {
        oDataService.checkEndPoint()
            .then(function(result) {
                expect(result).toEqual(true);
            }, function() {
                expect(false).toBe(true);
            });
        httpBackend.expectGET(url)
            .respond(200, null);
        httpBackend.flush();
    });
});

The expectation is set in the “expectGET” method call. The service will block on returning until I call flush, which fires the result based on the expectation I set, which was to return a 200-OK status with no content. You can see the failure example in the jsFiddle source.

The Integration Test

The rub comes with an integration test. I’d love to use the mock library because it sets up my module, the injector, and other components beautifully, but I’m stuck with an $http service that relies on the backend. I want the “real” $http service. What can I do?

The answer is that I can use dependency injection to my advantage and play some tricks. In the context of my app, the injector will provide the mocked service. However, I know the core module has the live service. So how can I grab it from the main module and replace it in my mocked module without changing the mocks source?

Before I grab the service, it is important to understand dependencies. $http relies on the $q service (I promise!). The $q service, in turn, relies on the digest loop. In the strange world of a mocked test object, if I manage to call the real $http service it is not going to respond until a digest loop is called.

“Easy,” you might say. “Get the $rootScope and then call $apply.”

“Not so fast,” is my reply. The $rootScope you probably plan to grab won’t be the same $rootScope used by the $http service we get from the injector. Remember, that is a different “container” that we are accessing because it hasn’t been overwritten in our current container that has the mocks library!

This is easier to explain with code:

beforeEach(function () {
    var i = angular.injector(["ng"]),
        rs = i.get("$rootScope");
    http = i.get("$http");
    flush = function () {
        rs.$apply();
    }
    module("test", function ($provide) {
        $provide.value("$http", http);
        $provide.value("$rootScope", rs);
    });
});

Angular’s modules overwrite their contents with a “last in wins” priority. If you include two module dependencies with the same service, the last one wins and you lose the original. To get the live $http, I need to create a new container. That container is completely isolated from the one I’m using to test. Therefore I need to grab that container’s $rootScope as well. The flush method gives me a reusable function I can easily use throughout my code. When I mock the module for the integration test, I intercept the provider by using the $provide service to replace the ones already there.

The replacement of $rootScope is important. It’s not good enough to capture the flush method because that will just run a digest in the original container. By making that $rootScope part of my current container, I ensure it is the one used all of the way down the dependency chain (and for digests in my tests). If I reference $q in my tests I’ll overwrite it from the original container too.

Now my test doesn’t configure expectations but is a true integration test. I am expecting the sample service to be up, and for the service call to return “true.” Notice that I need to call this asynchronously, so I take advantage of the oh-so-easy asynchronous syntax in the latest Jasmine (“pass done, then call it when you are.”)

it("should return true to verify the service is up",
    function (done) {
    oDataService.checkEndPoint()
        .then(function (result) {
            expect(result).toEqual(true);
            done();
        }, function () {
            expect(false).toBe(true);
            done();
        });
    flush();
});

That’s it. Using this method, I can take advantage all mocks has to offer while still integrating with my live web API to ensure service calls are working. This is what I call a “beneath the surface” test. I’m testing through the data bound model, ensuring that the test flows through to the database, etc., but again I’m testing what a function does, not how it is wired to the UI.

To see the unit test and integration test in action, check out the jsFiddle. Check out your network and notice the service is really being called for the integration test.

network

If you see several lines, it’s because it redirects to generate a “session” for the service and then makes the final call (307 ► 302 ► 200).

If you are a fan of integration tests and struggled with this in your AngularJS tests I hope this was helpful. If you have a different approach then please share it in the comments below!

Sunday, February 8, 2015

Generate Mazes in AngularJS with 8-Bit Algorithms

I started programming on 8-bit machines. Back then it was normal to do amazing things with a mere 64 kilobytes of memory on a CPU that clocked in at 2.6 Megahertz. No, those aren’t typos, and I meant “mega-” not “giga-”. In those days we were forced to do a lot with a little, and “optimizing code” meant choosing op codes based on how many cycles they took to execute.

maze

It is amazing to me to look back and see what we were able to accomplish. Software exists to perform everything from advanced compression to generating a fractal Mandelbrot set. In addition to programming I spent plenty of time playing adventure games. One of my favorite games I referenced in a recent talk. It rendered a 3D landscape for a first-person view. The sky would change color from day to night and it could even rain in this Alternate Reality.

To create fun and playable games, developers often had to resort to creative coding techniques such as using algorithms to generate the game world. Disk access was notoriously slow (how many of you remember playing the Ultima series and spending most of your time waiting to load the next city or dungeon?) so generating terrain or worlds “on the fly” made for better game play.

I remember an old arcade game called Berserker and was reminded of the game when I watched a documentary called Chasing Ghosts. It followed video game “champions” (or perhaps addicts) from 1982 to present. In it, they reviewed the game and talked briefly about how it generated its famous mazes. There is a great write-up at the Robotron site about the algorithm.

To create a “room” the game essentially assumed entrances on all four walls with eight “pillars” in the middle. Each pillar was assigned a random number, and the number determined if a wall would extend from the pillar in the north, south, east, or west position. The concept is so simple, I decided to implement it in Excel. To generate new mazes or see the underlying formulas, simply choose the Data option and select “Calculate Workbook” or edit any cell if you open it locally.

Once that was done, it was fairly straightforward to move the concept to AngularJS. The bulk of the solution is more related to the algorithm and Angular is more the means to display it. I decided to use pure CSS to draw the maze, rather than SVG or some other technology, so really all that I need are a pillar and a wall:

.pillar {
    width: 15px;
    height: 5px;
    margin: 0;
    background: red;
}
.wall
{
    width: 15px;
    height: 5px;
    margin: 0;
    background: green;
}

In the game mazes are exhaustive and interconnect “rooms.” For this example I decided to create two “rooms” so there is a single maze with an entrance and exit (always rendered on the top middle and lower right). The maze is 26 x 16 with 16 columns, represented as an array of rows that each contain an array of columns. Generating the basic maze is as simple as:

for (row = 0; row < 25; row += 1) {

    cells = [];
    maze.push(cells);
    for (col = 0; col < 16; col += 1) {
        border = row === 0 || row === 24 ||
            row === 12 || col === 0 ||
            col === 15;
        cells.push(border ? 1 : 0);
    }
}

The convention is a “0” for empty space, a “1” for a wall and a “2” for a pillar. Using the CSS and Angular, this is easily rendered with the following HTML:

<table>
    <tr ng-repeat="row in ctrl.maze track by $index">
        <td ng-repeat="col in row track by $index"
              ng-class="{ 'wall' : col === 1, 'pillar' : col === 2 }">            
        </td>
    </tr>
</
table>

I then “knock out” the doors (defined as an array of row/col coordinates):

angular.forEach(doors, function (door) {
    maze[door[0]][door[1]] = 0;
});

After that, it’s a simple question of iterating through the pillars to build the walls. Initially I wasn’t concerned about each pillar having a unique wall (i.e. they can overlap) just to keep the code simple. I store an array of values for each “direction” that indicates how to draw a wall relative to a pillar. For example, “up” means the previous three rows of the same column for the pillar:

up:
    [{
        row: -1,
        col: 0
    }, {
        row: -2,
        col: 0
    }, {
        row: -3,
        col: 0
    }]

For each pillar, I draw the pillar on the map, then generate a random number. Based on the random number, I return a direction:

function getDirection(seed) {
    if (seed < 0.25) return dirs.left;
    if (seed < 0.5) return dirs.up;
    if (seed < 0.75) return dirs.right;
    return dirs.down;
}

From the direction, I can then build the wall. The initial code for the “pillar” loop looked like this:

angular.forEach(pillarCoords.row,
    function (pillarRow) {
        angular.forEach(pillarCoords.col,
         function (pillarCol) {
                var dir, rand = Math.random();
                maze[pillarRow][pillarCol] = 2;
                dir = getDirection(rand);
                angular.forEach(dir,
                    function (offset) {
                        maze[pillarRow + offset.row]
                            [pillarCol + offset.col] = 1;
        });
    });
});

I then enhanced the algorithm beyond the Excel implementation to ensure each pillar gets a unique wall that doesn’t overlap with another pillar’s wall. This is done in a simple loop that checks to see if a wall already exists in that direction, then asks for a new direction. The way it is implemented, there is a low probability a pillar could end up without a wall due to the random generator (in the rare case it always returns the same direction).

That’s it. The majority of the implementation is done in pure JavaScript (my preference), so Angular just seeds the maze with a call to the generator, and binds a click event to generate a new maze. The controller code looks like this:

function MazeController() { }
 
angular.extend(MazeController.prototype, {
    maze: mazeGenerator(),
    newMaze: function () {
        this.maze = mazeGenerator();
    }
});


app.controller('mazeCtrl', MazeController);

You can view all of the source and see the working fiddle here.

I love this approach. A simple algorithm can generate a practically limitless configuration of mazes that are all solvable. In the 8-bit days we were forced to find simple, elegant solutions to complex problems and I see that trend happening all over again with the popularity of mobile platforms. After all, this maze will run without hesitation right on your phone!