Wednesday, September 28, 2011

The Top 10 Over-Engineering Mistakes in Silverlight Applications

Several weeks ago, I tweeted that a lot of people appear to be making their software far more complex than it needs to be. I was asked to elaborate and share details. The comment was prompted by reading dozens of forum posts by desperate developers in over their heads trying to apply enormous and complex frameworks to applications that really could use simple, straightforward solutions. I've witnessed this in projects I've taken over and worked with other developers and of course am guilty of making these same mistakes myself in the past. After years of working with line of business Silverlight applications, and speaking with several of my colleagues, what I thought might be a "Top 5" turned out to be a "Top 10" list. This is not necessarily in a particular order, but here are the ten most common mistakes I see developers make when they tackle enterprise Silverlight applications (and many of these apply to applications in general, regardless of the framework).

1. YAGNI

YAGNI is a fun acronym that stands for, "You aren't going to need it." You've probably suffered a bit from not following YAGNI and that is one mistake I've definitely been guilty of in the past. YAGNI is why my Jounce framework is so lightweight, because I don't want to have another bloated framework with a dozen features when most people will only ever use one or two. YAGNI is violated by the framework you build that has this awesome policy-based logging engine that allows dynamic configuration and multiple types of logs ... even though in production you always dump it to the same rolling text file. YAGNI is violated by the complex workflow you wrote using the Windows Workflow Engine ... when you could have accomplished the same thing with a dozen lines of C# code without having to spin up a workflow process. Many of the other items here evolve around YAGNI.

Learn how to avoid these mistakes in my new book, Designing Silverlight Business Applications: Best Practices for Using Silverlight Effectively in the Enterprise (Microsoft .NET Development Series)

A good sign that YAGNI is being violated is when the team spends three months building the "application framework" without showing a single screen. It has to be built just right with a validation engine, a configurable business rules engine, an XSLT engine and your own data access adapter. The problem is that trying to reach too far ahead into the project is a recipe for disaster. Not only does it introduce a bulk of code that may not be used and adds unnecessary complexity, but many times as the project progresses you'll find you guessed wrong and now have to go back and rewrite the code. I had a boss once who would throw up his hands in exasperation and say, "No, don't ask me for another refactoring." Often users only think they know what they want, and building software too far into the future means disappointment when they figure out they want something different after testing the earlier versions of the code.

Well-written software follows the SOLID principles. Your software should be made of small, compact building blocks. You don't have to boil the ocean. Instead, assume you aren't going to need it. That doesn't mean the logger or rules engine will never come into play, it just means you defer that piece of the software puzzle until it is relevant and comes into focus. Often you will find the users weren't really sure of what they really wanted, and waiting to place the feature until the software has been proven will save you lots of cycles and unnecessary overhead. Instead of pulling in the Enterprise Library, just implement an ILogger interface. You can always put the Enterprise Library behind it or refactor it later, but you'll often find that simple implementation that writes to the debug window is all you'll ever need.

2. Sledgehammer Framework Syndrome

This syndrome is something I see quite a bit with the Prism framework. I see a lot of questions about how to scratch your left ear by reaching around your back with your right arm using five layers of indirection. Contrary to what some people have suggested, I am a huge fan of Prism and in fact give quite a bit of credit to Prism concepts in Jounce. The problem is that while Prism provides guidance for a lot of features, few people learn to choose what features are relevant and most simply suck in the entire framework and then go looking for excuses to use the features. You don't have to replace all of your method calls with an event aggregator message and an application with five menu items doesn't always have to be chopped into five separate modules. I can't tell you how many projects I've seen pull in the Enterprise Library to use the exception handling block only to find there are only two classes that actually implement it and the rest do the same old try ... catch ... throw routine. Understand the framework you are using, and use only the parts that are relevant and make sense. If the source is available, don't even compile the features you aren't going to need ... there it is, YAGNI again.

The problem with pulling in the whole framework is what many of you have experienced. You jump into a new code base and find out there are one million lines of code but it just seems weird when the application only has a dozen pages. You started to work on a feature and find entire project libraries that don't appear to be used. You ask someone on the team about it, and they shrug and say, "It's working now ... we're afraid if we pull out that project, something might be broken that we won't learn about until after it's been in production for 6 months." So the code stays ... which is Not A Good Thing™.

3. Everything is Dynamic

The first question I often get about Jounce is "how do I load a dynamic XAP" and then there is that stare like I've grown a third eye when I ask "Do you really need to load the XAP dynamically?" There are a few good reasons to load a XAP dynamically in a Silverlight application. One is plug-in extensibility — when you don't know what add-ons may be created in the future, handling the model through dynamic XAP files is a great way to code for what you don't know. Unfortunately, many developers know exactly what their system will need and still try to code everything dynamic. "Why?" I ask. "Because it is decoupled." "But why is that good?" "Because you told me to follow the SOLID principles." The SOLID principles say a lot about clean separation of concerns, but they certainly don't dictate the need to decouple an application so much that you can't even tell what it is supposed to load.

Following SOLID means you can build the application as a set of compiled projects first, and then refactor modules into separate XAP files if and when they are needed. I mentioned one reason being extensibility. The other is managing the memory footprint. Why have the claims module referenced in your XAP file if you aren't going to use it? The thing is, there isn't much of a difference between delaying the creation of the claim view and the claim view model versus adding the complexity of loading it from a separate XAP file. If you profile the memory and resources, you'll find that most satellite XAP files end up being about 1% of the total size of the application. The main application is loaded with resources like images and fonts and brushes and controls, while the satellite XAP files are lightweight views composed of the existing controls and view models. Instead of making something dynamic just because it's cool, why not build the application as an integrated piece and then tackle the dynamic XAP loading only if and when it's needed?

4. Must ... Have ... Cache

Caches are great, aren't they? They just automatically speed everything up and make the world a better place by reducing the load on the network. That sounds good, but it's a tough sell for someone who actually profiles their applications and is looking to improve performance. Many operations, even with seemingly large amounts of data, end up having a negligible impact on network traffic. What's worse, a cache layer adds a layer of complexity to the application and another level of indirection. In Silverlight, the main option for a cache is isolated storage. Writes to isolated storage are slower than slugs on ice due to the layer between the isolated storage abstraction and the local file system.. Often you will find that your application is taking more time to compute whether or not a cached item has expired and de-serializing it from isolated storage than it would have taken to simply request the object from the database over the network. Obviously, there are times when a cache is required such as when you want the application to work in offline mode. The key is to build the cache based on need, and sometimes you may find that you aren't going to need it. As always, run a performance analysis and measure a baseline with and without the cache and decide based on those results whether or not the cache is necessary — don't just add one because you assume it will speed things up.

5. Optimistic Pessimistic Bipolar Synchronization

Synchronization is a problem that has been solved. It's not rocket science and there are great examples of different scenarios that deal with concurrency. Many applications store data at a user scope, so the "concurrency" really happens between the user and, well, the user. If you are writing an application that works offline and synchronizes to the server when it comes online, be practical about the scenarios you address. I've seen models that tried to address "What if the user went offline on their phone and updated the record, then updated the same record on their other offline phone then they went to their desktop in offline mode and updated the same record but the time stamp on the machine is off, and now both go online - what do we do?!" The reality is that scenario has about a 1 in 1,000,000 likelihood. Most users simply aren't offline that much and when they are, it's an intermittent exception case. Field agents who work in rural areas will be offline more often, but chances are they are using your application on one offline device, not multiples. It simply doesn't make sense to create extremely complex code to solve the least likely problem in the system, especially when it's something that can be solved with some simple user interaction. Sometimes it makes more sense to simply ask the user, "You have multiple offline updates. Synch with your phone or your desktop?" rather than trying to produce a complex algorithm that analyzes all of the changes and magically constructs the target record.

6. 500 Projects

I'm a big fan of planning your projects carefully. For example, it often does not make sense to include your interfaces in the same project as your implementations. Why? Because it forces a dependency. If you keep your interfaces (contracts) in a separate project, it is possible to reference them across application layers, between test and production systems, and even experiment with different implementations. I've seen this taken to the extreme, however, with applications that contain hundreds of projects and every little item is separated out. This creates a convoluted mass of dependencies and building the project can take ages. Often the separation isn't even needed because groups of classes are often going to be updated and shipped together.

A better strategy is to keep a solid namespace convention in place. Make sure that your folder structure matches your namespaces and create a folder for models, contracts, data access, etc. Using this approach enables you to keep your types in separate containers based on namespaces, which in turn makes it easy to refactor them if you decide that you do need a project. If you have a project called MyProject with a folder called Model and a class called Widget, the class should live in the MyProject.Model namespace. If you find you need to move it to a separate project, you can create a project called MyProject.Model, move the class to it and update references, and you're done - just recompile the application and it will work just fine.

Designing Silverlight Business Applications: Best Practices for Using Silverlight Effectively in the Enterprise (Microsoft .NET Development Series)

7. No Code Behind!

This is one that amazes me sometimes. Developers will swear MVVM means "no code behind" and then go to elaborate lengths to avoid any code-behind at all. Let's break this down for a minute. XAML is simply declarative markup for object graphs - it allows you to instantiate types and classes, set properties and inject behaviors. Code-behind is simply an extension of those types and the host class that contains them. The idea of MVVM is to separate concerns - keep your presentation logic and view-specific behaviors separate from your application model so you can test components in isolation and reuse components across platforms (for example, Windows Phone 7 vs. the new WinRT on Windows 8). Having business logic in your code-behind is probably NOT the right idea because then you have to spin up a view just to engage that logic. What about something completely view-specific, however? For example, if you want to kick off a storyboard after a component in the view is loaded, does that really have to end up in a view model somewhere? Why? It's view-only logic and doesn't impact the rest of the application. I think it's fine to keep concerns separated, but if you find you are spending 2 hours scouring forums, writing code and adding odd behaviors just so you can take some UI code and shove it into a view model, you're probably doing it wrong. Code-behind is perfectly fine when it makes sense and contains code that is specific to the view and not business logic. A great example of this is navigation on Windows Phone. Because of the navigation hooks, some actions simply make sense to write in the code-behind for the view.

8. Coat of Many Colors

Have you ever worked on a system where your classes wear coats of many colors? For example, you have a data base table that is mapped to an Entity Framework class, which then gets shoved inside a "business class" with additional behaviors, that is then moved into a lightweight Data Transfer Object (DTO) to send over the wire, is received using the proxy version of the DTO generated by the service client, and then pushed into yet another Silverlight class? This is way too much work just to move bits over the wire. Modern versions of the Entity Framework allow you to create true POCO classes for your entities and simply map them to the underlying data model. Silverlight produces portable code that you can share between the client and server projects, so when you define a service you can specify that the service reuses the type and de-serializes to the original class instead of a proxy object. WCF RIA Services will handle all of the plumbing for sharing entities between the client and the server for you. You know you are a victim of this problem if you ask someone to add a new entity into the mix and they moan for 10 minutes because they know it's going to take forever to build the various incarnations of the object. When there is too much ritual and ceremony involved with moving a simple entity from the server to the Silverlight client, it's time to step back and re-evaluate. In some cases it might make sense to keep the entities but use code-generation techniques like T4 templates to simplify the repetive tasks, but in many cases you can probably get away with reusing the same class across your entire stack by separating your models into a lightweight project that you reference from both sides of the network pond.

9. Navigation Schizophrenia

Are you building a web site in Silverlight, or an application? The presence of the navigation framework has led many projects down the path of using URL-driven navigation for line of business applications., To me, this is a complete disconnect. Do I use URLs in Excel? What does the "back" button mean in Word? The point is that some applications are well-suited to a navigation paradigm similar to what exists on the web. There is a concept of moving forward and "going back." Many line of business applications are framed differently with nested menus, multiple areas to dock panels and complex graphs, grids, and other drill-downs. It just doesn't make sense to try to force a web-browser paradigm on an application just because it is delivered over the web. Sometimes you have no choice - for example, navigation is an intrinsic part of the Windows Phone experience, and that's fine. Just make sure you are writing navigation based on what your application needs, rather than forcing a style of navigation on your application simply because there is a template for it.

10. Everything is Aggregated

The final issue is one that I've seen a few times and is quite disturbing. If you are publishing an event using the event aggregator pattern, and receiving the same event on the same class that published it, there's something wrong. That's a lot of effort to talk to yourself. The event aggregator is a great pattern that solves a lot of problems, but it shouldn't be forced to solve every problem. I've always been a fan of allowing classes communicate with peers through interfaces. I don't see an issue with understanding there is a view model that handles the details for a query, so it's OK to expose an interface to that view model and send it information instead of using the event aggregator. I still expose events on objects as well. For example, if I have a repository that is going to raise a notification when the collection changes, I'll likely expose that as an event and not as a published message. Why? Because for that change to be interesting, the consumer needs to explicitly understand the repository and have an established relationship. The event aggregator pattern works great when you have messages to publish that may have multiple subscribers, impact parts of the system that may not be explicitly aware of the class publishing the message, and when you have a plug-in model that requires messages to cross application boundaries. Specific messages that are typically shared between two entities should be written with that explicit conversation in mind. In some cases you want the coupling to show the dependency because it is important enough that the application won't work well without it. There is nothing wrong with using the event aggregator, just understand the implications of the indirection you are introducing and determine when a message is really an API call, a local notification, or a global broadcast.

Conclusion

I love writing line of business software. I've been doing it for well over a decade across a variety of languages ranging from C++, Java, VB6, JavaScript and XSLT (yes, I called XSLT a language, if you've worked with systems driven by XSLT you know what I mean) ... and I've been guilty of most of the items I listed here. One thing I learned quickly was that most people equate "enterprise software" to "large, clumsy, complex and difficult to maintain software" and that doesn't have to be the case. The real breakthrough for me happened when I started to focus on the tenants of SOLID software design as well as DRY (don't repeat yourself) and YAGNI. I learned to focus on simple building-block elements and working with what I know and not spending too much time worrying about what I don't know. I think you'll find that keeping the solution simple and straightforward creates higher quality software in a shorter amount of time than over-engineering it and going with all of the "cool features" that might not really be needed. If there is nothing else you take away from this article, I hope you learn two things: first, don't code it unless you know you need it, and second, don't assume - measure, spike, and analyze, but never build a feature because you THINK it will benefit the system, only build it when you can PROVE that it will.

Want to avoid these mistakes? Read about lessons learned from over a decade of enterprise application experience coupled with hands-on development of dozens of line of business Silverlight applications in my new book, Designing Silverlight Business Applications: Best Practices for Using Silverlight Effectively in the Enterprise (Microsoft .NET Development Series).

Jeremy Likness

11 comments:

  1. Amen brother. Love this post.

    ReplyDelete
  2. I always found myself over-engineering whenever I read about pattern X and how everybody should use it to get clean, professional, testable, robust, decoupled, maintainable, "you name it" - code. And then you think, "Oh man I wanna be one of those out of this world programmer".

    Great article!
    Thanks!

    ReplyDelete
  3. Nice article, Jeremy.

    I recently started a new project and have definitely been overkilling the architecture. It's good to hear some of these points from someone with lots of experience.

    ReplyDelete
  4. Great post. We can certainly tick off quite a few of these on our current project!

    ReplyDelete
  5. How refreshing -- a pragmatic enterprise architect! Can't wait to read the book.

    ReplyDelete
  6. How about top 10 under-engineering mistakes? Most of the things you wrote about aren't even considered where I am now.

    ReplyDelete
  7. hehehe ya most place are still doing procedural 2 layer or 3 layer spaghetti sucky table base code....

    Tired of this, very very tired of fixing bugs of in under-engineering projects

    ReplyDelete
  8. Thanks for sharing this post!

    One question about your new book; will it also be available as an book?

    Hoping so!

    Thanks again,

    ~Dave

    ReplyDelete
  9. Yes, it will be available as a hard copy book and available through Amazon, retail stores, etc.

    ReplyDelete