Daniel Cazzulino's Blog

Simplified INotifyPropertyChanged Implementation with WeakReference Support and Typed Property Access API

I've grown a bit tired of implementing INotifyPropertyChanged. I've tried ways to improve it before (like this "ViewModel" custom tool which even generates strong-typed event accessors).

But my fellow Clarius teammate Mariano thought it was overkill and didn't like that tool much. He mentioned an alternative approach also, which I didn't like too much because it relied on the consumer changing his typical interaction with the object events, but also because it has a substantial design flaw that causes handlers not to be called at all after a garbage collection happens. A very simple unit test will showcase this bug.

I also looked at the new WeakEvent Patterns page in MSDN but it's even worse in terms of implementing and exposing it to consumers.

So, with my ever growing love for lambdas and my strong-typed reflection approach (used by the first alternative too, btw), I thought I could do better :). Here's the result of that, which I think improves all the above choices.

Why you need weak reference support

The importance of this cannot be understated. A delegate that you pass around has a strong reference to the instance that exposes it. This is the Target property on the delegate class. What this means is that even if the subscribing object goes out of scope and is ready to be collected, it will not be as long as the event source (the object exposing the PropertyChanged event, for example) holds a reference to it. And as long as the event subscription is there, the reference will be there too. That's why it is typically important to remove your reference once you're ready to "go" (i.e. on Dispose, you detach from the events you're listening). Needless to say, this is a repetitive, error-prone activity.

Another typical side-effect of this is that you cannot use anonymous delegates or lambdas if you need to unsubscribe, as you need to keep a reference to the originally subscribed lamdba in order to unsubscribe:

var target = new Foo();

target.PropertyChanged += (sender, args) => Console.WriteLine(args.PropertyName);

// How do you unsubscribe now??

// This clearly doesn't work because even if the actual source is the same, 
// the delegate is still a brand-new one.
target.PropertyChanged -= (sender, args) => Console.WriteLine(args.PropertyName);

// So you need to keep the lambda around:
PropertyChangedEventHandler handler = (sender, args) => Console.WriteLine(args.PropertyName);

// Just so you can use that to subscribe/unsubscribe:
target.PropertyChanged += handler;
target.PropertyChanged -= handler;

// So typically you're better off just adding a full instance method on your 
// consumer just so you have a clear pointer for unsubscribing:
target.PropertyChanged += OnTargetPropertyChanged;

// But now if you need contextual state in the event handler that exists 
// at subscription time, you need to promote that state to class fields
// so that you can use that in the event handler
this.someState = currentMethodState;

// This is looking like .NET 1.0 already ;)

The PropertyChangeManager way

I therefore decided to take a TDD approach to the issue with the following requirements:

The programming model for consumers must not involve creating any new objects. They already have the object that will be raising property change events.
The "old style" way of attaching to property changed events must still work, but add the weak reference support that's so badly needed. And this must be transparent to consumers.
A new style should involve using lambdas to avoid property names as strings
The new style should be trivial to implement for an author exposing INotifyPropertyChanged.

So I came up with these BDD-style test specifications:

WhenSubscriberIsAlive_ThenNotifiesSubscriber
WhenSubscriberIsNotAlive_ThenDoesNotNotifySubscriber
WhenAddingPropertyChangedHandler_ThenNotifiesSubscriber
WhenAddedPropertyChangedHandlerTargetIsNotAlive_ThenDoesNotNotify
WhenRemovingPropertyChangedHandler_ThenDoesNotNotifySubscriberAnymore

These should cover all use cases.

Consuming PropertyChangeManager-enabled objects

The fact that a given object is internally (remember requirement 1.) using this PropertyChangeManager is completely hidden from the consumer:

var source = new Foo();

source.SubscribeChanged(
    x => x.Name,
    foo => Console.WriteLine(foo.Name));

The first argument specifies which property you're interested in, and the second is an Action<Foo> in this case for the callback when the property changes. It can of course point to a class method:

source.SubscribeChanged(
    x => x.Name,
    this.OnRenamed);

Optionally, if unsubscribing from the event will be needed at some point, you can just keep a reference to the returned IDisposable object from the call to SubscribeChanged:

// this could be assigned to a field, for example.
IDisposable onRenameSubscription = source.SubscribeChanged(
    x => x.Name,
    foo => Console.WriteLine(foo.Name));

// at some later point (i.e. IDisposable.Dispose implementation of the consumer)
onRenameSubscription.Dispose();
// now the subscription is removed, even if I didn't keep the lambda around!

The source object still implements INotifyPropertyChanged, but now does so explicitly to support databinding infrastructure. The consumer can still cast the object to INotifyPropertyChanged if he wants to use the "unsafe" property name strings.

Implementing INotifyPropertyChanged with PropertyChangeManager

The implementer defines a private field to hold a reference the manager:

public class Foo : INotifyPropertyChanged
{
    private PropertyChangeManager<Foo> propertyChanges;
    private string name;
    private int value;

    public Foo()
    {
        this.propertyChanges = new PropertyChangeManager<Foo>(this);
    }

Note that the manager is generic and receives the type of the "change event source", in this case Foo.

Next, your properties need to be turned into "old-style" .NET properties with a backing field, because you need to add a call to the manager in the property setter:

public string Name
{
    get { return name; }
    set { name = value; this.propertyChanges.NotifyChanged(x => x.Name); }
}

Note that the call to the manager also leverages lambdas to avoid using strings.

In order to provide a custom implementation of the INotifyPropertyChanged.PropertyChanged event, you need to implement the interface explicitly and pass-through the implementation to the manager:

event PropertyChangedEventHandler INotifyPropertyChanged.PropertyChanged
{
    add { this.propertyChanges.AddHandler(value); }
    remove { this.propertyChanges.RemoveHandler(value); }
}

This is a restriction in the language, which prevents this event from being public. But it's not as bad as it sounds, as you want to encourage adoption of safer lambda-version subscription, which is the last bit to implement:

public IDisposable SubscribeChanged(Expression<Func<Foo, object>> propertyExpression, Action<Foo> callbackAction)
{
    return this.propertyChanges.SubscribeChanged(propertyExpression, callbackAction);
}

How PropertyChangeManager works

The manager works by dismembering the received delegates into their actual target and method info, to be able to weakly reference the former, while remaining able to call the latter. It's a plain list internally, which is scavenged every time an action is performed in the manager (this could be optimized somehow to only happen on Notify, but it simplified the implementation a bit, and it's not like property change performance is a big issue in UIs anyway).

Here's the full source.

Some tooling such as a custom tool, item template, or code snippets would be nice, I'll try to provide those in the future.

Enjoy!

How to set the startup program for debugging a project for the entire team

You surely have set the startup application for a project countless times:

But that setting goes your user options file, the rest of the team doesn't get to reuse the setting. And what if you repave your machine or start working on a new virtual machine and just got the sources from source control? You have to re-set this value again and again.

Turns out that this setting goes to a file named after your project file plus the ".user" extension. This file is just a fragment of an MSBuild file, and would look something like:

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Debug|AnyCPU'">
    <StartAction>Program</StartAction>
    <StartProgram>C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\devenv.exe</StartProgram>
    <StartArguments>/rootSuffix Exp</StartArguments>
  </PropertyGroup>
</Project>

And because this just plain MSBuild properties, you can copy the entire PropertyGroup to your main project file, delete this .user, and check-in your change. From now on, everyone on the team will have this setting enabled, and you will have it too if you get a clean environment eventually :)

How to install Reactive Extensions for .NET 4.0 Beta 2 on VS2010 RC

Get the .NET 4.0 Beta 2 download from MSDN.
Open the downloaded .exe with 7zip (i.e. right-click on file, select 7-Zip > Open Archive)
Navigate to the .rsrc\RCDATA\ "folder" and open the CABINET file:
Extract the contained MSI, install and enjoy!

I thought I'd need to crack the MSI open with the good old Orca tool, but turns out I didn't have to!

Stay tuned, my Reactive Framework Extensions Generator will be soon updated to RC too :)

Enjoy!

How to quickly setup the best free Diff/Merge tool with VS 2010

First go get the tool. It's free and it rocks.

Next, save this XML to a file with a .vssettings extension:

<UserSettings>
    <ApplicationIdentity version="10.0"/>
    <Category name="Source Control_TeamFoundation" Category="{2A718788-A6D9-44C5-90EF-438BF5B06A74}" Package="{4CA58AB2-18FA-4F8D-95D4-32DDF27D184C}" RegisteredName="Source Control_TeamFoundation" PackageName="Microsoft.VisualStudio.TeamFoundation.VersionControl.HatPackage, Microsoft.VisualStudio.TeamFoundation.VersionControl, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
        <PropertyValue name="UserTool1" extension=".*" operation="Compare" command="C:\Program Files (x86)\SourceGear\DiffMerge\DiffMerge.exe" arguments="/title1=%6 /title2=%7 %1 %2"/>
        <PropertyValue name="UserTool2" extension=".*" operation="Merge" command="C:\Program Files (x86)\SourceGear\DiffMerge\DiffMerge.exe" arguments="/title1=%6 /title2=%8 /title3=%7 /result=%4 %1 %3 %2"/>
    </Category>
</UserSettings>

Finally, go to Tools > Import and Export Settings in VS and import that file by clicking Browse on the third and final page.

What this does is set the great SourceGear DiffMerge tool as the diff and merge tool to use for all your files. I find it much more usable and smart than the built-in TFS one.

For the ultimate collection of settings for diff/merge tools in VS, see James' blog post.

Enjoy.

Software Development Productivity

Scott Bellware has an interesting series of posts where he discusses how to get back to productive development teams. As usual in his writing (IMO), in a rather verbose way he brings up quite a few good points. Please go ahead and read them. He links from the first entry to the next so you can follow the flow.

I agree with the analysis that unnatural organizational structures kill productivity, motivation and leadership. And I believe this is one of the reasons why even big companies turn to so-called "boutique development shops" (shameless plug there): by being small and very cohesive, these shops offer creativity and productivity levels that "mere big" ISVs can only dream of.

And it's not always only a matter of design principles, I'd add. Sometimes you need a specific area of expertise which you're better off outsourcing (i.e. Visual Studio extensibility, hardcore WCF, framework/runtime libraries, WPF/Silverlight/Blend UEX, etc.). Small shops of highly specialized professionals can save you tons of money and time. But your own dev team will certainly benefit from applying sound design principles for what matters most to your business: the business rules and logic.

How to transform old properties to automatic properties with a simple search and replace

Say you have a (typically autogenerated) class with properties like:

public partial class Project : IExtensibleDataObject
{
    public System.Runtime.Serialization.ExtensionDataObject ExtensionData
    {
        get
        {
            return this.extensionDataField;
        }
        set
        {
            this.extensionDataField = value;
        }
    }

Now you can fire up the Find & Replace dialog in VS and enter the following "simple" expression in Find What:

\n:b*\{[:b\n]*get[:b\n]*\{[.:b\n]*.*[.:b\n]*\}[.:b\n]*set[:b\n]*\{[.:b\n]*.*[.:b\n]*\}[.:b\n]*\}

And use the following expression for the replace:

 { get; set; }

Don't forget to set the Use: Regular expressions option.

Running it will get you automatic properties for the entire file (or files):

public partial class Project : IExtensibleDataObject
{
    public System.Runtime.Serialization.ExtensionDataObject ExtensionData { get; set; }

    [System.Runtime.Serialization.DataMemberAttribute()]
    public int AffiliateId { get; set; }

    [System.Runtime.Serialization.DataMemberAttribute()]
    public Galleries.Domain.Model.Category[] Categories { get; set; }

Yeah, I know, that expression looks like crap ;)

How to get out of the GAC all the registered assemblies

You know how annoying the GAC shell extension makes it to access the actual assemblies:

Utterly useless.

Of course, you surely know that you can get to those elusive assemblies via the command-line and side-step the shell extension:

But, now you need to go to each assembly folder, then its version, and so the actual assemblies are scattered through various locations.

This one-liner powershell command will get them all out in a folder of your choosing for easy Reflector-ing (create the target before running it):

Get-ChildItem C:\Windows\assembly\GAC_MSIL -filter *.dll -recurse | Copy-Item -destination C:\GAC

Why Windows Media Center is dead

Windows Media Center (WMC) is based on a relatively simple (albeit awfully implemented) principle: you have ONE "server" PC holding and running your media, and then you associate any number of Media Center Extenders to it that are typically (except for the XBox 360) single-purpose devices that can only act as such and are fancy and silent enough to deserve a place in your living room.

I guess back in 2005, the entire model and most of Microsoft design decisions on this product may have be justifiable. 5 years later, none of them make any sense and IMO mean that WMC is currently a totally flawed, doomed and generally useless product for most common needs.

Why it (kind of) made sense back then

Hardware: In 2005, you wouldn't dare subject your family to the noise, ugliness, quirkiness, uex, power comsumption, and cost of a full-blown "Home Theater PC" (HTPC) or an XBox power sucker. The Media Center Extender model made sense because you couldn't buy a full PC that was silent and nice-looking enough for the price of an extender. (but the extenders weren't without limitations either, a good review of it at the time at the supersite)
The user experience is quite cool (but the extenders' lower processing profile meant their UI rendering capabilities were significantly worse)

Why it doesn't make sense anymore

Hardware: nowadays you can buy a full blown PC, small, silent and power-efficient for $199-$249. Why would you want a crippled, single-purpose device when you can have the full power of a desktop, with the capability to play back any weird format you can possibly throw at it, provided you just install the right codec. No more hardware limitations and being stuck with an obsolete device.
Software: back then, the Media Center team invented their own markup language to accommodate disparate rendering capabilities between a full PC, the XBox and a crippled extender. Nowadays we have WPF and Silverlight, no need to learn a new UI markup language. The extenders UI is sluggish at best, compared with what's possible in a PC with WPF/Silverlight.
OS: Windows 7 is a kick-ass operating system, with a ton of features. It's a snappy, sleek, touch-friendly, smart connected (i.e. "Play To"/DLNA, HomeGroup, etc. etc.) platform. And it runs great on the smallest chips out there (it gave new life an 'old' Dell Mini 9 I had, for example).
Home Server: having all your media in a "plain" PC is risky. You need to take care of backups, redundancy, fault-tolerance, etc. Today, it's a no-brainer to buy a Windows Home Server and let it manage all this for you, automatically, transparently, and just dead easily.

This combination makes desktop media management software a viable alternative and can certainly power your living room and projector. I believe we're at the beginning of a revolution in smart homes and connected media, just because of this new synergy and convergence.

If I had to start a project like that right now, I'd make it so that:

It runs full WPF on the HTPCs connected to every TV.
It supports a centralized repository for the entire house (which could be a regular PC or a Windows Home Server-WHS)
It has a built-in extensibility model based on some dependency injection technology (i.e. MEF), and *everything* including the core functionality is a plugin
It leverages WHS to provide mostly the same UI over the internet by leveraging Silverlight (single programming model for both), so that I can access all the same content from everywhere, and automatically takes advantage of Smooth Streaming for videos.
It integrates with home security cameras and provides those both in the home (i.e. monitor the baby upstairs on a PiP window) or via the web (i.e. watch my home while on vacations) leveraging Live Smooth Streaming.
It empowers a huge plugin community that can create cool advanced features like face detection on photos and videos and security recordings, provide filtered views of those, map pictures with geocoding, provide a "home connection point" where a GPS could submit road trip info, etc. etc.

For now, I got my Dell Zino HD (shipping soon) and I'm looking forward to selling the assorted Frankenstein media setup I have right now.

The future looks bright for the smart home :)

How to mock extension methods

. without paying for a TypeMock Isolator license to do it ;-)

There's going to be no magic here. You have to explicitly design for testability. That's one of the things I like about mocking: if you can mock a dependency, then it means your design is loosely coupled (e.g. not tied to a particular implementation of that dependency), and you're not "cheating or taking any shortcuts. If a test can replace a dependency at test-time, your'll surely be able to replace the real implementation with something different when/if time comes to do so.

Extension methods are tricky because they are static methods, really just syntactic sugar for a "good" old static class with static methods (typically a "helper" of some sort. But what is special about them, is that they show up (provided you have the right usings/imports) in the target type API as if they were its own instance methods. This is significant, because it also means that it's very easy to pollute the target type API as you (and other referenced libraries) keep piling up these methods on it.

I therefore prefer an approach where you group your extensions under a single API, and access that API via a single extension method on the target type that becomes the entry point to your extensions. For example, say you have an IPerson interface, and you want to add secutity-related helper methods that you can use consistently from other logic or services. One way of adding a GetPermissions security helper extension method (that will get the rights of the user over a given resource, for example), would be to plug it right into IPerson as an extension method:

[Flags]
public enum Permissions
{
    Read = 0, 
    Modify = 1, 
    Delete = 2, 
    FullControl = Read | Modify | Delete
}

public static class SecurityExtensions
{
    public static Permissions GetPermissions(this IPerson person, Uri resource)
    {
        // get permissions for the given resource from somewhere
    }
}

This would get you the API right on the IPerson object:

If you go overboard with this "extension in the root" approach, you might end up with an ugly looking assorted collection of helpers attached to the main class, such as what happened to the UML APIs in Visual Studio 2010. Look at the before and after we reference and import the assembly containing the extensions to IClass (the interface that represents a class in a UML Logical Class Diagram):

After referencing Microsoft.VisualStudio.Uml.Extensions.dll and adding the corresponding using/import:

(I pasted together four pages of intellisense for added effect, but you see the point, rigth?)

Extension Interfaces

Another alternative is to group extension methods that provide related functionality under what I'd extension interfaces. In our person example, we might find out that the security-related methods start growing, and we'd like to keep the main API clean, so we introduce the ISecurity extension interface, and provide an extension method that is the entry point to it:

public interface ISecurity
{
    Permissions GetPermissions(Uri resource);
}

public static class SecurityExtensions
{
    public static ISecurity Security(this IPerson person)
    {
        throw new NotImplementedException();
    }
}

Now, the consumer of the IPerson object can see that there is security-related behavior available to him, but won't see what those are unless he needs to:

Typically, the SecurityExtensions class will instantiate a non-public implementation of ISecurity that does the real work, passing the person to work on as a constructor argument:

internal class Security : ISecurity
{
    private IPerson person;

    public Security(IPerson person)
    {
        this.person = person;
    }

    public Permissions GetPermissions(Uri resource)
    {
        // get the real permissions for the person 
        // field we have received, for the given resource
        throw new NotImplementedException();
    }
}

public static class SecurityExtensions
{
    public static ISecurity Security(this IPerson person)
    {
        return new Security(person);
    }
}

The attentive reader might have noticed that, being a regular interface, the extension interface now can expose properties where it makes sense, overcoming the lack of support for extension properties in C#. For example we could add a Roles property quite easily:

And because the extension interface is nothing more than a fluent interface in the end, you can use the trick I blogged about before to hide those pesky System.Object members:

How to mock extension methods, again?

Back to the testing subject, in order to test the Update method above, we'll need to replace the ISecurity implementation, right? Because we followed the extension interface design approach, we now have a single extension method to replace, the one that creates the default implementation of the ISecurity inteface. And if you look at that method, all it is being a factory for creating an implementation of ISecurity given an IPerson:

public static class SecurityExtensions
{
    public static ISecurity Security(this IPerson person)
    {
        return new Security(person);
    }
}

So for testing purposes we can just add an internal factory that will isolate the extension method behavior, and that we'll replace for testing purposes:

public static class SecurityExtensions
{
    internal static Func<IPerson, ISecurity> SecurityFactory = person => new Security(person);

    public static ISecurity Security(this IPerson person)
    {
        return SecurityFactory(person);
    }
}

The test for the Update method above, which will have internals visibility enabled from the project under test, will simply replace the factory with one that returns a mock:

public void WhenPersonHasNoModifyPermission_ThenThrowsSecurityException()
{
    var security = new Mock<ISecurity>();
    security.Setup(x => x.GetPermissions(It.IsAny<Uri>())).Returns(Permissions.Read);

    SecurityExtensions.SecurityFactory = person => security.Object;

    var controller = new DocumentsController();

    Assert.Throws<SecurityException>(() => 
        controller.Update(new Mock<IPerson>().Object, new Uri("http://foo")));
}

And voila! We have just passed a test where the class under test uses an extension method:

public class DocumentsController
{
    public void Update(IPerson person, Uri document)
    {
        if (!person.Security().GetPermissions(document).HasFlag(Permissions.Modify))
            throw new SecurityException();

        // ...

Also note that the intenal Security class that implements ISecurity can also be fully tested in isolation.

Happy moqing!

PS: remember the focus of this post is on mocking extension methods, not on how to design security for your domain ;-). Feel free to mentally replace ISecurity with IFoo, GetPermissions with DoSomething, etc., as needed.

Resetting Visual Studio Experimental Instance to its super-clean initial state

If you are doing Visual Studio extensibility (VSX) work, you are probably aware of the existence of the Visual Studio "Experimental" instance. This is basically an instance of VS that has its own isolated registry, settings, extensions, etc. This allows you to test your extensions to VS without polluting your main development environment.

Sometimes, the environment might get corrupted for whatever reason, or it might be that you just want to test your extension with a clean environment after messing with it for a while.

The Visual Studio SDK does come with a tool to reset the experimental instance, available from your Start menu with the name "Reset the Microsoft Visual Studio 2010 Experimental instance". That will not, however, give you the pristine environment you got the first time you start the experimental instance to test your first extension.

In order to get a super-clean environment, here's what you need to do:

Close any running instance of VS Experimental.
Delete the entire folder %LocalAppData%\Microsoft\VisualStudio\10.0Exp
Run regedit.exe
Delete the registry key HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\10.0Exp
Delete the registry key HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\10.0Exp_Config
Run the command "Reset the Microsoft Visual Studio 2010 Experimental instance" from your start menu.

Now you'll have a fully restored environment. Also, any existing extensions you have in your main environment will be copied over to the experimental instance, with one caveat: you'll have to manually enable them from the Extension Manager UI from a running experimental instance to get them running.

If you've been doing VSX work in previous versions of VS, you'll sure appreciate the drastically simplified install/reset experience enabled by the new deployment mechanism in VS2010 via Extension Manager and VSIX files. If you have not, then this doesn't sound so scary anymore, does it? ;-)

Happy extending!

Clarius Visual T4 70% off!

http://twitter.com/vga/status/6204238634

Go get it :)

Reactive Framework Extensions Generator

You probably know already that the Reactive Framework Extensions (Rx) is a new library on top of .NET 4.0 and Silverlight that allows developers to leverage the expressiveness and power of LINQ for .NET events. It brings an entirely new paradigm for doing event-driven apps, and therefore shines in WPF/Silverlight scenarios.

Read more about Rx at the team blog, the project home page and Matthew excelent blog series.

Even with the general availability of the bits for VS2010 beta2 at DevLabs, there's still quite a bit of work you need to do in order to leverage the extensions. Specifically, you need to turn your events into IObservables that can then use the Rx extensions for querying and subscribing. This is a lot of repetitive and boring code that can be easily automated.

That's precisely what this Clarius Labs provided extension does, enabling Reactive Framework Extensions for arbitrary assemblies, without writing any code. It basically traverses all public types in a given assembly (i.e. PresentationFramework.dll, for WPF) and generates a "Reactive" assembly for it (i.e. PresentationFramework.Reactive.dll) which contains the necessary extension methods for all public types that expose generic or custom delegate events that can be automatically converted to IObservables. With that in place, you can simply use the Reactive() extension method on your classes and access in a strong-typed fashion all events of that type as IObservables:

In order to get the extensions assembly generated, you simply right-click on a project or assembly reference, and select "Create Reactive Extensions":

and a new assembly will be generated and referenced automatically:

which will enable you to use LINQ operators and the new Observable APIs for all events exposed on all public types for the given assembly.

Matthew sample using our generator looks like:

var mouseMoves = from mm in mainCanvas.Reactive().MouseMove
                 let location = mm.EventArgs.GetPosition(mainCanvas)
                 select new { location.X, location.Y };

var mouseDiffs = mouseMoves
    .Skip(1)
    .Zip(mouseMoves, (l, r) => new { X1 = l.X, Y1 = l.Y, X2 = r.X, Y2 = r.Y });

var mouseDrag = from _ in mainCanvas.Reactive().MouseLeftButtonDown
                from md in mouseDiffs.Until(
                    mainCanvas.Reactive().MouseLeftButtonUp)
                select md;

Go to the Visual Studio Gallery and give it a try!

The myth that TDD or test-first slows you down is true

I'm sad to say it, but it is true. It slows you down. But not everytime, and not for everything. So let's be more specific on the cases where it DOES slow you down noticeably:

Cowboy or Duct Tape Programming Mode: If you code all day like crazy in a "cowboy" or "duct tape" programming mode, then it WILL slow you down, significantly. In this mode, you're hacking together stuff that works by pure luck, you're only superficially and typically manually "testing" the thing, before calling it "done" and move on. Someone will come after some day when the customer complains about a bug, and figure out what to do. Not your problem though, for sure. For all you care about, you might even have another job by then. Your productivity kicks ass, you're the fastest guy in the team, and can get "complete" features to your boss in record time. Everyone (who hasn't worked with you long enough) thinks you're a genious, 'cause you can get something that works on the first shot. Brilliant!

It may also be just the way you are, and it's not that you don't care. It's just that your fingers fly and the itch to get the code done is SO strong that you just can't help it. Once I overheard an architect saying "I'm gonna tie your fingers with rubber bands!", hehe.

TDD WILL SLOW YOU DOWN, A **LOT**.
Visual Studio Extensibility or Mobile Development: the red/green/refactor cycle is just so brutally slow that it's a real endurance test to try to keep doing TDD in many cases. Yes, you can isolate the IDE/Mobile environment, mock stuff here and there, but at the end of the day, the mock does not guarantee that things will actually work on the real thing. So you need to run tests on the actual environment more often than not. You'll spend a great deal of time trying to isolate slow areas, trying to write more code before you run just to avoid the slow turnaround (and then fixing multiple things in a single pass when multiple fail), etc. A total TDD-anti-climax for sure :(

Show Customer Value, Quickly: there are some cases where the domain you're tackling is not very well explored, or the customer doesn't have a very clear idea of what to expect, or doesn't even believe you can deliver something valuable or innovative at all. In these cases, it's much more important to have a lot of tangible value/features soon, than to have little but very robust and well designed via TDD.

Of these, the first one is one that requires careful attention and mentoring. Being a speedy programmer certainly is a good quality and an important one to being great, I think. There are times when you just need to be bloody fast and get the job done quickly (third case). Such a developer is invaluable in an organization. But he must understand the constraints of the current project. Getting rock-solid, maintainable and well-designed code might be much more important than raw amount of features/code.

The second and third cases are just a matter of setting the right expectations, and understand how you're going to approach development. In either, you might skip TDD altogether, or do it only in more complicated areas only. Or you might just have what you can as integration tests only.

And yes, being pragmatic is also a very good quality to have. Shipping is indeed a feature.

Writing meaningful, self-documenting, behavior-oriented tests

Over the years I've come to realize that the one-fixture-per-class approach to unit testing just doesn't scale. As the amount of variations in state and interactions increases, that file starts becoming a big soup of "Should" methods that are increasingly difficult to traverse and find later on. Essentially, since every test is doing the first "A" in AAA (Arrange-Act-Assert) too, that means the context is also part of the test method.

You can only make a method so long and remain understandable at a glance: IfRepositoryContainsACustomerAndAddingANewOneWithSameIdButDifferentAliasThenThrowsInvalidArgumentException. So, a while ago at Clarius we started exploring some of the concepts behind BDD (Behavior Driven Development), Context/Specification, etc., while working on an internal project.

I was more than pleased with the compromise we made to accommodate our current tools, and to land in a place that is not totally alien for the regular "barebones"TDD guy. I'm convinced that changing paradigms just for this isn't probably worth it.

In my last post I hinted at one of the core values I see with TDD/unit testing: self-documenting code. Any kind of non-trivial code will involve a series of back and forth with the customer, numerous changes to a written spec (if you're lucky to have one) and its increasing and inevitable obsolescence. In most cases where I'm given enough freedom (i.e. the customer knows us well and trusts our development process and guaranteed quality), I usually get very few written specs, if any. A lot of details are just talked over standup (or IP), and many details are left for us to figure out. And that's fine: that's the reason you're the one with the keyboard writing code, and with the brains to make reasonable design choices if you understand the problem space well enough.

You could get picky with the customer and start asking for explicit definitions for everything. Tried that too, doesn't get you too far. In a short term, you're either out of the project 'cause you're too much of an overhead to an already busy person, or the customer gets even more exasperatingly vague and confusing as he tries hard to explain in detail something that doesn't even know yet for sure, while leaving crucial execution paths out, at which point you either quit politely, or go "don't worry, let me figure that out for you". Back to square zero.

The fact that the customer didn't know quite yet what he wanted, does certainly mean he doesn't want to know what he's getting. What you thought it was that he needed. So, you need to write whatever you write in such a way that it's easy to transform into a human-readable form that you can hand out to your customer.

So, without further ado, here's the way we write tests:

namespace Runtime.Workflow.JoinSpec
{
   [TestClass]
   public class GivenAJoinWithTwoPredecessorsAndOneSuccessor
   {
     // ctor builds up the context

     [TestMethod]
     public void WhenAPredecessorIsAvailable_ThenJoinIsBlocked()
     {
       // set predecesor state,
       // verify the join has the given state
     }

     [TestMethod]
     public void WhenBothPredecessorsAreFinished_ThenJoinIsFinished()
     {
       // set predecesor state
       // verify the join changed state to Finished
     }
   }
}

How it works:

The last part of the namespace becomes the logical grouping of the tests. This typically is the name of the class under test plus the "Spec" suffix.
The test class starts with "Given" and the phrase that follows describes what's instantiated in the constructor and typically stored in fields for use by tests. The Given is the Arrange in AAA.
Test methods have two parts: "When" and "Then", separated by an underscore.
- When: describes the action or state change that is caused in the context to perform the test. This is the Act in AAA. This is typically just one operation, but it could be more if changing the state/acting requires so.
- Then: the Assert in AAA. Typically just one Assert or mock Verify, but there could be more than one if verifying the state/interactions require so. But in either case, the Then should describe what you're asserting.

Key benefits of this approach:

This is plain MSTest code. You could as well use xUnit, NUnit, etc. No new paradigms to learn, just some naming conventions.
The only additional "overhead" is having a separate context (Given) class to group related tests (those tests that use the same setup).
Having a convention in place for how to write tests has proven immensely valuable on its own. I can navigate our tests and not tell the difference on who wrote which tests.
It triggers good practices about test complexity almost automatically: because context + tests have to make sense as an english phrase, sometimes you realize that a given test is testing too much (the test method becomes TOOOO long to write).
It's trivial to write code that uses reflection to render this as a document

We use this as a guideline. There's no requirement that we have a context class. Sometimes, it's just not worth it because you're testing a very small unit. In this case, the *Spec becomes the class, such as:

namespace Runtime.Workflow
{
   [TestClass]
   public class FinalSpec
   {
   }
}

This is typically more the exception than the rule, though.

To render specs I quickly put together a query that uses reflection:

public class RenderSpecs
{
    public void Render()
    {
        // Change and run with TestDriven.NET to get the specs for a given 
        // namespace:
        Render("Runtime.Workflow", Console.Out);
    }

    public void RenderAllSpecs()
    {
        using (var stream = File.Open(@"..\..\Specs.txt", FileMode.Create))
        using (var writer = new StreamWriter(stream))
        {
            Render("", writer);
        }
    }

    private void Render(string withinNamespace, TextWriter output)
    {
        var specs = (from type in this.GetType().Assembly.GetTypes()
                     where type.Namespace != null && type.Namespace.StartsWith(withinNamespace) &&
                         type.GetCustomAttributes(true).OfType<TestClassAttribute>().Any()
                     from method in type.GetMethods()
                     where method.GetCustomAttributes(true).OfType<TestMethodAttribute>().Any() &&
                        method.Name.StartsWith("When")
                     orderby type.Namespace, type.Name
                     select new
                     {
                         Type = type, 
                         Method = method,
                         //Phrase = method.Name,
                         When = ToPhrase(method.Name.Substring(0, method.Name.IndexOf('_'))),
                         Then = ToPhrase(method.Name.Substring(method.Name.IndexOf('_') + 1)),
                     })
                    .GroupBy(x => x.Type)
                    .OrderBy(x => x.Key.FullName)
                    .GroupBy(x => x.Key.Namespace);

        foreach (var ns in specs)
        {
            output.WriteLine(new string('-', 50));
            output.WriteLine(ToPhrase(ns.Key.Split('.').Last(), false));

            foreach (var context in ns)
            {
                output.WriteLine("    " + ToPhrase(context.Key.Name));
                foreach (var spec in context.OrderBy(spec => spec.When).ThenBy(spec => spec.Then))
                {
                    output.WriteLine( "        " + spec.When + ", " + spec.Then);
                    //Console.WriteLine("\t" + spec.Phrase);
                }
            }
        }
    }

    private static string ToPhrase(string pascalCasedPhrase)
    {
        return ToPhrase(pascalCasedPhrase, true);
    }


    private static string ToPhrase(string pascalCasedPhrase, bool toLower)
    {
        var builder = new StringBuilder();
        builder.Append(pascalCasedPhrase.First());

        for (int i = 1; i < pascalCasedPhrase.Length; i++)
        {
            if (Char.IsUpper(pascalCasedPhrase[i]))
                builder.Append(" ");

            builder.Append(pascalCasedPhrase[i]);
        }

        var phrase = builder.ToString();

        if (toLower)
        {
            phrase = phrase.ToLower(CultureInfo.CurrentCulture);
            // Make only When and Then upper case
            phrase = phrase.Replace("given", "Given").Replace("when", "When").Replace("then", "Then");
        }

        return phrase;
    }
}

Note that this class is not a test class or test method. That's because we render on-demand. When I need to discuss or explain how a given area works, I'll go and render the specs first, email it and then meet. To run the Render method, I use TestDriven.NET which can run any method on any class (with a default constructor). I use it to run all tests too :), it's SOOOO much speedier than the VS runner...

Are you smart enough to do without TDD

Ayende wrote a controversial post titled ~~I'm so smart I don't need TDD~~ Even tests has got to justify themselves ;-). It's important to read it, because it reinforces many of the reasons why "regular developers" (i.e. NOT *you* if you're even reading blogs as you are) continue to see "us" as some kind of unreachable and infallible elite of "hero programmers" who will eventually show up (i.e. be hired for big bucks, which we surely do want :)) and save the day.

You see, Ayende appears to say that if you're smart enough, you'll just know what code to write, just like that. Ergo, if you don't know, maybe you're not that smart and hence you would need this technique for losers called Test Driven Design/Development.

That's not how it works, at least for me. Far from it. I've been doing TDD for years on several projects and with varying degrees of similarity. And I can tell you that even for those where I already had a very clear idea of an initial design, I always ended up with something (however slightly) different after doing it TDD-style. It consistently enriches my APIs by providing me a users point of view that an integration/scenario test would never give me.

Spikes

With regards to design uncertainty (which is what Ayende mentions as his only motivator for doing TDD), I usually take a different approach altogether: run a quick time-boxed spike (or several), to test a couple design choices quickly, without the "overhead" of doing it "right". These are throw-away spikes that you learn from. When I'm done with the learning, I go back to doing it with TDD, and it's almost guaranteed it will not look 100% like the spikes, and that it will be much more robust and user-friendly.

Documentation

A new appreciation I'm developing for TDD when done with certain consistent naming conventions (i.e. Given, When, Then style), is the ability to have a human readable and always up to date specification of what the various components do. Yes, this is not something you'll show your end users, but it IS something the developer or project lead coming after you can certainly learn from. Ayende assumes everyone will be equally smart as he is and immediately grasp his software designs, 'cause you know, there's only one way it could have been done right :P. In order to fix bugs and maintain non-trivial software, you need to know what individual components are doing.

Proof

I'll refer to one case of each situation where TDD provided value.

Moq vs Rhino Mocks: he read the (useless IMO) literature on mocks vs stubs vs fakes, had apparently a clear idea of what to do, and came up with Rhino's awkward, user unfriendly and hard to learn API with a myriad of concepts and options, and a record-replay-driven API (ok, I'm sure it was not his original idea, but certainly it's his impl.) which two years ago seemed to him to stand at the core of mocking. Nowadays not only he learned what I've been saying all along, that "dynamic, strict, partial and stub... No one cares", but also is planning to remove the record / playback API too.

I'm pretty sure that if he had sat down with a blank project, two years ago, and rebuilt Rhino Mocks using TDD and a fresh mind, he would have ended with something very similar to Moq, way earlier, rather than seeming to be playing catch-up.

Moq on the other hand started from a blank slate, purely TDD-driven, with no preconceptions whatsoever on its API (other than the conviction that we just need one word, "mock"). I'm obviously biased, but users seem to love its simplicity too.

Complex behavior: on a farily complex workflow-related implementation, I recently got asked what the behavior of a Join node was in our project. I could just run a test that does a very simple reflection-driven query and get the following answer to the project lead:

Join Spec
  Given a join with two predecessors and one successor
     When a predecessor is available, Then join is blocked
     When a predecessor is blocked, Then join is blocked
     When a predecessor is in progress, Then join is blocked
     When a predecessor is unknown, Then join is blocked
     When a successor state changes, Then join state does not change
     When both predecessors are finished, Then join is finished

That is simply invaluable. Anyone coming later to the project only needs to read that to grasp an immediate understanding of the intended behavior. And it's isolated and unit-tested. How does the test code look like? Well, pretty much like your regular TDD-style, but with some naming conventions:

namespace JoinSpec
{
   [TestClass]
   public class GivenAJoinWithTwoPredecessorsAndOneSuccessor
   {
     // ctor builds up the context
     [TestMethod]
     public void WhenAPredecessorIsAvailable_ThenJoinIsBlocked()
     {
       // set predecesor state, and verify the join is with the given state
     }
   }
}