Don’t write spec documents, throw away any you have near you, they’re next to useless anyway. It might seem like a hearsay to some but it’s true and deep down even the most most anal planner in all of us knows it. Two of the biggest reasons for this are:

• size and scope: In order to address every aspect of your project a full specification document has to, well address every aspect of your project and it’s going to be huge as a result.
• lack of vision: For most people relating what’s written on paper to a vision of a design in their heads is difficult if not impossible.

Loose documents don’t work well either, because leaving the obvious unsaid just leads to problems, chiefly not everything is obvious to everyone. The devil is in the details and avoiding them until the end of a project is just going to cause trouble.

“Why haven’t we implemented x, that should have been obvious!”

create the interfaces

Prototype the interfaces and use them as your spec. Working this way has so many benefits, try it once and I promise you’ll be converted. Before you write a single line of backend code, sit down and work out your interfaces. Build each screen, put all the buttons, fields and elements on the page

I normally start with index cards and write a few simple requirements for each screen or page - just a few simple sentences listing what the page is expected to achieve.

My next step is to turn those requirements into interfaces.

in HTML, not photoshop

Photoshop is great, but let’s face it it’s no better for trying to work out a functional interface prototype than some paper and a pencil. It’s very easy to focus on making things look pretty over making them functional and you risk missing something important out.

The best way to avoid both of these traps is to write your interfaces in HTML, mark them up semantically as they will be marked up in the release and use them as your working spec.

I know good design but I’m no designer

Another benefit of coding up your interfaces in HTML before you start writing code is you can hand them off to designers so they can make them look pretty as you’re working on making them work. I have a pretty basic idea of design principals and the tools - but I find it hard work. So with interface first development I don’t have to think about it.

let the testing begin

Another thing which is made infinitely easier with interface first development is functional testing, because your testers don’t have to wait for a release before they can start developing a test suite. You are using an automated test suite aren’t you? By generating interfaces as developers add functionality testers can already have been through the interfaces and written a whole series of failing tests.

updating later

The great thing about using HTML as your design and communication tool is that when it comes to making a change to a page you have the most up to date working copy there ready to modify in the form of your site. Just save the page, add the change to the HTML and you’ve got your interfaces ready for discussion and later implementation. What could be easier, faster or more clear for everyone?

it’s not a panacea

There have been times when I’ve been working with notoriously difficult individuals where nothing was going to satisfy. An exmaple that springs to mind is a multi month project where we produced an interface design early in the project. The design was a set of PDF images not HTML (see above), but the delivered product was a pixel perfect implementation of it. Only once the project was delivered was there any input on the design or functioning of the project - I’m not just talking things that a PDF can’t show you like what clicking a link looks like. I’m taking basic things like colours, layout and the size of elements!

Would a HTML mock up have made any difference, I actually doubt it. I take blame for miscommunication and resulting mistakes when it’s my fault but in this case even after the site was fully developed and deployed to a staging area for testing none of these issues were highlighted. It was only after the site was put into production that these “critical issues” were discovered.

I’ve been trying to figure this out for months, and thought it should have been simple. All I wanted to do was write a set of unit tests to test my java code, have them run whenever I hit test. Next I wanted to have a set of HTTP Unit, Selenium or similar tests run to test that the actual application is working when it’s built and deployed to a container using the cargo plugin.

I didn’t really want to have a separate project to do this because it seemed like a massive pain in the ass to maintain the whole thing. I didn’t see any reason why it shouldn’t be easy with Maven either - it does have a phase for integration-tests already.

My first try looked good to me:

...
<build>
    ...
    <plugins>
        ...
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <configuration>
                <excludes>
                    <exclude>**/integration/**</exclude>
                </excludes>
            </configuration>
            <executions>
                <execution>
                    <id>integrationtest</id>
                    <phase>integration-test</phase>
                    <goals>
                        <goal>test</goal>
                    </goals>
                    <configuration>
                        <excludes>
                            <exclude>none</exclude>
                        </excludes>
                        <includes>
                            <include>**/integration/**/*Test*.java</include>
                            <include>**/integration/**/*Test.java</include>
                            <include>**/integration/**/*TestCase.java</include>
                        </includes>
                    </configuration>
                </execution>
            </executions>
        </plugin>
        ...
    </plugins>
    ...
</build>
...

I tried to run the tests, but unfortunately when it gets to the second round of tests - which should run my integration tests - it runs the same sets of as it ran the first time. I tried adding the combine.children="append" attribute to the includes, and excludes but that didn’t work either. Finally I came across a source file XppDom, part of plexus which maven uses. XppDom allowed the combine.children attribute as well as another I haven’t seen mentioned anywhere else childMergeOverride. I added that instead and it worked!

...
<build>
    ...
    <plugins>
        ...
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <configuration>
                <excludes>
                    <exclude>**/integration/**</exclude>
                </excludes>
            </configuration>
            <executions>
                <execution>
                    <id>integrationtest</id>
                    <phase>integration-test</phase>
                    <goals>
                        <goal>test</goal>
                    </goals>
                    <configuration>
                        <excludes childMergeOverride="true">
                            <exclude>none</exclude>
                        </excludes>
                        <includes childMergeOverride="true">
                            <include>**/integration/**/*Test*.java</include>
                            <include>**/integration/**/*Test.java</include>
                            <include>**/integration/**/*TestCase.java</include>
                        </includes>
                    </configuration>
                </execution>
            </executions>
        </plugin>
        ...
    </plugins>
    ...
</build>
...

Setting it up for Cargo

I’ve done this on a few projects, the first was a project that built a series of taglibs which are used across a number of applications. The release for the project is a jar so I have a separate test web-app structure in ${baseDir}/src/test/web-app. The code for generating this war looks like this:

...
<build>
    ...
    <plugins>
        ...
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-war-plugin</artifactId>
            <executions>
                <execution>
                    <id>generate-test-war</id>
                    <phase>pre-integration-test</phase>
                    <goals>
                        <goal>war</goal>
                    </goals>
                </execution>
            </executions>
            <configuration>
                <warSourceDirectory>${basedir}/src/test/webapp</warSourceDirectory>
                <warName>${project.artifactId}-test</warName>
                <webappDirectory>${basedir}/target/${project.artifactId}-test</webappDirectory>
                <primaryArtifact>false</primaryArtifact>
            </configuration>
        </plugin>
        ...
    </plugins>
    ...
</build>
...

During the pre-integration-test phase the above simple generates a test war. If you’re project is a web-app and it already generates a war you can skip the above as one will be generated for you already.

The next step for was then to get Cargo to deploy the application:

...
<build>
    ...
    <plugins>
        ...
        <plugin>
            <groupId>org.codehaus.cargo</groupId>
            <artifactId>cargo-maven2-plugin</artifactId>
            <configuration>
                <wait>false</wait>
                <container>
                    <containerId>tomcat5x</containerId>
                    <zipUrlInstaller>
                        <url>${integrationtests.tomcatURL}</url>
                        <installDir>${installDir}</installDir>
                    </zipUrlInstaller>
                    <output>
                        ${project.build.directory}/tomcat5x.log
                    </output>
                    <log>${project.build.directory}/cargo.log</log>
                </container>
                <configuration>
                    <home>
                        ${project.build.directory}/tomcat5x/container
                    </home>
                    <properties>
                        <cargo.logging>high</cargo.logging>
                        <cargo.servlet.port>8080</cargo.servlet.port>
                    </properties>
                </configuration>
            </configuration>
            <executions>
                <execution>
                    <id>start-container</id>
                    <phase>pre-integration-test</phase>
                    <goals>
                        <goal>start</goal>
                        <goal>deploy</goal>
                    </goals>
                    <configuration>
                        <wait>false</wait>
                        <deployer>
                            <deployables>
                                <deployable>
                                    <location>${basedir}/target/${project.artifactId}-test.war</location>
                                    <type>war</type>
                                    <pingURL>http://localhost:8080/${project.artifactId}-test/index.html</pingURL>
                                    <pingTimeout>300000</pingTimeout>
                                    <properties>
                                        <context>${project.artifactId}-test</context>
                                    </properties>
                                </deployable>
                            </deployables>
                        </deployer>
                    </configuration>
                </execution>
                <execution>
                    <id>stop-container</id>
                    <phase>post-integration-test</phase>
                    <goals>
                        <goal>stop</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
        ...
    </plugins>
    ...
</build>
...

Most of this is pretty self explanatory, I try to use parameters to configure as much as I can and try to maintain it in higher level POMs wherever appropriate. This is especially true for the tomcat URL which changes on a regular basis. The seem to post new releases and remove the older ones, and it can be a chore to keep up in multiple projects. You may want to look at the cargo plugin documentation, you can ignore some of what I’m doing above if you’re project’s default artifact (the one package creates) is a web-app. 

So there you have it, eventually I got there after a lot of digging. I’ve been pretty brief in my explanations and have assumed a fairly good understanding of maven. if you have any questions by all means leave a comment and I’ll do my best to make it clearer.

Related articles by Zemanta

• Testing Is Not Technically Hard, It Is Hard Because It Requires Clear Thought and Understanding
• Why do we (not) write tests

In a recently conversation I was told of a situation which I’ve experienced in the past and I’m sure others have as well.

While on holiday the code I’m responsible for has been modified by an external contractor. Before my holiday I pointed out that we should make sure version control is used by anyone making any change. It wasn’t. Now I’m frustrated and pissed off.

The problem isn’t that someone else was editing the code, or that I somehow lost control of a fiefdom or anything like that. It’s that I’ve just spent hours I could be using to do something else tracking down file modification dates and then diff’ing them against my versions out of version control. Time I shouldn’t have to had spent doing anything

The bigger issue is now I’m under pressure by my boss to deliver some changes at the same time as I have to deal with this, and no extra time has been allowed for it.

You pay contractors to do work for you, a large part of that should be to insist they do it in a way that is consistent with your working practices. You wouldn’t let them commit changes to a PHP website in ASP, Perl or Python and equally you should insist that changes are delivered consistently and in a manageable format.

Warning flags get raised if someone tells me they don’t use version control. Not knowing how and asking questions is one thing; I have no problem helping someone, a contractors included, learn our version control system and practices because I see our relationship as a long term one, so any investment in that relationship is worthwhile. If a contractor tells me they cannot or will not use our version control system though, we have a problem and usually we end the relationship then and there. It may sound like a hardline call based on a single criteria but it is a deal breaker for me.

For a contractor there could of course be some concern that they’ve submitted the work and may not be paid. Version control is not the cause of this - it’s dishonesty, plain and simple and if the person you’re working for is that dishonest submitting code via email before the cheque clears is equally risky.  

Some ground rules

Our most basic rule is commit at least once a day while you’re working on the project. This is especially the case if the work is being done on a day rate, but also if the project was quoted on a fixed rate basis. We use the data to judge if a project is on time, we’re completely upfront about why we want this: Knowing the current velocity for changes gives us a really good indication of where we are in the development cycle. With a current version committed to our repository we can just checkout, deploy and see. It removes the chances of miscommunication, misunderstandings and estimate errors.

Committing early and often by contractors is much like releasing early and often to users. It allows us to make regular builds and run QA on the project. The benefits of this are that we can start to run our testing regime long before we would otherwise be able to test. We try to encourage external developers to follow similar methodologies to those we use internally, the primary goal of any new project is to make a deployable release as quick as possible, then add features to it in a sensible manner. This coupled with regular commits allows us to test features as they’re added, not weeks down the line days or hours before the project is to go live.

Once bitten, twice shy

We’ve had a few projects in the past where we outside contractors for a variety of reasons failed to deliver on time. This has lead to our losing a follow up contract on at least one occasion we’re aware of. By getting code committed to our repository we can see where a project is, and if necessary step in early with concerns about the deadline.

Both daily commits and QA help immensely with this and all but eliminate estimate errors because we can have an honest objective conversation about how far we’re going to miss milestones very early in the process because a miss becomes obvious when you can see the velocity. The earlier you can do this, the better your chance delivering on time.

Rules for some, but not others

Sometimes it doesn’t make sense to go the whole forcing version control down someone’s throat route. For some projects and changes we’ll just bang a few files into an zip archive, email it and get the same back, then integrate it ourselves. It usually depends on the scope of the changes and how long they’re likely to be working on the project in question. We would only work with someone familiar or willing to learn version control though, even in the above scenario they’d would have to be willing to use it if we’d asked them to.

How have your experiences using version control with contractors gone? Post a comment and tell us about it!

I think tomorrow I’ll post a bit of more hands on article on setting up version control for contractors. I’ll try to focus a little more on how to build, install and implement subversion with access controls. See you then.

The address bar is at the top of every user’s browser and it’s often totally overlooked by developers. We all know we should think a little when choosing a domain name, well mostly, but after we’ve done that normally let the our toolkit or internal organisation work against the user and clutter up the address bar with crap.

it should make sense to the user

All developers primary motivation, for the address bar as with the rest of the ui, should be something that’s useful to a user. It’s there on every browser, on top of every page of every visitor to your site. Make them short and simple, easy to see where you are, easy to remember and easy to type.

http://mysite.com/products

http://mysite.com/services

http://mysite.com/services/urls

http://mysite.com/contactus
These are all good URLs, easy to see where you are and easy to remember.

I’ve seen a number of applications or frameworks that want URLs in the form http://somesite.com/site/PAGES/EN/index.php. It irritates me just seeing it. None of that cruft has anything to do with anything a user cares about. It’s inattention to detail, it’s endemic on the Internet and there is no excuse for it.

case sensitivity

A URL should not under normal circumstances be case sensitive. Storing something at the same place with the same name only differing in case is very confusing and counter intuitive to users. Don’t do it.

http://mysite.com/HOME

http://mysite.com/Home

http://mysite.com/home

These are all the same to most users and you wouldn’t actually put different pages at each of them would you? So why not do the sensible thing for the user and make sure that no matter what case they use it just works?

The case to use depends on a variety of factors. For most uses all lowercase is probably the best default choice, however using CamelCase in a wiki, casing for adherence to brand standards or common conventions are all reasonable situations where preserving case may be the prefered option. Preserve the case yes, but don’t make the URL case sensitive.

In the 37 signals book Defensive Design for the Web [amazon.co.uk] they point out a classic from a usability best:

I typed “www.apple.com/iTunes” in my browser’s address bar but instead of information on the MP3 application, I received a “Page Not Found” error. Even though the application is called “iTunes” apple’s site only accepts “www.apple.com/itunes” (all lowercase).

This is an unfortunate branding situation for Apple. Customers are actually punished for adhering to the brand’s naming conventions

At the time of this post on Apple has fixed it, both URLs will now give you the correct page but I think it’s a great example. As well as a great excuse to link to the excellent 37 signals book. It’s worth pointing out that the apple page currently redirects to the lowercase www.apple.com/itunes url so it’s not case preserving or using the correct case to adhere to the brand’s naming conventions, but at least it works.

file extentions

Why do so many websites have pages that end in html, asp, php, jsp, .do? Users don’t care about what language you’re using, telling them is pointless and worse makes it hard for them to figure out URLs from memory or by making an educated guess. Take the time to configure your server to at the very least drop the extensions.

The exception to the rule

There are times when a file extention is important, and developers should know when to apply the above rules and when for everyone’s sake not to- when a file extention dictates the mime type, or when you’re providing a file for a user to download you obviously want to maintain the extention. There is nothing wrong with having .pdf, .exe, .dmg, .gif, .jpg, .avi, .mov etc. Most of those files are embedded, or downloaded and should maintain their extensions.

Querystrings

I hate them. I really hate them. What the is index.php?ID=123 anyway, what does that mean to me? Keep your database IDs in the database and give me something useful. Blogs get this right a lot of the time, or close anyway. Forums are notorious for getting this wrong.

A querystring is alright, if you’re running a query, a search or something similar where various parameters are going to affect the outcome. It’s not alright if you’re pulling one element out of a database. It’s alright if you’re using an API. If you’re pulling a single item and the only identifier is a numeric id you can still do better than a query string. How about: http://mysite/items/12345/

why should we care?

It seems like a small thing to worry about when you’ve got a whole application to build. The URL is like a lot of little things, it shows attention to detail. It’s easy to make them sensible if you choose to and it will make some users lives easier. The short term pain of making all your URLs sensible will in the long run improve usability- you might even retain a few more users here and there and eventually that will add up.

Take for example something like a property sales application. Let’s give it two URLs for the sake of argument.

1. http://joespropertysite.com/PAGES/EN/SITE/property.php?ID=233223
2. http://joespropertysite.com/property/spain/malaga/2Bed/StunningTownHouse

I hammed it up a bit on the last one to prove a point- they’re both a similar number of characters but the second is so much better even though it’s longer!

• The second is much more user friendly.
• The second URL is much more information rich than the fist. You’re looking at a property, on a page, on a site written in php in english - or - You’re looking at a property, in Malaga, Spain with 2 bedrooms and it’s a ‘Stunning town house’.
• To quickly broaden a search for related two bedroom properties in malaga it’s obvious you can just remove the last part of the URL on the second. The first you’ll have to use the search function.

No excuse for… regular features on blogs

This might turn into a regular feature, I’ve got a growing list of things and there is no excuse for them. I think what I’ll do is queue them up as I write them then post them for a Friday afternoon read.

Jeff Atwood was pleased to find that his coding conventions, the ones we all settle into, have been tending towards those of a style of programming called Spartan Programming. Which is minimalist programming, as in “a spartan lifestyle”. Check his post out for a more in depth explination but suffice to say the aim is to reduce complexity in any way possible.

I’ve got to agree that more and more I’ve tended in that direction too. I don’t remember the last time I had more than a handful of conditionals or loops in a method. My methods are terse and accomplish one thing, sub tasks are split out into other methods. It’s come very naturally, but it has taken time. I don’t think it’s something you can be taught, or learn. It’s definitely a practice thing.

One aspect of Spartan Programming where I disagree is with the terse variable names. I think they could end up introducing more accidental and avoidable errors. They lessen the utility of some key IDE features. And generally would just make my life harder, so I’ve gone the other way. Once upon a time I would happily create the following code:

public void sendMessage() {
    MimeMessage m = new MimeMessage();
    m.setBody("This is a test");
    m.send();
}

These days I try to avoid calling variables things like m and prefer to call them what they are:

public void sendMessage() {
    MimeMessage message = new MimeMessage();
    message.setBody("This is a test");
    message.send();
}

I know the arguments and logic:

• The method should be simple enough that the variable name m is clear enough.
• The shorter name reduces complexity and therefore errors.
• Less characters means lines don’t get too long

Of these three the third is the only one I really buy though. Long variable names and too many characters on a line can push some stuff too far to the right potentially hiding errors or overly complex code; but it’s an edge case and there are other ways to solve the problem.

While true that in a short minimalist method a variable m would be clear enough, a variable called message is equally clear. The shortening doesn’t gain anything in clarity. The argument for less characters reducing errors in the era of modern IDEs doesn’t wash with me either - My IDE tells me of I mistype a variable and auto completes them furthermore by typing actual words I actually improve the accuracy even more.

Shorter names on the other hand can tend more towards a crowded namespace and can result in properly typing the wrong variable. This is especially a problem in a weakly typed language.

Consider the following:

function sendInviteMessages(n)
    for i = 0 to n
        Set m = Server.CreateObject("MimeMessage)
        m.Body = "Welcome to our fancy new site!"
        m.Send
        Set n = Nothing
    next
end function

What you see here is something that wouldn’t happen, or not as often with a variable name like number instead of n and message instead of m. Worse still, depending on the language and how it treats null variables it might even pass a quick test with a single address and appear to be working!

The one place I totally disagree with the terse variable naming conventions though, is the method parameters. That actually makes life harder when working with code completion. Which is easier to follow?

There are plenty of other code completion and helpful shortcuts in a modern IDE where the parameter names are used to make our lives easier. All of which lost, with no real gain.

I’m not arguing that variable names shouldn’t be terse and sensible. Going back to the first example the following is just stupid and is far worse than using just m, it’s practically illegible and very difficult to just eyeball the code to see what is going on.

public void sendMessage() {
    MimeMessage theMessageToSend = new MimeMessage();
    theMessageToSend.setBody("This is a test");
    theMessageToSend.send();
}

Other times I go the opposite way and have adopted conventions for making some variables extremely terse. InputStream and OutputStream variables for example I always call is and os. So I do try and be as terse as possible, but never terseness at the expense of obviousness of intent.

While I agree with most of the practices and like Jeff was nodding along as I read about Spartan Programming, there are a few I don’t fully follow and some I’m against. In the end though I tend to agree with his assertion that Minimalism isn’t always the right choice, but it’s rarely the wrong choice as long as it’s not at the expense of clarity of intent.

Agile methodologies have recently been getting something of a bad name, it’s not unwarranted but it is unfortunate. The problem isn’t actually the process, an agile process will work very well - if the infrastructure supporting it is capable of of being agile as well. By supporting infrastructure for a process here I don’t mean servers and software I mean the people and the organisational foundations. The organisation trying to use an agile process must it’s self be agile, quick to react to change. Agile processes that work have one thing in common, they encourage features to be released early and released often.

A pretty basic overview of our process goes something like this:

Big Bag of Features

Our projects and releases go through a number of stages, the first we call The Big Bag of Features. When a project is really just an idea, it’s something we might want to build and we basically rough out some ideas and start to grow them into all sorts of crazy future versions, we talk about all the features we’ll need to achieve them and how we’ll go about it.

Minimum Stable Feature-set

From the BBoF we usually end up with at least one, more often than not a number of possible outcomes and development paths. By talking through all of the options, all the features and all the future versions we’d like to be able to build leads us naturally to something we can build; a vision for a product.

Citrus is a good example of this practice, we’re at the stage where we’ve decided the vision for the project, have a big bag of features we’d like to implement and are now in the process of creating our MSF

From the vision we decide what we need to make a product work; less is definitely more at this stage. We normally take each feature we’ve decided would be nice to have and try to think about who would use it and why. If we can’t come up with a compelling argument for most of our users using it, we normally chuck it back into the bag to worry about later. We refer to the result as theMinimum Stable Feature-set (MSF) or Minimum Launch Feature Set.The bare minimum we need to launch a working product.

The MSF becomes the next iteration, and once it’s set we don’t alter it. Nothing is added or taken away. On the surface this might seem either silly or a bit high and mighty, but we can do it because we put the work in ahead of time. We’ve discussed how we envision the product being used and we make sure we’ve covered all the bases. New great ideas just aren’t important enough to include, no matter how good they are. We leave them for another iteration and another day.

These ideas can be politically disastrous in some organisations, it’s downright heresy to tell the boss, or the sales director or a colleague what amounts to: their newest great idea just isn’t good enough. It’s important for an organisation to have focus though, this is where the MSF and short iterations come into their own. Releasing early and often means that you’re quickly on to the next round of updates and those great ideas can be looked at objectively alongside anything else in the Big Bag of Features.

Getting started

Finally we start coding the interfaces first, then backend to meet our MSF. In the case of a new project partial interim releases are made for testing. We try to work out an entire area of the project first then release that to our testing audience for feedback. As the iterations contiune towards release we add all the features from our MSF.

Updates

When it comes to updating projects we throw away the Big Bag of Features and start all over again.You heard it right, throw them away, gone. A long time ago I learned that feature lists just cause trouble. You can’t help but schedule them, the schedules are very loose and they’re never met but still promises are made based on them. In the end they start to read more like a list of failures, things your application doesn’t do. More importantly, things your application will more than likely never do. So we throw them away and start again with a new Big Bag of Features, the most important features will bubble to the top of your memory anyway and will end up right back in this new BBoF.

For an update our MSF is usually a single feature, or small feature set if they work best that way. The aim is to keep the releases small and concise, targeting one area of functionality. This doesn’t mean that we don’t set milestones or visions for the future development of the product, we normally make a milestone accomplish some greater goal or vision for the product. But we try not to roll a list of disparate fixes to various parts of the application into a single release, we break them up into many smaller releases.

Rinse and Repeat

We continue in this way for the life of the project releasing features to users early and often so bugs are spotted quicker and we never go too far down a dead end, creating something users won’t use. It’s during this stage that we really work hard to measure how an application is being used, or not used so we can focus our energy only on the most important parts. We never try to be everything to everyone, we’re not afraid to disappoint some people or to just say no. We want our application to be great at what it does, not mediocre at everything.

All software is a work in progress none is ever perfect. It will always have bugs, sometimes it might even seem that the software is nothing but bugs an it’s in these dark times when we normally say fuck it, let’s start over. Not always, some of the good excuses I hear for a rewrite are:

• New blood Someone new comes along and the first thing they suggest is a complete rewrite - It’s alarming the number of times I hear this suggested. Sometimes it’s contractors, which is somewhat exusable - it’s more billable time for them. Other times it’s by people newly in charge or new to the team.

• Team members have left - rightly or wrongly they’ve been blamed for the code, and coding them out seems like a good idea

• Too many bugs - An effort to hammer out the bugs feels like it would take longer than to rewrite from the ground up

• New tools! - A shiny new framework is out and you want to use it, it will solve all your problems and put all your spare cycles to work sorting out poverty and war.

Common to all the above is the fact that everything seems like it would be easier if only you could get out from under the crushing weight of legacy. And it’s a very attractive fallacy; Bugs could be solved by reworking the architecture; Features would be easier if only the code wasn’t written that way; That new framework would fix the bugs we’re seeing because of the one we’re using.

resist the temptation

They sound like good reasons, but that’s the problem with attractive things that are bad for us - they’re attractive. It’s so much easier to believe that the grass would be greener if we re-turfed the lawn than to work out how to fix those brown patches.

it almost always takes longer than you expect

A complete from the ground up rewrite will take longer than you think. It will probably take almost as long as it took you to get where you are now. It’s tempting to believe that you don’t need planning meetings to work out what has to be done. It’s right there a spec in the current implantation, just get started! So you’re initial estimate is just a stab in the dark anyway.

Add to this the further false belief that since you implemented the features once already, you’ll be able to do it faster this time around. You won’t, well you might - but you won’t implement them as optimistically quick as you think you will.

Throughout the process you’ll also be further from a releasable product than you think you are - much further. In a rewrite you have to rewrite every feature all those “all we have left in component x to do is…” bullet points add up.

All of this leads to one last problem with timescales for rewrites - in order to get anyone else to buy into them they have to be wildly optimistic about the schedule. So just when you need everyone most motivated and excited, as you’re gearing up for a release, you’ve set them up to be demoralised by missing an increasing number of deadlines.

you’re reinventing the [buggy] wheel

Programmers are people, people are creatures of habit. You’ll inevitably end up implementing part of your radical new system in a very similar way to your existing system. Then you’ll be forced to make the same compromises as you’ve made before and you’ll undoubtedly introduce some bug that’s already been patched in your existing software.

Some of your old code will also be good code, at best you might cut and paste it into the new code base but for the most part though trying to hammer it into place and wire it up is going to be as long and as buggy a process as rewriting it. So instead you follow the rewrite it all logic and rewrite it all.

you’re standing still

When you’re rewriting your software you’re stainding still in almost every way possible. If you’re rewriting everything, you’re spending an awful amount of time not moving your application or business forward. Personally and professionally you’re not learning much new (unless a new framework is your excuse).

Even if you’re squeezing some features into your rewrite, probably because it’s been so long since a release you’d be lynched by a mob if you refused, it’s so long before the users are actually going to see them that you might as well not be. The perception of your software isn’t going to be sometimes shaky, but getting better all the time - it’s going to that it’s buggy and the bugs are never fixed.

you’ll probably end up no better off

The sad and somewhat painful truth is if a team can’t fix something - getting them to make a new one is probably not going to fare much better. Your new version is probably going to be no better than the old, will certainly be worse than an itteratively developed and incrementally improved version of what you had could have accomplished in the same time.

Also problems you’re having with a framework or tool probably aren’t going to be magically solved by using a new one. Most of the these arguments are born out of a naive belief that because the framework isn’t doing what the developer is expecting, the framework has bugs, changing frameworks should then solve the bugs. The problem is of course developer expectation versus reality and more often than not due to some underlying misunderstanding of the platform the framework or tool is abstracting - a reality that a new framework cannot change.

how to sensibly go about a whole rewrite of your software

While it’s not sensible to start a full rewrite of your software, it’s also ludicrous to decide that nothing written should be changed. Software development is a process not an event. If it’s not working fix it; if it’s complicated and bug prone, refactor it; if it’s shit, code it out. Just don’t throw the good out with the bad.

Start where it hurts

In the case of bug riddled software start with an area that is causing the most trouble. Patch the big bugs first. It can be a painful process, I won’t lie. It’s demoralising to everyone to have buggy software and it’s not exciting or glamorous to fix it.

Bug fixes are one of the most thankless tasks developers are asked to perform, so make sure you break it up with some of more challenging and interesting tasks, throw a few features into each iteration.

if I could do it all over I’d change…

These are the generally the things that are hurting the developers rather than the user. Some of them are far reaching, architectural or framework choices. They can all be fixed though, and you don’t need to rewrite the whole application to get there.

Again, start with the ones that hurt the most, if it’s part of the architecture refactor it. If it’s a framework you’d like to integrate to solve problems, try it out on part of the code and work to roll it out incrementally.

I promise it will be quicker, a lot quicker to go down the incremental route than it will be to rewrite the whole thing in one go.

Bugs, in an awful lot of organisations they’re a four letter word, they just shouldn’t happen. But unfortunately despite the shouting and stomping, they still will.

your software has bugs

Your software has bugs, go ahead, say it, try it on. Does it scare you? It doesn’t scare me. I know mine does, this isn’t a shock, revelation or epiphany for me. I had that years ago:

One dark stormy night with a start I sat bolt upright and realised I wasn’t perfect, that I couldn’t write perfect code. I sat through the rain and the thunder awake and frightened, but as the sun broke the next morning it a clear new day dawned on a world washed free of it’s sins. I felt renewed, rejuvenated; I bounded to the office, spearheaded an overhaul of our development process, got a more agile workflow implemented with fast iterations and complete management support.

Ok, it wasn’t quite like that. I was actually up all night because someone somewhere had been freaking out about yet another bug the day before. It was affecting one user, and it would only be a matter of time before it would be affecting others - then like dominoes all of them; soon they’d be the rats leaving our sinking ship upstream and they’d take the paddles with them - in short: DOOM!

It’s very easy to find a bug, say it’s obvious and should have been tested, it’s quite another to test all the functionality and find all of the bugs before launch. My software has bugs, so does yours and everyone else too.

get over it and stop blaming others for your problems

Bugs are a result of code, written by programmers- but it’s not always just the programmers who are to blame. Organisational pressures leading to scope and feature creep, poor design and processes or a lack of any sort of roadmap will all contribute to the problem.

So many times I’ve been working to a deadline (beginning, middle or end - it doesn’t really matter once a timeline is fixed any changes will incur a cost), then someone comes along and starts a sentence with “What about…” and totally blows the schedule. The problem is people rarely take into account the very simple fact that the schedule wasn’t set to include this new task, or the next next one.

This schedule pressure then leads to rushing either the design, the testing, the coding or all of them - this more or less ensures there will be more bugs.

Common excuses I hear for this are:

• It’s obvious, and should have been thought of. Sometimes this is a valid comment, but I have never been part of a team where the obvious was actually missed. In my experience this has almost always been used as some rhetorical comment to excuse someone for not mentioning a feature until launch or later.

• If we don’t have that for Friday’s launch, we won’t get customer X< The short simple answer should always be, if this goes into the current iteration- no customers will get a launch Friday.

• Adding would mean getting so many more customers This is such a great trap for a business- so good I’ll be writing much more about it in the future. Oftentimes it’s political or the result of an organisation without any clear goal setting or planning.

Schedule pressure is often further exacerbated by not making any allowances for bugs. If you don’t make allowances for fixing them post launch, you’ve already missed your next deadline.

what can we actually do about them?

There are a few things we can do organisationally and individually to lessen their impact. Start by not being afraid of them, accept they’re a fact of life and get over it. Shit happens, and it piles up - get a shovel or get out of the way. Look to the processes and people and streamline the development workflow. Everyone involved needs to think things through. Finally once you’ve accepted bugs, worked to minimised them in development deal with them calmly and rationally, if at all.

deal with the workflow issues

If your development processes seem to be creating a lot of bugs, they’re the fist place to look. These are going to be the hardest to change, but they’ll also give you the biggest returns. A few of simple steps to take:

• Design the interfaces first Changes to desgin mid-stream cost at any stage of the cycle, but the earlier they’re discovered the less of an impact it will have. At least wireframing your interfaces means everyone gets a good look at them before development starts, and can say “what about…” early.

• Remember changes cost When someone asks for a change, make sure you point out that changes will affect delivery. Ask them why it wasn’t thought of earlier. Don’t take anything as read and make sure nobody else does either.

• Iterate often Small quick iterations mean features go into use quickly and mean less changed code and less places for bugs to hide. This also allows bugs to form part of the next iterations work.

all bugs are not created equal

Not all bugs will affect people equally. Minor display issues are not as important as an issue where user data is randomly disappearing. An issue where the front page of your website is full of gibberish is far more important that a newsletter not being sent on time.

All bugs should enter a triage system, where the scope of the problem is analysed and a prioritisation decision is taken. There is no room for rhetoric, blame or politics in this process and clear descriptions of the problems are key. The title of this post is verbatim the subject line from quite a few emails I’ve seen in my life, the body is usually not much more helpful and generally descends into politics and conflict. A minor display issue affecting some users is hardly “irreparably damaging”, “actionable”, cause for anyone to “seek any and all legal recourses for reimbursement or compensation” or cause to run off and offer the person our most heart felt and abject apologies for the inconvenience.

We need to step back, look at the extent of the problem and respond appropriately. Going off on one while a wild wacky tale to tell alter, isn’t really professional or productive.

Which leads us to…

not all bugs need to be fixed immediately, or even at all

In a perfect world, we’d fix every bug as soon as it was reported. Then again, the world wouldn’t be so perfect because we’ve got a bug in our code… in a slightly more perfect, yet undeniably imperfect world we’d like to fix bugs as soon as they happen. This isn’t always going to be possible and for various reasons it may not make sense to fix a bug immediately.

Take for example the following completely hypothetical situation. You’ve got a bug in some obscure import routine written for one customer a year and a half ago. The sales team promised it was going to be the next big thing, would open the door on a whole raft of customers that used that package - it turned out to be just the one and they may or may not renew their contract in six months time. Politics aside, given certain situations the import fails and you might choose to just not fix the bug

• They’re unlikely to renew and fixing it isn’t going to sway them or get you any new customers Don’t fix it and just remove the whole broken import from a future iteration. Chalk it up to learning and plan new features better so you’re addressing the needs of your users

• A work around exists Your time might be better spent elsewhere, especially if the problem only affects a single or small number of customers

• Planned developments will obsolete the feature A reworking of the feature, section or other areas of your website, the integration of a new API or a new workflow or way of doing things may mean the bug just dissapears in a future revision. One caution with this, as with any bug fix the promise should be delivered to the cusqtomer on time