Some progress on continuous deployment

Posted on March 2, 2010

As I posted a couple of weeks ago I’m working on working a continuous deployment setup so that I can push changes live quicker and easier for the projects I’m working on.

I’ve installed and configured Hudson, which was a good first step. It’s shown me that my builds are a bit fragile, they’re heavily based on packages that I’ve compiled and installed into my local Maven repository. To solve the problem and to make my build a little more sensible I’m getting Artifactory working.

At this stage I’m just trying to get a very basic system in place so that I have an idea of how long a deployment will take. It’s a little more complicated because I’m using Java and the WARs contain all the libraries. I could just use the defaults and push a deployment directly to the target servers, but that doesn’t work. First wars are big, they can easily weigh in at 20Mb.

There are a couple of things I could do here. Ideally I could unroll all the jar libs and only include the classes I need. That would drastically reduce the size, as a bonus I could probably also have it give me a warning that I’m including an unused jar, even better if I could work out the path to the maven dependency. But there’s the whole problem of resources, the fact that using Springframework means most of the classes aren’t being explicitly called and the fact that since I don’t know anything that does all of this this, I’d have to write it all.

So I’ve written that whole idea off, and added a shorter to-do item to make sure I’m only including jars I need. Instead I’ve gone for a far easier solution: deploy exploded wars, and use rsync so I can push only changes to the server.

The next step is to get trigger a Hudson build with a commit-hook and use a successful Hudson build to trigger a script that does the rsync. Then I’ll have very basic continuous deployments up and running albeit without any automatic roll backs of a failed deployment which ultimately is a must, but it’s a start.

Continuous Deployment

Posted on February 23, 2010

I’ve been working lately trying to implement some of the things Eric Ries talks about. Primarily the idea of continuous deployments for lean startups and getting away from the fear of release.

On the surface and looking back on some of the disastrous releases I’ve seen (deployed) it’s a frightening concept. The problem with most of those releases though hasn’t so much been that there was a bug or a problem, it was there was no fall back position – the release was made and there was no way to unmake it. Also working in Java releasing WARs has been a pretty big issue – you have to upload the whole release to fix even a small bug like a typo. One bug in these cases can mean you’re 15-20 minutes from uploading a fix.

It’s a very bad place to be and I want to avoid it at all costs in the future, both the problems and as a result the fear of a release.

With that in mind I’ve been fashioning a system for Sproozi where a release is made as an exploded war using Hudson. On commit it’s tested, deployed to a single server in the cluster and tested some more. The server joins the main cluster and it’s monitored as the release is pushed to the rest of the servers to make sure it’s stable.

The only problem I’m having conceptually is how do I treat a commit that comes in before a release is finished – I guess I have to sit on it, but what if the release fails and I have to lock the repository? What about if the release is a success but in the meantime I have 2, 3 or even more commits that I’m sitting on? Do I deploy them one at a time until they’re either all successful or one fails?

How has anyone else practising dealt with these issues?

Stop asking questions.

Posted on July 19, 2009

I’ve had an ongoing conversation with a few people over the course of a few projects about the questions to ask when people sign up on the web or what hoops you make them jump through.

Ask most people what they need to know about user’s and you’ll end up with a list of information from the useful like name to the truly useless like job title or title. Give me one good reason for either. Are you going to be sending Mr John Smith or Dr. Sarah Jones a letter using their formal titles?

When it comes to things like Job Title, the reason I usually hear is that it’ll be useful for marketing and demographics later. Maybe, but I’ve never actually seen that later come to anything. So I don’t buy it.

If the signup form is the depth of your engagement with your user you might as well drop the pretense of a web application and go back to sending newsletters. Just don’t do it. Ask your user the bare minimum number of questions you need to get them started using what your product does now and stop throwing up barriers to entry.

Aside from just the questions that are asked there are other things, the hoops almost every site makes you jump through before you can start using it. Probably the best example is the ubiquitous confirm your email step. Do we need to be doing this? In some circumstances I’ll admit it makes sense, if your delivering content via email for example or signing up for a mailing list. Of course you want to make sure the user wants what you’re sending them.

But for every other site, I’m not sold. It’s just another step they have to take to start using your service and all the little steps all add up to users hating signing up for new things.

Also it begs the question, why even bother? For 99% of websites the only reason to make sure a user has entered their email address correctly is so that you can let them recover a password. You’re adding the step for people that have lost their password and haven’t managed to get their email right. That’s going to be a fairly small proportion of your users; even less if you let login with their email address, then they’ll know it’s wrong anyway as soon as they try it.

Do we pander to the idiots or make things as quick and easy for everyone else? I’ll take quick an easy any day.

I’m finding myself not just asking what any given field in a form is for and making it justify it’s self, I’m trying to figure out how to get by without it. I’m leaning towards going beyond just the bare minimum and into the realm of how can I modify this service to work with even less.

Release early – Release often

Posted on July 9, 2008

Agile methodologies have recently been getting something of a bad name, it’s not unwarranted but it is unfortunate. The problem isn’t actually the process, an agile process will work very well – if the infrastructure supporting it is capable of of being agile as well. By supporting infrastructure for a process here I don’t mean servers and software I mean the people and the organisational foundations. The organisation trying to use an agile process must it’s self be agile, quick to react to change. Agile processes that work have one thing in common, they encourage features to be released early and released often.

A pretty basic overview of our process goes something like this:

Big Bag of Features

Our projects and releases go through a number of stages, the first we call The Big Bag of Features. When a project is really just an idea, it’s something we might want to build and we basically rough out some ideas and start to grow them into all sorts of crazy future versions, we talk about all the features we’ll need to achieve them and how we’ll go about it.

Minimum Stable Feature-set

From the BBoF we usually end up with at least one, more often than not a number of possible outcomes and development paths. By talking through all of the options, all the features and all the future versions we’d like to be able to build leads us naturally to something we can build; a vision for a product.

Citrus is a good example of this practice, we’re at the stage where we’ve decided the vision for the project, have a big bag of features we’d like to implement and are now in the process of creating our MSF

From the vision we decide what we need to make a product work; less is definitely more at this stage. We normally take each feature we’ve decided would be nice to have and try to think about who would use it and why. If we can’t come up with a compelling argument for most of our users using it, we normally chuck it back into the bag to worry about later. We refer to the result as theMinimum Stable Feature-set (MSF) or Minimum Launch Feature Set.The bare minimum we need to launch a working product.

The MSF becomes the next iteration, and once it’s set we don’t alter it. Nothing is added or taken away. On the surface this might seem either silly or a bit high and mighty, but we can do it because we put the work in ahead of time. We’ve discussed how we envision the product being used and we make sure we’ve covered all the bases. New great ideas just aren’t important enough to include, no matter how good they are. We leave them for another iteration and another day.

These ideas can be politically disastrous in some organisations, it’s downright heresy to tell the boss, or the sales director or a colleague what amounts to: their newest great idea just isn’t good enough. It’s important for an organisation to have focus though, this is where the MSF and short iterations come into their own. Releasing early and often means that you’re quickly on to the next round of updates and those great ideas can be looked at objectively alongside anything else in the Big Bag of Features.

Getting started

Finally we start coding the interfaces first, then backend to meet our MSF. In the case of a new project partial interim releases are made for testing. We try to work out an entire area of the project first then release that to our testing audience for feedback. As the iterations contiune towards release we add all the features from our MSF.

Updates

When it comes to updating projects we throw away the Big Bag of Features and start all over again.You heard it right, throw them away, gone. A long time ago I learned that feature lists just cause trouble. You can’t help but schedule them, the schedules are very loose and they’re never met but still promises are made based on them. In the end they start to read more like a list of failures, things your application doesn’t do. More importantly, things your application will more than likely never do. So we throw them away and start again with a new Big Bag of Features, the most important features will bubble to the top of your memory anyway and will end up right back in this new BBoF.

For an update our MSF is usually a single feature, or small feature set if they work best that way. The aim is to keep the releases small and concise, targeting one area of functionality. This doesn’t mean that we don’t set milestones or visions for the future development of the product, we normally make a milestone accomplish some greater goal or vision for the product. But we try not to roll a list of disparate fixes to various parts of the application into a single release, we break them up into many smaller releases.

Rinse and Repeat

We continue in this way for the life of the project releasing features to users early and often so bugs are spotted quicker and we never go too far down a dead end, creating something users won’t use. It’s during this stage that we really work hard to measure how an application is being used, or not used so we can focus our energy only on the most important parts. We never try to be everything to everyone, we’re not afraid to disappoint some people or to just say no. We want our application to be great at what it does, not mediocre at everything.