OAuth 2.0

Posted on May 20, 2010

I’ve been updating my OAuth library to support OAuth 2.0 mostly so I can add Facebook to Announce.ly and Sproozi, but more on that later. OAuth 2.0 is similar to 1.0 but changes a few key things fundamentally and isn’t backwards compatible.

What’s wrong with 1.0, doesn’t it work?

It does, but probably the biggest issue is the fact that you have to sign the message knowing all it’s content beforehand. This works well if the data is on the querystring in a GET request or for simple operations but isn’t optimal if your data is part of the POST body. It also means you have to construct your requests in a certain way, which is a bad thing.

Take photo, audio or video data – to post that you’ll need to sign the whole request and it’s not clear how it should work with multipart data. There are several extensions to the spec that deal with some of these issues, but the fact that there are non standard extensions to do something pretty standard kinda says it all.

Even if you’re not dealing with these issues you still have to work with your requests as units where you know the whole content beforehand.

What’s new in OAuth 2?

OAuth 2.0 in it’s simplest form works over HTTPS connections and simply asks for a token – the security and trust are built in to the protocol. It’s that easy.

OAuth 2.0 sill lets users sign messages to transmit them over insecure channels, plain HTTP, but the signing methods are much easier to implement. Gone is the complicated parameter normalisation algorithm and in it’s place is a much simpler version that doesn’t require POST data in the signature. So even with multipart submissions it should just work.

At the moment I’m cleaning things up and preparing the oauth library to work with oauth 2.0 and changing the way it works to reflect the simpler way oauth 2.0 does. You can check it out on GitHub [http://github.com/andrewmccall/oauth]

Another Open Source Library.

Posted on April 6, 2010

I’m having a bit of a clear out, taking a look at some of the code I’ve written and I’ve been pushing some of the stuff I’m currently using up to GitHub under and Apache 2 licence. I’ve used things in Announce.ly, Sproozi and some other small projects and figure they may be useful to someone else. My only criteria has been to ask If I’m using it now in a project, if so I’m actively supporting it and I’ve started pushing that stuff to GitHub, everything else is dormant and I don’t want to release something I’m not actively supporting- it also occurs to me that if even I’m not using it, it can’t be all that worthwhile.

I’ve just pushed some code I’ve been testing for a few months in a couple of projects to GitHub. It’s an accounts package written for Spring, that ties my oAuth library and Twitter together with either Hibernate or Hbase as backend storage. In it’s simplest form when you login with twitter it creates you a new user and persists it and the oAuth access tokens you need to act on behalf of that user.

I’ll write some more about it, better documentation and probably throw a little more code up on GitHub over the course of the next couple of weeks as and when I get a chance.

My new open source Java OAuth library

Posted on March 19, 2010

I’ve just pushed out a new open source java OAuth library because I couldn’t find one that did what I needed. My key requirement was simplicity. I didn’t like the idea of using the library for HTTP stuff and there is no reason I should. Once I’ve obtained the Access Token all I’m doing with oAuth is signing my requests.

I want to use HttpClient directly and only use the oAuth library to sign the message for various reasons not the least of which being that I already have a HttpClient object setup in my IoC container.

The closest I found was signpost but it wasn’t very IoC friendly or thread-safe which meant every time I wanted to make a call I’d have to create new objects, or at the very least call a bunch of methods to set them up which highlights the third problem, there were no clear objects that I could store for later.

The library I’ve just release is a fork of the signpost code, that’s now thread-safe and should be more IoC friendly. You create your method calls as you would normally, and just before you call HttpClient.execute(HttpMethod) simply call OAuthConsumer.sign(HttpMethod, AccessToken);.

I’ve added a few new objects that handle most of the work. Service, RequestToken and AccessToken are all beans that you pass to a consumer depending on what you want to do. Starting with a Service you call

Service service = new Service();
service.setRequestTokenUrl("http://twitter.com/oauth/request_token");
service.setAccessTokenUrl("http://twitter.com/oauth/access_token");

service.setConsumerKey("b8sA385mBBNqOTD6Omlsw");
service.setSharedSecret("MD4Sve6AdaDasjdvOAsbpAJsA87S8s64e5rE4");

service.setMessageSigner(new PlainTextMessageSigner());
service.setSigningStrategy(new AuthorizationHeaderSigningStrategy());

RequestToken requestToken = oAuthConsumer.getRequestToken(twitter);

You’ll have to send the user off to twitter to check their credentials. When they come back
they’ll be given a verifier set it and trade the request token for an access token

requestToken.setVerifier(verifier):
AccessToken accessToken = oAuthConsumer.getAccessToken(requestToken);

Now you can store the accessToken to use later, when you want to simply setup your http method as you would normally.

HttpUriRequest request...
// do your HttpClient stuff here

oAuthConsumer.sign(request, accessToken);
HttpResponse response = httpClient.execute(request);

There is also code in there for the Jetty HttpClient, but it’s a bit rough and I haven’t used it. Have play with it and let me know what you think.

UPDATE: Forgot to link to it… Dumb. It’s on GitHub here.

Some progress on continuous deployment

Posted on March 2, 2010

As I posted a couple of weeks ago I’m working on working a continuous deployment setup so that I can push changes live quicker and easier for the projects I’m working on.

I’ve installed and configured Hudson, which was a good first step. It’s shown me that my builds are a bit fragile, they’re heavily based on packages that I’ve compiled and installed into my local Maven repository. To solve the problem and to make my build a little more sensible I’m getting Artifactory working.

At this stage I’m just trying to get a very basic system in place so that I have an idea of how long a deployment will take. It’s a little more complicated because I’m using Java and the WARs contain all the libraries. I could just use the defaults and push a deployment directly to the target servers, but that doesn’t work. First wars are big, they can easily weigh in at 20Mb.

There are a couple of things I could do here. Ideally I could unroll all the jar libs and only include the classes I need. That would drastically reduce the size, as a bonus I could probably also have it give me a warning that I’m including an unused jar, even better if I could work out the path to the maven dependency. But there’s the whole problem of resources, the fact that using Springframework means most of the classes aren’t being explicitly called and the fact that since I don’t know anything that does all of this this, I’d have to write it all.

So I’ve written that whole idea off, and added a shorter to-do item to make sure I’m only including jars I need. Instead I’ve gone for a far easier solution: deploy exploded wars, and use rsync so I can push only changes to the server.

The next step is to get trigger a Hudson build with a commit-hook and use a successful Hudson build to trigger a script that does the rsync. Then I’ll have very basic continuous deployments up and running albeit without any automatic roll backs of a failed deployment which ultimately is a must, but it’s a start.

Continuous Deployment

Posted on February 23, 2010

I’ve been working lately trying to implement some of the things Eric Ries talks about. Primarily the idea of continuous deployments for lean startups and getting away from the fear of release.

On the surface and looking back on some of the disastrous releases I’ve seen (deployed) it’s a frightening concept. The problem with most of those releases though hasn’t so much been that there was a bug or a problem, it was there was no fall back position – the release was made and there was no way to unmake it. Also working in Java releasing WARs has been a pretty big issue – you have to upload the whole release to fix even a small bug like a typo. One bug in these cases can mean you’re 15-20 minutes from uploading a fix.

It’s a very bad place to be and I want to avoid it at all costs in the future, both the problems and as a result the fear of a release.

With that in mind I’ve been fashioning a system for Sproozi where a release is made as an exploded war using Hudson. On commit it’s tested, deployed to a single server in the cluster and tested some more. The server joins the main cluster and it’s monitored as the release is pushed to the rest of the servers to make sure it’s stable.

The only problem I’m having conceptually is how do I treat a commit that comes in before a release is finished – I guess I have to sit on it, but what if the release fails and I have to lock the repository? What about if the release is a success but in the meantime I have 2, 3 or even more commits that I’m sitting on? Do I deploy them one at a time until they’re either all successful or one fails?

How has anyone else practising dealt with these issues?

EC2 Spot instances

Posted on December 14, 2009

The thing I really like about Amazon’s cloud stuff is they’re constantly undermining themselves with new innovations – spot instances are another great example. Taking the utility metaphor a step further you can now rent their services when nobody else is for cheaper, like buying electricity at night.

A lot of the tasks I’m envisioning for Sproozi aren’t really time dependent. While it’s important to show you a page in a timely manner, crawl and index a brand new website you add quickly and basically be interactive there is also a lot I have to do in the background. The huge and growing list of URLs people add all need to be re-crawled and re-indexed regularly is just one of many examples of processing vast amounts of data. These tasks are always running, always in the background.

Spot instances are a perfect fit – I can bid the price I want on extra capacity spin up some extra instances to join the cluster when they’re cheap. Over the next few weeks I’ll probably try to add some spot instances to my Hadoop dev cluster and see what happens.

Signups via twitter API

Posted on December 7, 2009

This is possibly big news, something I’ve hoped they would do for a long time and something Fred Wilson voiced his support for a few weeks ago: Creating user accounts via the twitter API.

Both the projects I’m working on full time could benefit from this. Sproozi of course because we could create user accounts for them and help users find interesting local people to follow. My oft promised side project which at it’s most basic level [the one I'll launch first] aims to make making announcements easier could benefit by making the signup process seamless – users wouldn’t have to already have to have a twitter account, they could create it all in one step with us.

That I think is what makes this such big news, sites that rely on users already having a twitter account to do something useful will be able to create that account as a seamless on site process. That will help drive users to the sites and drive users to twitter. Let’s hope they start rolling it out to other developers!

Related articles by Zemanta

• Twitter Extends Sign-Ups Off-Site As It Seeks New Users (mashable.com)

Useful commit messages

Posted on November 20, 2009

I realised that I’m not very good at this when I looked at my last commit for this theme. The message was “added a few things” a message I find myself using when I have nothing else to say, or if I’ve done too much work and either can’t remember all of what I did or be bothered writing it all out.

It’s bad form though, and I’m trying to learn my way into using git properly to make it happen less. Basically I’m still trying to find the right place between a commit for a unit of work and my previous more traditional view that all commits must compile and pass tests. At the same time I’m trying to reconcile that with a Kanban board way of organising what to do.

How does everyone else commit? When, how often and what ‘rules’ do you apply?

Downloading maven dependency source jars

Posted on November 3, 2009

I’ve been working on a new project that I’m planning to open source real soon – stay tuned.

When I’m implementing interfaces in a dependent package using Idea/Maven I want to tick the “copy javadoc” button to at least have the documentation from the intereface. The issue of course is that I don’t have the sources.

Run the following command:

mvn dependency:sources

Maven will download any sources it can find for in remote repositories for your dependencies and Idea finds them like magic. So now not only can you copy javadoc, you can also click the line number in the stack trace and get something meaningful – not “compiled code”.

Stop asking questions.

Posted on July 19, 2009

I’ve had an ongoing conversation with a few people over the course of a few projects about the questions to ask when people sign up on the web or what hoops you make them jump through.

Ask most people what they need to know about user’s and you’ll end up with a list of information from the useful like name to the truly useless like job title or title. Give me one good reason for either. Are you going to be sending Mr John Smith or Dr. Sarah Jones a letter using their formal titles?

When it comes to things like Job Title, the reason I usually hear is that it’ll be useful for marketing and demographics later. Maybe, but I’ve never actually seen that later come to anything. So I don’t buy it.

If the signup form is the depth of your engagement with your user you might as well drop the pretense of a web application and go back to sending newsletters. Just don’t do it. Ask your user the bare minimum number of questions you need to get them started using what your product does now and stop throwing up barriers to entry.

Aside from just the questions that are asked there are other things, the hoops almost every site makes you jump through before you can start using it. Probably the best example is the ubiquitous confirm your email step. Do we need to be doing this? In some circumstances I’ll admit it makes sense, if your delivering content via email for example or signing up for a mailing list. Of course you want to make sure the user wants what you’re sending them.

But for every other site, I’m not sold. It’s just another step they have to take to start using your service and all the little steps all add up to users hating signing up for new things.

Also it begs the question, why even bother? For 99% of websites the only reason to make sure a user has entered their email address correctly is so that you can let them recover a password. You’re adding the step for people that have lost their password and haven’t managed to get their email right. That’s going to be a fairly small proportion of your users; even less if you let login with their email address, then they’ll know it’s wrong anyway as soon as they try it.

Do we pander to the idiots or make things as quick and easy for everyone else? I’ll take quick an easy any day.

I’m finding myself not just asking what any given field in a form is for and making it justify it’s self, I’m trying to figure out how to get by without it. I’m leaning towards going beyond just the bare minimum and into the realm of how can I modify this service to work with even less.