Archive for the 'open source' tag

Downloading maven dependency source jars

Posted on November 3, 2009

I’ve been working on a new project that I’m planning to open source real soon – stay tuned.

When I’m implementing interfaces in a dependent package using Idea/Maven I want to tick the “copy javadoc” button to at least have the documentation from the intereface. The issue of course is that I don’t have the sources.

Run the following command:

mvn dependency:sources

Maven will download any sources it can find for in remote repositories for your dependencies and Idea finds them like magic. So now not only can you copy javadoc, you can also click the line number in the stack trace and get something meaningful – not “compiled code”.

Reblog this post [with Zemanta]

Scaling up vs scaling out

Posted on June 24, 2009

Jeff Atwood goes into some calculations about the cost of scaling up vs scaling out and makes an interesting point, it quickly becomes impractical if you’re not using open source software. I think Jeff slightly missed the point though, it’s not about open or closed source, it’s that scaling out is simply impractical if you’re paying traditional software licences.

This is something we came across when building Sproozi. If we wanted to store petabytes of data and run hundreds or thousands of concurrent processors there was no way we could ever afford to do it on machines running windows we were paying for by the box. But it’s not because we’d have to pay for software, per se, it’s how we’d have to pay for it.

Software has traditionally been licensed by machine, when machines got bigger vendors wanted to cash in so the licences got a little bigger. They had to cover their losses when you threw a few new processors in the machine rather than getting a new one to put alongside after all. It has always been in their best interest though for you to get a bigger box than to get more cheap ones – scaling out is very hard and the software doesn’t do it well. Most RDBMS just can’t do it well and they certainly can’t get anywhere near the the scale of something like Hadoop. If you want to scale out, forget SQL servers, you need software that’s going to scale out.

But let’s forget the specific software for the time being and just assume that the big boys (MS, Oracle, IBM) will have a scaling out solution soon – don’t worry this isn’t going to kill them, but it will change them. They will still want to licence an operating system and a data storage and retrieval system to you.

What I’m almost positive you’re going to see is these companies introduce new pricing schemes to meet the needs of the cloud, they have to or they’re going to lose all that revenue to the open source projects that have a head start on them. Just look at EC2, you can already provision MS and other software and I think that’s a trend that’s just going to continue.

So while Jeff is right that if I want to buy as many cheap boxes as I could for the hardware cost of a big iron server and put windows and SQL on them and it would all cost a small fortune. It’s not really a fair argument, you’re taking an old big iron way of thinking and trying to apply it to the cloud. What it fails to take into account is how much more powerful your new cloud cluster is than the big iron box, let the software vendors figure out the economics of making their software an attractive ROI when compared to OSS because if they want to compete in the cloud they’re going to have to.

Related articles by Zemanta
Reblog this post [with Zemanta]

Open Source for Business

Posted on June 8, 2009

I didn’t get this posted yesterday because the Internet crapped out in our area. Nothing but excuses, I know.

I’ve been working beyond the bleeding edge, using a version of the Nutch code that’s not even made it into the Apache SVN for the project yet. To celebrate the fact that my contributions will make it in I figure its a good time to get into open source and business.

To put it briefly, and as you can probably guess, I’m pro open source. I use it extensively and I push back as much as I can. When it comes to the most of the code I write there really isn’t much commercial benefit in keeping it hidden so it just makes sense to give back.

There are two types of business on the web, one where you provide a software service and that is the product and others where you provide access to data. It’s pretty easy to tell which camp you’re in.

37 Signals for example, they’re in the first and their software probably isn’t something they just want to let people download – unless they’re incredibly brave. Doing that would mean that they’d be competing on margins for the cheapest hosting, users would flock to the cheaper services, have a poor experience and blame the software.

Sproozi on the other hand is the second type, our data is what users mostly care about and we’re not planning to be precious about our code. I’ve already been pushing some of the changes I’ve made to Nutch back into the project and we’re planning open source projects of our own in the coming months.

One of our plans we have is to build iPhone, Andriod and other phone based applications for our service and release them as open source projects. We’re planning to write them (or have them written for us) and release ‘official’ versions. Then release that code as open source project to provide a framework for developers so that they can build great things from it and on our API.

If there are any experienced iPhone, Android, Blackberry, Symbian or Pre developers out there that want to get involved, drop me a line were a ways off yet but would love to chat about it and get some very early feedback.

Related articles by Zemanta
Reblog this post [with Zemanta]