Archive for the 'tests' tag

Some progress on continuous deployment

Posted on March 2, 2010

As I posted a couple of weeks ago I’m working on working a continuous deployment setup so that I can push changes live quicker and easier for the projects I’m working on.

I’ve installed and configured Hudson, which was a good first step. It’s shown me that my builds are a bit fragile, they’re heavily based on packages that I’ve compiled and installed into my local Maven repository. To solve the problem and to make my build a little more sensible I’m getting Artifactory working.

At this stage I’m just trying to get a very basic system in place so that I have an idea of how long a deployment will take. It’s a little more complicated because I’m using Java and the WARs contain all the libraries. I could just use the defaults and push a deployment directly to the target servers, but that doesn’t work. First wars are big, they can easily weigh in at 20Mb.

There are a couple of things I could do here. Ideally I could unroll all the jar libs and only include the classes I need. That would drastically reduce the size, as a bonus I could probably also have it give me a warning that I’m including an unused jar, even better if I could work out the path to the maven dependency. But there’s the whole problem of resources, the fact that using Springframework means most of the classes aren’t being explicitly called and the fact that since I don’t know anything that does all of this this, I’d have to write it all.

So I’ve written that whole idea off, and added a shorter to-do item to make sure I’m only including jars I need. Instead I’ve gone for a far easier solution: deploy exploded wars, and use rsync so I can push only changes to the server.

The next step is to get trigger a Hudson build with a commit-hook and use a successful Hudson build to trigger a script that does the rsync. Then I’ll have very basic continuous deployments up and running albeit without any automatic roll backs of a failed deployment which ultimately is a must, but it’s a start.

reblog_c.png


I have no idea what I’m doing

Posted on October 22, 2008

It’s very, very hard for us to admit that we’re wrong, let alone that we have no clue what we’re doing. The first step of course is being able to admit that you’ve got a problem and before you can admit you have a problem you need to be able to recognise that you’ve got one.

I came to the shocking realisation the other day, that I was guilty of not really know what I was doing. I came across a subtle bug in some of my code; my unit tests were passing but in production things weren’t working as I expected. I dug deeper and got a vague idea of what was happening, then I engaged in shotgun programing.

I knew I was doing it too. I had no idea what I was doing, and I knew it. I just started hacking away at my code, solving one problem and introducing another. Basically I knew where the problem was and as if it were some small animal rustling in the bushes nearby I just blasted away its general direction and just hoped I hit it, whatever it was.

I eventually solved the problem, and eventually understood what was going on; but it was touch and go there for a while. The problem wasn’t so much my trial and error approach, that did eventually solve the problem. It was more that I had no idea why I was trying the solutions I was trying, more specifically why I thought they were plausible I was just pointing in the general direction of the problem and letting loose.

Related articles by Zemanta
Reblog this post [with Zemanta]

Integration testing in maven – With Maven, Cargo, httpunit and Selenium

Posted on September 22, 2008

I’ve been trying to figure this out for months, and thought it should have been simple. All I wanted to do was write a set of unit tests to test my java code, have them run whenever I hit test. Next I wanted to have a set of HTTP Unit, Selenium or similar tests run to test that the actual application is working when it’s built and deployed to a container using the cargo plugin.

I didn’t really want to have a separate project to do this because it seemed like a massive pain in the ass to maintain the whole thing. I didn’t see any reason why it shouldn’t be easy with Maven either – it does have a phase for integration-tests already.

My first try looked good to me:

 

...
<build>
    ...
    <plugins>
        ...
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <configuration>
                <excludes>
                    <exclude>**/integration/**</exclude>
                </excludes>
            </configuration>
            <executions>
                <execution>
                    <id>integrationtest</id>
                    <phase>integration-test</phase>
                    <goals>
                        <goal>test</goal>
                    </goals>
                    <configuration>
                        <excludes>
                            <exclude>none</exclude>
                        </excludes>
                        <includes>
                            <include>**/integration/**/*Test*.java</include>
                            <include>**/integration/**/*Test.java</include>
                            <include>**/integration/**/*TestCase.java</include>
                        </includes>
                    </configuration>
                </execution>
            </executions>
        </plugin>
        ...
    </plugins>
    ...
</build>
...

 

I tried to run the tests, but unfortunately when it gets to the second round of tests – which should run my integration tests – it runs the same sets of as it ran the first time. I tried adding the combine.children="append" attribute to the includes, and excludes but that didn’t work either. Finally I came across a source file XppDom, part of plexus which maven uses. XppDom allowed the combine.children attribute as well as another I haven’t seen mentioned anywhere else childMergeOverride. I added that instead and it worked!

 

...
<build>
    ...
    <plugins>
        ...
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <configuration>
                <excludes>
                    <exclude>**/integration/**</exclude>
                </excludes>
            </configuration>
            <executions>
                <execution>
                    <id>integrationtest</id>
                    <phase>integration-test</phase>
                    <goals>
                        <goal>test</goal>
                    </goals>
                    <configuration>
                        <excludes childMergeOverride="true">
                            <exclude>none</exclude>
                        </excludes>
                        <includes childMergeOverride="true">
                            <include>**/integration/**/*Test*.java</include>
                            <include>**/integration/**/*Test.java</include>
                            <include>**/integration/**/*TestCase.java</include>
                        </includes>
                    </configuration>
                </execution>
            </executions>
        </plugin>
        ...
    </plugins>
    ...
</build>
...

 

 

Setting it up for Cargo

I’ve done this on a few projects, the first was a project that built a series of taglibs which are used across a number of applications. The release for the project is a jar so I have a separate test web-app structure in ${baseDir}/src/test/web-app. The code for generating this war looks like this:

 

...
<build>
    ...
    <plugins>
        ...
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-war-plugin</artifactId>
            <executions>
                <execution>
                    <id>generate-test-war</id>
                    <phase>pre-integration-test</phase>
                    <goals>
                        <goal>war</goal>
                    </goals>
                </execution>
            </executions>
            <configuration>
                <warSourceDirectory>${basedir}/src/test/webapp</warSourceDirectory>
                <warName>${project.artifactId}-test</warName>
                <webappDirectory>${basedir}/target/${project.artifactId}-test</webappDirectory>
                <primaryArtifact>false</primaryArtifact>
            </configuration>
        </plugin>
        ...
    </plugins>
    ...
</build>
...

 

During the pre-integration-test phase the above simple generates a test war. If you’re project is a web-app and it already generates a war you can skip the above as one will be generated for you already.

The next step for was then to get Cargo to deploy the application:

 

...
<build>
    ...
    <plugins>
        ...
        <plugin>
            <groupId>org.codehaus.cargo</groupId>
            <artifactId>cargo-maven2-plugin</artifactId>
            <configuration>
                <wait>false</wait>
                <container>
                    <containerId>tomcat5x</containerId>
                    <zipUrlInstaller>
                        <url>${integrationtests.tomcatURL}</url>
                        <installDir>${installDir}</installDir>
                    </zipUrlInstaller>
                    <output>
                        ${project.build.directory}/tomcat5x.log
                    </output>
                    <log>${project.build.directory}/cargo.log</log>
                </container>
                <configuration>
                    <home>
                        ${project.build.directory}/tomcat5x/container
                    </home>
                    <properties>
                        <cargo.logging>high</cargo.logging>
                        <cargo.servlet.port>8080</cargo.servlet.port>
                    </properties>
                </configuration>
            </configuration>
            <executions>
                <execution>
                    <id>start-container</id>
                    <phase>pre-integration-test</phase>
                    <goals>
                        <goal>start</goal>
                        <goal>deploy</goal>
                    </goals>
                    <configuration>
                        <wait>false</wait>
                        <deployer>
                            <deployables>
                                <deployable>
                                    <location>${basedir}/target/${project.artifactId}-test.war</location>
                                    <type>war</type>
                                    <pingURL>http://localhost:8080/${project.artifactId}-test/index.html</pingURL>
                                    <pingTimeout>300000</pingTimeout>
                                    <properties>
                                        <context>${project.artifactId}-test</context>
                                    </properties>
                                </deployable>
                            </deployables>
                        </deployer>
                    </configuration>
                </execution>
                <execution>
                    <id>stop-container</id>
                    <phase>post-integration-test</phase>
                    <goals>
                        <goal>stop</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
        ...
    </plugins>
    ...
</build>
...


Most of this is pretty self explanatory, I try to use parameters to configure as much as I can and try to maintain it in higher level POMs wherever appropriate. This is especially true for the tomcat URL which changes on a regular basis. The seem to post new releases and remove the older ones, and it can be a chore to keep up in multiple projects. You may want to look at the cargo plugin documentation, you can ignore some of what I’m doing above if you’re project’s default artifact (the one package creates) is a web-app. 

So there you have it, eventually I got there after a lot of digging. I’ve been pretty brief in my explanations and have assumed a fairly good understanding of maven. if you have any questions by all means leave a comment and I’ll do my best to make it clearer.

Related articles by Zemanta
Reblog this post [with Zemanta]