Maven 2 – An Overview

Here’s a high level Maven2 cheat sheet which just explains what Maven 2 is and the steps that are executed when Maven does a build.

What is Maven?
It’s a build tool, basically! It also manages dependencies and produces build reports. It uses xml syntax to create build files (called POMs), and the build files support the concept of inheritance. Maven uses plugins to extend basic functionality and perform cool tricks. It’s mainly used in the Java world but apparently there’s also a .Net plugin (I haven’t actually tried it though).

Why Use Maven?
I would recommend using maven if you need a build tool which will:

  • Take over dependency management
  • Enforce a (fairly) rigid format and practice on developers

The dependency management system uses repositories to store binaries. You can have external (third party) binaries as well as internal dependencies. You can reference them from external repositories (like the maven central repository) or you can put them in your own repository. I actually really like the way Maven enforces it’s own best practices on things like naming conventions and project structures, it’s a real time-saver in the long run.

Installing Maven
Installing maven is a doddle. Simply download it from here. Check the installation pre-requisites (you basically need JDK installed).

For instructions on installing maven on Mac OSX look here

For instructions on installing maven on Windows look here

For instructions on installing Maven on Linux you basically do the same as you do for Mac OSX. That is:

Extract the archive you downloaded.

Add an environment variable called M2_HOME and point it to your Maven directory (I installed mine in /home/maven/maven2).

Add the bin directory ${M2-HOME}/bin to your PATH

Add these to your bash_profile by simply adding these 2 lines:

export M2_HOME=/home/maven/maven2

export PATH=${M2_HOME}/bin:${PATH}

To confirm Maven is installed on your system, simply type: mvn -version

What Happens When I Run a Build?
A lot. And it varies between different types of builds. You’ll need to have a POM file for your project in order to be able to run a build, and I’m going to assume you’ve already got one. If you haven’t, try checking out this post for a couple of examples to get you started.
When you run Maven, it steps through a series of phases in a build lifecycle. You can also call goals defined in plugins. So essentially it works like this:

A lifecycle is a sequence of phases, phases can call plugins, plugins contain goals.

Clear? Yeah, it’s not exactly crystal, but stick with it.

There are 3 basic lifecycles: clean, default and site. You call these when you do a bild, like this:

mvn clean deploy site-deploy

I’ll explain exactly what’s happening in this command a little later, but first I’ll briefly explain what each of the three lifecycles do:

Clean, as the name suggests, does the cleaning work – it deletes the build output directory.

Default does all the build work. It compiles the code and the tests, runs the unit tests, packages up the build, and deploys it to the repository.

The Site lifecycle generates all of the build reports and uploads them to a site for your project.

So, back to our mvn command:

mvn clean deploy site-deploy

What’s actually happening here is we’re calling the clean lifecycle by passing the command “clean”. So far so easy.

Next up is “deploy”. Where has this come from? Well, deploy is a phase in the Default lifecycle. In fact, it’s the last phase in the default lifecycle (for a full list of all the phases that run in all 3 lifecycles take a look here. Because it’s the last phase in the default lifecycle, it automatically includes the execution of all of the others, so simply by calling “deploy” we are ensuring that the phases for compiling our source code, compiling our tests, running our tests and packaging our code are also run. It’s a bit like using the “depends” feature in ant targets, but it’s implicit. In other words, it’s a shortcut way of making sure all the phases in the default lifecycle are executed.

Finally we have site-deploy, and again this is the final phase of the site lifecycle, so we’re implicitly calling the other preceeding site phases.

The Maven Release Plugin
The Release plugin is commonly used when making an official Release Candidate build. In a nutshell it increments the build version and tags your code in SCM. It has 2 main goals. relase:prepare and release:perform.

Release:prepare does the following:

  • It checks that your source is up to date
  • It checks the POM to make sure there’s no occurances of the word SNAPSHOT in the dependencies section (the idea here being that you should never make a release build using snapshot dependencies)
  • It removes the word SNAPSHOT from your version
  • It compiles the code to make sure it works
  • It tags the code and commits the POM to scm
  • It then increments the version and re-adds “SNAPSHOT”, and commits this back to the original SCM location (i.e. not the tags location)

Once this completes successfully, release:perform is executed. Release:perform checks out the code from the tag location, and then runs the standard maven goals “deploy site-deploy” which has the effect of recompiling the code, running the tests, packaging and uploading etc etc (see above for detail on what the deploy phase actually does).

Summary
In general, I think Maven is a good build tool, it does a lot of the hard work for you. It’s far from easy to understand and follow though, unless you’re prepared to dedicate a bit of time to reading through the documentation.

In the past, I’ve spent quite a  bit of time and effort creating build systems using Ant, which, through my own design, have enforced some standards and best practices on the build process, and at the end of the day, I ended up with something fairly identical to what you get with Maven. I would definitely use Maven again, especially if standardisation and repository management were on my to-do list.

Maven POM Templates

Maven uses POM (Project Object Model) files to define its build and deploy tasks. You can end up with a lot of POM files (one for every app or api for instance) so I encourage the use of parent POMs, because they support inheritance and will end up saving a lot of typing and replication.
Here are a couple of examples of an application POM and its parent POM:

First, the app POM:

<?xml version="1.0" encoding="UTF-8"?>
<modelVersion>4.0.0</modelVersion>
<groupId>...</groupId> <!-- e.g. org.apache.maven -->
<artifactId>APPNAME</artifactId>
<packaging>PACKAGETYPE</packaging> <!-- e.g. jar -->
<version>...</version> <!-- e.g. 1.0.0.0-SNAPSHOT -->
<name>APPNAME</name>

<description>Description about this application</description>

<!-- The properties can be added to for each individual pom, 
this is just an example. You can add your own properties here -->
  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <org.springframework.version>3.0.2.RELEASE</org.springframework.version>
    <org.slf4j.version>1.6.0</org.slf4j.version>
  </properties>

<!-- list the parent pom here - if there is one -->
<parent>
  <groupId>...</groupId><!-- Group ID of Parent POM -->
  <artifactId>...</artifactId><!-- Name of parent pom file -->
  <version>1.0.0.0</version>
</parent>

<developers>
  <developer>
    <id>1</id>
    <name>NAME</name>
    <email>EMAIL@EMAIL.COM</email>
    <organization>MYCOMPANY</organization>
    <organizationUrl>http://www.mycompany.com</organizationUrl>
  </developer>
</developers>

<profiles>
<!-- This section is optional  -->
</profiles>

<dependencies>
<!-- List all app specific dependencies here -->
<!-- use RELEASE as the version if you always want it to use the latest version, but note that this is NOT supported in Maven 3 -->
  <dependency>
    <groupId>BLA.BLA.BLA</groupId>
    <artifactId>foo-api</artifactId>
    <version>RELEASE</version>
  </dependency>
</dependencies>

<build>
  <plugins>

<!-- you might need to enter some specific plugins here, however it will be better to have them in parent pom-->

    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-ejb-plugin</artifactId>
      <version>2.1</version>
      <configuration>
        <ejbVersion>3.0</ejbVersion>
      </configuration>
    </plugin>
  </plugins>
</build>

</project>

 

And now for the parent POM:

<?xml version="1.0" encoding="UTF-8"?>
<project>
<modelVersion>4.0.0</modelVersion>
<groupId>...</groupId>
<artifactId>parentpom</artifactId>
<packaging>pom</packaging>
<name>Master POM for apps</name>
<version>1.0.0.0</version>

<description>
This is parent pom for apps, where all the generic configurations are defined
</description>

<ciManagement>
  <system>Bamboo</system>
  <url>http://mybuildserver</url>
  <notifiers>
    <notifier>
      <address>admin@jamesbetteley.com</address>
    </notifier>
  </notifiers>
</ciManagement>

<licenses>
  <license>
  <name>myname</name>
  <url>http://www.myname/licenses/LICENSE.txt</url>
  <distribution>http://mybuildserver:8080/my-repo/</distribution>
  </license>
</licenses>

<organization>
  <name>MyCompany</name>
  <url>http://www.mycompany.com</url>
</organization>

<issueManagement>
  <system>GForge</system> <!-- list your issue management system here -->
  <url>http://My.gforge.url/</url>
</issueManagement>

<developers>
  <developer>
    <id>1</id>
    <name>James Betteley</name>
    <email>james.betteley@jamesbetteley.com</email>
    <organization>mycompany</organization>
    <organizationUrl>http://www.mycompany.com</organizationUrl>
  </developer>
</developers>

<scm> <!-- your source control information -->
  <connection>scm:svn:http://127.0.0.1/svn/my-project</connection>
  <developerConnection>scm:svn:https://127.0.0.1/svn/my-project</developerConnection>
  <tag>HEAD</tag>
  <url>http://127.0.0.1/websvn/my-project</url>
</scm>

<properties> <!-- You can define your own properties here, like this one -->
  <halt.on.fail>false</halt.on.fail>
</properties>

<!--	Build Configuration and Plugin Definition-->

<build>
  <finalName>${artifactId}-${version}</finalName>
  <outputDirectory>build/maven/${artifactId}/target/classes</outputDirectory>
  <testOutputDirectory>build/maven/${artifactId}/target/test-classes</testOutputDirectory>
  <directory>build/maven/${artifactId}/target</directory>
  <resources>
    <resource>
    <directory>src/main/java</directory>
      <includes>
        <include>**/*.xml</include>
      </includes>
    </resource>
    <resource>
    <directory>src/main/resources</directory>
    </resource>
</resources>
  <testResources>
    <testResource>
    <directory>src/test/java</directory>
    <includes>
      <include>**/*.xml</include>
      <include>**/*.properties</include>
    </includes>
    </testResource>
    <testResource>
    <directory>src/test/resources</directory>
    </testResource>
  </testResources>
<extensions>
  <extension>
    <groupId>org.apache.maven.wagon</groupId>
    <artifactId>wagon-ssh-external</artifactId>
    <version>1.0-alpha-5</version>
  </extension>
</extensions>

<pluginManagement>
<!-- List all the optional plugins here, for example, 
the war plugin. Child POMs can override these. -->
<!-- For a child to inherit these, 
it must reference them from within their plugin section -->
  <plugins>
    <plugin>
      <artifactId>maven-war-plugin</artifactId>
      <executions>
        <execution>
        <phase>package</phase>
        <goals>
          <goal>war</goal>
        </goals>
        </execution>
      </executions>
    </plugin>
  </plugins>
</pluginManagement>

<plugins> <!-- define all plugins that are common to your app poms. 
All of these will be inherited by the child poms -->
  <plugin>
  <artifactId>maven-compiler-plugin</artifactId>
  <configuration>
    <source>1.6</source>
    <target>1.6</target>
  </configuration>
  </plugin>
  <plugin>
  <artifactId>another-plugin</artifactId>
  <configuration>
    <source>etc</source>
    <target>etc</target>
  </configuration>
  </plugin>
</plugins>
</build>

<repositories>
  <repository>
    <snapshots>
      <enabled>false</enabled>
    </snapshots>
    <id>Central</id>
      <url>http://repo1.maven.org/maven2</url>
  </repository>
  <repository>
    <snapshots>
      <enabled>false</enabled>
    </snapshots>
    <id>3rd-Party-Repository</id>
      <url>http://mybuildserver:8080/maven-repo</url>
  </repository>
  <repository>
    <snapshots>
      <enabled>false</enabled>
    </snapshots>
    <id>my-Repository</id>
      <url>http://mybuildserver:8080/my-repo</url>
  </repository>
  <repository>
    <snapshots>
      <enabled>true</enabled>
    </snapshots>
    <id>my-Snapshot-Repository</id>
      <url>http://mybuildserver:8080/my-snap</url>
  </repository>
</repositories>



<distributionManagement>
  <repository>
  <id>my-Repository</id>
    <url>sftp://mybuildserver/home/maven/my-repo</url>
  </repository>
  <snapshotRepository>
  <id>my-Snapshot-Repository</id>
    <url>sftp://mybuildserver/home/maven/my-snap</url>
  </snapshotRepository>
  <site>
  <id>my-Reports-Repository</id>
    <url>sftp://mybuildserver/home/maven/maven-sites/${groupId}</url>
  </site>
</distributionManagement>

<reporting> <!-- List your reporting plugins here, such as cobertura	-->
<outputDirectory>build/maven/${artifactId}/target/site</outputDirectory>
<plugins>

  <plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>cobertura-maven-plugin</artifactId>
  <version>2.4</version>
    <configuration>
      <formats>
        <format>html</format>
        <format>xml</format>
      </formats>
    </configuration>
  </plugin>
</plugins>
</reporting>

</project>


 

Continuous Integration – Start at the beginning

Sometimes it’s easier to create a good Continuous Integration system from scratch than it is to improve a system which has been all bent out of shape to support bad practices or legacy issues. But if you’re going to take a wrecking ball to your CI system and start again, exactly how do you actually start? Well, in my experience, the best implementation I’ve worked on has been when the customer’s requirements were clearly understood first.

 

Start at the Beginning

Rather than go ahead and start by choosing your favorite CI tools and defining a standard system, start by picking up a pen and paper and go and speak with as many people involved in project delivery as you possibly can. This should include BAs, Project Managers, Testers, Developers and Systems Architects. Ask them what inputs and outputs they would expect to see and receive from the build and deploy system. Ask them what manual steps they do – you never know, your CI system might be able to help automate some of their work, even for the BAs. Quite often, Project Managers want to see quality metrics from their projects, and to get these they often rely on a manual process, perhaps expecting QA to produce bug statistics or asking a developer to produce some build reports. It seems that quite often, people are unaware of exactly what a CI system can really do for them.

 

Collaborative Design

Using the feedback from these sources, start to define a system on paper and continue to get feedback on your design. This needn’t be a long task, but it will help to stop you from going down the wrong path early on. Concentrate on simplicity and consistency. An over elaborate system will be hard for people to understand and then support, and ultimately that can be damaging. Don’t have several systems all doing the same thing, just have the one, reduce wasteful repetition. Don’t have 2 different types of build server in use unless you have to. Don’t support 2 types of static analysis tool that effectively measure the same thing, instead, settle on one.

In your design, be sure to include:

  • A Source Control system
  • A build tool or tools
  • Unit test framework(s)
  • A static analysis system
  • A reporting mechanism (possibly a dashboard)
  • Integration and Acceptance testing
  • A deployment mechanism

Design your system with the aim of making it work seamlessly, all the constituent parts should ultimately work together in an automated flow. You will find, as the system gets used, that there are likely to be several manual inputs into your system which it may be possible to automate further. For instance, I have seen an automated Hudson/Bamboo plan creation system using selenium, where users have only to input a fraction of the number of details than they would otherwise need to!

Target bottlenecks. Invest a proportionate amount of time addressing the high priority, high value parts of the system. Work out where the bottlenecks were in the old system (or the manual system you are hoping to replace) and invest the right amount of time working on those areas. There’s no point producing a system that can reliably deploy software to any OS when you only need to support one, especially if your build process is shaky and could do with more attention!

 

Choose your Tools Carefully

When deciding what tools to use for your system, there are a number of things to consider. The obvious ones are cost, suitability for purpose, performance and scalability. But consider other factors as well, such as how readily available the support for your tools is, check out the user communities presence and their support coverage. Also consider personal preference! If a whole team really want to use a particular tool for some reason, perhaps they’ve been excited about it for some time, then at least consider using it, you know it will get that team’s support!

 

Sell it!

Before you start the hard (but fun) work of implementing your system, be sure you get the buy-in you need from the management. I’m not just talking about getting approval, I literally mean buy-in. Management should feel invested in this system. Sell the system to them by highlighting the existing issues that it is going to help solve. The more the management teams buy into Continuous Integration, the more support it will get, the more it will be used, and the better we become at making software. If we fail to get management buy-in, we can easily be left with a system that is all too easily overlooked and ignored, and we can end up with a Continuous Integration System, but not actually practicing Continuous Integration.

 


Installing Sonar on the CI server

I’ve been trying out Sonar and it looks great – it’s much more presentable than trawling through maven sites to find build reports. Anyway, I decided to install it on the live build server today, and this is how it happened:

What you’ll need:

Maven

Java JDK

Download Sonar from http://www.sonarsource.org/downloads/

Copy the zip archive over to the build server. I unzipped it in /home/maven/sonar

I’m running a linux x86 32 bit system, so to start Sonar, I cd to:

/home/maven/sonar/sonar-2.5/bin/linux-x86-32

and run:

./sonar.sh start

Sometimes it can take a few minutes for the server to start up, so be patient. Eventually you’ll find your Sonar site up and running at http://{SERVERNAME}:9000 (where {SERVERNAME} is the name of your build server where you installed Sonar. It should look a bit like this:

Next, you have to configure Maven. I’m running with the default Apache Derby database which ships with Sonar, so I added the following section to my maven settings.xml (which I found under /home/maven/maven-2.0.9/conf). You need to add this to the <profiles> section:

<profile>
<id>sonar</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<properties>

<sonar.jdbc.url>
jdbc:derby://localhost:1527/sonar;create=true
</sonar.jdbc.url>
<sonar.jdbc.driver>org.apache.derby.jdbc.ClientDriver</sonar.jdbc.driver>
<sonar.jdbc.username>sonar</sonar.jdbc.username>
<sonar.jdbc.password>sonar</sonar.jdbc.password>
<sonar.host.url>http://localhost:9000</sonar.host.url>

</properties>
</profile>

Then you will need to run your first project against sonar! Go to the root of one of your projects on the build server (for me they were in /home/maven/Bamboo/xml-data/build-dir/PROJ_NAME) and run:

mvn clean install sonar:sonar

Go to http://{SERVERNAME}:9000 and you should now see your project listed. Click on it and revel in sonar goodness:

I’ll migrate to a MySQL db next week, and put an update here about what to do.

UPDATE:

Using a MySql db is a doddle. Once you’ve installed the MySql you simply comment out the Derby db connection details and uncomment the MySql section in the sonar.properties file (which lives in the conf directory of your sonar installation)

sonar.jdbc.url:                            jdbc:mysql://localhost:3306/sonar?useUnicode=true&characterEncoding=utf8
sonar.jdbc.driverClassName:                com.mysql.jdbc.Driver
sonar.jdbc.validationQuery:                select 1

And that’s it!

Changing Guest OS Resolution in VirtualBox

I just installed VirtualBox on my laptop, so that I can run one or two VMs on there, and the installation and setup are really quite straightforward. The “First Steps” section in the documentation here is good enough to see you through the, er, first steps, but there are a couple of other things they really ought to put in this section, rather than later on in the documentation.

One of the first things I wanted to do was change my screen resolution, because it defaulted to 800×600 and didn’t give me any better options. So, it turns out (if you bother to read as far as chapter 4 of the online docs) that to do this you must install some “Guest Additions”… and this is how you do it:

Open a terminal and cd to where your virtual CD drive appears, in my case it was in /media/VBOXADDITIONS_4.0.2_69518/ (see image below – click on image for larger pic)

 

 

 

 

 

 

 

 

Next, run the correct installer. I was running a Linux VM so I chose VBoxLinuxAdditions.run. Run this file by typing “sudo sh ./VBoxLinuxAdditions.run”, as shown below:


Once this has completed it’ll give you a message telling you to restart the guest system, as shown below:

So do as it says, shut down the VM, open VirtualBox and start up your VM again. Now we can go into System > Monitor Preferences, and you should see something like the screen below, with the new (better) screen resolution available to you: