Automate Configuration Management Using Tokens!

Devops engineers are often tasked with the job of managing deployments of code to multiple environments. Each one may have different environmental settings such as server name/ip address, URL, subnet name and different connection settings such as db connection strings and app layer connections to name but a few. In all, there’s a truck load of differences. These differences, for convenience sake, are usually stored in config and ini files…

Usually they’re a nightmare (sorry, a challenge) to manage. But here’s a solution that has worked well for me…..

  • Use “master” config files that have ALL environmental details replaced with tokens
  • Move copies of these files to folders denoting the environments they’ll be deployed to
  • Use a token replacement operation to replace the tokens
  • Deploy over the top of your code deployments, in doing so replacing the default config files

All the above can be automated very easily, and here’s how:
First off, make tokenised copies of your config files, so that environmental values are replaced with tokens, e.g.
change things like:

<add key=”DB:Connection” value=”Server=TestServer;Initial Catalog=TestDB;User id=Adminuser;password=pa55w0rd”/ >

to

<add key=”DB:Connection” value=”Server=%DB_SERVER%;Initial Catalog=%DB_NAME%;User id=%DB_UID%;password=%DB_PWD%”/ >

Then save a copy of these tokens, and their associated values in a sed file. This sed file should contain values specific to one environment, so that you’ll end up with 1 sed file per environment. These files act as lookups for the tokens and their values.

The syntax for these sed files is:

s/%TOKEN%/TokenValue/i

So here’s the contents of a test environmemt sed file (testing.sed)

s/%DB_SERVER%/TestServer/i

s/%DB_NAME%/TestDB/i

s/%DB_UID%/Adminuser/i

s/%DB_PWD%/pa55w0rd/i

And here’s live.sed:

s/%DB_SERVER%/LiveServer/i

s/%DB_NAME%/LiveDB/i

s/%DB_UID%/Adminuser/i

s/%DB_PWD%/Livepa55w0rd/i

Next up, we want to have a section in our build script which renames the web_master.config files and copies them, and then runs the token replacement task….so here it is:

<target name=”moveconfigs” description=”renames configs, copies them to respective prep locations”>

<delete file=”${channel.dir}\web.config” verbose=”true” if=”${file::exists (webconfig)}” />

<move file=”${channel.dir}\web_Master.config” tofile=”${channel.dir}\web.config” if=”${file::exists (webMasterConfig)}” />

<delete file=”${channel.dir}\web.config” verbose=”true” if=”${file::exists (webconfig)}” />

<move file=”${channel.dir}\web_Master.config” tofile=”${channel.dir}\web.config” if=”${file::exists (webMasterConfig)}” />

<mkdir dir=”${build.ID.dir}\configs\TestArea” />

<mkdir dir=”${build.ID.dir}\configs\Live” />

<copy todir=”${build.ID.dir}\configs\TestArea\${channel.output.name}” >

<fileset basedir=”${channel.dir}” >

<include name=”**\*.config” />

<exclude name=”*.bak” />

</fileset>

</copy>

<copy todir=”${build.ID.dir}\configs\Live\${channel.output.name}” >

<fileset basedir=”${channel.dir}” >

<include name=”**\*.config” />

<exclude name=”*.bak” />

</fileset>

</copy>

</target>

<target name=”EditConfigs” description=”runs the token replacement by calling the sed script and passing the location of the tokenised configs as a parameter” >

<exec program=”D:\compiled\call_testarea.cmd” commandline=”${build.ID.dir}” />

<exec program=”D:\compiled\call_Live.cmd” commandline=”${build.ID.dir}” />

</target>

As you can see, the last target calls a couple of cmd files, the first of which looks like this:

xfind “%*\TestArea” -iname *.* |xargs sed -i -f “D:\compiled\config\testing.sed”

xfind “%*\TestArea” -iname *.* |xargs sed -i s/$/\r/

This is the sed command to read the config file, pipe the contents to sed and run the script file against it, and edit it in place. the second line handles Line Feeds so that the file ends up in a readable state. Essentially we’re telling sed to recursively read through the config file, and replace the tokens with the relevant value.

The advantage that this method has over using Nant’s “replacetokens” is that we can call the script for any number of files in any number of subdirectories using just one call, and the fact that the tokens and values are extracted from the build script. Also, the syntax means that the sed files are a lot smaller than a similar functioning Nant script would be.

And that’s about it.

Advertisement

ClickOnce and Nant – The Plot Thickens

Turns out that these ClickOnce deployment builds aren’t as piss-easy as I once thought.
Turns out that the builds need to be customised for different environments, nothing new there, but (and here comes the catch), all the environmental settings have to be applied at BUILD TIME!!! Why? I hear you ask, and the answer is: because if you edit the config files post-build it changes some checksum jiggery pokery wotnot and then the thingumyjig goes and fails!!! Typical. (Basically the details you need to configure are held in files you cannot edit post build because the manifest file will do a checsum evaluation and see that someone has edited the file, and throw errors).
So what I’ve decided to do is this….

  1. Copy all configurable files to seperate environmentally named folders pre-build.
  2. Use SED to replace tokens for each environment in these files
  3. Copy 1 of them back to the build folder
  4. Compile
  5. Copy the output to the environmentally named folder

redo steps 3,4,5 for all the environments.
And hey presto, this works.

<target name=”changeconfigs”>
<!–This bit sets up some folders where I’ll do the prep work for each environment–>
<delete dir=”${config.dir}\${project.name}” verbose=”true” if=”${directory::exists(config.dir+’\’+project.name)}” />
<mkdir dir=”${config.dir}\${project.name}\TestArea” />
<mkdir dir=”${config.dir}\${project.name}\DevArea” />
<mkdir dir=”${config.dir}\${project.name}\Staging” />
<mkdir dir=”${config.dir}\${project.name}\UAT” />
<mkdir dir=”${config.dir}\${project.name}\Live” />

<!–This bit moves a tokenised config file to these folders–>
<copy file=”${source.dir}\App_Master.config” tofile=”${config.dir}\${project.name}\TestArea\app.config” />
<copy file=”${source.dir}\App_Master.config” tofile=”${config.dir}\${project.name}\DevArea\app.config” />
<copy file=”${source.dir}\App_Master.config” tofile=”${config.dir}\${project.name}\Staging\app.config” />
<copy file=”${source.dir}\App_Master.config” tofile=”${config.dir}\${project.name}\UAT\app.config” />
<copy file=”${source.dir}\App_Master.config” tofile=”${config.dir}\${project.name}\Live\app.config” />

<!–This bit calls sed, which replaces the tokens with relevant values for each environment, more on sed another time!–>
<exec program=”${sedUAT.exe}” commandline=”${sedParse.dir}” />
<exec program=”${sedTestArea.exe}” commandline=”${sedParse.dir}” />
<exec program=”${sedDevArea.exe}” commandline=”${sedParse.dir}” />
<exec program=”${sedStaging.exe}” commandline=”${sedParse.dir}” />
<exec program=”${sedLive.exe}” commandline=”${sedParse.dir}” />
</target>

<!–This bit copies the edited file back to the build directory–>
<target name=”prepTestArea”>
<delete file=”${source.dir}\app.config” />
<copy file=”${config.dir}\${project.name}\TestArea\app.config” tofile=”${source.dir}\app.config” />
</target>

<!–This bit builds the ClickOnce project–>
<target name=”publishTestArea” >
<msbuild project=”${base.dir}\Proj1\ClickOnce.vbproj”>
<arg value=”/t:Rebuild” />
<arg value=”/property:Configuration=Release”/>
<arg value=”/p:ApplicationVersion=${version.num}”/>
<arg value=”/p:InstallUrl=http://testarea/ClickOnce/”/>
<arg value=”/t:publish”/>
<arg value=”/p:UpdateRequired=true”/>
<arg value=”/p:MinimumRequiredVersion=${version.num}”/>
</msbuild>
</target>

<!–This bit copies the output to an environment-named folder, ready for deployment–>
<target name=”copyfilesTestArea”>
<mkdir dir=”${versioned.dir}\TestArea” />
<copy todir=”${versioned.dir}\TestArea” includeemptydirs=”true”>
<fileset basedir=”${base.dir}\Proj1\bin\Release\”>
<include name=”**.publish\**\*.*” />
</fileset>
</copy>
</target>

REPEAT LAST 3 TARGETS FOR THE OTHER ENVIRONMENTS.

Now that wasn’t too hard, and it doesn’t take up too much extra time.
I suppose I’d better mention some of the arguments I’m passing in the MSBuild calls:

<arg value=”/t:Rebuild” /> – I do this because it must re build the .deploy files each time, or you get the previous builds environment settings left in there because MSBuild decides to skip files that haven changed….

<arg value=”/property:Configuration=Release”/> – Obvious

<arg value=”/p:ApplicationVersion=${version.num}”/> – ClickOnce apps have a version stamped on them for various reasons, one of them being for use in automatic upgrades – people with installshield knowledge will know what a joke that can be!

<arg value=”/p:InstallUrl=http://testarea/ClickOnce/”/> – A pretty important one this, it stamps the URL for the download onto the manifest or application file.

<arg value=”/t:publish”/> – just calls the publish task, I do this because this makes the setup.exe

<arg value=”/p:UpdateRequired=true”/>
<arg value=”/p:MinimumRequiredVersion=${version.num}”/> – These 2 together mean the app will do a forced upgrade when a new version becomes available

So far, so good. My next trick will hopefully be how to get 2 installations working side-by-side. Currently it doesn’t work because one will overwrite the other. I’m working on it okay!!??

Building ClickOnce Applications with NAnt

Since I don’t like to actually do any work, and would much rather automate everything I’m required to do, I decided to automate a ClickOnce application build, because doing it manually was taking me literally, er, seconds, and this is waaaaaay too much like hard work. So, I naturally turned to NAnt, which is so often the answer to all my deployment questions….The answer came in the form of using NAnt to call MSBuild and pass the publish target, along with the version number. So, this is what you need to do:

Add a property to your nant script containing your build number (you can get this from CruiseControl.Net if you’re using CCNet to do your builds)
<property name=”version.num” value=”1.2.3.4″/>
Then just compile the project using NAnt’s MSBuild task, and call the publish target:

<target name=”publish” >

<msbuild project=”${base.dir}\ClickOnce.vbproj”>

<arg value=”/property:Configuration=Release”/>

<arg value=”/p:ApplicationVersion=${version.num}”/>

<arg value=”/t:publish” />

</msbuild>

</target>

The next thing you need to do is create or update the publish.htm file. What I’ve done for this is to take a copy of a previously generated publish.htm, and replace the occurrences of the application name with a token. Then in the NAnt script, I replace the token with the relevant application name with a version number. I do this because the version number will change with each build, and rather than manually update it, which is much too complicated for me, I’d rather just automate it so that I can go back to sleep while it builds.I tokenised the application name because of a much darker, more sinister reason that I’ll maybe explain at another time, but the world’s just not ready for that yet.
Anyway, here’s all that in NAntish:

<copy todir=”${config.dir}\${project.name}”>

<fileset basedir=”.”>

<include name=”publish.htm” />

</fileset>

<filterchain>

<replacetokens>

<token key=”VERSION” value=”${version.num}” />

<token key=”APPNAME” value=”${appname}” />

</replacetokens>

</filterchain>

</copy>

Fixing java heap issue with maven sites

I’ve suddenly started getting a few java heap (OutOfMemory) errors with my maven builds, mainly when I run the mvn site phase, but also sometimes when I run sonar:sonar.

I’m running the builds on both linux (centos) and windows.

To fix the issue on Windows:

Edit mvn.bat (this lives in your maven bin directory) and add

set MAVEN_OPTS=-Xmx512m

In theory you could add an environment variable called MAVEN_OPTS and give it the same value as above (Xmx512m) but this didn’t actually work very well for me.

To fix on linux:

Edit your mvn file (which for me was in /usr/local/maven/bin/) and add:

export MAVEN_OPTS=”-Xms256m -Xmx512m”

You could of course add this to your bash profile (don’t forget to source it afterward) or add it to etc/profile, but I found adding it to the mvn file to work best.

To fix on Continuous Integration Servers:

I’ve been getting this error on a number of our CI servers as well, so rather than go around adding “export MAVEN_OPTS” all over the place, I am passing it via the CI system. Hudson, Jenkins, Bamboo and Go all have simple UIs for adding extra parameters to your build commands.

Installing Go (crusie) build agents on linux

This is just an easy at-a-glance reference for installing the Go cruise agent on Linux because I’ve done it a few times and just want to have the instructions in one place. I’m using centos for my OS, but these instructions are true for most rpm supporting linux varieties.

Download the rpm:

You have to download the agent from the website here. Copy this to somewhere sensible on the target box, like /tmp for example.

Create User and Extract rpm:

After following the standard instructions a couple of times I noticed that the group and user “cruise” were not being created correctly on my servers. This could be an issue with the rpm I was using or an issue with the VM servers. Either way, to get around this issue I just manually create the group and user before extracting the rpm:

useradd cruise
groupadd cruise
useradd -G cruise cruise

Next I just install the rpm as root:

sudo rpm -i cruise-agent-2.0.0-11407.noarch.rpm

N.B. The latest rpm at this point in time is actually “go-agent-2.1.0-11943.noarch.rpm”.

The files are installed here:

/etc/default/cruise-agent

/var/lib/cruise-agent

/var/log/cruise-agent

/usr/share/cruise-agent

Connecting the Agent with the Server:

The file /etc/default/cruise-agent needs to be edited so that the cruise agent knows how to connect to the cruise server.

Open this file in vim or something similar. it should look like this:

CRUISE_SERVER=192.168.xxx.xx
export CRUISE_SERVER
CRUISE_SERVER_PORT=8153
export CRUISE_SERVER_PORT
AGENT_WORK_DIR=/var/lib/cruise-agent
export AGENT_WORK_DIR
DAEMON=Y
VNC=N

Simply change the IP address of the CRUISE_SERVER to the IP address of the cruise server! You might also need to change the port number if you’ve installed your cruise server to a non default port.

Next you need to start the agent:

/etc/init.d/cruise-agent start

And that’s about it. You should now see the agent appear in your agents list on the cruise server. Put a tick next to it and click enable and this will add the new agent to your cruise-config.xml, where you can assign resources or add it as an environment.

 

 

 

 

JDepend design metrics in CI

This article is intended to give the reader enough information to understand what JDepend is, what it does, and how to use it in a maven build. It’s a kind of cheat sheet, if you like.

What is it?

JDepend is more of a design metric than a code metric, it gives you information about your classes with regards to how they’re related to each other. Using this information you should be able to identify any unwanted or dubious dependencies.

How does it do that?

It traverses Java class files and generates design quality metrics, such as:

  • Number of Classes and Interfaces
  • Afferent Couplings (Ca) – What is this?? Someone probably feels very proud of themselves for coming up with this phrase. Afferent coupling means the number of other packages which depend on the package being measured, in a nutshell. JDepend define this as a measure of a package’s “responsibility”
  • Efferent Couplings (Ce) – Sort of the opposite of Ca. It’s a measure of the number of other packages that your package depends on
  • Abstractness (A) – The ratio of abstract classes to total classes.
  • Instability (I) – The ratio of efferent coupling (Ce) to total coupling (Ce + Ca)
  • Distance from the Main Sequence (D) – this sounds fairly wishy-washy and I’ve never paid any attention to it. It’s defined as: “an indicator of the package’s balance between abstractness and stability”. Meh.

 

To use JDepend with Maven you’ll need Maven 2.0 or higher and JDK 1.4 or higher. You don’t need to install anything, as maven will sort this out for you by downloading it at build time.

Here’s a snippet from one of my project POMs, it comes from in the <reporting> section:


 

<plugin>

    <groupId>org.codehaus.mojo</groupId>

    <artifactId>jdepend-maven-plugin</artifactId>

    <configuration>

        <targetJdk>1.6</targetJdk>

        <outputDirectory>build/maven/${pom.artifactId}/target/jdepend-reports</outputDirectory>

    </configuration>

</plugin>

 

What you’ll get is a JDepend entry under the project reports section of your maven site, like this:

project-reports-page

Maven Project Reports Page

 

And this is what the actual report looks like (well, some of it):

jdepend-report

jdepend report

Summary:

JDepend isn’t something I personally use very heavily, but I can understand how it could be used to good effect as a general measure of how closely related your classes are, which, in certain circumstances could prompt you to redesign or refactor your code.

I don’t think this sort of information is required on a per commit basis, so I’d be tempted to only include it in my nightly reports. However, I also use Sonar, and that has a built-in measure of afferent coupling, so if you’re only interested in that measurement and you’re already running Sonar, then JDepend is probably a bit of an unnecessary overhead. Also, Sonar itself has some good plugins which can provide architectural and design governance features, at least one of which I know implemented JDepend.

Sonar errors with Maven 2.2.1

I’m running maven 2.2.1 and trying to get my build to work with sonar 2.7, but I keep getting this error:

“Error resolving version for ‘org.codehaus.mojo:sonar-maven-plugin’: Plugin requires Maven version 3.0”

The Sonar FAQ rather unhelpfully says:
“This error means that you’re using Maven 2.0.10 or Maven 2.0.11.”
Er, no I’m not, actually!

The suggested workaround is to use 1.0-beta-2 in my POM, but when I do that I get:

“Unable to find resource ‘org.codehaus.mojo:sonar-maven-plugin:maven-plugin:1.0-beta-2′”

It’s one of those maven errors that says it can’t find it in the central repo even though it’s right there in front of your eyes…

Anyway, as a second workaround, I’ve referenced version 1.0-beta-1 and it seems to pick this up fine.

Hope this helps.

Build issue with maven-project-info-reports-plugin

Yesterday had a whole bunch of issues with some of my Maven plugins. I’ll list some details here for reference.

I’m running maven 2.2.1 and (I’ve suddenly just noticed) java 1.5.

The first error I recieved was:

Deprecated API called – not org.apache.maven.doxia.sink.Sink
instance and no SinkFactory available. Please update this plugin.

Which I thought was coming from my maven-jxr-plugin. So I simply commented it out, it’s not that important to me.

Next up I got:

A required class is missing org/apache/maven/reporting/AbstractMavenReport

Which, after a little googling, appeared to have something to do with the maven-project-info-reports-plugin. So I decided to upgrade to the latest version of maven-project-info-reports-plugin. No joy. Maven claimed it couldnd locate version 2.3.1 from ANY repository, despite the fact that it was there in the maven central repo AND I installed it in my own repo. I think this was the error it gave me:

unable to load the mojo org.apache.maven.plugins:maven-project-info-reports-plugin

So to fix this I randomly tried referencing an older version of the plugin (namely v 2.1.1) and it seems to work now. My plugin reference now looks like this in my POM:


<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-project-info-reports-plugin</artifactId>
    <version>2.1.1</version>
    <configuration>
           <dependencyLocationsEnabled>false</dependencyLocationsEnabled>
    </configuration>
</plugin>


 

Right, I'm going to go update my java to 1.6, which it what it should have been all along, and hope that this wasn't the cause of all my woes yesterday 🙂