March 2010 Archives

At IDM, we use a lot of tools for software development. I want to show here how the high-level ones fit together:

The tools are separated in three categories:

• in green, tools which are directly used by developers
  
• in blue, more server-side tools which are part of the development environment
  
• in red, production tools.

The Java environment is the most standardized one:

• We use SpringSource Tool Suite as the IDE. It is a rebranded version of Eclipse with m2eclipse (a plugin to work with Maven) and other plugins. The Subclipse plugin is used as the SVN client.
• Source code is commited in the SVN repository.
• A continuous integration server (Hudson) regularly checks out modifications from SVN and builds the software. Emails are sent in case of build or test failures.
  
• Hudson appends build details to Mantis issues when an issue ID is contained in the commit message.
• When a Web application is built by Hudson, it is then deployed on a dedicated server running Tomcat. Application tests are then ran against this web-application using SeleniumHQ.
• The developer gets immediate build feedback thanks to Hudson Tray Tracker.
• When the software is ready for production, a developer uses the Maven Release plugin to perform a release. This release is stored in the Maven repository (powered by Nexus) with a clean version number, and a tag is associated in SVN.
  
• Administrators deploy release versions on production servers by taking archives in Nexus. Thanks to the version number and the associated SVN tag, a developer can debug the exact production version of the application.

For the Microsoft .Net environment, the main difference comes from the release process which relies more on the developer. Because it is not enforced by a tool, like the Maven Release plugin for Java, this process requires more attention. For instance, the developer must not forget to tag the version in SVN. This is critical because of the absence of repository for production versions.

What's next in this architecture:

• Automated deployment of snapshots Maven artefacts (Java only): each time Hudson builds an artefact which is in a snapshot version, it should be automatically deployed to Nexus. This would allow simpler development for other users of this dependency.
• Introduction of a code review tool: there are some parts of the software we work on which are critical, especially the domain part. Checking SVN commits need to be more integrated in the development process than simply looking at SVN emails. We are currently evaluating tools such as Review Board.
  

A new version of Hudson Tray Tracker is out: version 0.10. You can download it from here.

Please note that, from now on, HTT relies on the Microsoft .Net framework v3.5. Make sure you have it installed before upgrading.

Here is the change log:

• Feature (#1): implemented Basic authentication.
• Feature (#29): sound notifications.
  
• Change (#34): migrated to Visual Studio 2008 / .Net 3.5.
  
• Fix (#11): use UTF-8 as the encoding when getting data from Hudson.
• Fix (#32): manually running a build now triggers an immediate build (useful when a "quiet period" is configured on the server).
• Fix (#33): the settings form was sometimes displaying the list of projects of the wrong server.
• Fix (#36): the acknowledged status was sometimes clearing itself.

Here at IDM, we've been using Hudson (the continuous integration server) for about a year now. More than 80 jobs are configured to run on the master (Linux) instance or on the slave (Windows) instance.

The server running the master instance is regularly backed up, but it was hard to keep track of modifications and almost impossible to get an history of modifications.

We've now set up a job to back up the configuration of Hudson and of all the jobs by following instructions from Keeping your configuration and data in Subversion, and these problems are now gone.

What's great is that, when someone changes the configuration of a project, we now receive an email and we can check the modifications. Also, we can do mass updates on the configuration, commit the changes, get the updates on the Hudson server and ask Hudson to reload its configuration.

In the JEE world, an HTTP request in handled from the start to the end by a given thread. That's why we often use ThreadLocal variables to store variables in the context of a request.

In ASP.Net, the container might well decide to switch the thread in the middle of the processing of the request. This is called thread agility.

Here is an example of logs I extracted from one of our applications:

15:10:23,666 [4] BeginRequest /Case/WeightVariables
15:10:23,744 [8] BeginRequest /resources/images/date-trigger.gif
15:10:23,744 [8] EndRequest /resources/images/date-trigger.gif
15:17:35,900 [8] EndRequest /Case/WeightVariables

As you can see, the request to /Case/WeightVariables starts being processed in thread #4 and finishes in thread #8.

From what I read, thread switching can only occur during an asynchronous IO operation. File or network operations are then likely to trigger a thread switch.

The consequence is that the [ThreadStatic] attribute should not be used in an ASP.Net application. Otherwise, if you store a variable at the beginning of the request, the variable might well be no longer accessible at the end of the request. In particular, this can be an issue when working with NHibernate, if you store the NHibernate session in a thread static storage variable. You might randomly get errors such as:

NHibernate.LazyInitializationException - Initializing[...]-Could not initialize proxy - no Session.

As a replacement to the ThreadStatic attribute, you should use HttpContext.Current to store variables in the context of the HTTP request.

If you are using Spring.NET, you can use the LogicalThreadContext class which provides an abstraction over thread-static data. If you configure it correctly, it will use the HTTP context in a Web environment and thread-static storage in other applications.

The easiest way to configure the logical thread context in an ASP.Net application is to enable the WebSupportModule in the Web.Config file:

<system.web>
  <httpModules>
    <add name="Spring" type="Spring.Context.Support.WebSupportModule, Spring.Web"/>
  </httpModules>
</system.web>

That way, Spring.NET's OpenSessionInViewModule will store the NHibernate session in the HttpContext, and lazy initialization errors will no longer occur.


References:

• A tale of two techniques: The [ThreadStatic] Attribute and System.Web.HttpContext.Current.Items
• ASP.NET thread switching
• log4net Context problems with ASP.Net thread agility
• ThreadStatic, CallContext and HttpContext in ASP.Net
• Spring.NET: Threading and Concurrency Support
• Spring.NET: Automatic context loading and hierarchical contexts