Thursday, 15 August 2013

Continous Integration Tools Comparison

In this article of mine I will only be covering the comparison between some of the renowned tools used for Continuous integration.

What is Continues Integration?

Continuous integration (CI) implements continuous processes of applying quality control - small pieces of effort, applied frequently. Continuous integration aims to improve the quality of software, and to reduce the time taken to deliver it, by replacing the traditional practice of applying quality control after completing all development.

Continuous integration – the practice of frequently integrating one's new or changed code with the existing code repository – should occur frequently enough that no intervening window remains between commit and build, and such that no errors can arise without developers noticing them and correcting them immediately. Normal practice is to trigger these builds by every commit to a repository, rather than a periodically scheduled build. The practicalities of doing this in a multi-developer environment of rapid commits are such that its usual to trigger a short time after each commit, then to start a build when either this timer expires, or after a rather longer interval since the last build.
It helps in running all unit and acceptance tests, publishes build results, notifies developers if build breaks, labels successful builds in source repository
Some of the practices which should be followed sincerely once the project is in the Development phase
  • Maintain a code repository
  • Automate the build
  • Make the build self-testing
  • Every commit should be build able
  • Keep the build fast
  • Test in a clone of the production environment
  • Make it easy to get the latest deliverable
  • Everyone can see the results of the latest build
  • Automate deployment
Some of the advantages:
  • when unit tests fail or a bug emerges, developers might revert the - codebase back to a bug-free state, without wasting time debugging
  • Developers detect and fix integration problems continuously - avoiding last-minute chaos at release dates, (when everyone tries to check in their slightly incompatible versions).
  • early warning of broken/incompatible code
  • early warning of conflicting changes
  • immediate unit testing of all changes
  • constant availability of a "current" build for testing, demo, or release purposes
  • immediate feedback to developers on the quality, functionality, or system-wide impact of code they are writing
There are lot of tools already present in the market for Continuous Integration, one can choose the tool according to the requirements, and here is the comparison for some of commonly used tools
  • Cruise Control is a free and open source build scheduler; it works with any build tool that produces parse-able output. Some of the features of Cruise Control:
    • It is cross platform and works with Many of the VCSs
    • It is configured in a xml file which maps the build cycles to certain tasks
    • The results are published on the system tray or in the forms of emails
  • Hudson Build is a Continuous Integration Server which integrates build, unit tests, code coverage, and analysis, gives instant knowledge of status of builds. Some of the features of Hudson Build:
    • Easy Installation
    • Hudson can be configured entirely from its friendly web GUI with extensive on-the-fly error checks and inline help. There's no need to tweak XML manually(optional) anymore
    • Hudson can generate a list of changes made into the build from CVS/Subversion. This is also done in a fairly efficient fashion, to reduce the load on the repository
    • Monitor build results by RSS or e-mail to get real-time notifications on failures and many more...
  • TFS is a tool which integrates with Visual Studio in dock able panel "Team Explorer". In Team Explorer, it lists all opened projects and provides various options in context menu on right clicking the project. It builds on every check-in, gives instant knowledge of status of builds. Some of the features of TFS:
    • Team Foundation Server 2010 supports teams doing parallel development and includes many powerful features like atomic check-ins, branching and merging, shelving, labelling, concurrent check-outs, check-in policies, and the association of check-ins with work items
    • As mentioned it can help in associating the check-ins with work items. Work items can consist of requirements, tasks, bugs, issues, and test cases
    • It provides a robust and fully featured build automation server. You can customize Team Foundation Build and configure triggers for manual build, continuous integration, rolling builds, or scheduled builds
    • Provides features to write scripts to run test cases on each build. There are check-in policies, with those in place you can forbid checking-in something without all unit tests passing
    • Powerful reports and dashboards in TFS helps you to analyse and track progress
    • provides built-in support for Agile and Waterfall methodologies
  • TeamCity is a tool which automates code analysing, compiling, and testing processes, providing instant feedback on build progress, problems and test failures, all in simple and intuitive web-interface. Some of the features of TeamCity:
    • Run multiple builds and tests under different configurations and platforms simultaneously
    • Perform Pre-tested commits, helping the rest of a team sustain an uninterrupted workflow
Have build history insight with customizable statistics on build duration, success rate, code quality and custom metrics


Windows Builder
Java Builder
Source Code management System
IDE Integration
Cruise Control
MSBuild, NAnt, Visual Studio
Phing, Apache Ant, Maven
AccuRev SCM, Alienbrain, BitKeeper, CVS, File system, FTP, Git, Mercurial, MKS Integrity, Perforce, Plastic SCM, PVCS, Rational ClearCase, Robocopy, Surround SCM, Sourceanywhere, SourceGear Vault, StarTeam, Subversion, Telelogic Synergy, Visual Source Safe, TFS
E-mail, CCTray, RSS
Servlet Container
MSBuild, NAnt
Ant, Maven 2, Kundo
AccuRev SCM, Bazaar, BitKeeper, Clearcase, CMVC, CVS, Git, Mercurial, Perforce, PVCS, SourceSafe, StarTeam, Subversion, TFS
Android, E-mail, Google Calendar, IRC, XMPP, RSS, Twitter
Eclipse, IntelliJ IDEA, NetBeans
Windows, VSTM
MSBuild, NAnt, Visual Studio
Ant, Maven 2/3
Subversion, CVS, Git, Mercurial, AccuRev SCM, Clearcase, Perforce, SourceSafe, StarTeam, TFS, Vault
E-mail, XMPP, RSS, IDE, SysTray
Eclipse, Visual Studio, IntelliJ IDEA, RubyMine, PyCharm, PhpStorm, WebStorm
Servlet Container
Team Foundation Server
E-Mail, SOAP
Visual Studio

PS – Hope you will see how to implement all of these in my next post.