Continous Integration Tools Comparison

In this article of mine I will only be covering the comparison between some of the renowned tools used for Continuous integration.

What is Continues Integration?

Continuous integration (CI) implements continuous processes of applying quality control - small pieces of effort, applied frequently. Continuous integration aims to improve the quality of software, and to reduce the time taken to deliver it, by replacing the traditional practice of applying quality control after completing all development.

Continuous integration – the practice of frequently integrating one's new or changed code with the existing code repository – should occur frequently enough that no intervening window remains between commit and build, and such that no errors can arise without developers noticing them and correcting them immediately. Normal practice is to trigger these builds by every commit to a repository, rather than a periodically scheduled build. The practicalities of doing this in a multi-developer environment of rapid commits are such that its usual to trigger a short time after each commit, then to start a build when either this timer expires, or after a rather longer interval since the last build.

It helps in running all unit and acceptance tests, publishes build results, notifies developers if build breaks, labels successful builds in source repository

Some of the practices which should be followed sincerely once the project is in the Development phase

• Maintain a code repository
• Automate the build
• Make the build self-testing
• Every commit should be build able
• Keep the build fast
• Test in a clone of the production environment
• Make it easy to get the latest deliverable
• Everyone can see the results of the latest build
• Automate deployment

Some of the advantages:

• when unit tests fail or a bug emerges, developers might revert the - codebase back to a bug-free state, without wasting time debugging
• Developers detect and fix integration problems continuously - avoiding last-minute chaos at release dates, (when everyone tries to check in their slightly incompatible versions).
• early warning of broken/incompatible code
• early warning of conflicting changes
• immediate unit testing of all changes
• constant availability of a "current" build for testing, demo, or release purposes
• immediate feedback to developers on the quality, functionality, or system-wide impact of code they are writing

There are lot of tools already present in the market for Continuous Integration, one can choose the tool according to the requirements, and here is the comparison for some of commonly used tools

• Cruise Control is a free and open source build scheduler; it works with any build tool that produces parse-able output. Some of the features of Cruise Control:
• It is cross platform and works with Many of the VCSs
• It is configured in a xml file which maps the build cycles to certain tasks
• The results are published on the system tray or in the forms of emails
• Hudson Build is a Continuous Integration Server which integrates build, unit tests, code coverage, and analysis, gives instant knowledge of status of builds. Some of the features of Hudson Build:
• Easy Installation
• Hudson can be configured entirely from its friendly web GUI with extensive on-the-fly error checks and inline help. There's no need to tweak XML manually(optional) anymore
• Hudson can generate a list of changes made into the build from CVS/Subversion. This is also done in a fairly efficient fashion, to reduce the load on the repository
• Monitor build results by RSS or e-mail to get real-time notifications on failures and many more...
• TFS is a tool which integrates with Visual Studio in dock able panel "Team Explorer". In Team Explorer, it lists all opened projects and provides various options in context menu on right clicking the project. It builds on every check-in, gives instant knowledge of status of builds. Some of the features of TFS:
• Team Foundation Server 2010 supports teams doing parallel development and includes many powerful features like atomic check-ins, branching and merging, shelving, labelling, concurrent check-outs, check-in policies, and the association of check-ins with work items
• As mentioned it can help in associating the check-ins with work items. Work items can consist of requirements, tasks, bugs, issues, and test cases
• It provides a robust and fully featured build automation server. You can customize Team Foundation Build and configure triggers for manual build, continuous integration, rolling builds, or scheduled builds
• Provides features to write scripts to run test cases on each build. There are check-in policies, with those in place you can forbid checking-in something without all unit tests passing
• Powerful reports and dashboards in TFS helps you to analyse and track progress
• provides built-in support for Agile and Waterfall methodologies
• TeamCity is a tool which automates code analysing, compiling, and testing processes, providing instant feedback on build progress, problems and test failures, all in simple and intuitive web-interface. Some of the features of TeamCity:
• Run multiple builds and tests under different configurations and platforms simultaneously
• Perform Pre-tested commits, helping the rest of a team sustain an uninterrupted workflow

Have build history insight with customizable statistics on build duration, success rate, code quality and custom metrics

Comparison:

Name	Platform	License	Windows Builder	Java Builder	Source Code management System	Notification	IDE Integration
Cruise Control	Cross-Platform	Free	MSBuild, NAnt, Visual Studio	Phing, Apache Ant, Maven	AccuRev SCM, Alienbrain, BitKeeper, CVS, File system, FTP, Git, Mercurial, MKS Integrity, Perforce, Plastic SCM, PVCS, Rational ClearCase, Robocopy, Surround SCM, Sourceanywhere, SourceGear Vault, StarTeam, Subversion, Telelogic Synergy, Visual Source Safe, TFS	E-mail, CCTray, RSS	Eclipse
Hudson	Servlet Container	MIT	MSBuild, NAnt	Ant, Maven 2, Kundo	AccuRev SCM, Bazaar, BitKeeper, Clearcase, CMVC, CVS, Git, Mercurial, Perforce, PVCS, SourceSafe, StarTeam, Subversion, TFS	Android, E-mail, Google Calendar, IRC, XMPP, RSS, Twitter	Eclipse, IntelliJ IDEA, NetBeans
TeamCity	Windows, VSTM	Proprietary	MSBuild, NAnt, Visual Studio	Ant, Maven 2/3	Subversion, CVS, Git, Mercurial, AccuRev SCM, Clearcase, Perforce, SourceSafe, StarTeam, TFS, Vault	E-mail, XMPP, RSS, IDE, SysTray	Eclipse, Visual Studio, IntelliJ IDEA, RubyMine, PyCharm, PhpStorm, WebStorm
TFS	Servlet Container	Proprietary	MSBuild	Unknown	Team Foundation Server	E-Mail, SOAP	Visual Studio

PS – Hope you will see how to implement all of these in my next post.