Continuous Integration: Going Beyond Builds

Establishing a build loop is the first thing you do when you install a continuous integration platform like TeamCity or Jenkins. You have a branch where a bunch of developers check in code frequently. They are bound to break the build often. You configure CI platform to pull code from your code repository via some trigger, probably at every check-in and then build the code. If the build breaks, you know exactly who broke it so everyone can go yell at the person till he fixes it. And that it where continuous integration seems to stop for most people. Like a rusted clock, it keeps on ticking and once in a while manages to tell the correct time. The original definition of continuous integration states it is the process of applying quality control in small chunks, applied frequently to improve the over all quality of the product replacing the traditional practice of applying quality control after the product development is done. Somehow this definition is lost in favor of finding build failures.

 

So what more can Continuous Integration do for you. With a little research and elbow grease, lots more !

  • Unit Test Cases - Test early, test often. Every developer knows that or should know that. However not all developers put that in practice. Developers should be encouraged to write unit test cases for their code. Unit tests should be inherent part of every build. All the best CI systems are good at collecting and parsing Unit test data. They will also show historical data and performance of the Unit tests over time thus showing and highlighting problems with code as it grows.
  • Functional Test Cases - This is where the elbow grease comes in. Depending on the complexity of your software this may or may not be possible. But a limited amount of functional test should be part of the build process. However to successfully execute functional tests, the software needs to be deployed to a test environment. A good Release Manager should be able to automate this process from within the build. These results can be combined with the Unit test data and will reduce the load on the QA team.
  • Code Metrics - Generating metrics on your code with every build and then interpreting them is a very useful thing to do. This can tell you the changes that are happening to your code in the background and even the second and third order consequences of some changes. Some of the common metrics are Test coverage, i.e how much of your code is covered by your test cases, Line of Codes, cyclomatic complexity, etc. However any code metric has to be interpreted in the context of your application and should not be treated as an absolute. Here is an excellent article I found that talks about it.
  • Environment Management - Lastly for the adventurous, you can configure and manage all your environments from within your CI. At the last count Jenkins boasts over 600+ plugins. Extending TeamCity and Jenkins and writing your own plugins is also easy. If configured correctly, your CI can be the central place from which you can manage, control, promote and demote your code, configuration and database changes. This one time effort will create an automation framework that you can leverage across all your infrastructure.

In all the places that I have configured CI, I have always configured it to the level of providing code metrics but was never quite able to configure it to make the whole infrastructure. What is your experience with CI ? Let me know in the comments below.