Continuous Integration - emphasis on tests not builds?
•
Eric Minick just wrote this post on how Continuous Integration is fixated on the act of building, and not testing. It's interesting: he posits that we all talk about builds, and wrap our tests in builds, kick off builds to do deployments to QA environments, etc. when what really gives us value is testing.
Eric works for Urbancode, who develop AnthillPro. There, they take a different approach to CI, and it's compelling: with AnthillPro you can deploy and/or test the same build many different times. In the post he's pointing out that in most CI implementations, the software gets compiled over and over again as you do further testing:
How does one relate the results of various slow builds to each other and the results of the commit build? These types of problems tend to be difficult to solve, often requiring a good deal of cleverness, the use of excessive source control labelingtagging, and at best remain only partly solved. Further, in order to practice CI speed is of the essence; however, the staged build approach reruns the same compilation several times when running different tests with each build. The extra building wastes resources that could be running tests.
I spent a reasonable amount of my time at ThoughtWorks frustrated that we didn't really solve this problem. We wrote papers about the Build Assembly Line but didn't manage to translate that thinking into implementing a multi-stage build that we could use on projects to use the technique for allowing the use of exhaustive automated functional tests, which TW was a penchant for.
Compilation, which Eric mentions in the snippet above is less of a problem for me. It's not too hard to keep the build artifacts around - in fact if you do venture down the path of multi-stage builds then you really want to know that everything compiles in the first step (it's really annoying to see the second stage of the build fail with a compilation error). Even if you don't buy that, compilation isn't generally the thing that cripples projects - it's slow tests.
He's pointed to the Martin Fowler article on CI as the source of this influence in thinking. I'll be interested to see what response ThoughtWorks have, especially as they will shortly be a competitor to Urbancode.
Anyway, well worth a read.