Survey: Ideal Continuous Integration
One of the things we do here at p&p, in addition to providing guidance, tools, and reference implementations, is try to evolve the processes we use to write software. Last week, over lunch, the developers here at p&p had an interesting discussion about continuous integration. We were trying to decide what changes we should make to our CI process, if any. At som point I’ll report back on what we are doing internally, but at this point, I’d like to hear from the rest of the world on what you would do. Before I ask for input though, I want to make sure we are on the same page. Here is how I view CI. A basic CI process, started on every check-in, includes:
Get the latest source
Build the components
Build and running the unit tests
Report success or failure of the build and unit test run (If any unit test fail, the build fails)
Experience has shown me the need to inject a few more steps in the process, so it looks more like this:
Get the latest source
Build the components
Build the deployment mechanism (MSIs, usually)
Build and running the unit tests
Report success or failure of the build and unit test run (If any unit test fail, the build fails)
Then there is my view of an ideal scenario. I have yet to see this implemented in the real world on a full-scale project:
Get the latest source
Build the components
Build the deployment mechanism (MSIs, usually)
Build and running the unit tests
Deploy to a development environment
Run a build verification test suite against the system deployed in the development environment
Run a full regression test pass against the deployed system
Run a full acceptance test pass against the deployed system
Report success or failure of the build, unit test run, deployment, build verification, regression tests, and acceptance tests (If any step in the process fails, or any test fails, the build fails)
So, readers, I have a few questions:
What do you think about the ideal scenario?
How would your ideal scenario work?
What do you actually do in your CI process?
