- Code Auditing via Continuous Integration: Overview
- Code Auditing via Continuous Integration: Components
To continue the discussion about “Code Auditing via Continuous Integration” from my last entry, here are a few useful tips about each of the components for CI.
Version Control System
Subversion with the Sventon repository browser.
- In most cases, to avoid costly code-merge activities, development of the major release should be done in the trunk.
- When a large refactoring is planned that can disrupt the work of the rest of the team, a developer should create a branch to keep his changes separate from the trunk. A developer should take into account the efforts to merge the end result in planning his approach.
- Commit comments should be useful, with references to the bug tracking or requirement gathering systems.
- Minor releases (patches) should be committed in branches.
- Every release has to be tagged whether a nightly snapshot for internal testing, or an official public release(see below)
- Code has to be committed every day. You can easily remember cases when somebody had lost results of their work due to hardware or software crashes
Maven and, in rare cases, Ant.
- It should be configurable for execution in different environments.
- It should be modular to allow incremental builds and faster execution of automated tests.
- Build scripts should be OS and IDE independent!
Maven is much more than just a unified build script. Features like project archetypes, artifacts repository, and dependency management can save a lot of valuable development time, especially for large geographically distributed projects.
Two different types of builds are recommended for any project:
- Continuous builds kick in every time a developer commits code. An email is sent out to the developer who commits the code and their manager. Code is not just compiled, but unit tests are also executed. A commit is considered successful only if the code compiles and all tests are passed.
- Nightly builds run every day. This type of build not only compiles the source code, but also launches functional and integration automated tests and code analysis. Also, it collects statistics and publishes them.
Besides executing build scripts based on various algorithms, a build server should be integrated with:
- The version control system to checkout the source code
- The version control viewer to browse the version control system
- The bug tracking system to update issue statuses and provide links to issues fixed in a build
- The release management software to publish release artifacts
- Reporting tools that display the results of the code analysis and trends
Build artifacts, including snapshots, have to be published in repositories. Together with a modular build structure, it can dramatically reduce compile and total build time.
Any of these: JUnit, TestNG, Selenium, and JMock.
There are various types of tests like unit and integration. The main rule is that tests have to be meaningful. The tests have to address a real system or business problem. A very good case for a functional test would be to automatically validate a bug fix. After all, if something was broken before, who can guarantee it will never happen again?
Certain integration tests require an actual database schema in order to work properly. It may not always be possible, or it may require additional resources. As a solution, mock objects can be used instead of DAO or Services implementations.
Such tools as Selenium can be used to create and execute scripts to test the UI.
Any code coverage is better than none. Realistically, it is advisable to achieve at least 80% code coverage.
(Code coverage makes sense only if automated tests are meaningful.)
Static Source Code Analysis
PMD, Findbugs, and CheckStyle
A development team can easily be overwhelmed by the sheer number of issues reported by default. To manage this situation, you should concentrate on reducing the number of error types by selecting only those that are meaningful to you. Over time this ruleset will evolve as the development team learns what’s bad and what’s good.
My favorite set of rules would contain anything related to:
- “stdout” (use loggers!)
- Exceptions (I’ve seen too many caught exceptions that were not handled properly.)
- “nullpointers” (the most frequent runtime exception)
- Copy-and-paste (Think before you press Ctrl+C. Maybe a separate method or procedure can be created?)
One comment on the last rule: I truly believe that developers should learn procedural programming before diving into OOP.
Reporting and Trends
- There should be a dashboard displaying the overall health of all active projects.
- It should provide the ability to drill down into the results of the build execution.
- It should allow custom metrics to be defined.
- There should be the ability to display a metrics trend over time.
When implemented, CI components create an ecosystem where coding errors are flushed out, code is always ready for testing, and product time-to-market is shortened.