Introduction
Can Continuous Integration, or CI, really make you and your team look awesome? We had a customer visit us recently for a progress meeting. We logged into our Continuous Integration platform and within just a couple of clicks showed him charts of algorithm performance over time. He thought the improvements were pretty awesome, but he also thought our CI setup was pretty awesome too. Did we look awesome? I didn’t ask him, but we felt a bit awesome!
In our last blog, we introduced the concept of Continuous Integration and then discussed some of its most popular applications. In this second blog of the series, we’ll move beyond the basics and explore some ways of taking your Continuous Integration setup to the next level.
In future blogs, we plan to explore some of the more complex ideas in greater depth, so follow us on Linkedin, Twitter or Facebook to find out more. Also, if you are interested in attending a workshop on CI, or receiving more information on this topic, please sign up to the ITDev Continuous Integration Interest Group.
Top 5 Next level CI Ideas
So, you’ve got the basics in place, what’s worth doing now?
As we said in the last blog, it’s worth noting that projects vary in complexity, team size, quality requirements, timescales and so on. Not every idea will deliver the same benefits on every project, but hopefully this list gives you food for thought. If you’d like some advice on likely costs and benefits based on our experience of CI, both internally and with clients, do get in touch. We’re more than happy to provide an initial consultation free of charge.
1. Peer Review Integration
If you’re using on-line peer review tools, such as Review Board, then it’s well worth considering how they integrate with your CI workflow. If your policy is that you run CI on your trunk/master branch, and you must always peer review changes before merging them back to that branch, then you may be reviewing changes that have not yet been verified by your CI system. This can lead to wasted time, with issues being found manually that could have been automatically detected and with extra review iterations associated with correcting CI failures.
At ITDev, we have been able to integrate Review Board with our CI system, so that CI tests are run on the code that is submitted for review and the results are published back to Review Board before any manual code review takes place.
2. The Earlier, the Better
CI is all about finding issues as quickly as possible, so whilst dedicated CI software is great at this, don’t forget that it’s not the only tool in the box. With most revision control systems, it is possible to set up hook scripts that run on changes before they are accepted to the repository. This is no alternative to CI software, as there are many reasons why it would be impractical to run a full suite of tests before accepting any changes; your engineers’ sanity for one! It is, however, great for simple checks like enforcing a no tabs policy, although it’s usually a good idea to provide a means of waiving certain checks for sets of files such as imported code from third parties. Text editors and IDEs with a level of built-in linting can also be very useful for catching issues as early as you possibly can.
3. Hardware-Based Testing
If you’re working on an embedded system, as we often are, then testing on the target hardware provides a further level of integration testing. For this, you will need to be able to transfer your compiled code to your target device or devices and then initiate some kind of test. This is highly dependent upon the nature of your system and may involve automation of surrounding test equipment, special test modes within your software/hardware and/or a test server providing scripted control/status communications with your hardware and streaming and capture of test data.
This approach requires more significant effort to implement, but for larger, longer running projects it can easily pay for itself.
4. Build Statistics Analysis
Automating your builds is step one, but there are opportunities to extend this further and get more value for limited additional effort. The simplest is just monitoring the time it takes to run a build, which many CI tools do automatically, but this information can be very helpful when you discover your project takes much longer than you expected to build, as you can quickly identify when this started to happen and which change it was that caused it.
If code size is important to you, then you can add scripts to monitor this over time and even plot it on a chart. With a feel for how much of your software you have implemented so far and how much you have left to do, you can quickly identify how tight things are going to get as the project progresses. Again, if you see a sudden unexpected step up, you can identify the change and investigate the cause.
Anywhere you have limited resources, CI can often provide a means to monitor your usage of them. For example, some compilers, or extensions thereof, can provide worst case stack size, other tools can be used to estimate memory requirements and so on.
5. Branch Management & Testing
The standard starting point for CI is to have it running on the trunk or master branch of your code repository. This is great for bringing issues to attention, but wouldn’t it be better to prevent them from reaching that shared branch in the first place? It is common practice to ask developers to run unit tests and so forth before merging back to trunk/master, but that’s a manual step and if we’ve invested in a system to automatically run all of those tests in a consistent and controlled manner, then why run them manually?
We mentioned earlier how we use Review Board to trigger CI testing of changes, but we also allow branches to be easily tested directly from the CI environment. This is especially beneficial when we are using CI for performance tests, as it allows us to run a full suite of automated performance tests and then compare the results between branches and different branch revisions. To achieve this, we have created special parameterised jobs that can be launched from our CI server that automatically generate new CI jobs for the specified branch. Another approach we took with one client who had more unusual requirements, was to create a bespoke tool to measure and analyse a wide range of performance metrics across multiple code branches and revisions.
How ITDev Can Help
As a provider of software and electronics design services, we have been using CI tools and Agile processes at ITDev since 2010. We have extensive experience in deploying them on a wide range of projects spanning embedded systems, software development and FPGA design. We have used them to analyse complex performance trade-offs and to ensure high quality and high efficiency in our development.
We offer advice and assistance to companies considering investing in CI and other workflow improvements. We can also use our experience and library of in-house scripts to quickly deliver ready to use CI environments customised to fit your own project requirements. Initial discussions are always free of charge, so if you have any questions, or would like to find out more, we would be delighted to speak to you. If you are interested in attending a workshop on CI, or receiving more information on this topic, please sign up to the ITDev Continuous Integration Interest Group.
Follow us on Linkedin, Twitter or Facebook to find out about our future blog posts containing tips for getting even more from your CI environment. Finally, if you’re interested in how CI can be used in FPGA design, why not also take a look at our FPGA Design Flow tour.
