Advanced Continuous Integration Ideas to make you look Awesome!

Call: +44 (0)23 8098 8890
E-mail:

Posted 22nd March 2018, By Andy C

Introduction

Can Continuous Integration, or CI, really make you and your team look awesome? We had a customer visit us recently for a progress meeting. We logged into our Continuous Integration platform and within just a couple of clicks showed him charts of algorithm performance over time. He thought the improvements were pretty awesome, but he also thought our CI setup was pretty awesome too. Did we look awesome? I didn’t ask him, but we felt a bit awesome!

In our last blog, we introduced the concept of Continuous Integration and then discussed some of its most popular applications. In this second blog of the series, we’ll move beyond the basics and explore some ways of taking your Continuous Integration setup to the next level.

In future blogs, we plan to explore some of the more complex ideas in greater depth, so follow us on Linkedin, Twitter or Facebook to find out more. Also, if you are interested in attending a workshop on CI, or receiving more information on this topic, please sign up to the ITDev Continuous Integration Interest Group.

Top 5 Next level CI Ideas

So, you’ve got the basics in place, what’s worth doing now?

As we said in the last blog, it’s worth noting that projects vary in complexity, team size, quality requirements, timescales and so on. Not every idea will deliver the same benefits on every project, but hopefully this list gives you food for thought. If you’d like some advice on likely costs and benefits based on our experience of CI, both internally and with clients, do get in touch. We’re more than happy to provide an initial consultation free of charge.

1. Peer Review Integration

If you’re using on-line peer review tools, such as Review Board, then it’s well worth considering how they integrate with your CI workflow. If your policy is that you run CI on your trunk/master branch, and you must always peer review changes before merging them back to that branch, then you may be reviewing changes that have not yet been verified by your CI system. This can lead to wasted time, with issues being found manually that could have been automatically detected and with extra review iterations associated with correcting CI failures.

At ITDev, we have been able to integrate Review Board with our CI system, so that CI tests are run on the code that is submitted for review and the results are published back to Review Board before any manual code review takes place.

2. The Earlier, the Better

CI is all about finding issues as quickly as possible, so whilst dedicated CI software is great at this, don’t forget that it’s not the only tool in the box. With most revision control systems, it is possible to set up hook scripts that run on changes before they are accepted to the repository. This is no alternative to CI software, as there are many reasons why it would be impractical to run a full suite of tests before accepting any changes; your engineers’ sanity for one! It is, however, great for simple checks like enforcing a no tabs policy, although it’s usually a good idea to provide a means of waiving certain checks for sets of files such as imported code from third parties. Text editors and IDEs with a level of built-in linting can also be very useful for catching issues as early as you possibly can.

3. Hardware-Based Testing

If you’re working on an embedded system, as we often are, then testing on the target hardware provides a further level of integration testing. For this, you will need to be able to transfer your compiled code to your target device or devices and then initiate some kind of test. This is highly dependent upon the nature of your system and may involve automation of surrounding test equipment, special test modes within your software/hardware and/or a test server providing scripted control/status communications with your hardware and streaming and capture of test data.

This approach requires more significant effort to implement, but for larger, longer running projects it can easily pay for itself.

4. Build Statistics Analysis

Automating your builds is step one, but there are opportunities to extend this further and get more value for limited additional effort. The simplest is just monitoring the time it takes to run a build, which many CI tools do automatically, but this information can be very helpful when you discover your project takes much longer than you expected to build, as you can quickly identify when this started to happen and which change it was that caused it.

If code size is important to you, then you can add scripts to monitor this over time and even plot it on a chart. With a feel for how much of your software you have implemented so far and how much you have left to do, you can quickly identify how tight things are going to get as the project progresses. Again, if you see a sudden unexpected step up, you can identify the change and investigate the cause.

Anywhere you have limited resources, CI can often provide a means to monitor your usage of them. For example, some compilers, or extensions thereof, can provide worst case stack size, other tools can be used to estimate memory requirements and so on.

5. Branch Management & Testing

The standard starting point for CI is to have it running on the trunk or master branch of your code repository. This is great for bringing issues to attention, but wouldn’t it be better to prevent them from reaching that shared branch in the first place? It is common practice to ask developers to run unit tests and so forth before merging back to trunk/master, but that’s a manual step and if we’ve invested in a system to automatically run all of those tests in a consistent and controlled manner, then why run them manually?

We mentioned earlier how we use Review Board to trigger CI testing of changes, but we also allow branches to be easily tested directly from the CI environment. This is especially beneficial when we are using CI for performance tests, as it allows us to run a full suite of automated performance tests and then compare the results between branches and different branch revisions. To achieve this, we have created special parameterised jobs that can be launched from our CI server that automatically generate new CI jobs for the specified branch. Another approach we took with one client who had more unusual requirements, was to create a bespoke tool to measure and analyse a wide range of performance metrics across multiple code branches and revisions.

How ITDev Can Help

As a provider of software and electronics design services, we have been using CI tools and Agile processes at ITDev since 2010. We have extensive experience in deploying them on a wide range of projects spanning embedded systems, software development and FPGA design. We have used them to analyse complex performance trade-offs and to ensure high quality and high efficiency in our development.

We offer advice and assistance to companies considering investing in CI and other workflow improvements. We can also use our experience and library of in-house scripts to quickly deliver ready to use CI environments customised to fit your own project requirements. Initial discussions are always free of charge, so if you have any questions, or would like to find out more, we would be delighted to speak to you. If you are interested in attending a workshop on CI, or receiving more information on this topic, please sign up to the ITDev Continuous Integration Interest Group.

Follow us on LinkedinTwitter or Facebook to find out about our future blog posts containing tips for getting even more from your CI environment. Finally, if you’re interested in how CI can be used in FPGA design, why not also take a look at our FPGA Design Flow tour.


Andy

Engineering Director

MEng - Electrical & Electronic Engineering, Imperial College London - CEng FCMI MIET ACGI

I am ITDev’s Engineering Director and have held this role for more than ten years.

I am passionate about maintaining a great company culture at ITDev and ensuring that the engineering team have all of the support they need to deliver outstanding results. I also enjoy building long term customer relationships and helping to deliver ground-breaking products. Having helped ITDev's clients to develop the first commercial implementations of several evolving standards, I have experience of delivering award winning products in the face of changing requirements.

Prior to joining ITDev, I held roles at Philips Semiconductors and Semtech, where I gained experience in digital and mixed signal design, as well as software development and project management.

I grew up in Bournemouth and now live in Winchester with my wife and two children. Much of my time, outside of work, is currently spent managing the design and build of our new house, which we hope to move into next year.

View all posts by Andy    |    View all ITDev team members

IET Enterprise Partners logo

Latest Blog Posts

VPK120 development board pictured wearing ear defenders with silent smiley on the fan
Posted 13th June 2023, By Aysa D
This blog contains the final steps for adding the minimal IP to send the necessary SYSMON data to the System Controller for controlling the fan on the AMD ...more
VPK120 development board pictured wearing ear defenders
Posted 25th May 2023, By Aysa D
Whilst developing on an AMD Versal VPK120 you will want to control the fan speed to keep the noise at manageable levels. This guide captures the steps taken to ...more
Robin on a bird table
Posted 20th January 2023, By Andy C
The customer is always right, and he hasn't changed his requirements, but the consultant says they're in constant flux. Who's right? Let's explore further ...
JonO interviewing Matthew
Posted 30th June 2022, By Jon O
30th June and it's Matthew's 2 year anniversary of joining as a full-time member of staff. Matthew spent 2 summers working for us as an intern before joining ...more

Latest News

Posted 12th September 2023
ITDev is proud to announce the launch of a new FPGA video IP core. The core allows integrators to quickly and easily add high-quality colour space conversion ...more
Shot of Sydney Harbour Bridge
Posted 3rd March 2023
We welcome David back from Australia and review some of the actvities we've been engaged with in February 2023.
Posted 9th August 2022
Last month we attended the second TechNES FPGA Frontrunners event, here's our write up from the day ...
Posted 28th July 2022
Here's a short report on our attendance at the ECS Taster week Careers Fair On Tues 26th July. A chance to promote industrial opportunties to year 11 students ...more