Advanced Continuous Integration Ideas to make you look Awesome!

Call: +44 (0)23 8098 8890
E-mail:

Posted 22nd March 2018, By Andy C

Introduction

Can Continuous Integration, or CI, really make you and your team look awesome? We had a customer visit us recently for a progress meeting. We logged into our Continuous Integration platform and within just a couple of clicks showed him charts of algorithm performance over time. He thought the improvements were pretty awesome, but he also thought our CI setup was pretty awesome too. Did we look awesome? I didn’t ask him, but we felt a bit awesome!

In our last blog, we introduced the concept of Continuous Integration and then discussed some of its most popular applications. In this second blog of the series, we’ll move beyond the basics and explore some ways of taking your Continuous Integration setup to the next level.

In future blogs, we plan to explore some of the more complex ideas in greater depth, so follow us on Linkedin, Twitter or Facebook to find out more. Also, if you are interested in attending a workshop on CI, or receiving more information on this topic, please sign up to the ITDev Continuous Integration Interest Group.

Top 5 Next level CI Ideas

So, you’ve got the basics in place, what’s worth doing now?

As we said in the last blog, it’s worth noting that projects vary in complexity, team size, quality requirements, timescales and so on. Not every idea will deliver the same benefits on every project, but hopefully this list gives you food for thought. If you’d like some advice on likely costs and benefits based on our experience of CI, both internally and with clients, do get in touch. We’re more than happy to provide an initial consultation free of charge.

1. Peer Review Integration

If you’re using on-line peer review tools, such as Review Board, then it’s well worth considering how they integrate with your CI workflow. If your policy is that you run CI on your trunk/master branch, and you must always peer review changes before merging them back to that branch, then you may be reviewing changes that have not yet been verified by your CI system. This can lead to wasted time, with issues being found manually that could have been automatically detected and with extra review iterations associated with correcting CI failures.

At ITDev, we have been able to integrate Review Board with our CI system, so that CI tests are run on the code that is submitted for review and the results are published back to Review Board before any manual code review takes place.

2. The Earlier, the Better

CI is all about finding issues as quickly as possible, so whilst dedicated CI software is great at this, don’t forget that it’s not the only tool in the box. With most revision control systems, it is possible to set up hook scripts that run on changes before they are accepted to the repository. This is no alternative to CI software, as there are many reasons why it would be impractical to run a full suite of tests before accepting any changes; your engineers’ sanity for one! It is, however, great for simple checks like enforcing a no tabs policy, although it’s usually a good idea to provide a means of waiving certain checks for sets of files such as imported code from third parties. Text editors and IDEs with a level of built-in linting can also be very useful for catching issues as early as you possibly can.

3. Hardware-Based Testing

If you’re working on an embedded system, as we often are, then testing on the target hardware provides a further level of integration testing. For this, you will need to be able to transfer your compiled code to your target device or devices and then initiate some kind of test. This is highly dependent upon the nature of your system and may involve automation of surrounding test equipment, special test modes within your software/hardware and/or a test server providing scripted control/status communications with your hardware and streaming and capture of test data.

This approach requires more significant effort to implement, but for larger, longer running projects it can easily pay for itself.

4. Build Statistics Analysis

Automating your builds is step one, but there are opportunities to extend this further and get more value for limited additional effort. The simplest is just monitoring the time it takes to run a build, which many CI tools do automatically, but this information can be very helpful when you discover your project takes much longer than you expected to build, as you can quickly identify when this started to happen and which change it was that caused it.

If code size is important to you, then you can add scripts to monitor this over time and even plot it on a chart. With a feel for how much of your software you have implemented so far and how much you have left to do, you can quickly identify how tight things are going to get as the project progresses. Again, if you see a sudden unexpected step up, you can identify the change and investigate the cause.

Anywhere you have limited resources, CI can often provide a means to monitor your usage of them. For example, some compilers, or extensions thereof, can provide worst case stack size, other tools can be used to estimate memory requirements and so on.

5. Branch Management & Testing

The standard starting point for CI is to have it running on the trunk or master branch of your code repository. This is great for bringing issues to attention, but wouldn’t it be better to prevent them from reaching that shared branch in the first place? It is common practice to ask developers to run unit tests and so forth before merging back to trunk/master, but that’s a manual step and if we’ve invested in a system to automatically run all of those tests in a consistent and controlled manner, then why run them manually?

We mentioned earlier how we use Review Board to trigger CI testing of changes, but we also allow branches to be easily tested directly from the CI environment. This is especially beneficial when we are using CI for performance tests, as it allows us to run a full suite of automated performance tests and then compare the results between branches and different branch revisions. To achieve this, we have created special parameterised jobs that can be launched from our CI server that automatically generate new CI jobs for the specified branch. Another approach we took with one client who had more unusual requirements, was to create a bespoke tool to measure and analyse a wide range of performance metrics across multiple code branches and revisions.

How ITDev Can Help

As a provider of software and electronics design services, we have been using CI tools and Agile processes at ITDev since 2010. We have extensive experience in deploying them on a wide range of projects spanning embedded systems, software development and FPGA design. We have used them to analyse complex performance trade-offs and to ensure high quality and high efficiency in our development.

We offer advice and assistance to companies considering investing in CI and other workflow improvements. We can also use our experience and library of in-house scripts to quickly deliver ready to use CI environments customised to fit your own project requirements. Initial discussions are always free of charge, so if you have any questions, or would like to find out more, we would be delighted to speak to you. If you are interested in attending a workshop on CI, or receiving more information on this topic, please sign up to the ITDev Continuous Integration Interest Group.

Follow us on LinkedinTwitter or Facebook to find out about our future blog posts containing tips for getting even more from your CI environment. Finally, if you’re interested in how CI can be used in FPGA design, why not also take a look at our FPGA Design Flow tour.


Andy

Engineering Director

MEng - Electrical & Electronic Engineering, Imperial College London - CEng FCMI MIET ACGI

I am ITDev’s Engineering Director and have held this role for more than ten years.

I am passionate about maintaining a great company culture at ITDev and ensuring that the engineering team have all of the support they need to deliver outstanding results. I also enjoy building long term customer relationships and helping to deliver ground-breaking products. Having helped ITDev's clients to develop the first commercial implementations of several evolving standards, I have experience of delivering award winning products in the face of changing requirements.

Prior to joining ITDev, I held roles at Philips Semiconductors and Semtech, where I gained experience in digital and mixed signal design, as well as software development and project management.

I grew up in Bournemouth and now live in Winchester with my wife and two children. Much of my time, outside of work, is currently spent managing the design and build of our new house, which we hope to move into next year.

View all posts by Andy    |    View all ITDev team members

Latest Blog Posts

Posted 22nd March 2018, By Andy C
Can Continuous Integration, or CI, really make you and your team look awesome? We had a customer visit us recently for a progress meeting. We logged into our ...more
Posted 22nd February 2018, By Andy C
Within industry, the understanding and use of Continuous Integration, or CI, varies greatly. From no awareness, to “we'd like to be doing that”, to highly ...more
Posted 6th February 2018, By Jon O
Providing undergraduates with careers guidance and tips to secure the perfect first job, based on our experience of mentoring students and recruiting new ...more
Binary Code
Posted 3rd November 2017, By James H
Previous articles have explained one traditional aspect of an RTOS, responsive scheduling. This article details other factors to consider when choosing to use ...more

Latest News

Posted 19th April 2018
We are pleased to announce our upcoming workshop event on the topic of Continuous Integration, or CI, and its use in the development of embedded systems. This ...more
JonO on TechWorks award judges panel
Posted 21st December 2017
We were pleased to be invited to join the judges' panel for this year's TechWorks awards dinner. TechWorks is a leading UK industry association "at the core of ...more
Mentor - the missing piece of the puzzle.
Posted 13th November 2017
As a business, we are committed to the future of engineering in the UK. As part of this commitment, we look to engage with local universities at various levels ...more
Southampton Careers Fair
Posted 26th October 2017
Having reviewed our strategic plans we are pleased to have published our vacancies for graduates and summer internships for 2018. As a company, we invest time ...more