UVM to VUnit (v3.0): Test Benches in VHDL

Call: +44 (0)23 8098 8890
E-mail:

Posted 27th October 2018, By Hugh J

If you’ve worked on projects where a full UVM test bench was used you’ll probably have appreciated the benefits it gives you. Unfortunately, sometimes it is too much overhead or the project requires testing in VHDL. In this case you may end up writing a basic VHDL test bench and wish you could use some of the nice features from UVM. Luckily you are not alone and you are not the first. Some people have even put together some helpful libraries and frameworks to help you achieve a better test bench in pure VHDL.

VUnit, as the name might suggest, is a unit testing framework. It is open source and supports both VHDL and SystemVerilog. VUnit includes lots of useful features (some of which I’ll look at today), if you find this blog interesting it would be worth checking out their site on GitHub (https://vunit.github.io/).

First it is worth considering what features of UVM we will want for our VHDL based verification project. The high-level aim is to develop a comprehensive test bench comprising of generic modular sections that can be reused but minimising any wasteful overhead. With this in mind, I feel most projects will require at least:

  • A method of simply running multiple tests and collating the results.
  • A test bench written using transaction level modelling to ensure robust high-level checking.
  • A well-structured test bench allowing easy reusability of components and configurability of checking.

Test run/report framework

In addition to HDL libraries, VUnit provides a python framework that allows you to quickly set up an environment where it is easy to run all of your tests. VUnit will parse your top-level testbenches and extract a list of tests that can be run. By default all tests will be run but it is also possible to specify a list of in the form <Library>.<Testbench>.<Test Case> where * wildcards can be used to select multiple tests. For example, you could run with:

python run.py lib.*.test_small*

At the end of the run the tool will output a summary of your results similar to:

==== Summary ============================================
pass lib.tb_example.test_small_random    (0.1 seconds)
pass lib.tb_example.test_small_bug234    (0.1 seconds)
pass lib.tb_example.test_small_myfeature (0.1 seconds)
=========================================================
pass 3 of 3
=========================================================
Total time was 0.3 seconds
Elapsed time was 0.3 seconds
=========================================================
All passed!

Whilst this is a pleasant output, if you are using a tool such as Jenkins for Continuous Integration (CI) testing you’ll be pleased to hear that VUnit also has a simple flag (--xunit-xml test_output.xml) allowing you to create a standard xUnit file which can be parsed by a Jenkins xUnit Plugin.  xUnit is a standard format so it should work with many other tools as well.

VUnit additionally gives us the option to add different ‘configs’ in our run.py scriptwhich allows us to run a testbench with different values for generics. Configs offer a great way to verify that a design works in all supported modes, change testbench configurations (such as simulated memory latency) and vary design features such as FIFO depths that can help expose otherwise hard to reach bugs. Each configuration can have custom test names to allow generic values to be easily seen within the reports.

Transaction Level Modelling (TLM)

TLM is a great way to simplify your testbench and reduce the risk of bugs in your test environment. In languages such as SystemVerilog we can use the UVM libraries to have transaction objects that are passed through port objects behind the scenes. My approach to this in pure VHDL is to use the VUnit Communications library’s Message Passing method.

Using the VUnit Com library we can treat the message passing infrastructure like ports in UVM. The mapping between methodologies (although not directly equivalent) is shown below:

 

UVM VUnit Com
Analysis Port Publisher/Subscriber Pattern[1]
Unidirectional Port Basic Message Passing[2]
Bidirectional Port Client/Server Pattern[3]

  1. https://vunit.github.io/com/user_guide.html#publisher-subscriber-pattern
  2. https://vunit.github.io/com/user_guide.html#basic-message-passing
  3. https://vunit.github.io/com/user_guide.html#receiving-a-reply

 

VUnit’s message passing uses string types, which are passed using functions such as ‘send’ and ‘receive’. We want to pass transactions rather than strings, and will use the VHDL record type to contain each transaction, so how do we send our transactions as string? VUnit provides push() and pull() functions to encode and decode standard types into a msg. We can then wrap these in a procedure that would take in a record and call push for each of the elements within the record.

Let’s have a look at how some basic patterns in UVM can be translated to VUnit.

Unidirectional Port vs. Basic Message Passing

Producer Consumer
-- create transaction and send
variable t        : trans_t := new;
variable msg      : msg_t;

msg := new_msg;
push(msg, t); // our custom wrapper
send(net, find(“consumer”), msg);


-- create actor
variable self     : actor_t := create(“consumer”);
variable t        : trans_t := new;
variable msg      : msg_t;

-- receive the next message
receive(net, self, msg)
t := pop(msg);

 

First it is worth noting some terminology: an actor is the producer or consumer of a message; a message is a string representation of the data being sent. Actors are created and referenced by a unique name. In the example above the consumer creates a new actor using “consumer” as its unique name, whilst the producer finds this actor using the same unique string (you can imagine that ‘create’ adds an actor to the address book whilst ‘find’ looks up an actor in the address book). You may be wondering, if you’ve looked at the code, what ‘net’ is. This is the network over which the messages are sent. It is defined in the com package and is used in the send and receive functions. You will always use net here.

This basic example highlights some of the main features of message passing in VUnit and differences to UVM. UVM may use a ‘put port’ (a port that allows you to push data from one component to another) to send data to the consumer. This consumer will implement the put function/task which will be called by the producer. A slightly more complex version of this might see the producer doing a ‘put’ to a uvm_tlm_fifo object (a transactional queue or FIFO), whilst the consumer would do a get or peek operation on this same fifo. The VUnit example above has a lot of similarity with this second UVM case.

We can see the producer sends a transaction to “consumer”. The consumer is sitting in a process where it waits for a new message, and then pops the transaction from the message (receive is a blocking call making it is safe to put in a loop). At this point you would normally do something with the transaction.

Analysis Port vs. Publisher/Subscriber Pattern

Producer Consumer
-- create actor
variable self   : actor_t := create(“publisher”);

-- create transaction and send
variable t      : trans_t := new;
variable msg    : msg_t;

msg := new_msg;
push(msg, t);
publish(net, self, msg);





-- create actor
variable self     : actor_t := create(“subscriber”);
variable status   : status_t;
variable msg      : msg_t;
variable t        : trans_t;

-- subscribe to publisher
subscribe(self, find(“publisher”));

-- receive all the messages
loop
  receive(net, self, msg)
  t := pop(msg);
end loop;

 

Analysis ports are often used in UVM to allow components such as monitors to broadcast information to any units that are interested in it. This will often include the system scoreboard. In VUnit we have the publish and subscribe/unsubscribe procedures to help us with this. The producer will send messages using the publish command using just the producer’s actor and no destination actor (as it is broadcast). The only difference in the consumer to the previous example is that it must subscribe to the stream before it can receive any messages. Essentially the subscribe command tells the publish command to do a send to the subscriber. The publish procedure will send to all the actors that have previously subscribed. The loop receiving the messages is then the same as in the previous example.

Bidirectional Port vs. Client/Server Pattern

Producer Consumer
-- create actor
variable self    : actor_t := create(“client”);

-- create transaction and send
variable t       : trans_t := new;
variable r       : resp_t;
variable msg     : msg_t;
variable reply   : msg_t;

-- send transaction
msg := new_msg;
push(msg, t);
send(net, find(“server”), msg);

-- receive reply
receive_reply(net, self, reply);
r := pop(reply);
delete(reply);
-- create actor
variable self     : actor_t := create(“server”);
variable msg      : msg_t;
variable reply    : msg_t;

variable t        : trans_t; 
variable r        : resp_t := new;

-- receive the next messages
receive(net, self, msg)
t := pop(msg);

// send reply
reply := new_msg;
push(reply(r);
reply(net, self, reply);



This final example is again similar to the first example. The additions here are the reply and receive_reply procedures. The consumer in this example sends a response to the producer using the sender and ID information from the message. The producer then receives the reply (and will be blocked on receive_reply until there is a reply) using the ID information from the receipt of the original send procedure. If you are only using the reply to send a true/false acknowledgment you can use the acknowledge procedure which takes a boolean instead of a string and a matching receive_reply that receives a boolean.

By using the server/client pattern, we can synchronise the different components. As you can see in the adjacent sequence diagram, the driver will only send the reply to the sequence once the transaction has been completely sent on the interface. The sequence in turn will reply to the test only once the entire sequence is complete (in the diagram the sequence only sends a single transaction). For example, if a design has a write interface and a read interface, we may want to ensure the write sequence has completed before starting the read sequence. In this situation we would call receive_reply for the write before sending the read. On the other hand, we can start multiple sequences using send, and then wait for one or more of them to complete using receive_reply.

 

Structured Code / Reusability

Now that we have transactions being passed as messages, it is much easier to divide up our TB. One of the biggest benefits of a well-structured testbench is reusability. In this instance we will keep all of the interface specific code in the agent as agent and interface will always be reused together. For example, an AXI-Streaming agent may be useful for many designs, whereas the test and scoreboard for the design as a whole will be less reusable. The connections to the agent from the rest of the testbench (the dotted purple lines in the image) can be made using Message Passing, whilst the DUT connections (solid green arrows) are ports out of the agent connected to the DUT ports in the TB top level.

If we have multiple instances of the same agent in a testbench, we will need to be able to reference them uniquely (so that we can send messages to each of them individually). The easiest way is to pass an instance name through a generic in the agent. This name can then be used to create the actor in the sequence allowing the tests to communicate with the sequence. This allows us to instance the same agent in a new test bench without being constrained by the naming while ensuring that each agent creates a unique actor.

When writing a new testbench we can now instance the selection of agents we need, giving each a unique instance name. Next, we can write the tests that will use the server/client pattern above to kick of sequences in each of the agents, using their unique instance name to ‘find’ the actors. The scoreboard can also use the unique instance names, this time to subscribe to the messages from the monitors and using the transactions driven by the master monitors to determine the validity of transactions from the slave monitors. This may seem like a small amount of overhead but it will save time and effort overall as well as producing a more effective and robust testbench.

Conclusion

UVM is an important methodology with a very useful set of libraries in SystemVerilog. We have seen that we can take ideas from this methodology and use the VHDL verification libraries available to create well structure and efficient testbenches. Hopefully this blog will help verification engineers familiar with UVM consider using structured VHDL-only TBs using the VUnit library for simple projects. Or perhaps you might find a new interest in both VUnit and UVM. If you have found this post interesting there is lots of other information about the VUnit libraries on their site: https://vunit.github.io/. Also, I’ve gone through how some of the features relate to the UVM equivalent but if you want to read about message passing without the comparisons, there is a great blog on the VUnit site at Improving VHDL Testbench Design with Message Passing, and some great updates from version 3.0 described in VUnit 3.0.

In a future blog we will be detailing how we link our design and verification tooling in a Jenkins based Continuous Integration environment. For a preview of our environment you can sign up for interactive demo at smartfpgadesign.co.uk.

To be notified when the blog is available sign up to our Continuous Integration Interest Group

Follow us on LinkedinTwitter or Facebook to find out about our future blog posts.

Hugh

Senior Engineer

MEng - Electronic Engineering, University of Southampton

I started working at ITDev in April 2017, working in the electronics design department. I work on IP design, FPGAs and embedded software.

I enjoy working with a team of enthusiastic and friendly, like-minded people with the opportunity to work on a range of different customer projects.

Outside of work I am a keen runner and enjoy woodworking.

View all posts by Hugh    |    View all ITDev team members

Latest Blog Posts

Posted 27th November 2018, By James H
Generally distributed devices will need to agree on time in order to effectively coordinate tasks but getting everyone to tell the same time is challenging! ...more
ITDev internship image
Posted 14th November 2018, By Orianne T
As a company, ITDev invests in the future of engineering by offering summer internships to undergraduates. Matthew wanted to share some advice to other ...more
Posted 27th October 2018, By Hugh J
If you’ve worked on projects where a full UVM test bench was used you’ll probably have appreciated the benefits it gives you. Unfortunately, sometimes it is ...more
Partial team photo summer 2018
Posted 7th September 2018, By Jon O
Claudine was one of our interns this summer. Having had an enjoyable experience she captured a summary of her time at ITDev and agreed we could share it. Here' ...more

Latest News

TechWorks Awards Gala and Dinner November 2018
Posted 29th November 2018
As founding members of TechWorks, ITDev was delighted to attend the TechWorks Annual Industry Summit and Awards Dinner in November. The Industry Summit was an ...more
Posted 9th November 2018
On Wednesday 7th November, ITDev invited 15 STEM students from the University of Southampton to take part in a ‘Getting to know you’ event. The company ...more
IBC 2017 Registration queue
Posted 10th September 2018
The Broadcast and Pro-AV industries are key markets for us and some of our clients. In recognition of this we plan attendance at key industry events like ISE ( ...more
Welcome meal photo
Posted 7th August 2018
As mentioned in our last news item, we have 3 summer interns working with us for 10 weeks over the summer. We combined a celebration of their joining the team ...more