UVM to VUnit (v3.0): Test Benches in VHDL

Call: +44 (0)23 8098 8890
E-mail:

Posted 27th October 2018, By Hugh J

If you’ve worked on projects where a full UVM test bench was used you’ll probably have appreciated the benefits it gives you. Unfortunately, sometimes it is too much overhead or the project requires testing in VHDL. In this case you may end up writing a basic VHDL test bench and wish you could use some of the nice features from UVM. Luckily you are not alone and you are not the first. Some people have even put together some helpful libraries and frameworks to help you achieve a better test bench in pure VHDL.

VUnit, as the name might suggest, is a unit testing framework. It is open source and supports both VHDL and SystemVerilog. VUnit includes lots of useful features (some of which I’ll look at today), if you find this blog interesting it would be worth checking out their site on GitHub.

First it is worth considering what features of UVM we will want for our VHDL based verification project. The high-level aim is to develop a comprehensive test bench comprising of generic modular sections that can be reused but minimising any wasteful overhead. With this in mind, I feel most projects will require at least:

  • A method of simply running multiple tests and collating the results.
  • A test bench written using transaction level modelling to ensure robust high-level checking.
  • A well-structured test bench allowing easy reusability of components and configurability of checking.

Test run/report framework

In addition to HDL libraries, VUnit provides a python framework that allows you to quickly set up an environment where it is easy to run all of your tests. VUnit will parse your top-level testbenches and extract a list of tests that can be run. By default all tests will be run but it is also possible to specify a list of in the form <Library>.<Testbench>.<Test Case> where * wildcards can be used to select multiple tests. For example, you could run with:

python run.py lib.*.test_small*

At the end of the run the tool will output a summary of your results similar to:

==== Summary ============================================
pass lib.tb_example.test_small_random    (0.1 seconds)
pass lib.tb_example.test_small_bug234    (0.1 seconds)
pass lib.tb_example.test_small_myfeature (0.1 seconds)
=========================================================
pass 3 of 3
=========================================================
Total time was 0.3 seconds
Elapsed time was 0.3 seconds
=========================================================
All passed!

Whilst this is a pleasant output, if you are using a tool such as Jenkins for Continuous Integration (CI) testing you’ll be pleased to hear that VUnit also has a simple flag (--xunit-xml test_output.xml) allowing you to create a standard xUnit file which can be parsed by a Jenkins xUnit Plugin.  xUnit is a standard format so it should work with many other tools as well.

VUnit additionally gives us the option to add different ‘configs’ in our run.py scriptwhich allows us to run a testbench with different values for generics. Configs offer a great way to verify that a design works in all supported modes, change testbench configurations (such as simulated memory latency) and vary design features such as FIFO depths that can help expose otherwise hard to reach bugs. Each configuration can have custom test names to allow generic values to be easily seen within the reports.

Transaction Level Modelling (TLM)

TLM is a great way to simplify your testbench and reduce the risk of bugs in your test environment. In languages such as SystemVerilog we can use the UVM libraries to have transaction objects that are passed through port objects behind the scenes. My approach to this in pure VHDL is to use the VUnit Communications library’s Message Passing method.

Using the VUnit Com library we can treat the message passing infrastructure like ports in UVM. The mapping between methodologies (although not directly equivalent) is shown below:

 

UVM VUnit Com
Analysis Port Publisher/Subscriber Pattern[1]
Unidirectional Port Basic Message Passing[2]
Bidirectional Port Client/Server Pattern[3]

  1. https://vunit.github.io/com/user_guide.html#publisher-subscriber-pattern
  2. https://vunit.github.io/com/user_guide.html#basic-message-passing
  3. https://vunit.github.io/com/user_guide.html#receiving-a-reply

 

VUnit’s message passing uses string types, which are passed using functions such as ‘send’ and ‘receive’. We want to pass transactions rather than strings, and will use the VHDL record type to contain each transaction, so how do we send our transactions as string? VUnit provides push() and pull() functions to encode and decode standard types into a msg. We can then wrap these in a procedure that would take in a record and call push for each of the elements within the record.

Let’s have a look at how some basic patterns in UVM can be translated to VUnit.

Unidirectional Port vs. Basic Message Passing

Producer Consumer
-- create transaction and send
variable t        : trans_t := new;
variable msg      : msg_t;

msg := new_msg;
push(msg, t); // our custom wrapper
send(net, find(“consumer”), msg);


-- create actor
variable self     : actor_t := create(“consumer”);
variable t        : trans_t := new;
variable msg      : msg_t;

-- receive the next message
receive(net, self, msg)
t := pop(msg);

 

First it is worth noting some terminology: an actor is the producer or consumer of a message; a message is a string representation of the data being sent. Actors are created and referenced by a unique name. In the example above the consumer creates a new actor using “consumer” as its unique name, whilst the producer finds this actor using the same unique string (you can imagine that ‘create’ adds an actor to the address book whilst ‘find’ looks up an actor in the address book). You may be wondering, if you’ve looked at the code, what ‘net’ is. This is the network over which the messages are sent. It is defined in the com package and is used in the send and receive functions. You will always use net here.

This basic example highlights some of the main features of message passing in VUnit and differences to UVM. UVM may use a ‘put port’ (a port that allows you to push data from one component to another) to send data to the consumer. This consumer will implement the put function/task which will be called by the producer. A slightly more complex version of this might see the producer doing a ‘put’ to a uvm_tlm_fifo object (a transactional queue or FIFO), whilst the consumer would do a get or peek operation on this same fifo. The VUnit example above has a lot of similarity with this second UVM case.

We can see the producer sends a transaction to “consumer”. The consumer is sitting in a process where it waits for a new message, and then pops the transaction from the message (receive is a blocking call making it is safe to put in a loop). At this point you would normally do something with the transaction.

Analysis Port vs. Publisher/Subscriber Pattern

Producer Consumer
-- create actor
variable self   : actor_t := create(“publisher”);

-- create transaction and send
variable t      : trans_t := new;
variable msg    : msg_t;

msg := new_msg;
push(msg, t);
publish(net, self, msg);





-- create actor
variable self     : actor_t := create(“subscriber”);
variable status   : status_t;
variable msg      : msg_t;
variable t        : trans_t;

-- subscribe to publisher
subscribe(self, find(“publisher”));

-- receive all the messages
loop
  receive(net, self, msg)
  t := pop(msg);
end loop;

 

Analysis ports are often used in UVM to allow components such as monitors to broadcast information to any units that are interested in it. This will often include the system scoreboard. In VUnit we have the publish and subscribe/unsubscribe procedures to help us with this. The producer will send messages using the publish command using just the producer’s actor and no destination actor (as it is broadcast). The only difference in the consumer to the previous example is that it must subscribe to the stream before it can receive any messages. Essentially the subscribe command tells the publish command to do a send to the subscriber. The publish procedure will send to all the actors that have previously subscribed. The loop receiving the messages is then the same as in the previous example.

Bidirectional Port vs. Client/Server Pattern

Producer Consumer
-- create actor
variable self    : actor_t := create(“client”);

-- create transaction and send
variable t       : trans_t := new;
variable r       : resp_t;
variable msg     : msg_t;
variable reply   : msg_t;

-- send transaction
msg := new_msg;
push(msg, t);
send(net, find(“server”), msg);

-- receive reply
receive_reply(net, self, reply);
r := pop(reply);
delete(reply);
-- create actor
variable self     : actor_t := create(“server”);
variable msg      : msg_t;
variable reply    : msg_t;

variable t        : trans_t; 
variable r        : resp_t := new;

-- receive the next messages
receive(net, self, msg)
t := pop(msg);

// send reply
reply := new_msg;
push(reply(r);
reply(net, self, reply);



This final example is again similar to the first example. The additions here are the reply and receive_reply procedures. The consumer in this example sends a response to the producer using the sender and ID information from the message. The producer then receives the reply (and will be blocked on receive_reply until there is a reply) using the ID information from the receipt of the original send procedure. If you are only using the reply to send a true/false acknowledgment you can use the acknowledge procedure which takes a boolean instead of a string and a matching receive_reply that receives a boolean.

By using the server/client pattern, we can synchronise the different components. As you can see in the adjacent sequence diagram, the driver will only send the reply to the sequence once the transaction has been completely sent on the interface. The sequence in turn will reply to the test only once the entire sequence is complete (in the diagram the sequence only sends a single transaction). For example, if a design has a write interface and a read interface, we may want to ensure the write sequence has completed before starting the read sequence. In this situation we would call receive_reply for the write before sending the read. On the other hand, we can start multiple sequences using send, and then wait for one or more of them to complete using receive_reply.

 

Structured Code / Reusability

Now that we have transactions being passed as messages, it is much easier to divide up our TB. One of the biggest benefits of a well-structured testbench is reusability. In this instance we will keep all of the interface specific code in the agent as agent and interface will always be reused together. For example, an AXI-Streaming agent may be useful for many designs, whereas the test and scoreboard for the design as a whole will be less reusable. The connections to the agent from the rest of the testbench (the dotted purple lines in the image) can be made using Message Passing, whilst the DUT connections (solid green arrows) are ports out of the agent connected to the DUT ports in the TB top level.

If we have multiple instances of the same agent in a testbench, we will need to be able to reference them uniquely (so that we can send messages to each of them individually). The easiest way is to pass an instance name through a generic in the agent. This name can then be used to create the actor in the sequence allowing the tests to communicate with the sequence. This allows us to instance the same agent in a new test bench without being constrained by the naming while ensuring that each agent creates a unique actor.

When writing a new testbench we can now instance the selection of agents we need, giving each a unique instance name. Next, we can write the tests that will use the server/client pattern above to kick of sequences in each of the agents, using their unique instance name to ‘find’ the actors. The scoreboard can also use the unique instance names, this time to subscribe to the messages from the monitors and using the transactions driven by the master monitors to determine the validity of transactions from the slave monitors. This may seem like a small amount of overhead but it will save time and effort overall as well as producing a more effective and robust testbench.

Conclusion

UVM is an important methodology with a very useful set of libraries in SystemVerilog. We have seen that we can take ideas from this methodology and use the VHDL verification libraries available to create well structure and efficient testbenches. Hopefully this blog will help verification engineers familiar with UVM consider using structured VHDL-only TBs using the VUnit library for simple projects. Or perhaps you might find a new interest in both VUnit and UVM. If you have found this post interesting there is lots of other information about the VUnit libraries on their site. Also, I’ve gone through how some of the features relate to the UVM equivalent but if you want to read about message passing without the comparisons, there is a great blog 'Improving VHDL Testbench Design with Message Passing' on the VUnit site as well as some great updates from version 3.0 described in VUnit 3.0.

In a future blog we will be detailing how we link our design and verification tooling in a Jenkins based Continuous Integration environment. For a preview of our environment you can sign up for interactive demo at smartfpgadesign.co.uk.

To be notified when the blog is available sign up to our Continuous Integration Interest Group

Follow us on LinkedinTwitter or Facebook to find out about our future blog posts.

IET Enterprise Partners logo

Latest Blog Posts

Balancing the benefits of contractors vs consultants
Posted 19th November 2024, By Jon O
Recently I was in discussion with a potential client about the challenges he was having with resourcing one of his projects. His automatic assumption was to ...more
Laurence at his desk in the office
Posted 29th August 2024, By Jon O
At ITDev, we are passionate about the work we do and the impact it has. Recently, we had the pleasure of welcoming a bright year 10 pupil, Laurence, for a 7 ...more
Joseph UKESF profile
Posted 5th August 2024, By Joseph S
We sponsored Joseph as a UKESF student, offering him a 1 year work placement. This is his report on the experience and what you can expect on a placement with ...more
VPK120 development board pictured wearing ear defenders with silent smiley on the fan
Posted 13th June 2023, By Aysa D
This blog contains the final steps for adding the minimal IP to send the necessary SYSMON data to the System Controller for controlling the fan on the AMD ...more

Latest News

IABM logo
Posted 30th August 2024
ITDev are pleased to announce becoming a silver member of the IABM. With a long-standing knowledge of creating custom solutions for the ...more
Posted 12th September 2023
ITDev is proud to announce the launch of a new FPGA video IP core. The core allows integrators to quickly and easily add high-quality colour space conversion ...more
Shot of Sydney Harbour Bridge
Posted 3rd March 2023
We welcome David back from Australia and review some of the actvities we've been engaged with in February 2023.
Posted 9th August 2022
Last month we attended the second TechNES FPGA Frontrunners event, here's our write up from the day ...