VHDL Testbench Generator - Example

Introduction

This example will generate a testbench for a simple AXI stream pipeline stage. The implementation itself is unimportant, as the only information required is the entity declaration. Our test block has a clock, 3 inputs, and 3 outputs. axis_in_tdata and axis_out_tdata are the only 8-bit signals, all the others are single-bit:

entity axis_pipe_stage is
    port (
        clk             : in std_logic;
        axis_in_tready  : out std_logic;
        axis_in_tvalid  : in std_logic;
        axis_in_tdata   : in std_logic_vector (7 downto 0);
        axis_out_tready : in std_logic;
        axis_out_tvalid : out std_logic;
        axis_out_tdata  : out std_logic_vector (7 downto 0)
    );
end entity; -- axis_pipe_stage

Define Tests

WaveJSON

The waveforms are written in WaveJSON (tutorial here), although additional requirements need to be met for the testbench generator to function:

• All signals must have a name and wave attribute.
• All signals must have a dir attribute, which can be in or out depending on whether the signal is an input to or an output from the block being tested.
• All signals must have the same length in cycles (but different tests can have different lengths).
• All signals belonging to the block being tested must be specified in every test case.
• All signals must have a consistent same name, dir and width in every test case.
• Only the clock signal can use edges.
• The clock signal must be a series of edges of only one type, for example p......., p...PPpP and n....N.. are all valid but p...n... and 1p...... are not. This type needs to be consistent accross all tests.
• Multi-bit signals must have a width and array attributes.
• Multi-bit signals can have a radix attribute, which can be 1 or 2 characters. By default the data is assumed to be unsigned decimal, but the following options are available:
  • signed: s
  • binary: b
  • decimal: d
  • hexadecimal: h
• The period, phase and code features are not currently supported.

The Editor

Figure 1: screenshot of tool

The editor is simple and straightforward to use. The main panel shows the WaveJSON definition of the waveform, which can be edited at any time. At the bottom is a graphical representation of the WaveJSON, which is automatically generated on each edit and can be downloaded as an image by right-clicking. The right shows a list of open files. Each test needs to be defined in a separate file, following the requirements above.

There are four buttons above the main window: - The first creates a new file for a new test. - The second lets a user upload a .yaml file as a new test. - The third lets the user download the currently selected test files (all test files are included in the results email with the testbench). - The fourth lets the user rename the currently selected test file.

Example

For this example, we are using the 2 test cases below. Note that all the signals named in the entity declaration have been included, and they have matching directions and widths. The signals also have consistent attributes across the tests, although their values differ. Here, the first test lasts 10 cycles and the multi-bit signals are using signed decimal, but the second test lasts 7 cycles and uses hexadecimal.

axis_pipe_stage_timing.yaml:

{
  signal: [
    {dir: 'in',  name: 'clk',              wave: 'p.....PPp..'},
    {dir: 'out', name: 'axis_in_tready',   wave: '1.01..0.1..'},
    {dir: 'in',  name: 'axis_in_tvalid',   wave: '01..01.010.'},
    {dir: 'in',  name: 'axis_in_tdata',    wave: 'x==.x==..x.', width: 8, data: "99 -23 4 107", radix: 'sd'},
    {dir: 'in',  name: 'axis_out_tready',  wave: '0..1..0.1..'},
    {dir: 'out', name: 'axis_out_tvalid',  wave: '0.1..01...0'},
    {dir: 'out', name: 'axis_out_tdata',   wave: 'xx=.=x=..=x', width: 8, data: "99 -23 4 107", radix: 'sd'},
  ],
} 

axis_pipe_stage_timing_2.yaml:

{
  signal: [
    {dir: 'in',    name: 'clk',             wave: 'p.....P'},
    ['In Group',
      {dir: 'out', name: 'axis_in_tready',  wave: '1.01..0'},
      {dir: 'in',  name: 'axis_in_tvalid',  wave: '01..01.'},
      {dir: 'in',  name: 'axis_in_tdata',   wave: 'x==.x=.', width: 8, data: ["AA","5F","6B"], radix: 'h'},
    ],
    {dir: 'in',    name: 'axis_out_tready', wave: '0..1..0'},
    {dir: 'out',   name: 'axis_out_tvalid', wave: '0.1..01'},
    {dir: 'out',   name: 'axis_out_tdata',  wave: 'xx=.=x=', width: 8, data: ["AA","5F","6B"], radix: 'h'},
  ],
}

Generate Testbench

Once the tests are described in the tool, there are some additional fields to complete in the form. This lets the user define their module, such as the name to use, which signal is the clock, and what period the clock uses. The user can also choose whether or not to enable VUnit support.

VUnit (https://vunit.github.io/) is a unit testing framework for VHDL. When the testbench has VUnit enabled, the files sent to the user will include a run_vunit.py script that will run the entire testbench from the command line. If VUnit support is not enabled, the testbench can be used by Modelsim or similar software as normal.

The testbench files are sent by email. The attached files include: all of the waveform definitions, the generated testbench file ([name]_tb.vhd), and a run_vunit.py script (if VUnit support was selected).

For this example, we called the module axis_pipe_stage, with a clock named clk with a period of 10 ns, and VUnit enabled. The generated testbench can be found at the bottom of this page.

Run Testbench

If 'Enable VUnit' was selected during generation, the simplest way to run the testbench is to run python run_vunit.py. This script has many options, which can be seen by running python run_vunit.py --help. In our case, we will run with multiple threads, and use the GUI option to open it in ModelSim, so we run python run_vunit.py -p 4 -g.

If you have selected a testbench without Vunit support, then just compile and simulate your testbench using your preferred flow (e.g. compile scripts or ModelSim GUI). Only the ieee.numeric_std and ieee.std_logic_1164 libraries are required to compile the testbench.

Example Testbench

The tests defined for this example resulted in this testbench:

--------------------------------------------------------------------------------
--
--! @file       axis_pipe_stage_tb.vhd
--! @brief      Generated testbench for the axis_pipe_stage module.
--! @author     ITDev Ltd. (http://www.itdev.co.uk)
--! @date       29/08/2018
--! @details    This is a self checking testbench produced automatically. The
--!             testbench is generated based on the source WaveJSON/YAML and
--!             includes VUnit support.
--
--------------------------------------------------------------------------------
--
--! @copyright  This work is licensed under the Creative Commons Attribution 4.0
--!             International License. To view a copy of this license, visit
--!             http://creativecommons.org/licenses/by/4.0/.
--
--------------------------------------------------------------------------------
--
-- ITDev are experts in the fields of software development, electronic
-- engineering and systems engineering.
--
-- Our electronics team has experience in developing digital IP for FPGA and
-- ASIC technologies, as well as systems and PCB design.
--
-- We have comprehensive experience of FPGA and digital ASIC design flows from
-- planning and HDL coding through to simulation, synthesis, post-layout
-- verification and evaluation.
--
-- Our design methodologies focus on quality at every stage and include detailed
-- peer reviews and high-coverage automated testing in both simulation and
-- evaluation.
--
-- Whatever your requirements, we are committed to providing a first class
-- service, so why not contact us for a free, no obligation, discussion of your
-- requirements. Call us at +44 (0)23 8098 8890 or send an email to
-- sales@itdev.co.uk.
--
--------------------------------------------------------------------------------

library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;

library vunit_lib;
context vunit_lib.vunit_context;

--------------------------------------------------------------------------------
--! axis_pipe_stage Testbench Top Level
--------------------------------------------------------------------------------

entity axis_pipe_stage_tb is
    generic (runner_cfg : string);
end entity; -- axis_pipe_stage_tb

architecture test of axis_pipe_stage_tb is
    constant CLK_PERIOD : time := 10 ns;

    signal clk : std_logic := '1';
    signal axis_in_tready : std_logic;
    signal axis_in_tvalid : std_logic := '0';
    signal axis_in_tdata : std_logic_vector (7 downto 0) := "XXXXXXXX";
    signal axis_out_tready : std_logic := '0';
    signal axis_out_tvalid : std_logic;
    signal axis_out_tdata : std_logic_vector (7 downto 0);

    component axis_pipe_stage is
        port (
            clk : in std_logic;
            axis_in_tready : out std_logic;
            axis_in_tvalid : in std_logic;
            axis_in_tdata : in std_logic_vector (7 downto 0);
            axis_out_tready : in std_logic;
            axis_out_tvalid : out std_logic;
            axis_out_tdata : out std_logic_vector (7 downto 0)
        );
    end component;

begin

    UUT : axis_pipe_stage
        port map (
            clk => clk,
            axis_in_tready => axis_in_tready,
            axis_in_tvalid => axis_in_tvalid,
            axis_in_tdata => axis_in_tdata,
            axis_out_tready => axis_out_tready,
            axis_out_tvalid => axis_out_tvalid,
            axis_out_tdata => axis_out_tdata
        );

    clk <= not clk after CLK_PERIOD / 2;

    ----------------------------------------------------------------------
    --! @brief Test Library
    --!
    --! This is where test inputs are applied. Outputs are then compared
    --! to expected values and an error is reported if they do not match.
    ----------------------------------------------------------------------
    process

        procedure check_cycle (
            constant axis_in_tready_value: in std_logic;
            constant axis_in_tvalid_value: in std_logic;
            constant axis_in_tdata_value: in std_logic_vector (7 downto 0);
            constant axis_out_tready_value: in std_logic;
            constant axis_out_tvalid_value: in std_logic;
            constant axis_out_tdata_value: in std_logic_vector (7 downto 0)
        ) is
        begin
            axis_in_tvalid <= axis_in_tvalid_value;
            axis_in_tdata <= axis_in_tdata_value;
            axis_out_tready <= axis_out_tready_value;
            wait until rising_edge(clk);
            if axis_in_tready_value /= 'X' then
                assert axis_in_tready = axis_in_tready_value
                report "axis_in_tready - expected '" & to_string(axis_in_tready_value) &
                    "' got '" & to_string(axis_in_tready) & "'";
            end if;
            if axis_out_tvalid_value /= 'X' then
                assert axis_out_tvalid = axis_out_tvalid_value
                report "axis_out_tvalid - expected '" & to_string(axis_out_tvalid_value) &
                    "' got '" & to_string(axis_out_tvalid) & "'";
            end if;
            if axis_out_tdata_value /= "XXXXXXXX" then
                assert axis_out_tdata = axis_out_tdata_value
                report "axis_out_tdata - expected " & to_string(to_integer(signed(axis_out_tdata_value))) &
                    " got " & to_string(to_integer(signed(axis_out_tdata)));
            end if;
        end procedure check_cycle;

    begin
        test_runner_setup(runner, runner_cfg);

        while test_suite loop

            if run("test_axis_pipe_stage_timing") then

                -- Cycle 0.
                check_cycle (
                    axis_in_tready_value => '1',
                    axis_in_tvalid_value => '0',
                    axis_in_tdata_value => "XXXXXXXX",
                    axis_out_tready_value => '0',
                    axis_out_tvalid_value => '0',
                    axis_out_tdata_value => "XXXXXXXX"
                );

                -- Cycle 1.
                check_cycle (
                    axis_in_tready_value => '1',
                    axis_in_tvalid_value => '1',
                    axis_in_tdata_value => std_logic_vector(to_signed(99, axis_in_tdata'length)),
                    axis_out_tready_value => '0',
                    axis_out_tvalid_value => '0',
                    axis_out_tdata_value => "XXXXXXXX"
                );

                -- Cycle 2.
                check_cycle (
                    axis_in_tready_value => '0',
                    axis_in_tvalid_value => '1',
                    axis_in_tdata_value => std_logic_vector(to_signed(-23, axis_in_tdata'length)),
                    axis_out_tready_value => '0',
                    axis_out_tvalid_value => '1',
                    axis_out_tdata_value => std_logic_vector(to_signed(99, axis_out_tdata'length))
                );

                -- Cycle 3.
                check_cycle (
                    axis_in_tready_value => '1',
                    axis_in_tvalid_value => '1',
                    axis_in_tdata_value => std_logic_vector(to_signed(-23, axis_in_tdata'length)),
                    axis_out_tready_value => '1',
                    axis_out_tvalid_value => '1',
                    axis_out_tdata_value => std_logic_vector(to_signed(99, axis_out_tdata'length))
                );

                -- Cycle 4.
                check_cycle (
                    axis_in_tready_value => '1',
                    axis_in_tvalid_value => '0',
                    axis_in_tdata_value => "XXXXXXXX",
                    axis_out_tready_value => '1',
                    axis_out_tvalid_value => '1',
                    axis_out_tdata_value => std_logic_vector(to_signed(-23, axis_out_tdata'length))
                );

                -- Cycle 5.
                check_cycle (
                    axis_in_tready_value => '1',
                    axis_in_tvalid_value => '1',
                    axis_in_tdata_value => std_logic_vector(to_signed(4, axis_in_tdata'length)),
                    axis_out_tready_value => '1',
                    axis_out_tvalid_value => '0',
                    axis_out_tdata_value => "XXXXXXXX"
                );

                -- Cycle 6.
                check_cycle (
                    axis_in_tready_value => '0',
                    axis_in_tvalid_value => '1',
                    axis_in_tdata_value => std_logic_vector(to_signed(107, axis_in_tdata'length)),
                    axis_out_tready_value => '0',
                    axis_out_tvalid_value => '1',
                    axis_out_tdata_value => std_logic_vector(to_signed(4, axis_out_tdata'length))
                );

                -- Cycle 7.
                check_cycle (
                    axis_in_tready_value => '0',
                    axis_in_tvalid_value => '0',
                    axis_in_tdata_value => std_logic_vector(to_signed(107, axis_in_tdata'length)),
                    axis_out_tready_value => '0',
                    axis_out_tvalid_value => '1',
                    axis_out_tdata_value => std_logic_vector(to_signed(4, axis_out_tdata'length))
                );

                -- Cycle 8.
                check_cycle (
                    axis_in_tready_value => '1',
                    axis_in_tvalid_value => '1',
                    axis_in_tdata_value => std_logic_vector(to_signed(107, axis_in_tdata'length)),
                    axis_out_tready_value => '1',
                    axis_out_tvalid_value => '1',
                    axis_out_tdata_value => std_logic_vector(to_signed(4, axis_out_tdata'length))
                );

                -- Cycle 9.
                check_cycle (
                    axis_in_tready_value => '1',
                    axis_in_tvalid_value => '0',
                    axis_in_tdata_value => "XXXXXXXX",
                    axis_out_tready_value => '1',
                    axis_out_tvalid_value => '1',
                    axis_out_tdata_value => std_logic_vector(to_signed(107, axis_out_tdata'length))
                );

                -- Cycle 10.
                check_cycle (
                    axis_in_tready_value => '1',
                    axis_in_tvalid_value => '0',
                    axis_in_tdata_value => "XXXXXXXX",
                    axis_out_tready_value => '1',
                    axis_out_tvalid_value => '0',
                    axis_out_tdata_value => "XXXXXXXX"
                );

            elsif run("test_axis_pipe_stage_timing_2") then

                -- Cycle 0.
                check_cycle (
                    axis_in_tready_value => '1',
                    axis_in_tvalid_value => '0',
                    axis_in_tdata_value => "XXXXXXXX",
                    axis_out_tready_value => '0',
                    axis_out_tvalid_value => '0',
                    axis_out_tdata_value => "XXXXXXXX"
                );

                -- Cycle 1.
                check_cycle (
                    axis_in_tready_value => '1',
                    axis_in_tvalid_value => '1',
                    axis_in_tdata_value => std_logic_vector(to_unsigned(16#AA#, axis_in_tdata'length)),
                    axis_out_tready_value => '0',
                    axis_out_tvalid_value => '0',
                    axis_out_tdata_value => "XXXXXXXX"
                );

                -- Cycle 2.
                check_cycle (
                    axis_in_tready_value => '0',
                    axis_in_tvalid_value => '1',
                    axis_in_tdata_value => std_logic_vector(to_unsigned(16#5F#, axis_in_tdata'length)),
                    axis_out_tready_value => '0',
                    axis_out_tvalid_value => '1',
                    axis_out_tdata_value => std_logic_vector(to_unsigned(16#AA#, axis_out_tdata'length))
                );

                -- Cycle 3.
                check_cycle (
                    axis_in_tready_value => '1',
                    axis_in_tvalid_value => '1',
                    axis_in_tdata_value => std_logic_vector(to_unsigned(16#5F#, axis_in_tdata'length)),
                    axis_out_tready_value => '1',
                    axis_out_tvalid_value => '1',
                    axis_out_tdata_value => std_logic_vector(to_unsigned(16#AA#, axis_out_tdata'length))
                );

                -- Cycle 4.
                check_cycle (
                    axis_in_tready_value => '1',
                    axis_in_tvalid_value => '0',
                    axis_in_tdata_value => "XXXXXXXX",
                    axis_out_tready_value => '1',
                    axis_out_tvalid_value => '1',
                    axis_out_tdata_value => std_logic_vector(to_unsigned(16#5F#, axis_out_tdata'length))
                );

                -- Cycle 5.
                check_cycle (
                    axis_in_tready_value => '1',
                    axis_in_tvalid_value => '1',
                    axis_in_tdata_value => std_logic_vector(to_unsigned(16#6B#, axis_in_tdata'length)),
                    axis_out_tready_value => '1',
                    axis_out_tvalid_value => '0',
                    axis_out_tdata_value => "XXXXXXXX"
                );

                -- Cycle 6.
                check_cycle (
                    axis_in_tready_value => '0',
                    axis_in_tvalid_value => '1',
                    axis_in_tdata_value => std_logic_vector(to_unsigned(16#6B#, axis_in_tdata'length)),
                    axis_out_tready_value => '0',
                    axis_out_tvalid_value => '1',
                    axis_out_tdata_value => std_logic_vector(to_unsigned(16#6B#, axis_out_tdata'length))
                );

            end if;
        end loop;

        test_runner_cleanup(runner);
    end process;

end architecture; -- test

Summary

In order to create your own VHDL testbench, visit the Testbench Generator Tool.