Split verification components

[eine]

As commented on gitter:

@1138-4eb Mar 30 10:05:

About verification components in general, they've been already a year in Beta. Is there any plan to change it in the near future? It would be handy to have some introduction article talking about the monitor, protocol checkers, etc.

@kraigher Apr 03 07:39:

Maybe the beta label on the verification components can be lifted but I think we need to track their version separate from the rest of VUnit. Maybe the verification components should be considered to be a separate project.

@1138-4eb Apr 05 18:08:

What's required to move VCs to a separate project? Just make vunit/vhdl/verification_components a submodule? As far as I understand it, this would allow to decouple the version of the VCs from the VUnit source, as explained by @joshrsmith. However, it would not be possible to use VCs without VUnit, would it? Is this something you want to support?

@kraigher Apr 05 19:37:

Regading making the verification components a separate project it is just to create a new repo and move the verification_components folder there. The verification_components would obviously depend on VUnit since they use the VUnit data structures and communication library internally. Of course the verification components test benches also use VUnit.

---

Apart from creating a new repo (say VUnit/verification_components or VUnit/vcs), I think that the following issues need to be considered:

• Whether to move the corresponding documentation to the submodule: http://vunit.github.io/verification_components/user_guide.html
• Whether to start the new repo by adding the content to a new clean initial commit, or to use filter-branch to extract the history of the files in verification_components.

[kraigher]

Ideally the documentation would move with the code to make it self contained. In this case there would be no tight coupling between the verification components and VUnit itself other than VUnit being a dependency. Probably this would mean that the Verification Components would need a separate part of the website or their own website for their documentation.

Regarding extracting the history it could be valuable to keep the old commits.

[eine]

Probably this would mean that the Verification Components would need a separate part of the website or their own website for their documentation.

I'll give it a try.

Regarding extracting the history it could be valuable to keep the old commits.

I think that I will be shown as the committer (not author) of all the history if I do it. Would you like to do it instead?

Related to this, it is also possible to remove all traces from the history of VUnit/vunit, but this would require rewriting the history, so I would suggest avoiding it.

[kraigher]

I can try to make a new repository and add the verification components code there as a starting point. We would probably need to stop bundling the verification components with VUnit to break free the life cycle of the verification components. Thus the verification components would become a separate download.

[kraigher]

@1138-4eb If you want to do more of this yourself it would be great. I am just not able to invest so much time in this right now. If you help start a new repository for the verification components I would support it. My hope would be that more people would have permission to merge pull requests in this repo to share the burden to review.

[eine]

Hi @kraigher, thanks for coming back. It is almost done already:

• Main repo after vcomponents are (re)moved: https://travis-ci.org/1138-4EB/vunit/builds/540795441
• New verification components repository: https://travis-ci.org/1138-4EB/vunit/builds/540874222

If you help start a new repository for the verification components I would support it.

For now, I am not using separate repositories, but just using separate branches on the same repo. Nonetheless, I would appreciate if you could tell me which name you expect to use for the repository: VUnit/verification_components, VUnit/vcomponents, VUnit/vc, VUnit/vcs, VUnit/vc_lib...

My hope would be that more people would have permission to merge pull requests in this repo to share the burden to review.

IMHO, even if I had permission to merge PRs, I would not do it without the review of another maintainer. Hence, I think it is not important for now.

[eine]

After the discussion in Gitter, this is the new proposal:

NOTE: These changes depend on #508 and #509.

• Main repo without AXI, UART, Avalon or Wishbone:
  • https://github.com/1138-4EB/vunit/tree/distributed-vcs
  • https://travis-ci.org/1138-4EB/vunit/builds/541391952
• Repo for AXI vcs:
  • https://github.com/1138-4EB/vunit/tree/vc_axi https://github.com/VUnit/vc_axi
  • https://travis-ci.org/1138-4EB/vunit/builds/541252008 https://travis-ci.org/VUnit/vc_axi/builds
  • Contains sources, acceptance tests and two examples. Two travis jobs.
• Repo for UART vcs:
  • https://github.com/1138-4EB/vunit/tree/vc_uart https://github.com/VUnit/vc_uart
  • https://travis-ci.org/1138-4EB/vunit/builds/541311819 https://travis-ci.org/VUnit/vc_uart/builds
  • Contains sources and one example, but no acceptance tests. Single travis job.
• Repo for Avalon vcs:
  • https://github.com/1138-4EB/vunit/tree/vc_avalon
  • https://travis-ci.org/1138-4EB/vunit/builds/541331757
  • Contains sources and acceptance tests, but no examples. Single travis job.
• Repo for Wishbone vcs:
  • https://github.com/1138-4EB/vunit/tree/vc_wishbone
  • https://travis-ci.org/1138-4EB/vunit/builds/541374163
  • Contains sources and acceptance tests, but no examples. Single travis job.

Further notes:

• As a result of removing AXI, UART, Avalon and Wishbone from the main repo, vcomponents tests no longer exists, because vunit/vhdl/verification_components/run.py contains tests which are only related to those.
• AXI, Avalon and Wishbone depend on VUnit only, but UART depends on both VUnit and AXI.
• As a result of using docker images, any user willing to add a new vc repo only needs to copy tox.ini (20 lines), .travis.yml (10-15 lines) and (optionally) run.py (40-70 lines) and/or examples/test_examples.py (60 lines).
  • It is possible to create additional docker images to be used when testing the 'third-party' verification components, which would already include VUnit. This can be achieved by either adding tox to ghdl/ext:vunit (or ghdl/ext:vunit-master), or by creating a vunit/dev:mcode-master image based on vunit/dev:mcode as a deployment job in the main VUnit repo. This can be discussed in vunit/docker.
• Each verification component repo has it's own vc_*_context.vhd which includes the core vc_context.vhd and extends it with the corresponding additional packages.
• As far as developers of multiple verification component libraries are aware of not creating conflicting names/symbols, there is no need to use separate library names for each repo. For example, in the UART tests and example, both AXI vcs and UART sources are compiled into vunit_lib. This allows to preserve most of the existing sources untouched, even though they are moved to separate repos. For example, this is the single commit that was added to the Avalon repo after splitting it: eine@dd085c1.
• I did neither add a README nor modify the copyright notices in the sources. @kraigher, @LarsAsplund I don't know how do you want to handle this.

Regarding the docs, none of the components which where split has any specific documentation. The single modification is that the references to three of the examples are removed from the user guide (precisely, those corresponding to UART or AXI). I think we can leave it like this for now. As we commented, the next step would be to make the documentation dynamically fetch content from a list of sibling repos. However, it is hard to implement until we know the names of the repos.

@kraigher, @LarsAsplund, should I create vunit/vc_axi, vunit/vc_uart, vunit/vc_avalon and vunit/vc_wishbone? Or do you want to discuss the location with other contributors?

[eine]

FTR, this is the script I used to split each of the new vc repos:

#!/bin/sh

split_move="$(dirname $0)/split_move.sh"

cat > "$split_move" <<-EOF
#!/bin/sh

dir_move() {
  if [ -d \$1 ]; then mkdir -p \$2 && mv \$1/* \$2; fi
}

file_move() {
  if [ -f \$1 ]; then mkdir -p \$(dirname \$2) && mv \$1 \$2; fi
}

<REPO SPECIFIC FILES>
EOF

git clone -b use-docker https://github.com/1138-4EB/vunit
cd vunit

git checkout -b vc_branch

git filter-branch -f --tree-filter "$split_move" HEAD

cp "tox.ini" "vcs/tox.ini"
cp ".gitattributes" "vcs/.gitattributes"
cp ".gitignore" "vcs/.gitignore"
cp ".travis.yml" "vcs/.travis.yml"
cp "vunit/test/acceptance/test_external_run_scripts.py" "vcs/examples/test_examples.py"

git add vcs
git commit -m "split verification components from main repository"

git filter-branch -f --prune-empty --subdirectory-filter vcs vc_branch

where <REPO SPECIFIC FILES> is:

dir_move "examples/vhdl/array_axis_vcs" "vcs/examples/array_axis_vcs"
dir_move "examples/vhdl/axi_dma" "vcs/examples/axi_dma"

d=vunit/vhdl/verification_components/src/
if [ -d \$d ]; then
  for f in \$(find \$d -name "axi_*.vhd"); do
    file_move "\$f" "vcs/src/\$(basename -- \$f)"
  done
fi

d=vunit/vhdl/verification_components/test/
if [ -d \$d ]; then
  for f in \$(find \$d -name "tb_axi_*.vhd"); do
    file_move "\$f" "vcs/test/\$(basename -- \$f)"
  done
fi

file_move "vunit/vhdl/verification_components/src/vc_context.vhd" "vcs/src/vc_context.vhd"
file_move "vunit/vhdl/verification_components/run.py" "vcs/run.py"

or

dir_move "examples/vhdl/uart" "vcs/examples/uart"

d=vunit/vhdl/verification_components/src/
if [ -d \$d ]; then
  for f in \$(find \$d -name "uart_*.vhd"); do
    file_move "\$f" "vcs/src/\$(basename -- \$f)"
  done
fi

file_move "vunit/vhdl/verification_components/test/tb_uart.vhd" "vcs/test/tb_uart.vhd"
file_move "vunit/vhdl/verification_components/src/vc_context.vhd" "vcs/src/vc_context.vhd"

or

d=vunit/vhdl/verification_components/src/
if [ -d \$d ]; then
  for f in \$(find \$d -name "avalon_*.vhd"); do
    file_move "\$f" "vcs/src/\$(basename -- \$f)"
  done
fi

d=vunit/vhdl/verification_components/test/
if [ -d \$d ]; then
  for f in \$(find \$d -name "tb_avalon_*.*"); do
    file_move "\$f" "vcs/test/\$(basename -- \$f)"
  done
fi

file_move "vunit/vhdl/verification_components/src/vc_context.vhd" "vcs/src/vc_context.vhd"
file_move "vunit/vhdl/verification_components/run.py" "vcs/run.py"

or

d=vunit/vhdl/verification_components/src/
if [ -d \$d ]; then
  for f in \$(find \$d -name "wishbone_*.vhd"); do
    file_move "\$f" "vcs/src/\$(basename -- \$f)"
  done
fi

d=vunit/vhdl/verification_components/test/
if [ -d \$d ]; then
  for f in \$(find \$d -name "tb_wishbone_*.*"); do
    file_move "\$f" "vcs/test/\$(basename -- \$f)"
  done
fi

file_move "vunit/vhdl/verification_components/src/vc_context.vhd" "vcs/src/vc_context.vhd"
file_move "vunit/vhdl/verification_components/run.py" "vcs/run.py"

[kraigher]

As a result of removing AXI, UART, Avalon and Wishbone from the main repo, vcomponents tests no longer exists, because vunit/vhdl/verification_components/run.py contains tests which are only related to those.

There are test still tests of the memory model itself which should still be part of the core in the VUnit repository.

As far as developers of multiple verification component libraries are aware of not creating conflicting names/symbols, there is no need to use separate library names for each repo. For example, in the UART tests and example, both AXI vcs and UART sources are compiled into vunit_lib. This allows to preserve most of the existing sources untouched, even though they are moved to separate repos. For example, this is the single commit that was added to the Avalon repo after splitting it: 1138-4EB@dd085c1.

I think the different vc should get new library names to prevent unnecessary namespace conflicts. Doing this only the library names can of VC can conflict which each other but not the files within.

should I create vunit/vc_axi, vunit/vc_uart, vunit/vc_avalon and vunit/vc_wishbone? Or do you want to discuss the location with other contributors?

I think lets start with vunit/vc_axi first and let that be the prototype.

[eine]

There are test still tests of the memory model itself which should still be part of the core in the VUnit repository.

Indeed. This was a misunderstanding on my side. It is fixed now.

I think the different vc should get new library names to prevent unnecessary namespace conflicts. Doing this only the library names can of VC can conflict which each other but not the files within.

I think this should be up to the maintainer/developer. For now, it is not required, and I think it will be easier to follow the refactoring if we avoid adding this constraint. Of course, as a maintainer of vc_axi, the first thing you can do is to add a README specifying the expected library name and adapt the two examples accordingly. Furthermore, I am open to submitting a PR to do it. But I'd prefer to do it after we have effectively split it. I.e., after the corresponding PR is merged to master and the new repo is created.

I think lets start with vunit/vc_axi first and let that be the prototype.

Since UART depends on AXI, it might be not straightforward to split AXI only. Do you mind if I create both vunit/vc_axi and vunit/vc_uart at the same time?

[kraigher]

I think this should be up to the maintainer/developer. For now, it is not required, and I think it will be easier to follow the refactoring if we avoid adding this constraint. Of course, as a maintainer of vc_axi, the first thing you can do is to add a README specifying the expected library name and adapt the two examples accordingly. Furthermore, I am open to submitting a PR to do it. But I'd prefer to do it after we have effectively split it. I.e., after the corresponding PR is merged to master and the new repo is created.

I am okay with doing it in a different order where the split is done before the new library name is introduced. But I think we need to introduce library names for each vc-repo to provide namespace protection and not encourage people to use vunit_lib.

Since UART depends on AXI, it might be not straightforward to split AXI only. Do you mind if I create both vunit/vc_axi and vunit/vc_uart at the same time?

It is ok as it is not avoidable

[eine]

Please, see #511 and the two new repos.

Travis does seem to need some time to pick the changes. But vc_uart is already being built: https://travis-ci.org/VUnit/vc_uart/builds/541420627

EDIT

PRs #495 and #425 are affected by these splits. The former would appy to vc_uart and the later should contributed separately to vc_avalon (planned but not created yet) and vc_spi (neither planned nor created yet).

[eine]

As commented in Gitter:

After having a look at #513, if @LarsAsplund and @kraigher agree, I think it is relevant enough to schedule it before #511. This will provide some time frame for @Aaron-Hartwig to update #495 and have it merged before #511 too.

Meanwhile, I added a mechanism to:

• Declare which third-party VCs are to be used in a design through ui.use_verification_components, whithout having to manage the location of each third-party repo separately. Although, optionally, it is possible to do so.
• Automatically include in the docs the list of VCs and examples provided by each third-party VC.

To do so:

• The list and locations of third-party VCs are declared in a vc_list.json file located at vunit/vhdl/verification_components/.
• Each third-party repository includes a vunit_cfg.json file.

Example of vc_list.json (see #511):

{
    "AXI": {
        "path": "vc_axi",
        "remote": "https://github.com/VUnit/vc_axi"
    },
    "UART": {
        "path": "vc_uart",
        "remote": "https://github.com/VUnit/vc_uart"
    }
} 

Examples of vunit_cfg.json (see VUnit/vc_axi and VUnit/vc_uart, respectively):

{
  "name": "AXI",
  "lib": "vunit_lib",
  "provides": [
    "Lite Master",
    "Read Slave",
    "Write Slave",
    "Stream Master",
    "Stream Monitor",
    "Stream Protocol Checker",
    "Stream Slave"
  ],
  "depends": {
    "VUnit": " distribute-axi-uart"
  }
}

{
  "name": "UART",
  "lib": "vunit_lib",
  "provides": [
    "Master",
    "Slave"
  ],
  "depends": {
    "VUnit": " distribute-axi-uart",
    "AXI": ""
  }
}

The examples in VUnit/vc_axi and VUnit/vc_uart are already adapted to use this mechanism. For example, array_axis_vcs: https://github.com/VUnit/vc_axi/blob/master/examples/array_axis_vcs/run.py

wh = vu.add_verification_components()
wh['AXI']['path'] = abspath(join(root, '..', '..'))
vu.use_verification_components(wh, ['AXI'])

In this case, we are using a custom setup for the AXI VCs, instead of letting VUnit automatically retrieve it. A regular user would do:

vu.use_verification_components(vu.add_verification_components(), ['AXI'])

which would automatically download VUnit/vc_axi to vunit/vhdl/verification_components/third_party, which is the default intall location for external VCs.

Nonetheless, it is straighforward to provide an alternative location. For example:

vu.use_verification_components(vu.add_verification_components(join('/tmp', 'vunit_vcs')), ['AXI'])

or

wh = vu.add_verification_components()
wh['third_party'] = join('/tmp', 'vunit_vcs')
vu.use_verification_components(wh, ['AXI'])

---

Note that, as commented previously in this thread, the lib field of vunit_cfg.json files should/will be changed to a custom name for each VC repo.

At the same time, the value of each field in depends indicates the minimum and/or maximum version that is required. But this is not implemented yet. ATM, master is cloned.