[bazel,fusesoc] Add a way to create a hash of all files used by fusesoc

[pamaury]

See commits. Currently, we provide filegroups as input to fusesoc which contain many more files than necessary which causes some frequent bazel cache misses even though the bitstream has not really changed. This PR creates a new target which always one to run the fusesoc --setup stage only and hash the content of the build directory. This way, only files which are really necessary appear in the hash and it is possible to know if the bitstream should be rebuilt or not.

Note that this hash does nothing by ideal, another PR integrating this hash into the CI flow must be created.
To build the hash:

./bazelisk.sh build //hw/bitstream/vivado:fpga_cw340_hash
# Display:
cat $(./bazelisk.sh cquery //hw/bitstream/vivado:fpga_cw340_hash)

[pamaury]

I was running out of time and create a bash command line which at least works but it would be better to use a zero separator I think, I couldn't figure out how to do that nicely.

[jwnrt]

Nit:

Suggested change

	If the src is a directory, its content will be hashes recursively.""",
	If the src is a directory, its content will be hashed recursively.""",

[jwnrt]

I've had problems using sort for determinism because of the locale being different on my laptop versus CI.

Would it be sensible to set LC_ALL=en_US.UTF-8 in the env?

[engdoreis]

Thanks for pulling this together so quickly.

When building the target //hw/bitstream/vivado:fpga_cw340_hash builds more dependencies than it should in my view like test_rom, is there a way to drop dependencies and make it faster?

The final hash contains a trailing -, can this be removed?

[pamaury]

Thanks for pulling this together so quickly.

When building the target //hw/bitstream/vivado:fpga_cw340_hash builds more dependencies than it should in my view like test_rom, is there a way to drop dependencies and make it faster?

The final hash contains a trailing -, can this be removed?

We currently include the test ROM in the OT build, I would be happy to remove it from the hash but I think for the build we cannot remove it otherwise Vivado optimizes the memory away? @gautschimi @andreaskurth
This would lead to a potentially confusing situation where the hash is the same but actualy final bitstream is different due to the ROM (even though it's usually reloaded via the backdoor loader), but we can always add a comment about it?

[andreaskurth]

We currently include the test ROM in the OT build, I would be happy to remove it from the hash but I think for the build we cannot remove it otherwise Vivado optimizes the memory away? @gautschimi @andreaskurth This would lead to a potentially confusing situation where the hash is the same but actualy final bitstream is different due to the ROM (even though it's usually reloaded via the backdoor loader), but we can always add a comment about it?

@pamaury Not sure I fully understand your question, sorry. Let me try to elaborate. The prim_rom instance certainly needs to be kept in the RTL, so that Vivado puts cells for it into the bitstream. But because we backdoor-load the ROM, I think the hash over all files required to build the bitstream should not include any ROM sources (neither test ROM nor real ROM), because ideally we don't rebuild the bitstream when only the ROM changes. The same holds for OTP and flash/RRAM. Does this answer your question?

Cc @gautschimi @thommythomaso

[gautschimi]

We currently include the test ROM in the OT build, I would be happy to remove it from the hash but I think for the build we cannot remove it otherwise Vivado optimizes the memory away? @gautschimi @andreaskurth This would lead to a potentially confusing situation where the hash is the same but actualy final bitstream is different due to the ROM (even though it's usually reloaded via the backdoor loader), but we can always add a comment about it?

@pamaury Not sure I fully understand your question, sorry. Let me try to elaborate. The prim_rom instance certainly needs to be kept in the RTL, so that Vivado puts cells for it into the bitstream. But because we backdoor-load the ROM, I think the hash over all files required to build the bitstream should not include any ROM sources (neither test ROM nor real ROM), because ideally we don't rebuild the bitstream when only the ROM changes. The same holds for OTP and flash/RRAM. Does this answer your question?

Cc @gautschimi @thommythomaso

I guess we need to provide a file to the ROM, otherwise vivado will think "ahh this is all zero, lets drop it". This might have changed with the second port from the bkdr util. Maybe worth a try to drop this from the dependencies. I think you will see very quick if it works or not