RHEL SBSA TF2 Backend Build

[fpetrini15]

Goal: Incorporate SBSA builds/tests and extended the smoke tests for x86 and SBSA to include L0_sequence_batcher.

Server changes: triton-inference-server/server#7595
PyTorch Backend changes: triton-inference-server/pytorch_backend#139
TensorRT Backend changes: triton-inference-server/tensorrt_backend#99

[mc-nv]

can't find utility make this more transparent?

Suggested change

	COMMAND if [ "${TRITON_TENSORFLOW_INSTALL_EXTRA_DEPS}" = "ON" ] \; then docker exec tensorflow_backend_deps sh -c "tar -cf - $<IF:$<BOOL:${RHEL_BUILD}>,/usr/local/cuda/targets/${LIBS_ARCH}-linux/lib/,/usr/lib/${LIBS_ARCH}-linux-gnu/>libnccl.so*" | tar --strip-components=3 -xf - -C ./tf_backend_deps \; fi
	COMMAND if [ "${TRITON_TENSORFLOW_INSTALL_EXTRA_DEPS}" = "ON" ] \; then docker exec tensorflow_backend_deps sh -c "find /usr/local/cuda/targets/*-linux/lib -name libnccl.so* -exec tar -cvf archive.tar {} \; fi "

[fpetrini15]

This is a great suggestion! Let me examine the tar commands for equivalency first.

[nv-kmcgill53]

If you can use find then please remove the

if(${RHEL_BUILD})
      set(LIBS_ARCH "sbsa")
    else()
      set(LIBS_ARCH "aarch64")
    endif()

[fpetrini15]

So I spent yesterday afternoon trying to get the right incantation for this, but have so far been unsuccessful. However, I've learned a lot more about what's going on here. From my understanding we are tar-ing each of the libnccl.so matches within the external tf container and then writing it to standard output (note the - [https://www.gnu.org/software/tar/manual/html_node/file.html]:

"tar -cf - $<IF:$<BOOL:${RHEL_BUILD}>,/usr/local/cuda/targets/${LIBS_ARCH}-linux/lib/,/usr/lib/${LIBS_ARCH}-linux-gnu/>libnccl.so*"

Which is then picked up in the subsequent command to expand the tar file in the build container stripping out the leading path ( /path/to/lib/libnccl.so --> libnccl.so):

tar --strip-components=3 -xf - -C ./tf_backend_deps

When we attempt to use the find command, nothing I've tried seems to match this behavior, however, I also think this is a really odd way to go about copying these files. I've tried a few alternate strategies such as mounting the tf_backend_deps folder and copying the libs in, which I thought would be a more straightforward approach, but that hasn't worked either.

I will continue trying to find a way to resolve this, but since this is not tested by default in our CI, I will prioritize getting the EA3 image out for now.

[nv-kmcgill53]

Address @mc-nv's comments but I have nothing new to add.

[fpetrini15]

Okay, I believe I ended up finding a solution that's more straightforward. We now pass the results of the find command directly into docker cp so that we can copy the libs directly into the target folder. No messing around with filepaths. No messing around with tar-ing. Let me know what you think.

[nv-kmcgill53]

neat!