[CI] Update drivers installation process into containers

[pvchupin]

• Install level_zero loader and headers as part of the drivers installation. Should be installed as dependency into containers, rather than built and shipped with the compiler.
• Add level_zero to update_drivers.py
• Fix CM install. Takes version from dependencies.json rather than always the latest.
• Exclude u18 CM package from consideration. Before it was installing u18 then u20 and produced a warning.
• Update "unstable" containers build by always taking the "latest" packages rather than relying on prior definition of staging versions.
• Clean up dependencies.json and erase specific staging versions. We stopped updating these earlier.

[bader]

Please, take a look at pre-commit failures.

[bader]

This URL points to Level Zero loader and I think the runtime is installed as part of compute-runtime package. Isn't loader installed with compute-runtime as well?
If not, I'm okay with installing the loader, but PR title should be corrected.

[pvchupin]

Loader is not installed as part of the compute-runtime.
Here is content for the record. Here is what is installed with compute-runtime package:

dpkg-query -L intel-level-zero-gpu
/.                                                        
/usr                                                      
/usr/include                                              
/usr/lib                                                  
/usr/lib/x86_64-linux-gnu                                 
/usr/lib/x86_64-linux-gnu/libze_intel_gpu.so.1.3.24278    
/usr/share                                                
/usr/share/doc                                            
/usr/share/doc/intel-level-zero-gpu                       
/usr/share/doc/intel-level-zero-gpu/changelog.gz          
/usr/share/doc/intel-level-zero-gpu/copyright             
/var                                                      
/var/lib                                                  
/var/lib/libze_intel_gpu                                  
/var/lib/libze_intel_gpu/pci_bind_status_file             
/var/lib/libze_intel_gpu/wedged_file                      
/usr/lib/x86_64-linux-gnu/libze_intel_gpu.so.1

And here is what this PR adds:

dpkg-query -L level-zero                       
/usr                                                                   
/usr/lib                                                               
/usr/lib/x86_64-linux-gnu                                              
/usr/lib/x86_64-linux-gnu/libze_loader.so.1.8.8                        
/usr/lib/x86_64-linux-gnu/libze_tracing_layer.so.1.8.8                 
/usr/lib/x86_64-linux-gnu/libze_validation_layer.so.1.8.8              
/usr/lib/x86_64-linux-gnu/libze_loader.so.1                            
/usr/lib/x86_64-linux-gnu/libze_tracing_layer.so.1                     
/usr/lib/x86_64-linux-gnu/libze_validation_layer.so.1                  

dpkg-query -L level-zero-devel                 
/usr                                                                   
/usr/include                                                           
/usr/include/level_zero                                                
/usr/include/level_zero/layers                                         
/usr/include/level_zero/layers/zel_tracing_api.h                       
/usr/include/level_zero/layers/zel_tracing_ddi.h                       
/usr/include/level_zero/layers/zel_tracing_register_cb.h               
/usr/include/level_zero/loader                                         
/usr/include/level_zero/loader/ze_loader.h                             
/usr/include/level_zero/ze_api.h                                       
/usr/include/level_zero/ze_ddi.h                                       
/usr/include/level_zero/zes_api.h                                      
/usr/include/level_zero/zes_ddi.h                                      
/usr/include/level_zero/zet_api.h                                      
/usr/include/level_zero/zet_ddi.h                                      
/usr/lib                                                               
/usr/lib/x86_64-linux-gnu                                              
/usr/lib/x86_64-linux-gnu/pkgconfig                                    
/usr/lib/x86_64-linux-gnu/pkgconfig/level-zero.pc                      
/usr/lib/x86_64-linux-gnu/pkgconfig/libze_loader.pc                    
/usr/lib/x86_64-linux-gnu/libze_loader.so                              
/usr/lib/x86_64-linux-gnu/libze_tracing_layer.so                       
/usr/lib/x86_64-linux-gnu/libze_validation_layer.so                  

[pvchupin]

Made quite a few changes, so please re-review. Note that generate test matrix fails because old version of dependencies.json don't have level_zero component defined.

[bader]

The nice thing about staging is that we know the versions of the dependencies. How do we know what "latest" is?

[pvchupin]

latest is a moving tag.
Exact versions can be found in a container build logs or from the inside of the container through package manager commands, like:

# apt list --installed | grep intel-igc-cm                                                                                        
intel-igc-cm/now 1.0.119 amd64 [installed,local]                               

[igchor]

@pvchupin @smaslov-intel is the plan to do the same with Unified Runtime loader (install into the containers)?

[smaslov-intel]

@pvchupin @smaslov-intel is the plan to do the same with Unified Runtime loader (install into the containers)?

Yes, eventually Unified Runtime would a side project installed separately and not by the compiler.