More XLA fixes for nightly support #4668

Job	Run time
fabric-cpu (macOS-11, lightning, 3.8, 1.11)	4m 13s
fabric-cpu (macOS-11, lightning, 3.9, 1.12)	4m 34s
fabric-cpu (macOS-11, lightning, 3.10, 1.13)	4m 12s
fabric-cpu (ubuntu-20.04, lightning, 3.8, 1.11)	3m 9s
fabric-cpu (ubuntu-20.04, lightning, 3.9, 1.12)	3m 21s
fabric-cpu (ubuntu-20.04, lightning, 3.10, 1.13)	4m 15s
fabric-cpu (windows-2022, lightning, 3.8, 1.11)	4m 38s
fabric-cpu (windows-2022, lightning, 3.9, 1.12)	4m 6s
fabric-cpu (windows-2022, lightning, 3.10, 1.13)	4m 34s
fabric-cpu (macOS-11, lightning, 3.10, 2.0)	4m 53s
fabric-cpu (ubuntu-20.04, lightning, 3.10, 2.0)	4m 15s
fabric-cpu (windows-2022, lightning, 3.10, 2.0)	5m 14s
fabric-cpu (macOS-11, lightning, 3.8, 1.11, oldest)	7m 1s
fabric-cpu (ubuntu-20.04, lightning, 3.8, 1.11, oldest)	5m 56s
fabric-cpu (windows-2022, lightning, 3.8, 1.11, oldest)	5m 32s
fabric-cpu (macOS-11, fabric, 3.8, 1.13)	3m 24s
fabric-cpu (ubuntu-20.04, fabric, 3.8, 1.13)	3m 2s
fabric-cpu (windows-2022, fabric, 3.8, 1.13)	3m 3s
	1h 19m 22s

Provide feedback