Edge web driver creation fails with "End of Central Directory record could not be found."

[Piedone]

Starting today, we get the following exception when calling DriverSetup.AutoSetUp("Edge"):

OpenQA.Selenium.WebDriverException : Creating the web driver failed with the message "End of Central Directory record could not be found.". This can mean that there is a leftover web driver process that you have to kill manually. Full exception: System.IO.InvalidDataException: End of Central Directory record could not be found.
	   at System.IO.Compression.ZipArchive.ReadEndOfCentralDirectory()
	   at System.IO.Compression.ZipArchive..ctor(Stream stream, ZipArchiveMode mode, Boolean leaveOpen, Encoding entryNameEncoding)
	   at System.IO.Compression.ZipFile.Open(String archiveFileName, ZipArchiveMode mode, Encoding entryNameEncoding)
	   at Atata.WebDriverSetup.DriverSetupExecutor.ExtractFromZip(String archiveFilePath, String destinationFilePath)
	   at Atata.WebDriverSetup.DriverSetupExecutor.ExtractDownloadedFile(String sourceFilePath, String destinationFilePath)
	   at Atata.WebDriverSetup.DriverSetupExecutor.DownloadDriverIfMissing(String version, String driverDestinationDirectoryPath, String driverDestinationFileName)
	   at Atata.WebDriverSetup.DriverSetupExecutor.SetUp(String version)
	   at Atata.WebDriverSetup.DriverSetupConfigurationBuilder.ExecuteSetUp()
	   at Atata.WebDriverSetup.DriverSetupConfigurationBuilder.SetUp()
	   at Atata.WebDriverSetup.DriverSetup.AutoSetUp(String browserName)

This only seems to be happening under GitHub-hosted Ubuntu runners of GitHub Actions. It doesn't happen on Ubuntu runners hosted by WarpBuild, or under GitHub-hosted Windows runners (nor can I reproduce it on my Windows machine).

I suppose something recently changed in how the Edge web driver is made available. This run on the 22nd succeeded, and this other one failed today, without any changes made to the affected code. Running the previously succeeding older run again also fails, confirming that this is something with the driver's download URL.

[YevgeniyShunevych]

Thanks @Piedone for reporting this. I'm also able to reproduce on Ubuntu on Azure Pipelines. I'm looking at this...

[Piedone]

Thank you for your quick reply, Yevgeniy, and for looking into this.

[YevgeniyShunevych]

While I was trying to solve this issue, seems that it is gone now. EdgeDriver for Linux ZIP appeared in Azure drivers storage and the library is able to use it. You can try on your side. But anyway, let me describe the problem.

So the library was trying to setup the driver for Edge. If you don't specify the version explicitly, the library tries to detects the version of Edge browser on a machine and use that version as desired EdgeDriver version. When it cannot detect the Edge version, then it gets the latest EdgeDriver version by URL https://msedgedriver.azureedge.net/LATEST_STABLE. In my case today both approaches returned version 126.0.2592.113, which is the current stable version according to https://developer.microsoft.com/en-us/microsoft-edge/tools/webdriver/. Then the library composes a URL to download ZIP file with the driver. In this case it was https://msedgedriver.azureedge.net/126.0.2592.113/edgedriver_linux64.zip. But at that moment there were no ZIP files for Linux and macOS, there were only ZIPs for Windows. The version specific drivers can be checked by such link: https://msedgewebdriverstorage.z22.web.core.windows.net/?prefix=126.0.2592.113/. So we were getting "404 (The specified blob does not exist.)" by requesting that URL. Your error "End of Central Directory record could not be found." is just a case of bad error handling in the library, it was trying to unzip "404 (The specified blob does not exist.)" text. I'm fixing the error handling for such cases.

So at the moment we are good with the current version of Edge. But in case of the similar problem in future I'm trying to find a fallback solution that will try to get the driver of the lower version, the closest one.

[Piedone]

I can confirm the previously failing runs pass now (see here and here).

Better error handling would be great here, thank you.

[sarahelsaig]

This has resurfaced just now, e.g. (I've seen this on multiple different PRs):

And here is the is the generated URL that's not found: https://msedgedriver.azureedge.net/127.0.2651.74/edgedriver_linux64.zip

Also, now it happens on all runners and on my local Linux machine as well. Not just the GitHub-hosted Ubuntu.

[YevgeniyShunevych]

I've just released v2.12.0 of Atata.WebDriverSetup package with Edge driver download improvement. Please upgrade and let me know how it works for you.

[Piedone]

Thank you! I've tested the updated package and it appears to be working. However, let me get back to you because due to the global Azure networking issues currently the download can randomly fail.

[Piedone]

Yeah, looks good both under Ubuntu and Windows when run in GHA. Locally, I could also get the tests to finally run, catching a break in the networking issues.

[YevgeniyShunevych]

Thanks for the information. I don't know what I can do for a case when Azure network has issues. Anyway, I think we handled the current issue and can close it.

[Piedone]

Yep, thanks.

Nothing to do, really, I just couldn't properly test first due to this outage.

[Piedone]

I got this now here:

OpenQA.Selenium.WebDriverException : Creating the web driver failed with the message "Failed to download Edge driver v127.0.2651.74 by URL 'https://msedgedriver.azureedge.net/127.0.2651.74/edgedriver_linux64.zip'.". This can mean that there is a leftover web driver process that you have to kill manually. Full exception: Atata.WebDriverSetup.DriverSetupException: Failed to download Edge driver v127.0.2651.74 by URL 'https://msedgedriver.azureedge.net/127.0.2651.74/edgedriver_linux64.zip'.
	 ---> System.Net.Http.HttpRequestException: Response status code does not indicate success: 404 (The specified blob does not exist.).
	   at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
	   at Atata.WebDriverSetup.HttpRequestExecutor.DownloadFile(String url, String filePath)
	   at Atata.WebDriverSetup.ReliableHttpRequestExecutor.<>c__DisplayClass4_0.<DownloadFile>b__0()
	   at Atata.WebDriverSetup.ReliableHttpRequestExecutor.ExecuteWithRetries[TResult](Func`1 operation)
	   at Atata.WebDriverSetup.ReliableHttpRequestExecutor.DownloadFile(String url, String filePath)
	   at Atata.WebDriverSetup.DriverSetupExecutor.DownloadDriverIfMissing(String version, String driverDestinationDirectoryPath, String driverDestinationFileName)
	   --- End of inner exception stack trace ---
	   at Atata.WebDriverSetup.DriverSetupExecutor.DownloadDriverIfMissing(String version, String driverDestinationDirectoryPath, String driverDestinationFileName)
	   at Atata.WebDriverSetup.DriverSetupExecutor.SetUp(String version)
	   at Atata.WebDriverSetup.DriverSetupConfigurationBuilder.SetUpDriver(DriverVersionResolver driverVersionResolver, DriverSetupExecutor setupExecutor, String driverVersion)
	   at Atata.WebDriverSetup.DriverSetupConfigurationBuilder.ExecuteSetUp()
	   at Atata.WebDriverSetup.DriverSetupConfigurationBuilder.SetUp()
	   at Atata.WebDriverSetup.DriverSetup.AutoSetUp(String browserName)
	   at Lombiq.Tests.UI.Services.WebDriverFactory.AutoSetup(String browserName) in /home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/test/Lombiq.UITestingToolbox/Lombiq.Tests.UI/Services/WebDriverFactory.cs:line 206
	   at Lombiq.Tests.UI.Services.WebDriverFactory.CreateDriverAsync[TDriver](String browserName, Func`1 driverFactory) in /home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/test/Lombiq.UITestingToolbox/Lombiq.Tests.UI/Services/WebDriverFactory.cs:line 190
	---- Atata.WebDriverSetup.DriverSetupException : Failed to download Edge driver v127.0.2651.74 by URL 'https://msedgedriver.azureedge.net/127.0.2651.74/edgedriver_linux64.zip'.
	-------- System.Net.Http.HttpRequestException : Response status code does not indicate success: 404 (The specified blob does not exist.).

I this expected and https://msedgedriver.azureedge.net/127.0.2651.74/edgedriver_linux64.zip for some reason now randomly giving a 404, or should it use a different URL?

[YevgeniyShunevych]

A Linux driver for 127.0.2651.74 is missing according to https://msedgewebdriverstorage.z22.web.core.windows.net/?prefix=127.0.2651.74/. Strange that Atata.WebDriverSetup decided to download that version.

[YevgeniyShunevych]

OK. I'm able to reproduce that. Let me try to fix that.

[Piedone]

Thank you!

[YevgeniyShunevych]

@Piedone, I've just released v2.13.0 of Atata.WebDriverSetup package with the fix.

[Piedone]

Thank you! I can confirm it works (see run here).

[wAsnk]

I got this now here:

OpenQA.Selenium.WebDriverException : Creating the web driver failed with the message "Failed to download Edge driver v127.0.2651.74 by URL 'https://msedgedriver.azureedge.net/127.0.2651.74/edgedriver_linux64.zip'.". This can mean that there is a leftover web driver process that you have to kill manually. Full exception: Atata.WebDriverSetup.DriverSetupException: Failed to download Edge driver v127.0.2651.74 by URL 'https://msedgedriver.azureedge.net/127.0.2651.74/edgedriver_linux64.zip'.
	 ---> System.Net.Http.HttpRequestException: Response status code does not indicate success: 404 (The specified blob does not exist.).
	   at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
	   at Atata.WebDriverSetup.HttpRequestExecutor.DownloadFile(String url, String filePath)
	   at Atata.WebDriverSetup.ReliableHttpRequestExecutor.<>c__DisplayClass4_0.<DownloadFile>b__0()
	   at Atata.WebDriverSetup.ReliableHttpRequestExecutor.ExecuteWithRetries[TResult](Func`1 operation)
	   at Atata.WebDriverSetup.ReliableHttpRequestExecutor.DownloadFile(String url, String filePath)
	   at Atata.WebDriverSetup.DriverSetupExecutor.DownloadDriverIfMissing(String version, String driverDestinationDirectoryPath, String driverDestinationFileName)
	   --- End of inner exception stack trace ---
	   at Atata.WebDriverSetup.DriverSetupExecutor.DownloadDriverIfMissing(String version, String driverDestinationDirectoryPath, String driverDestinationFileName)
	   at Atata.WebDriverSetup.DriverSetupExecutor.SetUp(String version)
	   at Atata.WebDriverSetup.DriverSetupConfigurationBuilder.SetUpDriver(DriverVersionResolver driverVersionResolver, DriverSetupExecutor setupExecutor, String driverVersion)
	   at Atata.WebDriverSetup.DriverSetupConfigurationBuilder.ExecuteSetUp()
	   at Atata.WebDriverSetup.DriverSetupConfigurationBuilder.SetUp()
	   at Atata.WebDriverSetup.DriverSetup.AutoSetUp(String browserName)
	   at Lombiq.Tests.UI.Services.WebDriverFactory.AutoSetup(String browserName) in /home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/test/Lombiq.UITestingToolbox/Lombiq.Tests.UI/Services/WebDriverFactory.cs:line 206
	   at Lombiq.Tests.UI.Services.WebDriverFactory.CreateDriverAsync[TDriver](String browserName, Func`1 driverFactory) in /home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/test/Lombiq.UITestingToolbox/Lombiq.Tests.UI/Services/WebDriverFactory.cs:line 190
	---- Atata.WebDriverSetup.DriverSetupException : Failed to download Edge driver v127.0.2651.74 by URL 'https://msedgedriver.azureedge.net/127.0.2651.74/edgedriver_linux64.zip'.
	-------- System.Net.Http.HttpRequestException : Response status code does not indicate success: 404 (The specified blob does not exist.).

I this expected and https://msedgedriver.azureedge.net/127.0.2651.74/edgedriver_linux64.zip for some reason now randomly giving a 404, or should it use a different URL?

Same error is happening again ☹ With the exact same error message.

OpenQA.Selenium.WebDriverException : Creating the web driver failed with the message "Failed to download Edge driver v127.0.2651.74 by URL 'https://msedgedriver.azureedge.net/127.0.2651.74/edgedriver_linux64.zip'.". This can mean that there is a leftover web driver process that you have to kill manually. Full exception: Atata.WebDriverSetup.DriverSetupException: Failed to download Edge driver v127.0.2651.74 by URL 'https://msedgedriver.azureedge.net/127.0.2651.74/edgedriver_linux64.zip'.
	 ---> System.Net.Http.HttpRequestException: Response status code does not indicate success: 404 (The specified blob does not exist.).
	   at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
	   at Atata.WebDriverSetup.HttpRequestExecutor.DownloadFile(String url, String filePath)
	   at Atata.WebDriverSetup.ReliableHttpRequestExecutor.<>c__DisplayClass4_0.<DownloadFile>b__0()
	   at Atata.WebDriverSetup.ReliableHttpRequestExecutor.ExecuteWithRetries[TResult](Func`1 operation)
	   at Atata.WebDriverSetup.ReliableHttpRequestExecutor.DownloadFile(String url, String filePath)
	   at Atata.WebDriverSetup.DriverSetupExecutor.DownloadDriverIfMissing(String version, String driverDestinationDirectoryPath, String driverDestinationFileName)
	   --- End of inner exception stack trace ---
	   at Atata.WebDriverSetup.DriverSetupExecutor.DownloadDriverIfMissing(String version, String driverDestinationDirectoryPath, String driverDestinationFileName)
	   at Atata.WebDriverSetup.DriverSetupExecutor.SetUp(String version)
	   at Atata.WebDriverSetup.DriverSetupConfigurationBuilder.SetUpDriver(DriverVersionResolver driverVersionResolver, DriverSetupExecutor setupExecutor, String driverVersion)
	   at Atata.WebDriverSetup.DriverSetupConfigurationBuilder.ExecuteSetUp()
	   at Atata.WebDriverSetup.DriverSetupConfigurationBuilder.SetUp()
	   at Atata.WebDriverSetup.DriverSetup.AutoSetUp(String browserName)
	   at Lombiq.Tests.UI.Services.WebDriverFactory.AutoSetup(String browserName) in /home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/test/Lombiq.UITestingToolbox/Lombiq.Tests.UI/Services/WebDriverFactory.cs:line 206
	   at Lombiq.Tests.UI.Services.WebDriverFactory.CreateDriverAsync[TDriver](String browserName, Func`1 driverFactory) in /home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/test/Lombiq.UITestingToolbox/Lombiq.Tests.UI/Services/WebDriverFactory.cs:line 190
	---- Atata.WebDriverSetup.DriverSetupException : Failed to download Edge driver v127.0.2651.74 by URL 'https://msedgedriver.azureedge.net/127.0.2651.74/edgedriver_linux64.zip'.
	-------- System.Net.Http.HttpRequestException : Response status code does not indicate success: 404 (The specified blob does not exist.).

Currently using 2.13.0 of Atata.WebDriverSetup.

[YevgeniyShunevych]

@wAsnk, I've released v2.14.0 of Atata.WebDriverSetup package with a fix. Please verify again using the latest.

[Piedone]

It works now, thank you.

[Piedone]

I hate to bring this up but we again see the same error, see here, with v2.14.0:

	OpenQA.Selenium.WebDriverException : Creating the web driver failed with the message "Failed to download Edge driver v129.0.2792.89 by URL 'https://msedgedriver.azureedge.net/129.0.2792.89/edgedriver_linux64.zip'.". This can mean that there is a leftover web driver process that you have to kill manually. Full exception: Atata.WebDriverSetup.DriverSetupException: Failed to download Edge driver v129.0.2792.89 by URL 'https://msedgedriver.azureedge.net/129.0.2792.89/edgedriver_linux64.zip'.
	 ---> System.Net.Http.HttpRequestException: Response status code does not indicate success: 404 (The specified blob does not exist.).
	   at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
	   at Atata.WebDriverSetup.HttpRequestExecutor.DownloadFile(String url, String filePath)
	   at Atata.WebDriverSetup.ReliableHttpRequestExecutor.<>c__DisplayClass4_0.<DownloadFile>b__0()
	   at Atata.WebDriverSetup.ReliableHttpRequestExecutor.ExecuteWithRetries[TResult](Func`1 operation)
	   at Atata.WebDriverSetup.ReliableHttpRequestExecutor.DownloadFile(String url, String filePath)
	   at Atata.WebDriverSetup.DriverSetupExecutor.DownloadDriverIfMissing(String version, String driverDestinationDirectoryPath, String driverDestinationFileName)
	   --- End of inner exception stack trace ---
	   at Atata.WebDriverSetup.DriverSetupExecutor.DownloadDriverIfMissing(String version, String driverDestinationDirectoryPath, String driverDestinationFileName)
	   at Atata.WebDriverSetup.DriverSetupExecutor.SetUp(String version)
	   at Atata.WebDriverSetup.DriverSetupConfigurationBuilder.SetUpDriver(DriverVersionResolver driverVersionResolver, DriverSetupExecutor setupExecutor, String driverVersion)
	   at Atata.WebDriverSetup.DriverSetupConfigurationBuilder.ExecuteSetUp()
	   at Atata.WebDriverSetup.DriverSetupConfigurationBuilder.SetUp()
	   at Atata.WebDriverSetup.DriverSetup.AutoSetUp(String browserName)
	   at Lombiq.Tests.UI.Services.WebDriverFactory.AutoSetup(String browserName) in /home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/test/Lombiq.UITestingToolbox/Lombiq.Tests.UI/Services/WebDriverFactory.cs:line 224
	   at Lombiq.Tests.UI.Services.WebDriverFactory.CreateDriverAsync[TDriver](String browserName, Func`1 driverFactory) in /home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/test/Lombiq.UITestingToolbox/Lombiq.Tests.UI/Services/WebDriverFactory.cs:line 201
	---- Atata.WebDriverSetup.DriverSetupException : Failed to download Edge driver v129.0.2792.89 by URL 'https://msedgedriver.azureedge.net/129.0.2792.89/edgedriver_linux64.zip'.
	-------- System.Net.Http.HttpRequestException : Response status code does not indicate success: 404 (The specified blob does not exist.).

[YevgeniyShunevych]

@Piedone, looking into this. Unfortunately, algorithm of resolution of driver version corresponding to Edge browser version is not perfect. Will try to find some solution for improvement.

[Piedone]

Great, thank you.