Android development is not cleaning up old files

[Juraj-Masiar]

Is this a feature request or a bug?

Bug

What is the current behavior?

Each execution of:

web-ext run --target=firefox-android --android-device=emulator-5554 --firefox-apk=org.mozilla.firefox

will create new directory on the SD card and not deleting it afterwards.
After few days of development this consumed 400MB of space on my device.

What is the expected or desired behavior?

There should be some clean-up.

Version information (for bug reports)

• Firefox version: 66.02
• Your OS and version: Win 10x64 1809
• Paste the output of these commands:

node --version && npm --version && web-ext --version

v12.0.0
6.8.0
3.0.0

https://www.dropbox.com/s/9hvb6d6rms0fspi/Screenshot%202019-04-29%2015.50.26.png?dl=0

[Rob--W]

Could you paste the output of web-ext run , with -v for extra verbosity?

I am expecting the following messages near the exit of web-ext run:

• "Stopping existing instances of ${selectedFirefoxApk}..."
• "Cleaning up artifacts directory on the Android device..."
• "Removing ${artifactsDir} artifacts directory on ${deviceId} device"

... followed by the actual removal of the files on the device.

[Juraj-Masiar]

Interesting, I can see that it actually works most of the times, but not always.
This is the log that after which the folder "web-ext-artifacts-1556622126212" is still on the SD card:

[extension-runners/firefox-android.js][debug] Uploading group_speed_dial-13.1 on the android device
[util/adb.js][debug] Pushing C:\Users\Juraj\AppData\Local\Temp\tmp-web-ext-435122YQogIRiDy50\group_speed_dial-13.1.zip to /sdcard/web-ext-artifacts-1556622126212/group_speed_dial-13.1.xpi on emulator-5554
[extension-runners/firefox-android.js][debug] Upload completed: /sdcard/web-ext-artifacts-1556622126212/group_speed_dial-13.1.xpi
[util/temp-dir.js][debug] Removing temporary directory: C:\Users\Juraj\AppData\Local\Temp\tmp-web-ext-435122YQogIRiDy50
[firefox/remote.js][debug] Received message from client: {"from":"root","type":"addonListChanged"}
Last extension reload: 13:03:13 GMT+0200 (GMT+02:00)[firefox/remote.js][debug]

[extension-runners/index.js][info]
Exiting web-ext on user request
[extension-runners/firefox-android.js][info] Stopping existing instances of org.mozilla.firefox_beta...
[util/adb.js][debug] Run adb shell command on emulator-5554: ["am","force-stop","org.mozilla.firefox_beta"]
[extension-runners/firefox-android.js][debug] Cleaning up artifacts directory on the Android device...
[util/adb.js][debug] Removing /sdcard/web-ext-artifacts-1556622126212 artifacts directory on emulator-5554 device
[util/adb.js][debug] Run adb shell command on emulator-5554: ["rm","-rf","/sdcard/web-ext-artifacts-1556622126212"]
[firefox/remote.js][debug] Received "end" from Firefox client
PS C:\Users\Juraj\git\addons\GroupSpeedDial\bin>

I will try to find a pattern to help reproducing it.

[Juraj-Masiar]

So the easiest way to reproduce it is sending "Ctrl + C" during add-on reload (when sources changes) or during the start.
However I can still see the lines about removing it, but the folders are not deleted:

[extension-runners/index.js][info] Press R to reload (and Ctrl-C to quit)
[firefox/remote.js][debug] Received message from client: {"from":"root","type":"tabListChanged"}
[extension-runners/index.js][info]
Exiting web-ext on user request
[extension-runners/firefox-android.js][info] Stopping existing instances of org.mozilla.firefox_beta...
[util/adb.js][debug] Run adb shell command on emulator-5554: ["am","force-stop","org.mozilla.firefox_beta"]
[firefox/remote.js][debug] Received "end" from Firefox client
[extension-runners/firefox-android.js][debug] Cleaning up artifacts directory on the Android device...
[util/adb.js][debug] Removing /sdcard/web-ext-artifacts-1556623222592 artifacts directory on emulator-5554 device
[util/adb.js][debug] Run adb shell command on emulator-5554: ["rm","-rf","/sdcard/web-ext-artifacts-1556623222592"]

Only difference is when I ctrl + c during the start, then it ends with this log:

[util/file-filter.js][debug] Resolved path C:\Users\Juraj\git\addons\GroupSpeedDial\firefox_dev\_locales\ru\messages.json with sourceDir C:\Users\Juraj\git\addons\GroupSpeedDial\firefox_dev to C:\Users\Juraj\git\addons\GroupSpeedDial\firefox_dev\_locales\ru\messages.json
[util/file-filter.js][debug] Resolved path C:\Users\Juraj\git\addons\GroupSpeedDial\firefox_dev\_locales\sk\messages.json with sourceDir C:\Users\Juraj\git\addons\GroupSpeedDial\firefox_dev to C:\Users\Juraj\git\addons\GroupSpeedDial\firefox_dev\_locales\sk\messages.json
[util/file-filter.js][debug] Resolved path C:\Users\Juraj\git\addons\GroupSpeedDial\firefox_dev\_locales\uk\messages.json with sourceDir C:\Users\Juraj\git\addons\GroupSpeedDial\firefox_dev to C:\Users\Juraj\git\addons\GroupSpeedDial\firefox_dev\_locales\uk\messages.json
[util/file-filter.js][debug] Resolved path C:\Users\Juraj\git\addons\GroupSpeedDial\firefox_dev\_locales\zh_CN\messages.json with sourceDir C:\Users\Juraj\git\addons\GroupSpeedDial\firefox_dev to C:\Users\Juraj\git\addons\GroupSpeedDial\firefox_dev\_locales\zh_CN\messages.json
[util/file-filter.js][debug] Resolved path C:\Users\Juraj\git\addons\GroupSpeedDial\firefox_dev\_locales\zh_TW\messages.json with sourceDir C:\Users\Juraj\git\addons\GroupSpeedDial\firefox_dev to C:\Users\Juraj\git\addons\GroupSpeedDial\firefox_dev\_locales\zh_TW\messages.json
[cmd/build.js][debug] Manifest declared default_locale, localizing extension name
[extension-runners/firefox-android.js][debug] Uploading group_speed_dial-13.1 on the android device
[util/adb.js][debug] Pushing C:\Users\Juraj\AppData\Local\Temp\tmp-web-ext-12576gmHs5qvwR10q\group_speed_dial-13.1.zip to /sdcard/web-ext-artifacts-1556623555093/group_speed_dial-13.1.xpi on emulator-5554
[extension-runners/firefox-android.js][debug] Upload completed: /sdcard/web-ext-artifacts-1556623555093/group_speed_dial-13.1.xpi
[util/temp-dir.js][debug] Removing temporary directory: C:\Users\Juraj\AppData\Local\Temp\tmp-web-ext-12576gmHs5qvwR10q
[program.js][error]
s: Exiting Firefox Remote Debugging socket discovery on user request
    at discoverRDPUnixSocket (C:\Users\Juraj\AppData\Roaming\npm\node_modules\web-ext\dist\webpack:\src\util\adb.js:284:32)
    at m.adbDiscoveryAndForwardRDPUnixSocket (C:\Users\Juraj\AppData\Roaming\npm\node_modules\web-ext\dist\webpack:\src\extension-runners\firefox-android.js:552:7)
    at async Promise.all (index 2)
    at m.run (C:\Users\Juraj\AppData\Roaming\npm\node_modules\web-ext\dist\webpack:\src\extension-runners\firefox-android.js:120:16)
    at async Promise.all (index 0)
    at p.run (C:\Users\Juraj\AppData\Roaming\npm\node_modules\web-ext\dist\webpack:\src\extension-runners\index.js:89:5)
    at h (C:\Users\Juraj\AppData\Roaming\npm\node_modules\web-ext\dist\webpack:\src\cmd\run.js:174:13)
    at $.execute (C:\Users\Juraj\AppData\Roaming\npm\node_modules\web-ext\dist\webpack:\src\program.js:215:11)

[program.js][debug] Command executed: run

[Rob--W]

This could also happen if the physical connection between the computer and the device is broken (i.e. the cable is unplugged).

I suggest that we check the sdcard for directories with the known web-ext-artifacts- prefix at startup, and output a suggestion to run something like web-ext cleanup-artifacts to remove those files (and then we should also implement this command of course).

[rpl]

I'll mentor this bug.

[onlywicked]

I would like to work on it.

[rpl]

@onlywicked once you have started to look into it feel free to ask more details if needed (@Rob--W provided some details in #1591 (comment))

[rpl]

Fixed by @Vishalghyv in #1965 (old artifacts are detected automaticallly, an additional --adb-remove-old-artifacts cli option can be used to automatically remove them), The fix is going to be included in the next major release (web-ext v5.0.0)