Existing chapters no longer being ignored

[ghost]

Describe the bug
Mango seems to be overwriting chapters instead of just ignoring the existing ones.

To Reproduce
Attempt to download an existing chapter.

Expected behavior
Before it ignored existing chapters when using the download manager and said something like "1 of 46 chapters added".

Environment (please complete the following information):

• OS: Manjaro 20.0.3
• Browser: Firefox 77.0.1
• Mango Version 0.6.1

[hkalexling]

This is the expected behavior. When you download a chapter, Mango will check if the same chapter is in the queue, and if it is, Mango will ignore the chapter. But Mango doesn't check if the chapter is already downloaded on your filesystem. In your case, I think you have deleted the download task from the queue, so Mango just downloads it as usual and overwrites the existing file.

But I agree it's arguably a poor design choice. @jaredlt you are a much better UX designer than I am. What do you think?

[jaredlt]

Just to clarify the current behaviour (as I haven't spent a lot of time in the Download area):

• If a chapter is in the queue (regardless of status ie. even if status = Downloaded) then this chapter will be ignored when you try to re-download it
• If you click Delete Completed Tasks in the Download Manager and then try to re-download an existing chapter, this chapter will be downloaded again and the original file overwritten

In this case the current behaviour is inconsistent from a user's perspective. A downloaded chapter which is no longer in the queue will not be ignored, but a downloaded chapter which is still in the queue will be ignored. Ideally we would have a consistent behaviour for all downloaded chapters.

Considerations:

• Is it possible to have multiple releases for the same chapter for the same language? Eg. multiple groups both release, or a later, more high quality release?
• How do you compare if something is already in the queue? Is it just be title and chapter or does a specific chapter for a specific group for a specific language have a unique id which you use?

Options:

• Do nothing :)
• Modify the 'queue check' when adding to Download Manager to only ignore if status is 'downloading' (this would re-download for 'completed' and 'failed') <-- this provides consistent behaviour but uses more bandwidth
• After checking the queue, also check the file system before adding to the Download Manager (the 'Considerations' above may play in to this though. If the 'match' is by title and chapter we could be preventing users from downloading a future HQ release but I don't know enough about the internal behaviour and how MangaDex works - also, do we need to think about this more generically for when MangaDex is just one of many providers?)

Other options (we could attack the problem from a different angle, or do this in parallel):

• On the Download page, we could expose which chapters are already downloaded. Perhaps greyed out or with a tick. Something to clearly and obviously show they already exist. Happy to think about the UI more if it's a direction you like
• We could also introduce a new button which says eg. Download All Missing - I don't love this terminology though. Needs more thought.

[hkalexling]

@jaredlt Thanks for your reply! Yup, that is indeed the current behavior.

MangaDex provides each chapter with a unique ID. When adding a download job to the queue, Mango compares the ID of the chapter with those in the queue to determine if it should be ignored. Multiple versions or releases are guaranteed to have different IDs, but they might have the same title and chapter/volume number, and therefore they might be saved under the same filename and overwrites each other.

Modify the 'queue check' when adding to Download Manager to only ignore if status is 'downloading' (this would re-download for 'completed' and 'failed')

I like this approach! Never thought about that. I will think about it and try to implement it.

While we are at it, we should probably find a better way to notify users when a download job is ignored. In the current form, we just show an alert box saying something like "2 of 3 chapters added to the download queue. Proceed to the download manager?". A better way would be to take the user to the download manager directly, and show a detailed error message there if a job is ignored (e.g., "an archive with the same path is already on your file system", or "a chapter with the same ID is already in the queue")

[jaredlt]

@hkalexling while this will provide a consistent behaviour I'm not sure it really solves the problem that @outnos brought up. From what I understand, the scenario would be:

You have a manga with some chapters already downloaded. You know you want to grab the rest and, without going to manually check which you have and which you don't, you basically want to 'Download All' and have Mango intelligently download only the missing chapters. I guess when we're talking about large batch jobs this could make a big difference so it doesn't re-download a lot of files (@outnos please correct me if I've misinterpreted).

If that is the case, then probably an approach from the Download UI is best. But you would need to compare the file system to the search results eg.

Select Missing would select first and last columns in the above example. But I also don't know if we need to convey that the 'chapter' has a match in the library, as opposed matching against that particular release group / language etc. So I don't know how feasible this approach is.

[hkalexling]

Having a "select missing" option would be great, but the problem is we don't know if a cbz file in the library is downloaded from MangaDex. Mango cannot differentiate the chapters downloaded with the built-in downloader and the chapters downloaded with external tools and then imported into the library. In a future version, we could save the manga and chapter IDs in the info.json file, but we won't be able to do the same for chapters that are already downloaded.

[jaredlt]

Just for clarity, what is concern about knowing whether a file was downloaded via MangaDex? Is it that you can't control the naming and it may be difficult to match? Or that you don't wan't to overwrite a file that was not downloaded via MangaDex?

[hkalexling]

@jaredlt It's impossible to reliably match the existing files. Even if the user never renamed the file, in the current state a chapter would be downloaded with a filename like "Vol.1 Ch.1 One Punch", but there could be multiple chapters on MangaDex that match this filename. It's even more complicated if we consider the possibility that users might rename the files and even customize the rename rules.

[ghost]

I'd personally say just attempting conversions would be fine as this is still in development, so breaking stuff is to be expected.
Then we could save chapter id's and generally more information on the manga to prevent such issues in the future.
EDIT:
This should maybe tie into the future plugin system.