Incorrect source detection

[hautecodure]

The current systems seems to incorrectly tag releases due to words matching closely with the presets.

For example a title containing the words: Camelot, Camera, Cambodia will be tagged as CAM regardless of the actual tag at the end.

AFAICS its due to python's in comparison here. A simple fix would be to match the title against the quality (its the currently doing the opposite).

• Commit: 2d08448
• Python Version [python -V]: 3.6.5

[nosmokingbandit]

What do you mean by matching the title against the quality? I can split the title on the year and match source terms only on the part of the string after the year, but I want to understand what you are saying in case that works better.

…

On Sat, Jul 7, 2018 at 7:02 AM endofline; ***@***.***> wrote: The current systems seems to incorrectly tag releases due to words matching closely with the presets. For example a title containing the words: Camelot, Camera, Cambodia will be tagged as CAM regardless of the actual tag at the end. AFAICS its due to python's in comparison here <https://github.com/nosmokingbandit/Watcher3/blob/master/core/searcher.py#L353>. A simple fix would be to match the title against the quality (its the currently doing the opposite). - Commit: 2d08448 <2d08448> - Python Version [python -V]: 3.6.5 — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#257>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKX1SZOmM7QhNSv_r3wSypkjyVn0RM7Yks5uEJVEgaJpZM4VGUcb> .

[hautecodure]

I meant reversing the match, instead of

'cam' in 'camera'

it may become:

'camera' in 'cam'

This coupled with the year split should minimize the chance for a false-positive/negative.

[nosmokingbandit]

But 'camera' in 'cam' can never be true. I might not understand completely what you mean.

Either way, 6673a0f should get everything working as expected. I tested it with a few "Camelot" movies and it fixed all the problems with source detection.

[hautecodure]

Sorry, probably I should've word it better or at least provide some examples.

A name like:

Some.Generic.Camera.Documentary.20XX.Resolution.CAM.Codec-Group

Split and iterated over, would have comparisons like:

'some' in 'cam'
'generic' in 'cam'
'camera' in 'cam'
'documentary' in 'cam'
'20XX' in 'cam'
'resolution' in 'cam'
'cam' in 'cam'
'codec' in 'cam'
'group' in 'cam'

compared to the current evaluation (pre-split on year), which would be:

'cam' in 'some'
'cam' in 'generic'
'cam' in 'camera'
'cam' in 'documentary'
'cam' in '20XX'
'cam' in 'resolution'
'cam' in 'cam'
'cam' in 'codec'
'cam' in 'group'

Changing the predicate, would reduce false-negatives with a much smaller chance of false-positives. But as you've already implemented a split on year, false-positives should be close to 0.

[nosmokingbandit]

I'd have to check a few releases but that might run into problems if the uploader doesn't follow scene release rules, which happens all the time. Some join with spaces, underscores, hyphens, periods, etc and it would have to iterate over the title split on all of those delimiters. I'll take a look at it, but it might add more complexity than I'd really like.

From a coding perspective that can be done a little quicker as well:

if any(source == i for i in title.split('.'):
    #do something

[chet-manley]

Found a small issue with the title split method today. Some queries pull IMDB metadata with an incorrect year, so the split never occurs, and _fuzzy_title fails with far less than 75%.

For instance, scene releases may contain a different year than the U.S. release date. Here is an example in IMDB. Since the IMDB metadata used to provide "year" to fuzzy_title does not match any of the scene release years, the title string does not get split, and matching occurs on the whole string, causing good results to be discarded. If we assume that every title will always contain a year, a better split method would be:

year_regex = re.compile(r'[ _.(](\d{4})[ _.)]') # this would go outside the loop
split_title = year_regex.split(result['title'])

This doesn't require the years to match exactly, and we can then check if the release's year is within a range of the search year:

rel_title_ss = split_title[0]
if len(split_title) == 3:
  rel_title_year = int(split_title[1])
  year_int = int(year)
  year_min = year_int - 1
  year_max = year_int + 1
  if not rel_title_year in [year_min, year_int, year_max]:
    logging.debug('{} year outside range {}-{}.').format(rel_title_ss, year_min, year_max)
    continue
else:
  # string was not split

I think +/-1 year is a pretty safe assumption that the correct release was found. The else is only strictly required if you also want to handle titles with no year in them. However, sending an unsplit title through _fuzzy_title still ends up with a discarded release, unless the search title is sufficiently long enough to still reach 75%.

[chet-manley]

Just thought of some edge cases. The above method will fail for movies with years in their names (1983, Class of 1984, 1985, Wonder Woman 1984, et al). Considering this, a few options could be:

• Make a further assumption that the year will always be in parens and never "bare". I have found very few releases where this was not the case.
• First check for parens. If true, then split using my first method. If false, assume any 4 consecutive digits surrounded by non-word characters to be the year.