dateparser not able to parse things like next tuesday.

[saroup]

It parses Tuesday to the date of the Tuesday of the current week but when input is next Tuesday it returns none.

[tannercollin]

I'm having the same issue except it's returning Tuesday of the previous week:

>>> parse('now').strftime('%a %Y-%m-%d')
'Mon 2019-10-21'
>>> parse('tuesday').strftime('%a %Y-%m-%d')
'Tue 2019-10-15'
>>> parse('next tuesday').strftime('%a %Y-%m-%d')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'NoneType' object has no attribute 'strftime'

[GargiVyas31]

Hi, I am Gargi Vyas. I am GSOC 2020 candidate and would like to work on this bug.

[tannercollin]

That'd be awesome, Gargi. Thanks!

[aditya-hari]

@Gallaecio, @noviluni
I have looked through the code and I think I understand the gist of it at this point. What would the recommended way to tackle this? Would appreciate some suggestions to get started. A separate function in FreshnessDateDataParser maybe?

[Gallaecio]

@aditya-hari Go ahead and propose an approach in a pull request. It’s easier to discuss over code 🙂

[aditya-hari]

@Gallaecio I haven't really come up with anything concrete in code yet, can't open a pull request.

Things like 'next tuesday' aren't identified with any locale, so there has to be some changes made in the locale info to sort that out. I am not entirely sure how to though.

I thought about just changing the date_string to something standard like "in x days/months" but that will obviously only work for English if implemented in that way.

[Gallaecio]

@aditya-hari I suggest you start from FreshnessDateDataParser.parse, go through what the code does keeping the target strings in mind (e.g. “Next Tuesday”), and make the required changes as you go. I see for example that ago and in are hardcoded in some parts, I guess you will need to add next there.

You could add a test for “Next Tuesday”, extend FreshnessDateDataParser.parse as needed until it is parsed successfully, and then make sure no other tests are broken after your changes.

[aditya-hari]

@Gallaecio

Sorry it is taking me this long, I have something sort of working, I will hopefully open a PR soon. However the way I am doing this won't be able to handle the "after 15 days" situation mentioned in #635

[Gallaecio]

Not a problem. It’s OK to just fix “Next ” for now, we can improve things later with additional, separate changes.

[jgtimestuff]

There are a lot of time-related translations available in the unicode-cldr xml or json files that could definitely be used to augment dateparser.py with things that handle all sorts of variations like 'Next Tuesday'. Of course, I'd also like to see something that cover 'Next Weekend' or 'on the weekend'... but it doesn't look like that's been defined as yet.

Anyway, what would it take to pull in the cldr datefields for each language and incorporate them?

https://github.com/unicode-cldr/cldr-dates-full/blob/master/main/ru/dateFields.json

[jgtimestuff]

Okay, just saw this issue from 2 years ago -- cldr_language_data | move data directory | 2 years ago
So, is it just that the cldr_language_data needs updating to include more variations of 'next'?

My apologies... it seems there is a script to do just this already in the code:
#487 CLDR script update - https://github.com/scrapinghub/dateparser/compare/cldr-script-update

Is the current dictionary up to date or is it just that the existing code isn't calling things like 'next' that already exist in the code?

[jgtimestuff]

In freshness_date_parser, I think we need to add something from calendar to get the right day of the week?

    td = relativedelta(**kwargs)

relativedelta arguments for 'next' + dayofweek needs to add a day, then check the calendar for the next one?

today = datetime.datetime.now() (happens to be Friday)
today + relativedelta.relativedelta(weekday=calendar.FRIDAY)

today + rld.relativedelta(weekday=calendar.FRIDAY)
datetime.datetime(2020, 3, 20, 8, 55, 7, 615746) [today, instead of next friday]

so, we have to add a day to today, then look for next Friday:

today + rld.relativedelta(days=+1)
datetime.datetime(2020, 3, 21, 8, 55, 7, 615746)
today = today + rld.relativedelta(days=+1)
today + rld.relativedelta(weekday=calendar.TUESDAY)
datetime.datetime(2020, 3, 24, 8, 55, 7, 615746)
today + rld.relativedelta(weekday=calendar.FRIDAY)
datetime.datetime(2020, 3, 27, 8, 55, 7, 615746) [ Next Friday ]

today = today + rld.relativedelta(days=+1)
today + rld.relativedelta(weekday=calendar.TUESDAY)
datetime.datetime(2020, 3, 24, 8, 55, 7, 615746) [ Next Tuesday ]