[WynandVStaden]

Closes #751.

What changed

A Gmail search now reads the value of after: and before: (and so of newer: and older:) the way real does:

1. A value real cannot read (abc, 2026/13/01, 2026/02/30, 2026.10.01, 2026-10-01T00:00, 2026/10-01, digits that are not ASCII) makes the search match nothing, also beside a value it can read (after:2026/10/01 after:abc). Before this it was skipped, so the search matched as if it were not there.
2. A date is three runs of ASCII digits split by one / or one -, tried as year, month, day, then month, day, year, then day, month, year: 10/01/2026 is 1 October, 01/10/2026 10 January and 15/10/2026 15 October. A year below 100 is in the 2000s (26/10/05), a first run of three digits or more is the year, and a month or day may carry leading zeros. Trailing /, ., , and : are dropped.
3. A count of seconds is read when above 0 and at most _GMAIL_SECONDS_MAX (10000-01-01), leading zeros and all; 0, -1, a quoted date (after:"2026/10/01"), '2026/10/01' and (2026/10/01) match nothing. Before this the quotes were stripped, 0 and -1 were read as seconds, and after:99999999999999999999 was a 500.
4. A - in front negates the operator: -after:X is before:X and -before:X is after:X (-newer: and -older: likewise), and a negation whose value does not read (-after:abc, -before:0) is dropped. Before this the - stayed in the free text.

after:- stays the search term after, as an empty after: already was. Reading a value after a space (after: 2026/10/01) is #781's and is left to it.

_gmail_q_date in backlot/routers/google.py reads a value from the search and returns the epoch seconds it names, which _parse_gmail_q stores; a value it returns None for goes under _GMAIL_NOTHING. _gmail_date is unchanged, so the bounds newer_than: and older_than: compute, which can be zero or negative, are read as before.

Why this is what the real API does

Two measurements against gmail.googleapis.com, neither of them mine; I have no Gmail token.

• 2026-10-15, in gmail: after: and before: skip a value they cannot read, miss month-first dates, read a value after a space as text and take -after: as after: #751: the table of values over a mailbox of 595 messages.
• 2026-10-17, by @nuriyoo in the review of this pull request, over the same mailbox (603 messages): day-first dates, two-digit years, longer runs of digits, the trailing characters, the bound on seconds, non-ASCII digits, and a negation whose value does not read. His review supplied the code and tests for those rows as suggestions, applied as written.

The comment above _GMAIL_DATE_KEYS records both, and names what is not modelled: a year past 9999, 00, a count of seconds between _GMAIL_SECONDS_MAX and 9223372036854, and values real answered with ids no reading gives (2026/10, 10/01, 2026/10/01x, 1790812800.5), each of which matches nothing here.

Verification

• Tests — pytest on a .[dev] install, Ubuntu 24.04 (WSL), Python 3.12.3: 7729 passed, 38 skipped. The Gmail tests of tests/test_search.py and tests/test_google.py: 531 passed, the count from the review. test_gmail_after_before_value, test_gmail_after_before_parse and test_gmail_after_before_over_the_wire fail with backlot/ from main and pass with this change. test_gmail_newer_older_are_after_before now compares the stored value with the epoch seconds _gmail_q_date gives.

• Optional surfaces that ran rather than skipped — none. This was a .[dev] install only, so tests/test_sdk.py, tests/test_s3.py, tests/test_mcp.py, tests/test_llamaindex.py and the reader, Google and mirage tests in tests/test_integrations.py sat out. The change touches backlot/routers/google.py and tests/test_search.py only.

• Lint — ruff check . && ruff format --check .: clean. python scripts/prose_check.py upstream/main: no findings.

• A test fails without this change

• A new endpoint serving corpus content is ACL-scoped, proved for both an admin and a scoped token

• Comments state what was measured, not the history of the fix

The second box is not ticked because no endpoint is added.

[nuriyoo]

Thank you for this, and for reading the issue's table so closely: every row of it holds on this branch. I replayed the reading on 2026-10-17 on the same Workspace mailbox (603 messages) and found more of it, which the suggestions below fold into _gmail_q_date as one loop:

• day-first dates once the month cannot be first (15/10/2026), two-digit years (26/10/05, 10/05/26), longer runs of digits (02026/10/01, 2026/010/01, 01790812800), and trailing //, ., , and : all read on real, where this branch answers nothing;
• \d reads full-width and Arabic-Indic digits as a date, where real answers nothing;
• after:99999999999999999999 is a 500 here (the bound overflows SQLite's integer), where real answers nothing;
• a negation whose value does not read (-after:abc, -before:0) answered every message on real, not nothing.

The space reading is the other open pull request's, which is #781, so the first suggestion drops the _GMAIL_OP hunk and the tests leave its rows to it.

On the suggestion tree, the Gmail tests of tests/test_search.py and tests/test_google.py pass (531), and 30 of the 33 mutations generated from the diff fail a test; the three left drop the value appended to the nothing marker, which no reader uses, trade two branches whose keys cannot both match, and narrow \s on the free-text line this diff leaves as it was.

[nuriyoo]

The space reading is the other open pull request's issue, which is #781, and that one was opened first; it reads all seven date and folder operators across a space (before: and older: too, measured 2026-10-17) and keeps a tab from reading a date. Dropping this hunk keeps the two from conflicting, and once it lands the space rows hold here as well:

Suggested change

	# The lookbehinds are the four `_GMAIL_ADDRESS_KEYS`; their comment says why.
	_GMAIL_OP = re.compile(
	r'(\w+):(?:(?:(?<=\bfrom:)|(?<=\bto:)|(?<=\bcc:)|(?<=\bbcc:))\s*)?("[^"]*"|\S*)', re.IGNORECASE
	r"(\w+):(?:(?:(?<=\bfrom:)|(?<=\bto:)|(?<=\bcc:)|(?<=\bbcc:)"
	r"|(?<=\bafter:)|(?<=\bbefore:)|(?<=\bnewer:))\s*)?"
	r'("[^"]*"|\S*)',
	re.IGNORECASE,
	)
	# The lookbehinds are the four `_GMAIL_ADDRESS_KEYS`; their comment says why.
	_GMAIL_OP = re.compile(
	r'(\w+):(?:(?:(?<=\bfrom:)|(?<=\bto:)|(?<=\bcc:)|(?<=\bbcc:))\s*)?("[^"]*"|\S*)', re.IGNORECASE
	)

[nuriyoo]

I replayed the reading on 2026-10-17 on the same Workspace mailbox (603 messages): every row of the issue's table holds, and the reading has more to it than the two patterns here.

value                       real                                 this branch
15/10/2026, 30/09/2026      15 October, 30 September (day first) nothing
26/10/05, 9/30/26           2026-10-05, 2026-09-30               nothing
10/05/26, 30/9/26           2010-05-26, 2030-09-26 (year first)  nothing
2026/010/01, 02026/10/01    1 October                            nothing
026/10/05, 2026/10/005      5 October                            nothing
01790812800                 1 October (seconds)                  nothing
2026/10/01//, 2026/10/01.   1 October                            nothing
1790812800,  2026/10/01:    1 October                            nothing
２０２６/１０/０１            nothing                              1 October (`\d` takes any digit)
99999999999999999999        nothing                              500 (OverflowError in the SQL bound)

So a date is three runs of ASCII digits tried as year, month, day, then month, day, year, then day, month, year, with a year below 100 in the 2000s and a run of three digits or more always the year; trailing /, ., , and : drop; and seconds stop at a bound (253402300800 read, 9223372036854 and up nothing). One loop reads all of it, and the comment above gets the table:

Suggested change

	# How real reads the value of `after:` and `before:` (and so of `newer:` and `older:`). Measured on
	# a Workspace mailbox of 595 messages on 2026-10-15, comparing the ids each query returned: a date
	# written year first or month first, with `/` or `-`, with or without leading zeros and with a
	# trailing `/` (`2026-10-1`, `10/1/2026`, `10-01-2026`, `2026/10/01/`), answered as
	# `after:2026/10/01`, and `01/10/2026` as 10 January; a positive count of seconds (`1790812800`)
	# answered as the instant it names, and `1` every message. `abc`, `2026/13/01`, `2026/02/30`,
	# `2026/10/32`, `2026.10.01`, `2026-10-01T00:00`, `0`, `-1`, `"2026/10/01"`, `'2026/10/01'` and
	# `(2026/10/01)` answered nothing, and so did a query holding one of them beside a value real
	# reads. The value is read after one or two spaces as well (`after: 2026/10/01`, `before:
	# 2026/10/01`; `_GMAIL_OP`). A `-` in front negates the operator: `-after:2026/10/01` answered as
	# `before:2026/10/01`, and `-before:X` as `after:X`. `after:-` is the search term `after`, as an
	# empty `after:` is.
	_GMAIL_DATE_KEYS = {"after", "before"}
	_GMAIL_YMD = re.compile(r"(\d{4})([/-])(\d{1,2})\2(\d{1,2})")
	_GMAIL_MDY = re.compile(r"(\d{1,2})([/-])(\d{1,2})\2(\d{4})")
	_GMAIL_NEGATED_DATE = re.compile(r"(?<!\S)-(after|before|newer|older):", re.IGNORECASE)
	_GMAIL_NEGATION = {"after": "before", "before": "after", "newer": "older", "older": "newer"}
	def _gmail_q_date(v: str) -> str | None:
	"""The epoch seconds an `after:` or `before:` value in a search names, as `_gmail_date` reads
	them, or None for a value real answers with nothing (the measurement at `_GMAIL_DATE_KEYS`)."""
	if v.endswith("/"):
	v = v[:-1]
	if re.fullmatch(r"[1-9][0-9]*", v):
	return v
	if m := _GMAIL_YMD.fullmatch(v):
	y, mo, d = m.group(1), m.group(3), m.group(4)
	elif m := _GMAIL_MDY.fullmatch(v):
	mo, d, y = m.group(1), m.group(3), m.group(4)
	else:
	return None
	try:
	day = datetime.datetime(int(y), int(mo), int(d), tzinfo=datetime.timezone.utc)
	except ValueError:
	return None
	return str(int(day.timestamp()))
	# How real reads the value of `after:` and `before:` (and so of `newer:` and `older:`), as
	# `_gmail_q_date` reads it. Measured on a Workspace mailbox, comparing the ids each query returned
	# with those of a date written `YYYY/MM/DD` (595 messages on 2026-10-15, 603 on 2026-10-17):
	#
	# read as seconds ASCII digits whose value is above 0 and at most `_GMAIL_SECONDS_MAX`,
	# leading zeros and all: `1`, `01`, `1790812800`, `01790812800`,
	# `253402300800`; `0`, `000`, `-1`, `+1790812800`, `1e9`, `9223372036854` and
	# `99999999999999999999` nothing
	# read as a date three runs of ASCII digits split by one `/` or one `-`, tried as year, month,
	# day, then month, day, year, then day, month, year, the first that is a date:
	# `10/1/2026` is 1 October, `01/10/2026` 10 January, `15/10/2026` 15 October;
	# a year above 0 and below 100 is in the 2000s (`26/10/05`, `9/30/26`), so
	# `10/05/26` is 26 May 2010; a first run of three digits or more is the year
	# (`026/10/05`, `02026/10/01`); a month or day may carry leading zeros
	# (`2026/010/01`, `2026/10/005`)
	# either after the trailing `/`, `.`, `,` and `:` are dropped (`2026/10/01//`,
	# `2026/10/01.`, `1790812800,`, `2026/10/01:`)
	# nothing anything else: `abc`, `2026/13/01`, `2026/02/30`, `2026.10.01`,
	# `2026-10-01T00:00`, `2026/10-01`, `"2026/10/01"`, `(2026/10/01)`, digits
	# that are not ASCII (`２０２６/１０/０１`), and a query holding one of them
	# beside a value real reads
	#
	# A `-` in front negates the operator when its value reads: `-after:2026/10/01` answered as
	# `before:2026/10/01`, and `-before:X` as `after:X`; `-after:abc`, `-after:0` and `-before:0`
	# answered every message. `after:-` is the search term `after`, as an empty `after:` is. Not
	# modelled here, each answered nothing: a year past 9999 (real read `10000/01/01` as a date after
	# every message), `00` (real read it), a count of seconds between `_GMAIL_SECONDS_MAX` and
	# `9223372036854`, and values real answered with ids no reading above gives (`2026/10`, `10/01`,
	# `2026/10/01x`, `1790812800.5`).
	_GMAIL_DATE_KEYS = {"after", "before"}
	# The largest count of seconds measured to read, 10000-01-01T00:00:00Z.
	_GMAIL_SECONDS_MAX = 253402300800
	_GMAIL_DATE = re.compile(r"([0-9]+)([/-])([0-9]+)\2([0-9]+)")
	_GMAIL_NEGATED_DATE = re.compile(r"(?<!\S)-(after|before|newer|older): *(\S+)", re.IGNORECASE)
	_GMAIL_NEGATION = {"after": "before", "before": "after", "newer": "older", "older": "newer"}
	def _gmail_q_date(v: str) -> str | None:
	"""The epoch seconds an `after:` or `before:` value in a search names, or None for a value real
	answers with nothing (the reading the comment at `_GMAIL_DATE_KEYS` measures)."""
	v = v.rstrip("/.,:")
	if re.fullmatch(r"[0-9]+", v):
	return str(int(v)) if 0 < int(v) <= _GMAIL_SECONDS_MAX else None
	m = _GMAIL_DATE.fullmatch(v)
	if m is None:
	return None
	a, b, c = m.group(1), m.group(3), m.group(4)
	for y, mo, d in [(a, b, c)] if len(a) > 2 else [(a, b, c), (c, a, b), (c, b, a)]:
	year = int(y) + 2000 if 0 < int(y) < 100 else int(y)
	try:
	day = datetime.datetime(year, int(mo), int(d), tzinfo=datetime.timezone.utc)
	except ValueError:
	continue
	return str(int(day.timestamp()))
	return None
	def _gmail_negated_date(m: re.Match) -> str:
	"""A negated date operator as the one it flips to, or nothing at all when its value does not
	read (the comment at `_GMAIL_DATE_KEYS`)."""
	if _gmail_q_date(m.group(2)) is None:
	return " "
	return f"{_GMAIL_NEGATION[m.group(1).lower()]}:{m.group(2)}"

A few values real answered with ids no reading gives (2026/10 193, 10/01 49, 2026/10/01x 38, 1790812800.5 39, 00 read as a date) stay as the nothing they are on this branch; the comment names them.

[nuriyoo]

-after:abc, -after:0 and -before:0 answered every message on real, so a negation whose value does not read is dropped rather than flipped into the nothing it would be unnegated. Reading the value after a space here too keeps -after: 2026/10/01 the before: it is on real:

Suggested change

	q = _GMAIL_NEGATED_DATE.sub(lambda m: _GMAIL_NEGATION[m.group(1).lower()] + ":", q)
	q = _GMAIL_NEGATED_DATE.sub(_gmail_negated_date, q)

[nuriyoo]

The rows for the readings above, the space rows left to the other pull request, and two for a negation whose value does not read and the bound on seconds:

Suggested change

	_OCT_1 = "1790812800" # 2026-10-01T00:00:00Z
	@pytest.mark.parametrize(
	"value, want",
	[
	*((v, _OCT_1) for v in ("2026/10/01", "2026-10-01", "2026/10/1", "2026-10-1")),
	*((v, _OCT_1) for v in ("10/01/2026", "10/1/2026", "10-01-2026", "2026/10/01/")),
	(_OCT_1, _OCT_1),
	("01/10/2026", "1768003200"), # 10 January
	("1", "1"),
	*(
	(v, None)
	for v in ("abc", "2026/13/01", "2026/02/30", "2026/10/32", "2026.10.01")
	+ ("2026-10-01T00:00", "0", "-1", '"2026/10/01"', "'2026/10/01'", "(2026/10/01)")
	),
	],
	)
	def test_gmail_after_before_value(value, want):
	"""Each value the comment on `_GMAIL_DATE_KEYS` measures, as `_gmail_q_date` reads it."""
	from backlot.routers.google import _gmail_q_date
	assert _gmail_q_date(value) == want
	def test_gmail_after_before_parse():
	"""A value real cannot read leaves nothing to match, beside a readable one too; the value is
	read after a space; a `-` negates the operator; `after:-` stays a search term."""
	from backlot.routers.google import _GMAIL_NOTHING, _parse_gmail_q
	for q in ("after:abc", "before:2026/02/30", "after:0", 'after:"2026/10/01"'):
	assert _GMAIL_NOTHING in _parse_gmail_q(q)[1], q
	assert _GMAIL_NOTHING in _parse_gmail_q("after:2026/10/01 after:abc")[1]
	for q in ("after: 2026/10/01", "after: 2026/10/01", "newer: 2026/10/01"):
	assert _parse_gmail_q(q) == ("", {"after": [_OCT_1]}), q
	assert _parse_gmail_q("before: 2026/10/01") == ("", {"before": [_OCT_1]})
	assert _parse_gmail_q("-after:2026/10/01") == ("", {"before": [_OCT_1]})
	assert _parse_gmail_q("-before:2026/10/01") == ("", {"after": [_OCT_1]})
	assert _parse_gmail_q("-newer:2026/10/01") == ("", {"before": [_OCT_1]})
	assert _parse_gmail_q("after:-") == ("after:-", {})
	def test_gmail_after_before_over_the_wire(tmp_path):
	"""The issue's reproduction: three messages, around 1 October."""
	from tests._helpers import client_for, tiny_corpus
	records = [
	{
	"source_type": "gmail",
	"doc_id": d,
	"mailbox": "ava",
	"title": t,
	"content": "Status notes.",
	"author_email": "bob@acme.com",
	"to": "ava@acme.com",
	"label_ids": ["INBOX"],
	"readers": ["ava@acme.com"],
	"created": c,
	}
	for d, t, c in [
	("sep", "September", "2026-09-15T09:00:00Z"),
	("oct1", "October 1", "2026-10-01T12:00:00Z"),
	("oct5", "October 5", "2026-10-05T09:00:00Z"),
	]
	]
	settings = tiny_corpus(tmp_path, records)
	with client_for(settings, reload=True) as c:
	h = {"Authorization": f"Bearer {settings.admin_token}"}
	def count(q):
	r = c.get("/gmail/v1/users/ava@acme.com/messages", headers=h, params={"q": q})
	return len(r.json().get("messages", []))
	for q in ("after:2026/10/01", "after:10/01/2026", "after:10-01-2026", "after:2026/10/01/"):
	assert count(q) == 2, q
	assert count("after: 2026/10/01") == 2
	for q in ("after:abc", "after:2026/13/01", "after:0", 'after:"2026/10/01"'):
	assert count(q) == 0, q
	assert count("after:2026/10/01 after:abc") == 0
	assert count("-after:2026/10/01") == count("before:2026/10/01") == 1
	_OCT_1 = "1790812800" # 2026-10-01T00:00:00Z
	@pytest.mark.parametrize(
	"value, want",
	[
	*((v, _OCT_1) for v in ("2026/10/01", "2026-10-01", "2026/10/1", "2026-10-1")),
	*((v, _OCT_1) for v in ("10/01/2026", "10/1/2026", "10-01-2026", "2026/10/01/")),
	(_OCT_1, _OCT_1),
	("01/10/2026", "1768003200"), # 10 January
	("1", "1"),
	("01", "1"),
	("01790812800", _OCT_1),
	("253402300800", "253402300800"),
	*((v, _OCT_1) for v in ("2026/10/01//", "2026/10/01.", "1790812800,", "2026/10/01:")),
	*((v, _OCT_1) for v in ("02026/10/01", "2026/010/01")),
	("15/10/2026", "1792022400"), # 15 October
	("26/10/05", "1791158400"),
	("026/10/05", "1791158400"),
	("2026/10/005", "1791158400"),
	("9/30/26", "1790726400"),
	("10/05/26", "1274832000"), # 26 May 2010
	("30/9/26", "1916611200"),
	*(
	(v, None)
	for v in ("abc", "2026/13/01", "2026/02/30", "2026/10/32", "2026.10.01")
	+ ("2026-10-01T00:00", "0", "-1", '"2026/10/01"', "'2026/10/01'", "(2026/10/01)")
	+ ("000", "+1790812800", "1e9", "2026/10-01", "0010/05/2026", "00/01/2026")
	+ ("9223372036854", "99999999999999999999")
	+ ("\uff12\uff10\uff12\uff16/\uff11\uff10/\uff10\uff11",)
	),
	],
	)
	def test_gmail_after_before_value(value, want):
	"""Each value the comment on `_GMAIL_DATE_KEYS` measures, as `_gmail_q_date` reads it."""
	from backlot.routers.google import _gmail_q_date
	assert _gmail_q_date(value) == want
	def test_gmail_after_before_parse():
	"""A value real cannot read leaves nothing to match, beside a readable one too; a `-` negates
	the operator, or drops it when its value does not read; `after:-` stays a search term."""
	from backlot.routers.google import _GMAIL_NOTHING, _parse_gmail_q
	for q in ("after:abc", "before:2026/02/30", "after:0", 'after:"2026/10/01"'):
	assert _GMAIL_NOTHING in _parse_gmail_q(q)[1], q
	assert _GMAIL_NOTHING in _parse_gmail_q("after:2026/10/01 after:abc")[1]
	assert _parse_gmail_q("-after:2026/10/01") == ("", {"before": [_OCT_1]})
	assert _parse_gmail_q("-after: 2026/10/01") == ("", {"before": [_OCT_1]})
	assert _parse_gmail_q("-after:abc") == _parse_gmail_q("-before:0") == ("", {})
	assert _parse_gmail_q("-before:2026/10/01") == ("", {"after": [_OCT_1]})
	assert _parse_gmail_q("-newer:2026/10/01") == ("", {"before": [_OCT_1]})
	assert _parse_gmail_q("after:-") == ("after:-", {})
	def test_gmail_after_before_over_the_wire(tmp_path):
	"""The issue's reproduction: three messages, around 1 October."""
	from tests._helpers import client_for, tiny_corpus
	records = [
	{
	"source_type": "gmail",
	"doc_id": d,
	"mailbox": "ava",
	"title": t,
	"content": "Status notes.",
	"author_email": "bob@acme.com",
	"to": "ava@acme.com",
	"label_ids": ["INBOX"],
	"readers": ["ava@acme.com"],
	"created": c,
	}
	for d, t, c in [
	("sep", "September", "2026-09-15T09:00:00Z"),
	("oct1", "October 1", "2026-10-01T12:00:00Z"),
	("oct5", "October 5", "2026-10-05T09:00:00Z"),
	]
	]
	settings = tiny_corpus(tmp_path, records)
	with client_for(settings, reload=True) as c:
	h = {"Authorization": f"Bearer {settings.admin_token}"}
	def count(q):
	r = c.get("/gmail/v1/users/ava@acme.com/messages", headers=h, params={"q": q})
	return len(r.json().get("messages", []))
	for q in ("after:2026/10/01", "after:10/01/2026", "after:10-01-2026", "after:2026/10/01/"):
	assert count(q) == 2, q
	for q in ("after:abc", "after:2026/13/01", "after:0", 'after:"2026/10/01"'):
	assert count(q) == 0, q
	assert count("after:2026/10/01 after:abc") == 0
	assert count("-after:2026/10/01") == count("before:2026/10/01") == 1
	assert count("-after:abc") == 3
	assert count("before:99999999999999999999") == 0

[WynandVStaden]

Thank you for replaying the reading and for the table. I applied the four suggestions as written in one commit, cfc619a, which drops the _GMAIL_OP hunk and leaves the space reading to #781. On it, the Gmail tests of tests/test_search.py and tests/test_google.py give 531 passed, your count, and pytest on a .[dev] install (Ubuntu 24.04, Python 3.12.3) gives 7729 passed, 38 skipped. ruff check ., ruff format --check . and python scripts/prose_check.py upstream/main are clean. I rewrote the description for the new reading, the dropped negation and the bound on seconds, and to credit your 2026-10-17 measurement.

[nuriyoo]

Thank you, and for applying the four suggestions so exactly: cfc619a is byte for byte the tree I tested, so this merges with no maintainer commit. Reading a date in the three orders real tries, with two-digit years, the trailing characters and the bound on seconds, and dropping a negation whose value does not read, settles every row of the issue's table and of the 2026-10-17 measurement. main was merged in on the way; at that head the suite passes (10684 passed, 2 skipped) and CI is green. The space reading stays with the other open pull request.