With transactional writes, Dataset.pushData() inside a request handler records a journal entry and returns; the actual write happens in commitJournalEntries() during commit. So a rejection can't be caught where the write was made:
try {
await pushData(item); // resolves fine
} catch (err) {
// never runs — the item is rejected at commit, after the handler returned
}
Our case is an oversized dataset item: we caught "Data item is too large", trimmed a few large fields, retried, and otherwise raised a non-retryable error with actionable advice. In v3 that worked for the HTTP and browser crawlers because writes were immediate, and for AdaptivePlaywrightCrawler we hooked commitResult, which is gone in v4 with no equivalent. Now the failure surfaces as an ordinary request error and gets retried to exhaustion.
With transactional writes, Dataset.pushData() inside a request handler records a journal entry and returns; the actual write happens in commitJournalEntries() during commit. So a rejection can't be caught where the write was made:
Our case is an oversized dataset item: we caught "Data item is too large", trimmed a few large fields, retried, and otherwise raised a non-retryable error with actionable advice. In v3 that worked for the HTTP and browser crawlers because writes were immediate, and for AdaptivePlaywrightCrawler we hooked commitResult, which is gone in v4 with no equivalent. Now the failure surfaces as an ordinary request error and gets retried to exhaustion.