ItemProcessListener.onProcessError() not called when using AsycnItemProcessor [BATCH-2371]

[spring-projects-issues]

Dominik Bartholdi opened BATCH-2371 and commented

We wanna speed it up our jobs with parallel processing. But when ever we turn on parallel processing, ItemProcessListener.onProcessError() is not called anymore, but only ItemProcessListener.beforeProcess() and ItemProcessListener.afterProcess() are called.

We need this to be able to report on processing errors.

I created a simple project showing the problem: https://github.com/imod/parallel-batch

---

Affects: 3.0.3

Reference URL: https://github.com/imod/parallel-batch

Issue Links:

• BATCH-2773 Document how to handle asynchronous processing errors

Referenced from: commits 9059a9e, 2e8e7f6

Backported to: 4.0.0.M1

[spring-projects-issues]

Michael Minella commented

When using the AsyncItemProcessor, there is no way for the framework to know that an error occurred (it would be reported by the Future that is passed to the ItemWriter. Because of this, there is no support for the ItemProcessListener#onProcessError callback. The documentation has been updated to communicate this limitation.

[spring-projects-issues]

Dominik Bartholdi commented

Michael Minella what do you tink about extending org.springframework.batch.integration.async.AsyncItemProcessor.process(I) to take an optional ItemProcessListener? If the listener is present, then it would call it in case of an error and then still throw the original exception?

...
			public O call() throws Exception {
				if (stepExecution != null) {
					StepSynchronizationManager.register(stepExecution);
				}
				try {
					return delegate.process(item);
				} catch (Exception exception) {
					if (listener != null) {
						listener.onProcessError(item, exception);
					}
					throw exception;
				} finally {
					if (stepExecution != null) {
						StepSynchronizationManager.close();
					}
				}
			}
...

I agree, this is probably not the perfect solution, but I'm sure this will help in most cases.

[spring-projects-issues]

Dominik Bartholdi commented

placed a PR: #366

[spring-projects-issues]

Michael Minella commented

I moved this issue to 3.1 so further discussion can occur and since any of the proposed solutions would be considered breaking changes, I'd defer their implementation to a major release.

[spring-projects-issues]

Michael Minella commented

Dominik Bartholdi, the issue is that the AsyncItemProcessor has no idea if an exception occurs or not. Only the AsyncItemWriter would have any insight into an exception being thrown in the delegate ItemProcessor when the Future is unwrapped. If we looked at putting your example in the AsyncItemWriter, we'd face lifecycle issues (the idea that the ItemWriteListener for an item would be called before the ItemProcessListener methods have been). I really think, at this point, a documentation update is the best option.

[spring-projects-issues]

Michael Minella commented

There is already documentation in the javadoc for the AsyncItemProcessor calling out the lifecycle limitations including this one with this technique: http://docs.spring.io/spring-batch/apidocs/org/springframework/batch/integration/async/AsyncItemProcessor.html