Skip to content

Problem when filtering with max fetch size > 1 #8792

Closed
@abraham-macias

Description

@abraham-macias

In what version(s) of Spring Integration are you seeing this issue?

Spring integration SFTP v6.1.3

Describe the bug

If you combine SftpSimplePatternFileListFilter (so you use *.trg files) and SftpPersistentAcceptOnceFileListFilter (so you only accept one file once) by using a ChainFileListFilter, the resulting filter has "supportsSingleFileFiltering" to false.
This causes the filtering to be done and rollback (necessary for SftpPersistentAcceptOnceFileListFilter) in AbstractRemoteFileStreamingMessageSource.listFiles() method.

When you set max fetch size to a number bigger than one (for example 5) and at a certain point it is necessary to "this.toBeReceived.clear()" inside "AbstractRemoteFileStreamingMessageSource.doReceive()", those removed elements from toBeReceived are not rolled back.

To Reproduce

  1. Use a SftpStreamingMessageSource with a maxFetchSize = 5 and a ChainFileListFilter filter composed by SftpSimplePatternFileListFilter and SftpPersistentAcceptOnceFileListFilter.
  2. Put 2 files in the folder and invoke SftpStreamingMessageSource.receive() method twice.
  3. Put 5 files in the folder and invoke SftpStreamingMessageSource.receive() method five times.
  4. The last two files won't be received.

Expected behavior

All the 7 files should be received.

Sample

See the related pull request.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions