Remote files sources should have the option not to read remote file contents

[fgabolde]

Expected Behavior

File message sources should not have to download the file contents. There are use cases for tracking the files themselves, but not their contents.

E.g. I want to write a service that monitors files on a remote file system and notifies a downstream service. It does not need to care about the file contents.

As far as I can tell, "all" that would be needed is to allow AbstractRemoteFileStreamingMessageSource to return remotePath as message payload, instead of session.readRaw(remotePath).

Current Behavior

Remote file sources always attempt to read the file contents.

Context

E.g. I want to write a service that monitors files on a remote file system and notifies a downstream service. It does not need to read the file contents.

This should also work for the AWS S3 integration, hopefully (which is my actual use case). Consumers of this message source would handle S3Object payloads instead of InputStream.

All the rest of the Spring Integration machinery (file filters, message stores, ...) is super useful to me, so I really want to take advantage of it, but the file downloading is not, so I'm currently copying the code of AbstractRemoveFileStreamingMessageSource and specializing it for my use case, but I feel that it would be generally beneficial to have.

[artembilan]

That is why it is called Streaming.
I think what you are looking for is a nlst command for the AbstractRemoteFileOutboundGateway: https://docs.spring.io/spring-integration/reference/6.3/ftp/outbound-gateway.html#using-the-nlst-command.

I don't think we will look into something like poll just for list names.
That simply could be achieved with simple poller for let's say remote directory name as a message payload and that gateway which would return you a list of names. Filters can be applied there as well.