In TIdFTPServer, 'RetrieveFile' is much slower than 'StoreFile'

[davidwed]

Upload ( StoreFile )

Download ( RetrieveFile )

I have created a sample project that can be used to simulate this scenario.

FtpServerDemo-Indy10

[rlebeau]

When receiving a file (StoreFile), the file data is received by the data channel's IOHander.InputBuffer dynamic memory buffer, which is read into the target TStream in 100Kb blocks in a loop. When sending a file (RetreiveFile), the source TStream is read into a fixed 100Kb memory buffer in a loop and written directly to the data channel. In both cases, after each block, the command channel is checked for an abort command.

So, off-hand I don't see anything that would adversely cause a RetrieveFile to be much slower than a StoreFile, as you have shown. I suggest you profile the code to see where it is actually spending its time during RetreiveFile to slow it down so much.

You might also consider using TBufferedFileStream rather than using CreateFile() with a THandleStream. But, if you must use CreateFile() directly, then consider creating a custom THandleStream-derived class that models the caching behavior of TBufferedFileStream. Alternatively, you could memory-map the file (CreateFileMapping() + MapViwOfFile()) and use the view with TIdReadOnlyMemoryBufferStream.

---

That being said, here are some additional observations about your code, not related to the speed issue:

• your TuneDataChannel() method is fairly useless. Tuning the buffer sizes of the TIdFTPServerContext.Binding affects only the command channel, not the data channel. Since the command channel doesn't exchange files, there is no point in tuning the command channel.

  You are also trying to tune the buffer sizes of TIdFTPServerContext.DataChannel, which is a standalone TIdDataChannel object, not a TIdTCPConnection or TIdIOHandlerSocket object. So, your attempts to tune TIdTCPConnection.Socket.Binding and TIdIOHandlerSocket.Binding will always be skipped.

  If you want to tune the data channel manually, use this instead:

  type
    TIdDataChannelAccess = class(TIdDataChannel)
    end;
  ...
  TuneBinding(TIdDataChannelAccess(Ctx.DataChannel).FDataChannel.Socket.Binding);
  

• StopButtonClick() doesn't need to disconnect the client sockets manually before deactivating the server. Especially since you are doing so outside of the Contexts lock, which is dangerous given TIdFTPServer's multi-threaded nature. The TIdFTPServer.Active setter already disconnects the clients internally for you. But if you must do it manually, at least do it inside of the Contexts lock to ensure TIdContext objects don't get destroyed behind your back.

  Something else TIdFTPServer does internally which you are not doing manually is making sure that when a client is being disconnected, any active file transfer for that client is also canceled.

  If you are just trying to not block the main UI thread while TIdFTPServer is deactivating, then use TIdAntiFreeze, or deactivate the server in a worker thread.

• you don't need to seek the VStream in the OnStoreFile event. TIdFTPServer handles that internally after the OnStoreFile handler exits.

[davidwed]

I have implemented your suggestions, but the transfer rate has not increased.