Skip to content

Latest commit

 

History

History
750 lines (571 loc) · 35.5 KB

sftp.md

File metadata and controls

750 lines (571 loc) · 35.5 KB
Raw

SFTP

Both client-side and server-side SFTP are supported. Starting from version 2.0, the SFTP related code is located in the sshd-sftp artifact, so one needs to add this additional dependency to one's maven project:

    <dependency>
        <groupId>org.apache.sshd</groupId>
        <artifactId>sshd-sftp</artifactId>
        <version>...same as sshd-core...</version>
    </dependency>

Server-side SFTP

On the server side, the following code needs to be added:

SftpSubsystemFactory factory = new SftpSubsystemFactory.Builder()
    ...with...
    ...with...
    .build();
server.setSubsystemFactories(Collections.singletonList(factory));

Note: the factory uses an ad-hoc CloseableExecutorService in order to spawn the necessary threads for processing the protocol messages. The user can provide a custom Supplier of such a service - however, it must be protected from shutdown if the user needs it to remain active between successive SFTP session. This can be done via the ThreadUtils#noClose utility:

CloseableExecutorService mySpecialExecutor = ...;
SftpSubsystemFactory factory = new SftpSubsystemFactory.Builder()
    .withExecutorServiceProvider(() -> ThreadUtils.noClose(mySpecialExecutor))
    .build();
server.setSubsystemFactories(Collections.singletonList(factory));

SftpEventListener

Provides information about major SFTP protocol events. The provided File/DirectoryHandle to the various callbacks can also be used to store user-defined attributes via its AttributeStore implementation. The listener is registered at the SftpSubsystemFactory:

public class MySfpEventListener implements SftpEventListener {
    private static final AttributeKey<SomeType> MY_SPECIAL_KEY = new Attribute<SomeType>();

    ...
    @Override
    public void opening(ServerSession session, String remoteHandle, Handle localHandle) throws IOException {
        localHandle.setAttribute(MY_SPECIAL_KEY, instanceOfSomeType);
    }

    @Override
    public void writing(
            ServerSession session, String remoteHandle, FileHandle localHandle,
            long offset, byte[] data, int dataOffset, int dataLen)
                throws IOException {
        SomeType myData = localHandle.getAttribute(MY_SPECIAL_KEY);
        ...do something based on my data...
    }
}


SftpSubsystemFactory factory = new SftpSubsystemFactory();
factory.addSftpEventListener(new MySftpEventListener());
sshd.setSubsystemFactories(Collections.<NamedFactory<Command>>singletonList(factory));

Note: the attached attributes are automatically removed once handle has been closed - regardless of whether the close attempt was successful or not. In other words, after SftpEventListener#closed has been called, all attributes associated with the handle are cleared.

SftpFileSystemAccessor

This is the abstraction providing the SFTP server subsystem access to files and directories. The SFTP subsystem uses this abstraction to obtain file channels and/or directory streams. One can override the default implementation and thus be able to track and/or intervene in all opened files and folders throughout the SFTP server subsystem code. The accessor is registered/overwritten in via the SftpSubSystemFactory:

SftpSubsystemFactory factory = new SftpSubsystemFactory.Builder()
    .withFileSystemAccessor(new MySftpFileSystemAccessor())
    .build();
server.setSubsystemFactories(Collections.singletonList(factory));

Note:

  • Closing of file channel/directory streams created by the accessor are also closed via callbacks to the same accessor

  • When closing a file channel that may have been potentially modified, the default implementation forces a synchronization of the data with the file-system. This behavior can be modified by setting the sftp-auto-fsync-on-close property to false (or by providing a customized implementation that involves other considerations as well).

Internal exceptions and error message handling

If an exception is thrown during processing of an SFTP command, then the exception is translated into a SSH_FXP_STATUS message using a registered SftpErrorStatusDataHandler. The default implementation provides a short description of the failure based on the thrown exception type. However, users may override it when creating the SftpSubsystemFactory and provide their own codes and/or messages - e.g., for debugging one can register a DetailedSftpErrorStatusDataHandler (see sshd-contrib) that "leaks" more information in the generated message.

Intercepting data sent via STDERR channel data from the client

If the registered handler implements ChannelSessionAware then it will also be informed of the registered ChannelSession when it is provided to the SftpSubsystem itself. This can be used to register an extended data writer that override the default (which ignores such data) and can handle data sent via the STDERR channel. Note: this feature is allowed according to SFTP version 4 - section 3.1:

Packets are sent and received on stdout and stdin. Data sent on stderr by the server SHOULD be considered free format debug or supplemental error information, and MAY be displayed to the user.

however, the current code provides no built-in support for this feature other than ignoring any such sent data.

If registering an extended data writer one should take care of any race conditions that may occur where (extended) data may arrive before the handler is informed of the existence of the ChannelSession. For this purpose one should configure a reasonable buffer size by setting the channel-session-max-extdata-bufsize property. This way, if any data arrives before the extended data handler is registered it will be buffered (up to the specified max. size). Note: if a buffer size is configured but no extended data handler is registered when channel is spawning the command then an exception will occur.

Sending custom data via STDERR channel data to the client

Same logic as the STDERR incoming data applies to the outgoing error I/O streams provided to the SftpSubsystem. If the handler implements the relevant CommandDirectErrorStreamAware and/or AsyncCommandErrorStreamAware interface then it will be provided with the relevant error stream when the SFTP subsystem is initialized. Note: the current SftpSubsystemimplementation uses asynchronous streams so AsyncCommandErrorStreamAware is the interface that will be invoked. However, in order to support possible future changes it is highly recommended that any custom code implement both interfaces.

Symbolic links handling

Whenever the server needs to execute a command that may behave differently if applied to a symbolic link instead of its target it consults the AbstractSftpSubsystemHelper#resolvePathResolutionFollowLinks method. By default, this method simply consultsthe value of the sftp-auto-follow-links configuration property (default=true).

Note: the property is consulted only for cases where there is no clear indication in the standard how to behave for the specific command. E.g., the lsetstat@openssh.com command specifically specifies that symbolic links should not be followed, so the implementation does not consult the aforementioned property. However, the final decision what LinkOption(s) to use is left to the SftpFileSystemAccessor#resolveFileAccessLinkOptions method (which by default does not interfere in the decision).

Client-side SFTP

In order to obtain an SftpClient instance one needs to use an SftpClientFactory:

try (ClientSession session = ...obtain session...) {
    SftpClientFactory factory = ...obtain factory...
    try (SftpClient client = factory.createSftpClient(session)) {
        ... use the SFTP client...
    }

    // NOTE: session is still alive here...
}

A default client factory implementations is provided in the module - see SftpClientFactory.instance()

If the intended use of the client instance is "one-shot" - i.e., the client session should be closed when the SFTP client instance is closed, then it is possible to obtain a special wrapper that implements this functionality:

// The underlying session will also be closed when the client is
try (SftpClient client = createSftpClient(....)) {
    ... use the SFTP client...
}

SftpClient createSftpClient(...) {
    ClientSession session = ...obtain session...
    SftpClientFactory factory = ...obtain factory...
    SftpClient client = factory.createSftpClient(session);
    return client.singleSessionInstance();
}

Using a custom SftpClientFactory

The code creates SftpClient-s and SftpFileSystem-s using a default built-in SftpClientFactory instance (see DefaultSftpClientFactory). Users may choose to use a custom factory in order to provide their own implementations - e.g., in order to override some default behavior - e.g.:

SshClient client = ... setup client...

try (ClientSession session = client.connect(user, host, port).verify(timeout).getSession()) {
    session.addPasswordIdentity(password);
    session.auth.verify(timeout);

    // User-specific factory
    try (SftpClient sftp = MySpecialSessionSftpClientFactory.INSTANCE.createSftpClient(session)) {
        ... instance created through SpecialSessionSftpClientFactory ...
    }
}

Version selection via SftpVersionSelector

The SFTP subsystem code supports versions 3-6 (inclusive), and by default attempts to negotiate the highest possible one - on both client and server code. The user can intervene and force a specific version or a narrower range.

SftpVersionSelector myVersionSelector = new SftpVersionSelector() {
    @Override
    public int selectVersion(ClientSession session, boolean initial, int current, List<Integer> available) {
        int selectedVersion = ...run some logic to decide...;
        return selectedVersion;
    }
};

try (ClientSession session = client.connect(user, host, port).verify(timeout).getSession()) {
    session.addPasswordIdentity(password);
    session.auth.verify(timeout);

    SftpClientFactory factory = SftpClientFactory.instance();
    try (SftpClient sftp = factory.createSftpClient(session, myVersionSelector)) {
        ... do SFTP related stuff...
    }
}

Note: the version selector is invoked twice - the first time in order to retrieve the initial version to be used when estabilishing the SFTP channel, and the second after having done so after receiving the server's version. The invocations are distinguished by the initial parameter value.

On the server side, version selection restriction is more complex - please remember that according to the protocol specification

The server responds with a SSH_FXP_VERSION packet, supplying the lowest (!) of its own and the client's version number

Currently at the server we support requiring a specific version via the SftpSubsystem#SFTP_VERSION configuration key. The same can be achieved for the CLI SSHD code by specifying -o sftp-version=N option.

For more advanced restrictions one needs to sub-class SftpSubSystem and provide a non-default SftpSubsystemFactory that uses the sub-classed code.

Intercepting data sent via STDERR channel data from the server

According to SFTP version 4 - section 3.1 the server MAY send error data through the STDERR pipeline. By default, the code ignores such data - however, users may register a SftpErrorDataHandler that will be invoked whenever such data is received from the server.

ClientSession session = ...establish a session...
SftpClientFactory factory = ...obtain a factory instance...

try (SftpClient client = factory.createSftpClient(session, new MySftpErrorDataHandler())) {
   ...
}

The same applies to the SftpFileSystem - users may provide a custom error data handler that will be invoked whenever such data is received from the server.

Note:

  • Error data handling must be short or it will cause the SSH session to hang - any long/blocking processing must be done in a separate thread.
  • The provided data buffer contents must be copied if they need to be used after the callback returns as the buffer contents might be re-used by the caller code.
  • Any exception thrown during handling of the data will cause the SFTP session to terminate.

Using SftpFileSystemProvider to create an SftpFileSystem

The code automatically registers the SftpFileSystemProvider as the handler for sftp:// URL(s). Such URLs are interpreted as remote file locations and automatically exposed to the user as Path objects. In effect, this allows the code to "mount" a remote directory via SFTP and treat it as if it were local using standard java.nio calls like any "ordinary" file system.

// "Mounting" a file system
URI uri = SftpFileSystemProvider.createFileSystemURI(host, port, username, password);
try (FileSystem fs = FileSystems.newFileSystem(uri, Collections.<String, Object>emptyMap())) {
    Path remotePath = fs.getPath("/some/remote/path");
    ...
}

// Full programmatic control
SshClient client = ...setup and start the SshClient instance...
SftpFileSystemProvider provider = new SftpFileSystemProvider(client);
URI uri = SftpFileSystemProvider.createFileSystemURI(host, port, username, password);
try (FileSystem fs = provider.newFileSystem(uri, Collections.<String, Object>emptyMap())) {
    Path remotePath = fs.getPath("/some/remote/path");
}

The obtained Path instance can be used in exactly the same way as any other "regular" one:

try (InputStream input = Files.newInputStream(remotePath)) {
   ...read from remote file...
}

try (DirectoryStream<Path> ds = Files.newDirectoryStream(remoteDir)) {
   for (Path remoteFile : ds) {
       if (Files.isRegularFile(remoteFile)) {
           System.out.println("Delete " + remoteFile + " size=" + Files.size(remoteFile));
           Files.delete(remoteFile);
       } else if (Files.isDirectory(remoteFile)) {
           System.out.println(remoteFile + " - directory");
       }
   }
}

It is highly recommended to close() the mounted file system once no longer necessary in order to release the associated SFTP session sooner rather than later - e.g., via a try-with-resource code block.

Caveat: Due to URI encoding of the username/password as a basic authentication, the system currently does not allow colon (:) in either one in order to avoid parsing confusion. See RFC 3986 - section 3.2.1:

Use of the format "user:password" in the userinfo field is deprecated ... Applications may choose to ignore or reject such data when it is received as part of a reference...

Configuring the SftpFileSystemProvider

When "mounting" a new file system one can provide extra configuration parameters using either the environment map in the FileSystems#newFileSystem method or via the URI query parameters. See the SftpFileSystemProvider for the available configuration keys and values.

// Using explicit parameters
Map<String, Object> params = new HashMap<>();
params.put("param1", value1);
params.put("param2", value2);
...etc...

URI uri = SftpFileSystemProvider.createFileSystemURI(host, port, username, password);
try (FileSystem fs = FileSystems.newFileSystem(uri, params)) {
    Path remotePath = fs.getPath("/some/remote/path");
    ... work with the remote path...
}

// Using URI parameters
Path remotePath = Paths.get(new URI("sftp://user:password@host/some/remote/path?param1=value1&param2=value2..."));
// Releasing the file-system once no longer necessary
try (FileSystem fs = remotePath.getFileSystem()) {
    ... work with the remote path...
}

Note: if both options are used then the URI parameters override the environment ones

Map<String, Object> params = new HashMap<>();
params.put("param1", value1);
params.put("param2", value2);

// The value of 'param1' is overridden in the URI
try (FileSystem fs = FileSystems.newFileSystem(
        new URI("sftp://user:password@host/some/remote/path?param1=otherValue1", params)) {
    Path remotePath = fs.getPath("/some/remote/path");
    ... work with the remote path...
}

Configuring the client session used to create an SftpFileSystem

It is possible to register a SftpFileSystemClientSessionInitializer with the provider instead of the default one and thus better control the ClientSession used to generate the file-system instance. The default implementation simply connects and authenticates before creating a default SftpFileSystem instance. Users may wish to override some options or provide their own - e.g., execute a password-less authentication instead of the (default) password-based one:

SftpFileSystemProvider provider = ... obtain/create a provider ...
provider.setSftpFileSystemClientSessionInitializer(new SftpFileSystemClientSessionInitializer() {
    @Override
    public void authenticateClientSession(
            SftpFileSystemProvider provider, SftpFileSystemInitializationContext context, ClientSession session)
                throws IOException {
        /*
         * Set up password-less login instead of password-based using the specified key
         *
         * Note: if SSH client and/or session already have a KeyPairProvider set up and the code
         * knows that these keys are already registered with the remote server, then no need to
         * add the public key identitiy - can simply call sesssion.auth().verify(context.getMaxAuthTime()).
         */
        KeyPair kp = ... obtain a registered key-pair...
        session.addPublicKeyIdentity(kp);
        sesssion.auth().verify(context.getMaxAuthTime());
    }
});

SFTP sent/received names encoding

By default, the SFTP client uses UTF-8 to encode/decode any referenced file/folder name. However, some servers do not properly encode such names, and thus the "visible" names by the client become corrupted, or even worse - cause an exception upon decoding attempt. The SftpClient exposes a get/setNameDecodingCharset method which enables the user to modify the charset - even while the SFTP session is in progress - e.g.:

try (SftpClient client = ...obtain an instance...) {
    client.setNameDecodingCharset(Charset.forName("ISO-8859-8"));
    for (DirEntry entry : client.readDir(...some path...)) {
        ...handle entry assuming ISO-8859-8 encoded names...
    }

    client.setNameDecodingCharset(Charset.forName("ISO-8859-4"));
    for (DirEntry entry : client.readDir(...some other path...)) {
        ...handle entry assuming ISO-8859-4 encoded names...
    }
}

The initial charset can be pre-configured on the client/session by using the sftp-name-decoding-charset property - if none specified then UTF-8 is used. Note: the value can be a charset name or a java.nio.charset.Charset instance - e.g.:

SshClient client = ... setup/obtain an instance...
// default for ALL SFTP clients obtained through this client
PropertyResolverUtils.updateProperty(client, SftpModuleProperties.NAME_DECODING_CHARSET.getName(), "ISO-8859-8");

try (ClientSession session = client.connect(...)) {
    session.addPasswordIdentity(password);
    session.auth().verify(timeout);

    // default for ALL SFTP clients obtained through the session - overrides client setting
    PropertyResolverUtils.updateProperty(session, SftpModuleProperties.NAME_DECODING_CHARSET.getName(), "ISO-8859-4");

    SftpClientFactory factory = SftpClientFactory.instance();
    try (SftpClient sftp = factory.createSftpClient(session)) {
        for (DirEntry entry : sftp.readDir(...some path...)) {
            ...handle entry assuming ISO-8859-4 (inherited from the session) encoded names...
        }

        // override the inherited default from the session
        sftp.setNameDecodingCharset(Charset.forName("ISO-8859-1"));

        for (DirEntry entry : sftp.readDir(...some other path...)) {
            ...handle entry assuming ISO-8859-1 encoded names...
        }
    }
}

On the server side, one can use the SftpFileSystemAccessor#putRemoteFileName to encode the returned file name/path using non-UTF8 encoding. However, this might break clients that expect UTF-8 - i.e., as long as both the client and server are somehow "aligned" on the encoding being used it will work. In this context, one might also need to consider implementing the filename-charset , filename-translation-control extensions as described in DRAFT 13 - section 6 on the server side - though it is not supported out-of-the-box in version 3 (which what most clients run).

Listing SFTP directories

Listing directories can be done in Java in various ways. With the Java NIO framework, a common approach is

public void processDirectory(Path directoryPath, Consumer<Path> process) throws IOException {
  try (DirectoryStream<Path> dir = Files.newDirectoryStream(directoryPath)) {
    for (Path path : dir) {
      process.accept(path); // Do whatever needs to be done with 'path'
    }
  }
}

This also works fine if the Path is an SftpPath obtained from an SftpFileSystem. But what if you also need the file attributes ?

Again, in plain Java NIO, one might do

public void processDirectory(Path directoryPath, BiConsumer<Path, BasicFileAttributes> process) throws IOException {
  try (DirectoryStream<Path> dir = Files.newDirectoryStream(directoryPath)) {
    for (Path path : dir) {
      BasicFileAttributes attributes = Files.readAttributes(path, BasicFileAttributes.class);
      process.accept(path, attributes);
    }
  }
}

This gets all the Paths of the files inside the directory, then reads their attributes one-by-one. On Unix, there is variation using Files.walkFileTree that may have much better performance:

public void processDirectory(Path directoryPath, BiConsumer<Path, BasicFileAttributes> process) throws IOException {
  Files.walkFileTree(directoryPath, EnumSet.noneOf(FileVisitOption.class), 1,
      new SimpleFileVisitor<Path>() {

          @Override
          public FileVisitResult visitFile(Path path, BasicFileAttributes attributes) {
            // Beware this is also called for the directory itself
            process.accept(path, attributes);
            return FileVisitResult.CONTINUE;
          }
      });
}

This typically performs better on Unix because the file system can deliver the file attributes together with the paths, and the standard Java implementation of FileTreeWalker takes advantage of this. On Windows you'll typically not see any improvement because the file system stores attributes differently and has to fetch them extra anyway.

This is important when remote file systems come into play. With an SftpFileSystem, the call to Files.readAttributes() is a remote call to the SFTP server, hence it's an expensive operation. Thus the first variant is slow, which may make processing a directory with many files excruciatingly slow.

SFTP has a directory model similar to Unix: a request for a directory listing always returns the file names and the file attributes. But Java's FileVisitor doesn't know this, and doesn't know about SftpFileSystem at all -- it's just a normal java.nio.file.FileSystem for it. Hence it doesn't use its internal optimization for Unix file systems and instead also calls Files.readAttributes() for each file under the hood. This makes the second variant also slow with an SftpFileSystem.

To get paths and attributes in an efficient way from an SftpFileSystem, one has to bypass the FileSystem abstraction. One can do it all manually using SFTP commands directly:

public void processSftpDirectory(SftpPath directoryPath, BiConsumer<Path, SftpClient.Attributes> process) throws IOException {
  try (SftpClient client = directoryPath.getFileSystem().getClient()) {
    for (SftpClient.DirEntry directoryEntry : client.readDir(directoryPath.toString()) {
      SftpClient.Attributes attributes = directoryEntry.getAttributes();
      String file = directoryEntry.getFilename();
      if (".".equals(file)) {
        // The directory itself.
        process(directoryPath, attributes);
      } else if ("..".equals(file)) {
        // The parent directory, if any
        process(directoryPath.getParent(), attributes);
      } else {
        process(directoryPath.resolve(file), attributes);
      }
    }
  }
}

Alternatively, one can also use the fact that Apache MINA sshd caches the SFTP file attributes it received from the server on the SftpPath objects it returns from a DirectoryStream:

public void processSftpDirectory(SftpPath directoryPath, BiConsumer<Path, SftpClient.Attributes> process) throws IOException {
  try (DirectoryStream<Path> dir = Files.newDirectoryStream(directoryPath)) {
    for (Path path : dir) {
      if (path instanceof SftpPath) {
        SftpClient.Attributes attributes = ((SftpPath) path).getAttributes();
        process.accept(path, attributes);
      } else {
        // A DirectoryStream on a directory given by an SftpPath always returns SftpPath instances as elements.
        throw new IllegalStateException("Path " + path + " has unexpected type " + path.getClass().getName());
      }
    }
  }
}

In either case, the behavior is undefined if files in the directory or their attributes are changed during the iteration.

File attributes, whether cached or not, and irrespective of the type of FileSystem, represent a snapshot taken at the time they were obtained; they do not reflect any changes made to the files after that. The cached attributes Apache MINA sshd provides through the SftpFileSystem reflect the state when the directory listing was obtained. Calling Files.readAttributes() on an SftpPath always is a remote call fetching a fresh snapshot.

Note: the same is true for any other operation that implicitly calls Files.readAttributes(), like Files.size(), Files.exists(), Files.getOwner(), Files.isDirectory(), and so on.

Even if an SftpFileSystem fulfills the general contract of a FileSystem, a client still has to be aware that it is a remote file system that may have quite different performance characteristics than a local file system.

SFTP aware directory scanners

The framework provides special SFTP aware directory scanners that look for files/folders matching specific patterns. The scanners support recursive scanning of the directories based on the selected patterns.

E.g. - let's assume the layout present below

    root
      + --- a1.txt
      + --- a2.csv
      + sub1
         +--- b1.txt
         +--- b2.csv
      + sub2
         + --- c1.txt
         + --- c2.csv

Then scan results from root are expected as follows for the given patterns

  • "**/*" - all the files/folders - [a1.txt, a2.csv, sub1, sub2, b1.txt, b2.csv, c1.txt, c2.csv]
  • "**/*.txt" - only the ".txt" files - [a1.txt, b1.txt, c1.txt]
  • "*" - only the files/folders at the root - [a1.txt, a2.csv, sub1, sub2]
  • "*.csv" - only a2.csv at the root

Note: the scanner supports various patterns - including regex - see DirectoryScanner and SelectorUtils classes for supported patterns and matching - include case sensitive vs. insensitive match.

// Using an SftpPathDirectoryScanner
FileSystem fs = ... obtain an SFTP file system instance ...
Path basedir = fs.getPath("/some/remote/path");
DirectoryScanner ds = new SftpPathDirectoryScanner(basedir, ...pattern...);
Collection<Path> matches = ds.scan();

// Using an SftpClientDirectoryScanner
SftpClient client = ... obtain a client instance ...
Strinng basedir = "/some/remote/path";
SftpClientDirectoryScanner ds = new SftpClientDirectoryScanner(basedir, ...pattern...);
Collection<ScanDirEntry> matches = ds.scan(client);

Extensions & custom file/folder attributes

Extending the SFTP protocol and/or the reported file/folder attributes

SFTP protocol extensions

Both client and server support several of the SFTP extensions specified in various drafts:

Furthermore several OpenSSH SFTP extensions are also supported:

  • fsync@openssh.com
  • fstatvfs@openssh.com
  • hardlink@openssh.com
  • posix-rename@openssh.com - only client side
  • statvfs@openssh.com
  • lsetstat@openssh.com
  • limits@openssh.com

On the server side, the reported standard extensions are configured via the SftpModuleProperties.CLIENT_EXTENSIONS configuration key, and the OpenSSH ones via the SftpModuleProperties.OPENSSH_EXTENSIONS.

On the client side, all the supported extensions are classes that implement SftpClientExtension. These classes can be used to query the client whether the remote server supports the specific extension and then obtain a parser for its contents. Users can easily add support for more extensions in a similar manner as the existing ones by implementing an appropriate ExtensionParser and then registering it at the ParserUtils - see the existing ones for details how this can be achieved.

// properietary/special extension parser
ParserUtils.registerExtension(new MySpecialExtension());

try (ClientSession session = client.connect(username, host, port).verify(timeout).getSession()) {
    session.addPasswordIdentity(password);
    session.auth().verify(timeout);

    SftpClientFactory factory = SftpClientFactory.instance();
    try (SftpClient sftp = factory.createSftpClient(session)) {
        Map<String, byte[]> extensions = sftp.getServerExtensions();
        // Key=extension name, value=registered parser instance
        Map<String, ?> data = ParserUtils.parse(extensions);
        for (Map.Entry<String, ?> de : data.entrySet()) {
            String extName = de.getKey();
            Object extValue = de.getValue();
            if (SftpConstants.EXT_ACL_SUPPORTED.equalsIgnoreCase(extName)) {
                AclCapabilities capabilities = (AclCapabilities) extValue;
                ...see what other information can be gleaned from it...
            } else if (SftpConstants.EXT_VERSIONS.equalsIgnoreCase(extName)) {
                Versions versions = (Versions) extValue;
                ...see what other information can be gleaned from it...
            } else if ("my-special-extension".equalsIgnoreCase(extName)) {
                MySpecialExtension special = (MySpecialExtension) extValue;
                ...see what other information can be gleaned from it...
            } // ...etc....
        }
    }
}

One can skip all the conditional code if a specific known extension is required:

try (ClientSession session = client.connect(username, host, port).verify(timeout).getSession()) {
    session.addPasswordIdentity(password);
    session.auth().verify(timeout);

    SftpClientFactory factory = SftpClientFactory.instance();
    try (SftpClient sftp = factory.createSftpClient(session)) {
        // Returns null if extension is not supported by remote server
        SpaceAvailableExtension space = sftp.getExtension(SpaceAvailableExtension.class);
        if (space != null) {
            ...use it...
        }
    }
}

Contributing support for a new SFTP extension

  • Add the code to handle the new extension in AbstractSftpSubsystemHelper#executeExtendedCommand

  • Declare the extension name in DEFAULT_SUPPORTED_CLIENT_EXTENSIONS or DEFAULT_OPEN_SSH_EXTENSIONS (same class) - according to the extension type (generic or OpenSSH one).

  • In the org.apache.sshd.sftp.client.extensions.helpers package implement an extension of AbstractSftpClientExtension for sending and receiving the newly added extension.

  • Add a relevant parser for reported extension data initial report (if necessary) in ParserUtils#BUILT_IN_PARSERS

See how other extensions are implemented and follow their example

Providing/processing file/folder custom attributes

According to SFTP - File Attributes it is possible to provide custom attributes for a referenced file/folder. The client-side code supports this via the Attributes#getExtensions call. On the server-side one needs to provide a custom SftpFileSystemAccessor that overrides the resolveReportedFileAttributes method (which by default simply returns the original attrbiutes as-is. A similar hook method has been provided in case a client attempts to apply custom attributes - simply need to provide a implementation that obverrides applyExtensionFileAttributes (which by default ignores the attributes).

class MyCustomSftpFileSystemAccessor implements SftpFileSystemAccessor {
    @Override
    public NavigableMap<String, Object> resolveReportedFileAttributes(
            SftpSubsystemProxy subsystem, Path file, int flags, NavigableMap<String, Object> attrs, LinkOption... options)
                throws IOException {
        Map<String, Object> extra = (Map<String, Object>) attrs.get(IoUtils.EXTENDED_VIEW_ATTR);
        if (extra == null) {
            extra = new HashMap<>();
            attrs.put(IoUtils.EXTENDED_VIEW_ATTR, extra)
        }
        extra.put("custom1", ...some string...);
        extra.put("custom", ...some byte[]...)
    }
    
    @Override
    public void applyExtensionFileAttributes(
            SftpSubsystemProxy subsystem, Path file, Map<String, byte[]> extensions, LinkOption... options)
                throws IOException {
        if (MapEntryUtils.isNotEmpty(extensions)) {
            ...process the extensions...
        }
    }
}

SftpSubsystemFactory factory = new SftpSubsystemFactory.Builder()
    .withFileSystemAccessor(new MyCustomSftpFileSystemAccessor())
    .build();

SshdServer sshd = ...setup...
sshd.setSubsystemFactories(Collections.singletonList(factory));

Note:

  • The code assumes that the extension name is a string - the draft specification actually allows an array of bytes as well, but we chose simplicity.

  • The value can be either a string or an array of bytes. If the value is neither (e.g., an integer) then the value's toString() will be used.

References