Command Interface
The server can receive the fallowing commands. Every command is formed by a name and optional arguments. The double quote of the command arguments in the examples, as in the return status examples, are not part of the command syntax, it is just for reading convenience.
UNKNOWN
This is the response sent by the server if the command is not recognized.
END command is sent by the client to the server in order to terminate the TCP connection.
[Supports TFPSFS]
ECHO command is for debug propose. Once sent the server replays back the exact same thing the client sent, including the ECHO word.
[Supports TFPSFS]
MKDIR command creates a directory at the specified path. The return status of this command could be:
OK
FAILED 4 : Directory already exist.
FAILED 1 : Access denied to location.
FAILED 30 : Directory is locked.
If the parameter is left empty the daemon protocol directory will be used. So it responds the FAILED 4 return status.
- Secured Directory
This feature allows the creation of a kind of directory that cannot be listed with any command. In order to access it, the name must be known in advance. The directory name may contain any character, except the decimal zero “0” and the path separator “/”, and must end with an “.sd” extension. The length of the directory name must not exceed 255 characters including the extension.
In order to create strong secured directories, the following technique could be used: Combine a user ́s name with a very strong password and apply to them the SHA256 or any other secure hash function.
The resulting hash must be scanned to remove or change from it any occurrence of the decimal zero ‘0’ -the null character- or the path separator ‘/’. If the hash is converted to hexadecimal format -which is usual- , it is guaranteed that the resulting string will contain only numbers and letters. If more security is needed, this process can be repeated recursively.
[Supports TFPSFS]
DEL command deletes the specified file. This is a special command because is the only one who can operate in a locked directory. If it is the case, then DEL can only delete exactly one file: The locking file. The return status of this command could be:
** OK **
** FAILED 1 : Access denied to location. **
** FAILED 9 : File does not exist. **
** FAILED 8 : Requested file is a directory. **
** FAILED 30 : Directory is locked. **
** FAILED **
If used without an argument the protocol daemon directory is used. So it return the FAILED 8 return status.
[Supports TFPSFS]
RMDIR removes a specified directory recursively. The return status of this command could be:
** OK **
** FAILED 1 : Access denied to location. **
** FAILED 30 : Directory is locked. **
** FAILED 15 : Directory to remove still exist. **
If “/path/to/dir/to/remove” has secure directories inside of it, they will be not removed.
If used without an argument the protocol daemon directory is used. So it returns the FAILED 15 return status due the impossibility to delete the daemon directory. However, every other directory will be deleted.
[Supports TFPSFS]
COPY copy the file indicated by the first parameter of the command to the file indicated by the second one. The return status of this command could be:
** OK **
** FAILED 16 : Missing parameter from command. **
** FAILED 1 : Access denied to location. **
** FAILED 19 : Directory can't be linked. **
** FAILED 12 : File already exist. **
** FAILED 17 : Source file does not exist. **
** FAILED 18 : Copy general error. **
** FAILED 30 : Directory is locked. **
TOUCH creates a new file in the specified directory. The return status of this command could be:
** OK **
** FAILED 1 : Access denied to location. **
** FAILED 12 : File already exist. **
** FAILED 24 : Error creating new file. **
** FAILED 30 : Directory is locked. **
If used without an argument the protocol daemon directory is used. So it return the FAILED 12 return status.
DATE returns the number of elapsed seconds since the epoch, and arbitrary point in the time continuum, which is the Gregorian calendar time Jan 1 1970 00:00 UTC. The return status of this command could be:
** OK “timestamp” **
UDATE returns the number of elapsed seconds and microseconds since the epoch -separated by dot-, and arbitrary point in the time continuum, which is the Gregorian calendar time Jan 1 1970 00:00 UTC. The return status of this command could be:
** OK “seconds.microseconds” **
NDATE returns the number of elapsed seconds and nanoseconds since the epoch -separated by dot-, and arbitrary point in the time continuum, which is the Gregorian calendar time Jan 1 1970 00:00 UTC. The return status of this command could be:
** OK “seconds.nanoseconds” **
DATEF returns the current date of the server in human readable format “yyyy-mm-dd HH:MM:SS” UTC. The return status of this command could be:
** OK “yyyy-mm-dd HH:MM:SS” **
DTOF converts date in Unix timestamp format -seconds since the epoch- in human readable format “yyyy-mm-dd HH:MM:SS” UTC.
The return status of this command could be:
** OK “yyyy-mm-dd HH:MM:SS” **
** FAILED 16 : Missing parameter from command. ** ** FAILED 26 : Date is not representable. **
FTOD converts date in human readable format “ yyyy-mm-dd HH:MM:SS” to its Unix timestamp format. The return status of this command could be:
** OK “timestamp” **
** FAILED 16 : Missing parameter from command. **
[Supports TFPSFS]
FSTAT returns statistics of a file or directory in the form "D | F | U FILE-SIZE LAST-ACCESS LAST-MODIFICATION" where D stands for directory; F stands for file; and U stands for unknown, only one of them is reported. The “FILE-SIZE” is reported in bytes in a integer of 64 bits and the number could be up to 20 digits long.
The “LAST-ACCESS” and “LAST-MODIFICATION” are both timestamps. The return status of this command could be:
** OK “D | F | U FILE-SIZE LAST-ACCESS LAST-MODIFICATION” **
** FAILED 16 : Missing parameter from command. **
** FAILED 1 : Access denied to location. **
** FAILED 9 : File does not exist. **
** FAILED 30 : Directory is locked. **
Supports TFPSFS
FUPD update the timestamps of the file or directory to server current time. The return status could be:
** OK ** ** FAILED 16 : Missing parameter from command. ** ** FAILED 1 : Access denied to location. ** ** FAILED 9 : File does not exist. ** ** FAILED 27 : Timestamp updating failed. ** ** FAILED 30 : Directory is locked. **
Supports TFPSFS
CPDIR copy recursively the source directory into a new created directory specified in the second parameter of the command. The return status of this command could be:
** OK **
** FAILED 16 : Missing parameter from command. **
** FAILED 1 : Access denied to location. **
** FAILED 20 : Source path is not a directory. **
** FAILED 4 : Directory already exist. **
** FAILED 21 : Error replicating directory tree. **
** FAILED 30 : Directory is locked. **
XCOPY copies the source file specified in the second parameter into every directory in the tree that matches “pattern” with the name “newname” specified in the first parameter.
This is a best-effort command instead of super-reliable one. If any one of the destinations are locked or the file exists instead of return a failed status code, XCOPY skips it silently. The return status of this command are relative to the source file/directory. The return status of this command could be:
** OK **
** FAILED 16 : Missing parameter from command. **
** FAILED 1 : Access denied to location. **
** FAILED 17 : Source file does not exist. **
** FAILED 23 : Source path is not a file. **
** FAILED 30 : Directory is locked. **
XDEL deletes all files that match “filename” in the specified path at first parameter and it does recursively starting at the specified directory.
This is a best-effort command instead of super-reliable one. If any one of the path are locked or even and error occurs, instead of return a failure return status, XDEL skips it silently. The return status of this command are relative to the path specified.
The return status of this command could be:
** OK **
** FAILED 16 : Missing parameter from command. **
** FAILED 1 : Access denied to location. **
** FAILED 20 : Source path is not a directory. **
** FAILED 30 : Directory is locked. **
XRMDIR deletes all directories recursively that match “directory- name” starting at the specified path at first parameter. This is a best-effort command instead of super-reliable one. If any one of the path are locked or even an error occurs, instead of return a failure return status, XRMDIR skips it silently. The return status of this command are relative to the path specified. The return status of this command could be:
** OK **
** FAILED 16 : Missing parameter from command. **
** FAILED 1 : Access denied to location. **
** FAILED 10 : Directory does not exist. **
** FAILED 20 : Source path is not a directory. **
** FAILED 30 : Directory is locked. **
XCPDIR copy recursively source directory specified in the second parameter to every directory in the tree that matches “destination- directory-pattern” with the name “new-directory-name” specified at first parameter.
This is a best-effort command instead of super-reliable one. If any one of the destinations are locked or the directory or file exists, instead of return a failure return code, XCPDIR skips it silently. The return status of this command are relative to the source file/directory The return status of this command could be:
** OK **
** FAILED 16 : Missing parameter from command. **
** FAILED 1 : Access denied to location. **
** FAILED 20 : Source path is not a directory. **
** FAILED 29 : Source path does not exist. **
** FAILED 30 : Directory is locked. **
LOCK specifies a filename that when it exists in a directory no operation can be done on it, except to delete the lock file. This allows a temporary blocking of a directory for any sort of operations except the one eliminating the locking file which is the command DEL with the path to the locking file. The above statement does not apply for commands that do not specify “FAILED 30” as possible return status. The return status of this command could be:
** OK **
Specify LOCK with an empty string “” or without argument is the way to turn off the locking mechanism regardless of how many locking files are installed in the directories. The notification system is not affected by the locking mechanism.
This is not a synchronization mechanism. A synchronization mechanism eliminates race conditions, LOCK does not. Don’t try to use it as such, there is a lot of windows in between. Even it is possible that some operations are completed successfully in the directory by other clients after it has been locked. The only way the client realizes of this is by comparing the files and directories timestamps against the lock file timestamp. If the lock file and any other file or directory in the locked directory has the same timestamp the client can decide whether to accept that file or directory as valid one.
Supports TFPSFS
SNDFILE sends a file to the server. The first parameter could be either “0” for false or “1” for true. If “/path/to/filename” it exists in the server, this flag indicates whether should be overwritten or not. Once the transmission loop starts there are three keywords to continue or stop the flow: ** CONT ** ** BREAK ** and ** OK ** . The ** CONT ** keyword followed by a whitespace “ ” and the payload tells to the server that the chunk of data is part of file. The ** BREAK ** keyword tells to the server to abort the operation and it deletes the already stored data. The ** OK ** keyword tells to the server that there is no more data to be send. Every time the server receives a chunk of data it responds with the flow control keyword ** CONT ** or and status returns in case of a failure. When the SNDFILE command is sent the server responds with ** CONT ** flow control to signal that is ready for receiving the file.
The return status of this command could be:
** CONT “To indicate the willingness to continue receiving the file” **
** OK “In response to the OK sent by the client signaling the EOF -end of file- condition or the BREAK to tell the server to stop receiving” **
** FAILED 13 : Quota exceeded. **
** FAILED 16 : Missing parameter from command. ** ** FAILED 1 : Access denied to location. ** ** FAILED 12 : File already exist. ** ** FAILED 8 : Requested file is a directory. ** ** FAILED 7 : File transfer API error. ** ** FAILED 30 : Directory is locked. **
As an illustration of the dynamic of this command we can use this example:
CLIENT SEND: ** SNDFILE 0 /path/to/new/file **
SERVER RESPONDS: ** CONT **
CLIENT SEND: ** CONT PAYLOAD-OF-DATA **
“This process repeats until the server returns and failure status or the client sends BREAK to stop the flow or OK to indicate that the operation is done. In this example the first parameter which is 0 tells to the server that if the /path/to/new/file exists must not be overwritten”. The maximum number of bytes for payload is BUFSZ
- CONT word and the whitespace. If buffer size is 512, then the payload cannot exceed 507 bytes. As benchmarks have been made we suggest that this command should be used only to transfer small files ‘100 MB or less’. In a1Gbps network testing the operation is completed in eleven seconds for a 100 MB file.
RCVFILE “0 | 1” “/path/to/filename”
Supports TFPSFS
RCVFILE receives a file from the server. The first parameter could be either “0” for false or “1” for true and tells the server whether the file must be deleted after successfully received by the client. Once the transmission loop starts there are three keywords to continue or stop the flow: CONT BREAK and OK. The CONT keyword followed by a whitespace “ ” and the payload tells to the client that the chunk of data is part of file. The BREAK keyword tells to the server to abort the operation. The OK keyword tells to the client that there is no more data to receive. Every time the client receives a chunk of data it responds with the flow control keyword CONT or BREAK to stop the operation. When the RCVFILE command is sent the server responds with CONT flow control and the payload or a return status for failure.
The return status of this command could be:
** CONT PAYLOAD-OF-DATA **
** OK “In response to the BREAK sent by the client or to signal the EOF -end of file- condition indicating that the operation is done.” **
** FAILED 16 : Missing parameter from command. **
** FAILED 1 : Access denied to location. **
** FAILED 9 : File does not exist. **
** FAILED 8 : Requested file is a directory. **
** FAILED 7 : File transfer API error. **
** FAILED 30 : Directory is locked. **
As an illustration of the dynamic of this command we can use this example:
CLIENT SEND: ** RCVFILE 0 /path/to/filename ** SERVER RESPONDS: ** CONT PAYLOAD-OF-DATA ** CLIENT SEND: ** CONT **
“This process repeats until the server returns OK for -end of file- condition or a failure status, or the client sends BREAK to stop the flow. In this example the first parameter which is 0 tells to the server that /path//to/filename must not be deleted after the operation is done”. The maximum number of bytes for payload is BUFSZ - CONT word and the whitespace. If buffer size is 512, then the payload cannot exceed 507 bytes. As benchmarks have been made we suggest that this command be used only to transfer small files ‘100 MB or less’. In a1Gbps network testing the operation is completed in eleven seconds for a 100 MB file.
Supports TFPSFS
LS command list the directory entries for the indicated path, if the argument is missing, it lists the root directory of the protocol daemon. The return value of this command is a file with the listed content. In fact it is like issuing the command RCVFILE to a temporary file with the listed content of the directory. The file returned by LS has the following syntax.
** F | D | U /path/to/file-or-directory **
The paths are relative -“path/to/directory/to/list” is omitted-. The F stands for “file”; the D for “directory” and the U for “unknown”. The return status of this command beside those for RCVFILE because RCVFILE works on behalf LS could be:
** FAILED 1 : Access denied to location. **
** FAILED 30 : Directory is locked. **
** FAILED 10 : Directory does not exist. **
** FAILED 11 : Failed to make a temporary file. **
Supports TFPSFS
This command is exactly like LS but it lists recursively the specified directory.
Supports TFPSFS
RENAM renames the file or directory specified at first parameter into the name specified at second parameter. RENAM operates atomically; there is no instant at which “newname” is non-existent between the operation’s steps if “newname” already exists. If a system crash occurs, it is possible for both names “oldname” and “newname” to still exist, but “newname” will be intact.
RENAM has some restrictions to operate.
-
“oldname” it must exist.
-
If “newname” is a directory must be empty.
-
If “oldname” is a directory then “newname” must not exist or it has to be an empty directory.
-
The “newname” must not specify a subdirectory of the directory “oldname” which is being renamed.
KEEPALIVE sets the configuration parameters for the TCP keepalive feature. This is especially useful for clients behind NAT boxes. If there is some idle time in the established connection -no data transmission- the NAT box could close or unset the connection without the peers knowing it. In contexts where it is predictable that an established connection could be ‘in silent’ for long periods of time, and it is possible that clients are behind NAT boxes, it is necessary to set the TCP keepalive packets.
The first parameter of the command could be 0 or 1, meaning on or off.
The second parameter is the time (in seconds) the connection needs to remain idle before TCP starts sending keepalive probes.
The third parameter is the time (in seconds) between individual keepalive probes.
The fourth parameter is the maximum number of keepalive probes TCP should send before dropping the connection.
The return status of this command could be:
** OK **
** FAILED 32: Failed to set TCP keepalive parameters. **
Whether the server implements a default timeout of keepalive to declare a connection dead, it is up to the implementation. In order to be sure that keepalive feature is turned off or on, it must be done explicitly.
There are two more commands ADDNTFY and STARTNTFY. We will see them in the next section.
PROCKEY retrieves a unique key generated by the server’s instance that communicates with the client. This unique key could be used later to identify that instance. One of these uses, but not the only one, is to test whether the server or even the socket communication line is still opened, in other words: the keepalive mechanism from the client side perspective.
The return status of this command could be:
** OK **
** FAILED 33 : Process without unique key. **
FREESP retrieves the available space in the partition where the protocol folder is located. This command should be used only after a failure to find out if such a failure is due to the lack of space. It should not be used to guess how much space remains, and then proceed with a writing operation. The described scenario could potentially lead to a race condition. If both clients at the same time retrieve the free space and then write according to that value, one of them will fail.
The return status of this command could be:
** OK “free-space” **
“free-space” will be a 64bit signed integer indicating how much space left -in bytes- in the protocol partition directory at the time of the command execution.
Supports TFPSFS
struct hpfile {
uint64_t offst;
}int64_t;
bufsz;PUT is part of the High Performance File Interface. It can be used to transfer files from the client to the server.
The first parameter is the file to be written, it could exist or not. If the file does not already exist, then it is created. If the file already exists, the command will overwrite it.
The second parameter goes after the “/path/to/filename” and a whitespace. It consists of two 64bit integers normalized ‘converted to network byte-order’. The first one should be unsigned and the second one, signed. The first one must range from 0 to 2^64-1. The second one must range from -(2^63) to 2^63-1.
The “offst” integer -the first one- is the desire offset in which the data should be written in the opened file, being a new file or an existing one.
The “bufsz” integer -the second one- is the proposed buffer size by the client.
Once the command is constructed like above, it can be sent to the server. On the server side, the protocol will evaluate the parameters and will return one of the following conditions:
** OK “definitive-buffer” **
** FAILED 1 : Access denied to location. **
** FAILED 16 : Missing parameter from command. **
** FAILED 34 : H-P interface failed to open file descriptor. **
** FAILED 35 : H-P interface failed to set file position. **
** FAILED 36 : H-P interface failed to allocate the buffer. **
“definitive-buffer” is a 64bit signed integer normalized -converted to network byte-order- with the buffer that the server finally decided to use. The server tries to create a buffer with the size indicated by the client. If that request cannot be satisfied, the server will create a buffer according to implementation-dependent criteria. In any case, if the server replies OK, the 64bit integer after the whitespace will be the buffer that should be used. The server will never return a buffer’s size greater than the one requested by the client.
After this, the server starts accepting the data flowing from the client. It works as follow:
The client must send a header before every payload. This header contains how many bytes will be sent next. The header is a 64bit signed integer normalized -converted to network byte-order.
This header is a window that allows the client to make some operations during the flow. The client can send the following codes using the header:
0 This is the HPFEND code which means that there is no more data to be sent.
-1 This is the HPFSTOP code which means that the client wants to stop sending data and will continue later. In this case the server just stops writing the file without removing it.
-2 This is the HPFCANCEL code which means that the client is canceling the operation. The server stops the operation and deletes the file.
-127 This is the HPFFIN code and is used to tell the server that the operation has finished.
No matter which code is used, the operation always must be ended by the HPFFIN code. Both sides must send it. First the server, then the client.
After the server receives one of the above codes -except HPFFIN-, it sends HPFFIN code, then the client responds with another HPFFIN code. Once this termination handshake is done, the server returns to listen again for new commands.
About cancellation:
This command is an asynchronous one in the sense that either side can send an HPFCANCEL code in the middle of the flow. As a result, the server can send, while receiving data, any code to request cooperation from the client. The server can send the following codes:
** HPFCANCEL **
Under certain conditions the server can decide to cancel the operation and will request the client to do so. After the server sends the HPFCANCEL code, all data received in the server will be discarded. The file will be removed too.
Once the HPFCANCEL code reaches the client, it must stop sending data. After the server sends this code, as stated before, it will send the HPFFIN code which indicates that the operation is done. Then the client responds with another HPFFIN code to confirm the end of the operation.
If the client sends, at any point, HPFSTOP, it can be safely assumed that the amount of data sent to the server is already stored in the file. The above assumption does not apply when after the client sends HPFSTOP it receives from the server an HPFCANCEL. In this case the server deletes the file.
Note of caution: As the server can cancel the operation asynchronously, the client may send some data before realizing the cancellation.
Supports TFPSFS
struct hpfile {
uint64_t offst;
int64_t bufsz;
};** GET “/path/to/filename” hpfile **
GET is part of the High Performance File Interface. It can be used to transfer files from the server to the client.
The first parameter is the file to be read.
The second parameter goes after the “/path/to/filename” and a whitespace. It consists of two 64bit integers normalized ‘converted to network byte-order’. The first one should be unsigned and the second one, signed. The first one must range from 0 to 2^64-1. The second one must range from -(2^63) to 2^63-1.
The “offst” integer -the first one- is the desire offset in which the data should start to be read in the opened file.
The “bufsz” integer -the second one- is the proposed buffer size by the client.
Once the command is constructed like the above, it can be sent to the server. On the server side, the protocol will evaluate the parameters and will return one of the following conditions:
** OK “definitive-buffer” **
** FAILED 1 : Access denied to location. **
** FAILED 16 : Missing parameter from command. **
** FAILED 34 : H-P interface failed to open file descriptor. **
** FAILED 35 : H-P interface failed to set file position. **
** FAILED 36 : H-P interface failed to allocate the buffer. **
“definitive-buffer” is a 64bit signed integer normalized -converted to network byte-order- with the buffer that the server finally decided to use. The server tries to create a buffer with the size indicated by the client, if that request cannot be satisfied, the server will create a buffer according to implementation-dependent criteria. In any case, if the server replies OK, the 64bit integer after the whitespace will be the buffer that should be used. The server will never return a buffer’s size greater than the one requested by the client.
After this, the server starts sending data to the client. It works as follows:
The server sends a header before every payload. This header contains how many bytes will be sent next. The header is a 64bit signed integer normalized -converted to network byte-order.
This header is a window that allows the server to do some operations during the flow. The server can send the following codes using the header:
0 This is the HPFEND code which means that there is no more data to be sent.
-2 This is the HPFCANCEL code which means that the server is canceling the operation.
-127 This is the HPFFIN code and is used to tell the client that the operation has finished.
No matter which code is used, the operation must always end with the HPFFIN code. Both sides must send it. Unlike the PUT command, first the client, then the server.
After the server sends one of the above codes -except HPFFIN-, it waits for HPFFIN code, then it responds with another HPFFIN code. Once this termination handshake is done, the server returns to listen again for new commands.
About cancellation: This command is an asynchronous one in the sense that either side can send a HPFCANCEL code in the middle of the flow. As a result, the client can send, while receiving data, any code to request cooperation from the server. The client can send the following codes:
HPFCANCEL
HPFFIN
Under certain conditions the client can decide to cancel the operation and will request the server to do so. After the client sends the HPFCANCEL code, all data received by the client should be discarded.
Once the HPFCANCEL code reaches the server, it will stop sending data. After the server receives this code, as stated before, it will wait for the HPFFIN code which indicates that the operation is done. Then the server responds with another HPFFIN code to confirm the end of the operation.
Note of caution: As the client can cancel the operation asynchronously, the server may send some data before realizing the cancellation.
Supports TFPSFS
struct hpfile {
uint64_t offst;
int64_t bufsz;
uint64_t canpt;
} ;** PUTCAN “/path/to/filename” hpfile **
PUTCAN is part of the High Performance File Interface. It can be used to transfer files from the client to the server. It differs from PUT because the cancellation, rather than asynchronous, are predefined points.
The first parameter is the file to be written, it could exist or not. If the file does not already exist, then it is created. If the file already exists, the command will overwrite it.
The second parameter goes after the “/path/to/filename” and a whitespace. It consists of three 64bit integers normalized ‘converted to network byte-order’. The first one should be unsigned; the second one, signed; and the third one, unsigned. The first one must range from 0 to 2^64-1. The second one must range from -(2^63) to 2^63-
- The third one must range from 0 to 2^64-1.
The “offst” integer -the first one- is the desire offset in which the data should be written in the opened file, being a new file or an existing one.
The “bufsz” integer -the second one- is the proposed buffer size by the client.
The “canpt” -the third one- is the number of transmitted buffers in which there will be a window to cancel the operation. 0 means no cancellation window.
Once the command is constructed like above, it can be sent to the server. On the server side, the protocol will evaluate the parameters and will return one of the following conditions:
** OK “definitive-buffer” **
** FAILED 1 : Access denied to location. **
** FAILED 16 : Missing parameter from command. **
** FAILED 34 : H-P interface failed to open file descriptor. **
** FAILED 35 : H-P interface failed to set file position. **
** FAILED 36 : H-P interface failed to allocate the buffer. **
“definitive-buffer” is a 64bit signed integer normalized -converted to network byte-order- with the buffer that the server finally decided to use. The server tries to create a buffer with the size indicated by the client. If that request cannot be satisfied, the server will create a buffer according to implementation-dependent criteria. In any case, if the server replies OK, the 64bit integer after the whitespace will be the buffer that should be used. The server will never return a buffer’s size greater than the one requested by the client.
After this, the server starts accepting the data flowing from the client. It works as follow:
The client must send a header before every payload. This header contains how many bytes will be sent next. The header is a 64bit signed integer normalized -converted to network byte-order.
This header is a window that allows the client to make some operations during the flow. The client can send the following codes using the header:
0 This is the HPFEND code which means that there is no more data to be sent.
-1 This is the HPFSTOP code which means that the client wants to stop sending data and will continue later. In this case the server just stops writing the file without removing it.
-2 This is the HPFCANCEL code which means that the client is canceling the operation. The server stops the operation and deletes the file.
About cancellation:
This command is intended for scenarios where multi-threading is not available, so asynchronous cancellation is not possible. In such a case, the client and the server must agree after how many bytes sent, there will be a chance to cancel the operation.
The “canpt” parameter specifies the number of buffers that will be sent before switching the way the data is flowing. Once the flow is inverted, the server has the chance to send one of the following codes:
** HPFCANCEL **
-3 HPFCONT (This code tells the other end that flow can continue).
Supports TFPSFS
struct hpfile {
uint64_t offst;
int64_t bufsz;
uint64_t canpt;
};** GETCAN “/path/to/filename” hpfile **
GETCAN is part of the High Performance File Interface. It can be used to transfer files from the server to the client. It differs from PUT because the cancellation, rather than asynchronous, are predefined points.
The first parameter is the file to be read.
The second parameter goes after the “/path/to/filename” and a whitespace. It consists of three 64bit integers normalized ‘converted to network byte-order’. The first one should be unsigned; the second one, signed; and the third one, unsigned. The first one must range from 0 to 2^64-1. The second one must range from -(2^63) to 2^63-
- The third one must range from 0 to 2^64-1.
The “offst” integer -the first one- is the desire offset in which the data should start to be read in the opened file.
The “bufsz” integer -the second one- is the proposed buffer size by the client.
The “canpt” -the third one- is the number of transmitted buffers in which there will be a window to cancel the operation. 0 means no cancellation window.
Once the command is constructed like the above, it can be sent to the server. On the server side, the protocol will evaluate the parameters and will return one of the following conditions:
** OK “definitive-buffer” **
** FAILED 1 : Access denied to location. **
** FAILED 16 : Missing parameter from command. **
** FAILED 34 : H-P interface failed to open file descriptor. **
** FAILED 35 : H-P interface failed to set file position. **
** FAILED 36 : H-P interface failed to allocate the buffer. **
“definitive-buffer” is a 64bit signed integer normalized -converted to network byte-order- with the buffer that the server finally decided to use. The server tries to create a buffer with the size indicated by the client, if that request cannot be satisfied, the server will create a buffer according to implementation-dependent criteria. In any case, if the server replies OK, the 64bit integer after the whitespace will be the buffer that should be used. The server will never return a buffer’s size greater than the one requested by the client.
After this, the server starts sending data to the client. It works as follows:
The server sends a header before every payload. This header contains how many bytes will be sent next. The header is a 64bit signed integer normalized -converted to network byte-order.
This header is a window that allows the server to do some operations during the flow. The server can send the following codes using the header:
0 This is the HPFEND code which means that there is no more data to be sent.
-2 This is the HPFCANCEL code which means that the server is canceling the operation.
About cancellation:
This command is intended for scenarios where multi-threading is not available, so asynchronous cancellation is not possible. In such a case, the client and the server must agree after how many bytes sent, there will be a chance to cancel the operation.
The “canpt” parameter specifies the number of buffers that will be sent before switching the way the data is flowing. Once the flow is inverted, the client has the chance to send one of the following codes:
** HPFCANCEL**
** -3 HPFCONT (This code tells the other end that flow can continue). **
LOGIN evaluates the combination of ‘user’ -as first parameter- and ‘password’ -as second parameter- against a database in the server. Note that there is a whitespace between the first and the second parameter. If a match is found, then the protocol sets the real-user id and real-group id of the process to that user. From this point on, all executed commands at the server are done on behalf of that logged user and the access to the file system will be restricted to the permission of that user. The return status of this command could be:
** OK **
** FAILED 39 : The process is already logged. **
** FAILED 16 : Missing parameter from command. **
** FAILED 40 : Failed to open users database. **
** FAILED 37 : Login failed. **
Supports TFPSFS
CHMOD changes the file access permission in the same fashion that the UNIX system call does it. The first parameter is the file or directory in which the access bits are about to be changed. The second parameter is the mode ‘permission’ in octal notation to be used. Between the first and the second parameter there is a whitespace.
The complete explanation of composing the mode is out of the scope of this document. But here are a few useful examples:
** CHMOD “path-to-file” 0400 **
Only owner can read the file.
** CHMOD “path-to-file” 0200 **
Only owner can write the file.
** CHMOD “path-to-file” 0100 **
Only owner can execute the file.
Now if we do summation we can obtain the following combinations:
** CHMOD “path-to-file” 0600**
Only owner can read and write the file.
** CHMOD “path-to-file” 0500
Only owner can read and execute the file.
** CHMOD “path-to-file” 0300
Only owner can write and execute the file.
So far, we have seen the second digit -from left to right- which represents the owner ́s permission. The third digit represents the group ́s permission. The fourth digit represents the permission for ‘others’ (anybody else). The same summation logic explained before for the second digit applies to the third and the fourth.
About the first digit, it could be:
4 (SUID) Set user-id bit.
2 (SGID) Set group-id bit.
1 Sticky bit.
Here the summation only applies to SGID (2) and Sticky bit (1). So, the value for the first digit could be: 4, 2, 1, or 3.
For the CHMOD command the first digit -if not needed- could be 0 or it can be completely omitted. For more information see the UNIX man pages.
The return status of this command could be:
** OK **
** FAILED 16 : Missing parameter from command. **
** FAILED 1 : Access denied to location. **
** FAILED 9 : File does not exist. **
** FAILED 41 : Failed to change file mode. **
Supports TFPSFS
CHOWN changes the ownership of a file or a directory. This command in some systems could be only executed by a privileged user (root). In other systems it can be executed too by the owner of the file or directory. The successful execution will depend of the logged user.
The return status of this command could be:
** OK **
** FAILED 16 : Missing parameter from command. **
** FAILED 1 : Access denied to location. **
** FAILED 42 : System user does not exist. **
** FAILED 43 : System group does not exist. **
** FAILED 44 : Failed to change file ownership. **
Supports TFPSFS
SHA256 makes a sha256 hash of a file indicated by ‘path’. This command is intended to apply a hash to a file before downloading it in order to guarantee its integrity. Be aware that on very large files the time to resolve the digest function could be potentially long. The return status of this command could be:
** OK “hash-in-hex-form like 0xff33...” **
** FAILED 45 : Failed to make SHA256 hash. **
** FAILED 16 : Missing parameter from command. **
** FAILED 1 : Access denied to location. **
NIGMA generate a random session key of the specified length in the first parameter. This length should be a number equal or greater than 8 and multiple of 4. If the command fails it return FAILED, otherwise OK. In case of success, after the OK, the server will send a 32bit integer converted to network byte-order indicating the payload size that coming next. This payload will be the new session key and the header must be equal to the keylen indicated by the first parameter. After the client receives the key it should be swapped with the old key, and the new key is the one should be used hereinafter to encrypt/decrypt the data. The return status of this command could be:
** OK **
-After OK the client must read a 32bit integer header and the session key that coming next, then should be swapped with the old session key-
** FAILED 46 : Failed to swap the session key. **
Supports TFPSFS
RMSECDIR removes a specified secure directory recursively. The first parameter is the secure token that allows to remove the directory specified in the second parameter. The secure token could be either a file or a directory named as the first parameter, inside the the directory specified as the second parameter. The return status of this command could be:
** OK **
** FAILED 1 : Access denied to location. **
** FAILED 30 : Directory is locked. **
** FAILED 15 : Directory to remove still exist. **
** FAILED 47 : Failed removing secure directory. **
If the second parameter is leaved in blank the protocol daemon directory is used. So it returns the FAILED 15 return status due the impossibility to delete the daemon directory. However, every other directory will be deleted is “secure_token” is found.
INJAIL in-jails the TFProtocol damon in the directory specified by the second parameter. This is achieved only if “secure_token”, specified as first parameter, is permitted for that directory. The return status of this command could be:
** OK **
** FAILED 1 : Access denied to location. **
** FAILED 16 : Missing parameter from command. **
** FAILED 48 : Failed to open ACL file in jail directory. **
** FAILED 49 : Invalid security token in jail directory. **
** FAILED 51 : The process is already in-jailed. **
** FAILED 50 : Invalid command before in-jail. **
TLB requests a tuple 2 ip/port from the Transfer Load Balancer pool. The clients can use this command -in ‘advisory’ way- to retrieve other servers in order to balance the overall traffic. By ‘advisory’ we mean that it’s up to the clients to agree in asking for a server from the pool before start any further interaction. The return status of this command could be:
** OK “4|6 ip;port” **
an example for IPv4 could be:
4 1.1.1.1;1234
an example for IPv6 could be:
6 ::1;1234
** FAILED 52 : Failed to retrieve TLB tuple. **
At this point, again, it is up to the client to disconnect from the server and connect to the retrieved one. It is guarantee that the hash and public key will remain the same.
Supports TFPSFS
SDOWN downloads the specified file in the command argument. Before every payload, a 32bit signed integer header converted to big-endian with the payload size is sent by the server. If there is an error a header with value -1 is sent. If the end of file is reached a header with value 0 is sent.
Supports TFPSFS
SUP uploads the specified file in the command argument. Before every payload, a 32bit signed integer header converted to big-endian with the payload size must be sent by the client. If there is an error a header with value -1 must be sent. If the end of file is reached a header with value 0 must be sent. If the file already exist in the server, it will be truncated to zero. If the client sent -1 indicating the upload canceling, the uploaded data is deleted. If the client sent 0 indicating that there is no more data to send -EOF-, then it has to wait for a header -32bit big-endian- from the server indicating the result of the operation. 0 means OK, -1 means failed. If -1 is sent by the server, the uploaded data is deleted.
Supports TFPSFS
FSIZE gets the file size -in bytes- of the specified file in the parameter. This command returns a header of 64bit signed integer converted to big-endian with the file size or an error code. The return status of this command could be:
File size in bytes (i.e 5403)
-1: indicates a failure executing the command.
-2: The indicated file is not a regular file. It could be a directory or any other special file.
Supports TFPSFS
Each line in the specified file as parameter represents a path to a file. FSIZELS gets the file size -in bytes- of each path. This command returns headers of 64bit signed integer converted to big-endian with the file sizes or error codes. For each line a header is returned with one of the following results:
File size in bytes (i.e 5403)
-1: indicates a failure executing the command for that line.
-2: The line is not a regular file. It could be a directory or any other special file.
If there are no more lines in the file, a header with value -3 is returned indicating it. This command always ends by sending -3.
Supports TFPSFS
LSV2 lists the directory indicated by the first parameter. It stores all listed files in the file indicated by the second parameter, one file per line. If a line ends with slash ‘/’ it means is a directory, otherwise it could be any other file. By any other file we mean a regular file, a socket, a character device, and so on. The @||@ token separates both paths. The return status of this command could be:
** OK **
** FAILED 16 : Missing parameter from command. **
** FAILED 20 : Source path is not a directory. **
** FAILED 1 : Access denied to location. **
** FAILED 30 : Directory is locked. **
** FAILED 53 : Failed running LSRV2. **
The paths are absolute -”path/to/list” is included-.
Supports TFPSFS
LSRV2 does the same that LSV2 but it does it recursively. The return status are the same.
Supports TFPSFS
FTYPE returns a byte indicating the type of the file that the first parameter points to. Once the command is sent to the server, the client must wait for a single byte containing one of the following results:
0 for a directory
1 for a character device
2 for a block device
3 for a regular file
4 for a FIFO or a pipe
5 for a symbolic link
6 for a socket
7 for any other type
-1 for a failure getting the type
Supports TFPSFS
FTYPELS works pretty much like FTYPE except it return a byte with the type per each line in the file indicated by the first parameter. Each line in the file passed as first parameter must be a path. The return values are the same as in FTYPE, except it sends -2 when there are no more types to receive.
Supports TFPSFS
FSTATLS returns a special structure per each line in the file indicated by the first parameter. Each line in that file must be a path. The structure is exactly 26 bytes and looks like the bellow:
struct fstathdr { char code; char type; uint64_t size; uint64_t atime; uint64_t mtime; };
As far as the client concerns it receives 26 bytes. The first byte -code member- contains one of the following values:
0 It means OK and there is relevant info in the rest of the members.
-1 A failure has occurred getting the statistics for that path. In this case there is not any relevant information in the rest of the members.
-2 It means that there is no more statistics structures coming. In this case there is not any relevant information in the rest of the members.
The second byte –the type member- contains the same possible values of the FTYPE command.
The next 8 bytes -the size member- is a 64bit unsigned integer converted to big-endian that contains the size of the file, directory, etc, indicated by the path.
The next 8 bytes -the atime member- is a 64bit unsigned integer converted to big-endian with the timestamp -Unix time in seconds- of the last access time.
The next 8 bytes -the mtime member- is a 64bit unsigned integer converted to big-endian with the timestamp -Unix time in seconds- of the last modification time.
Supports TFPSFS
INTREAD -Integrity Read-. It is intended to atomically download a file with its integrity checksum, for the case SHA256. It will return to the client a 32bit integer -big-endian- header with -1 to indicate an error or the file size even if is 0. After the server sent the header it will send the SHA256 checksum which it is exactly 66 bytes: 0x plus the hash formatted in hexadecimal format. After this, the server will send the payload. Unlike many commands, the entire file will be loaded in the server process space address, so it is not intended to manage vary large file.
Supports TFPSFS
INTWRITE -Integrity Write-. It is intended to atomically upload a file with the integrity checksum -SHA256- of the file indicated in the parameter, so the destination file must exist. The server will expect the hash, after receive it, which is exactly 66 bytes: 0x plus the hash formatted in hexadecimal format, the server will expect for a 32bit integer -big-endian- header with the fie size to be uploaded. After the this header is received, the client must send the payload. Once the file and the hash are at the server side, the server will compare that hash with the hash of the file indicated in the parameter, if both hashes are equal the server returns 0, otherwise -1, in a 32bit -big- endian- integer header. The main reason for the server to return -1 is that the hashes do not match, but it could be by many other situations. In any case, a return -1 must be considered as a hash match failure. If the server returns 0, truncates the destination file to 0 and write the file with the payload received. Unlike many commands, the entire file will be loaded in the server process space address, so it is not intended to manage vary large file.
NETLOCK creates an advisory record lock in the file specified by the second parameter with a timeout in seconds indicated in the first parameter. The system guarantee that each time the network detects activity, the timeout will be reset. By activity we meant any read or write operation from/to the network. However, attention must by pay specially when the lock is set-up and TFProtocol enters a subsystem. It is possible that the subsystem remains silent for long periods of times and the timeout expires. The client is responsible for send data periodically to avoid the timeout expiration. If the timeout expires and the client does not called NETUNLOCK, TFProtocol will close the connection releasing the lock on the file. The return status of this command could be:
** OK “lock-id” **
** FAILED 16 : Missing parameter from command. **
** FAILED 1 : Access denied to location. **
** FAILED 54 : Failed to acquire or release NETLOCK. **
If the command succeeded, the “lock-id” -without double quotes- is an integer index that represents the lock.
NETLOCK_TRY works exactly like NETLOCK but instead of blocking until obtain the lock, it will return the error FAILED 54.
NETULOCK releases the lock, if were in place, on the file represented by “lock-id” -without double quotes-. The return status of this command could be:
** OK ** ** FAILED 54 : Failed to acquire or release NETLOCK. **
Supports TFPSFS
NETMUTACQ_TRY tries to acquire the ownership of a synchronization object to enter a critical section. It work as follows: If the file indicated by the first parameter does not exist, the command will fail. If the file exist TFProtocol will acquire an advisory exclusive record lock on to it. If the file is zero bytes, then “token” -without double quotes- will be written in it. The mutex is acquired and OK will be returned to the client. If the file has something written that match exactly with “token”, then the mutex is considered as acquired too and OK will be returned to the client. If the file has something written that do not match with “token”, then the mutex is considered as non-acquired and a failure will be returned to the client. At this point the exclusive advisory record lock will be release on the file. Be aware that this kind of mutual exclusive synchronization object is persistent and will survive any network disconnection. Who acquires the mutex -”token”- must release it explicitly by calling NETMUTREL command, otherwise the mutex will be considered as acquire forever. The return status of this command could be:
** OK **
** FAILED 16 : Missing parameter from command. **
** FAILED 1 : Access denied to location. **
** FAILED 55 : Failed to acquire or release NETMUT. **
Summary: The mutex -”token”- is considered acquire if the file indicated by the first parameter either has 0 bytes or contains exactly ”token” - without double quotes-. Otherwise, the mutex is considered as non- acquire.
Supports TFPSFS
NETMUTREL releases the ownership of a mutex “token” previously acquired by NETMUTACQ_TRY. The first parameter indicates the file where the mutex was placed. The “token” -without double quotes- parameter tells TFProtocol what mutex has to be released. It works as follows: An exclusive advisory record lock is acquired on the file. If the file does not exist, it will be returned a failure to the client. If the file exist and it has 0 bytes or it contains exactly “token”, the file is truncated to 0 and OK is returned to the client. At this point the mutex is considered released. The return status of this command could be:
** OK ** FAILED 16 : Missing parameter from command. FAILED 1 : Access denied to location. FAILED 55 : Failed to acquire or release NETMUT. Summary: The mutex is considered released only if the file that contain it has 0 bytes or has written exactly “token”, in the later case TFProtocol will truncate it to 0. SETFSID “secureFileSystemIdentity” SETFSID sets the process identity that will be used in any operations that involves the TFProtocol Secure Filesystem. The return status of this command could be: OK SETFSPERM “secID:128” “path/to/securefs/directory” SETFSPERM sets the permission mask indicated after ‘:’ for the identity that precede it. If is the first time that a permission is set for the directory the “secID” becomes the owner with the permissions indicated after ‘:’. This means that only secID -secure filesystem identity- will be able to sets new permissions for others “secIDs”. If the directory has already permissions set, the process’s identity set by SETFSID must have the right permissions to call SETFSPERM. For further details see TFProtocol Secure Filesystem section. The return status of this command could be: OK FAILED 16 : Missing parameter from command. FAILED 57: Failed setting secure FS permission. FAILED 1 : Access denied to location. REMFSPERM “secID” “path/to/securefs/directory” REMFSPERM removes the secID indicated by the first parameter from the secure filesystem directory indicated by the second parameter. The process’s identity set by SETFSID must have the right permissions to call REMFSPERM. For further details see TFProtocol Secure Filesystem section. The return status of this command could be: OK FAILED 16 : Missing parameter from command. FAILED 58: Failed removing secure FS permission. FAILED 1 : Access denied to location. GETFSPERM “secID” “path/to/securefs/directory” GETFSPERM gets the secID’s permissions of the specified directory in the second parameter. The return status of this command could be: OK “permission-mask” FAILED 16 : Missing parameter from command. FAILED 59: Failed getting secure FS permission identity. FAILED 1 : Access denied to location. ISSECFS “path/to/directory” ISSECFS gets if “path/to/directory” belongs to the TFProtocol Secure Filesystem domain. The return status of this command could be: OK FAILED 60: The directory does not belongs to secure FS. FAILED 20 : Source path is not a directory. FAILED 1 : Access denied to location. LOCKSYS “path/to/directory/to/in/lock” LOCKSYS locks TFProtocol in the specified directory. Once the daemon is locked in a directory, there isn’t way back. This command it’s not intended to be called multiple times, instead, if TFProtocol is configured to in-lock processes this command should be called before any other, except: END; LOGIN; KEEPALIVE; PROCKEY; INJAIL. OK FAILED 9 : File does not exist. FAILED 10 : Directory does not exist. FAILED 61: Unable to use Folder Locking System.
