Home

GRIDA documentation

Clients

Using GridaClient

This does synchronous stuff, the commands are described bellow

This does not use pools and it does not do disk size verification.

Using GridaPoolClient

All operations (download/upload/replicate/delete) are asynchronous and are handled into dedicated pools. They are all identified by an operationId to monitor them.

In the case of file Download (not folder), a size check is done by checking the size to download and the disk free space (in PoolOperation::addOperation' so add the request time, not at the moment of download).

Technical implementations

All the commands coming from the two clients are managed and routed in the Executor class. In all cases the actual operations are done in the Operations implementation (Local or Dirac). OperationBusiness is an additional layer on top of Operations, common to all implementations. The difference between async / sync is the following:

• in sync, Executor calls the Command implementation (GetRemoteFile / UploadFile / etc), which calls the OperationBusiness methods. Of course all is synchronous, there is no database action.
• in async, Executor calls the AddOperationsCommand, which does a disk size check and an operation in database. Then different pools threads (PoolDownload / PoolUpload / etc) handle them and use OperationBusiness.

Caching

TODO

AddOperation Command

All asynchronous commands are managed here to add an entry in database. In all cases (upload, download, etc), a space check is done on disk (in case of error the operation is inserted on db as Failed). In case of downloads, the downloaded file(s) size are taken into account in the disk size check.

PoolDownload

• max parallel downloads configured in grida conf file
• basic use of OperationBusiness::downloadFile or OperationBusiness::downloadFiles on OperationBusiness::downloadFolder

PoolUpload

• max parallel uploads configured in grida conf file
• upload using same implementation of the UploadFile command, not doing any replication yet, but submitting a new replicate operation that will replicate on all SEs configured in the grida conf file

PoolReplicate

• max parallel replication configured in grida conf file
• replicates file on all SEs configured in the grida conf file

PoolDelete

• max parallel deletes configured in grida conf file
• delete using same implementation of the Delete command

OperationBusiness::downloadFile

• checks if dest file already exists on disk
• If absent or ousdated, calls Operations::downloadFile

OperationBusiness::downloadFolder

• walk through folder and recursively calls itself or OperationBusiness::downloadFile
• can zip the original folder (the folder is them deleted)

OperationBusiness::downloadFiles

• calls OperationBusiness::downloadFile on all files
• zip all the files, and then delete them

GetRemoteFile commmand

• Checks CacheBusiness.
• If absent or outdated, calls OperationBusiness::downloadFile

NO POOLING

NO DISK SIZE CHECK

GetRemoteFolder command

• calls OperationBusiness::downloadFolder

NO CACHE

NO POOLING

NO DISK SIZE CHECK

UploadFile Command

• using dput
• using SE configured in grida conf file (try in order in order, stop when 1 is ok)
• delete local file
• if SEs were given on the command, do a drepl on each one
• otherwise, add a Replicate operation in a pool (asynchronous)

NO POOLING

Delete Command

• do a dirac-dms-clean-directory if it is a directory
• do a dirac-dms-remove-files if it is a file

NO POOLING