Skip to content

Shaloo/ar 4947 chunksize #166

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: dev
Choose a base branch
from
Open

Shaloo/ar 4947 chunksize #166

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file removed .DS_Store
View file
Open in desktop
Binary file not shown.
19 changes: 17 additions & 2 deletions usage.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ Refer to the [Arcana Storage SDK Quick Start Guide](https://docs.beta.arcana.net
1. Install Storage SDK
2. Import `StorageProvider` from the Storage SDK package in the dApp. Call `init` method of `StorageProvider` and specify the Web3 wallet `provider` and the `appAddress` as input parameters. **Note:** Get the provider via the Auth SDK or third-party supported wallet. You can copy the appAddress from the [Arcana Developer Dashboard](https://docs.beta.arcana.network/docs/config_dapp) after registering your dApp
3. Use `StorageProvider` to:
- `upload` and push file data into the Arcana Store. **Note:** Save file DID that is returned after file upload operation is successful.
- `upload` and push file data into the Arcana Store. **Note:** Save file DID that is returned after the file upload operation is successful.
Copy link
Contributor

@mmjee mmjee Nov 14, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good opportunity to fix this, “Save file DID” is not English.

Suggested change
- `upload` and push file data into the Arcana Store. **Note:** Save file DID that is returned after the file upload operation is successful.
- `upload` and push file data into the Arcana Store. **Note:** The newly saved file's DID is returned once the file upload operation is successful.

- `download` a file from the Arcana Store using DID as input.
4. Use `StorageProvider.files` to:
- `delete` a file by specifying its DID.
Expand Down Expand Up @@ -102,6 +102,21 @@ After a file is successfully uploaded to the Arcana Store, it is assigned a uniq
let shareURL = await dAppStorageProvider.files.getPublicFileURL(did);
``` -->

### Upload Chunk Size

Arcana Storage SDK uses chunked uploading mode. Each file is split into chunks of default size 10MB and chunks are uploaded one by one. These chunks are further processed on the server, encrypted, and split into multiple parts before being stored in the Arcana Store. Chunked uploading helps to overcome the server's default upload size limitation and there is no server-side setting change required to upload a very large file size. Also, it helps in decentralizing data storage.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uses chunked uploading mode

What does that even mean? I suggest: “uploads files in chunks”.

chunks of default size 10MB

I recommend: “chunks (10 MiB by default)”.


You can change the default chunk size while uploading a file. Make sure the new chunk size is a multiple of 16 otherwise there may be issues in downloading the file.
Copy link
Contributor

@mmjee mmjee Nov 14, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there may be issues in downloading the file

Could be worded more precisely.

Suggested change
You can change the default chunk size while uploading a file. Make sure the new chunk size is a multiple of 16 otherwise there may be issues in downloading the file.
You can change the default chunk size while uploading a file. Make sure the new chunk size is a multiple of 16 (bytes) otherwise you may face unexpected issues if you try to download the file.


```ts
// Upload a file and set the chunk size to 16 MB
await dAppStorageProvider.upload(file, {chunkSize: 16 * 2 ** 20},{
onProgress: (bytesUploaded, bytesTotal) => {
console.log('Progress:', ((bytesUploaded / bytesTotal) * 100).toFixed(2), '%')
}
}).then((did) => console.log('File successfully uploaded in 16 MB chunks. DID:', did)).catch(e => console.error(e));
```

## Download File

```ts
Expand Down Expand Up @@ -290,4 +305,4 @@ returns `true` if required.

```js
const isPermissionRequired = await storage.checkPermission() -> boolean
```
```