[Storage] Async iterator for listing blobs and containers - storage-blob #3136
Conversation
HarshaNalluru
added
Client
HarshaNalluru
changed the title
HarshaNalluru
changed the title
| * @memberof BlobServiceClient | ||
| * | ||
| * @example | ||
| * for await (const container of blobServiceClient.listContainers()) { |
There was a problem hiding this comment.
Just wondering what will happen if Aborter aborted iteration
There was a problem hiding this comment.
Example -
// List containers - generator syntax
let iter2 = await blobServiceClient.listContainers({ abortSignal: Aborter.timeout(X) });
i = 1;
let containerItem = await iter2.next();
do {
console.log(`Container ${i++}: ${containerItem.value.name}`);
containerItem = await iter2.next();
} while (containerItem.value);[ About 6500 containers are present in the account ]
- When X = 50000 (large number), all the 6500 containers got listed(and their names are printed in the terminal)
- When X = 2000, only 5000 containers got listed and a message got printed saying
The request was aborted.
[ X = 10000 yielded the same result ] - When X < 1000, none of the containers got listed,
The request was abortedis printed in the terminal.
| * } while (blobItem.value); | ||
| * | ||
| */ | ||
| public async *listBlobs( |
There was a problem hiding this comment.
Or called listBlobsFlat to avoid confusing between flat and hierarchical listing?
HarshaNalluru
force-pushed
the
AsyncIteratorsStorageBlob
branch
from
d9a99c4
to
43d30c0
Compare
| blobClients.push(blobClient); | ||
| } | ||
|
|
||
| const iterator = await containerClient.listBlobsFlat({ |
There was a problem hiding this comment.
Do we need test against other usage scenarios for async iterator? such as async for loop
There was a problem hiding this comment.
Can we cover the cross segment scenario? Because implementation of listBlobsFlat needs to handle marker update, while existing test case doesn't cover that
There was a problem hiding this comment.
Can we cover the cross segment scenario? Because implementation of listBlobsFlat needs to handle marker update, while existing test case doesn't cover that
I'm not sure if we are doing it.
azure-sdk-for-js/sdk/storage/storage-blob/src/ContainerClient.ts
Lines 630 to 634 in bc932b3
The current implementation of listBlobsFlat doesn't take marker as an input.
We can make marker as an optional attribute to the method. I'm just not sure if we want to do it.
There was a problem hiding this comment.
- Sorry for the confusion.
- After an offline discussion with Jeremy, he suggested that you might be referring to the usage of more than one
listBlobFlatSegmentin the samelistBlobsFlatcall.
842de33 - Added a for-loop test case.
Outputs (console.logs) to verify your question.
[By passing maxresults in the options for listBlobsFlat ]
Case 1 - maxresults is not set
------------------------- called listBlobFlatSegment -------------------------
blockblob/0155925458974007856
blockblob/1155925459003608508
blockblob/2155925459032203396
blockblob/3155925459061100420
√ Verify AsyncIterator(for-loop syntax) for listBlobsFlat (2714ms)Case 2 - maxresults is set to 2
------------------------- called listBlobFlatSegment -------------------------
blockblob/0155925451934305295
blockblob/1155925451964308353
------------------------- called listBlobFlatSegment -------------------------
blockblob/2155925451993002640
blockblob/3155925452022007968
√ Verify AsyncIterator(for-loop syntax) for listBlobsFlat (3012ms)
| @@ -222,7 +222,7 @@ describe("ContainerClient", () => { | |||
| } | |||
| }); | |||
|
|
|||
| it("Verify AsyncIterator for listBlobsFlat", async () => { | |||
| it("Verify AsyncIterator(generator .next() syntax) for listBlobsFlat", async () => { | |||
There was a problem hiding this comment.
would the special characters ( and ) affect later record and playback test generation?
There was a problem hiding this comment.
No, they won't. We handled those things gracefully 😊
No description provided.