Skip to content

gcloud.storage.iterator.KeyDataIterator chunking is off-by-one #143

Closed
@tseaver

Description

@tseaver

'get_headers' computes the "Range:" header as '_bytes_written' - '_bytes_written + CHUNK_SIZE':

https://github.com/GoogleCloudPlatform/gcloud-python/blob/1f2aaf3606802f4f02eeb30e22e1082ef7f4594e/gcloud/storage/iterator.py#L215

and the '_bytes_written' gets updated to bump by the size of the chunk the server returns. Consider the initial read, assuming a CHUNK_SIZE of 100:

Range: bytes=0-100

Note that, per RFC2616. this will return up to 101 bytes (both bytes-position values are inclusive):

http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.35.1

Metadata

Metadata

Assignees

Labels

🚨This issue needs some love.api: storageIssues related to the Cloud Storage API.triage meI really want to be triaged.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions