Skip to content

feat: Adding API for ListParts #3359

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 18 commits into
base: mpu-feature-2
Choose a base branch
from
Open

feat: Adding API for ListParts #3359

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
a35e4d6
feat: Adding API for ListParts
ShreyasSinha Oct 22, 2025
07afc4e
chore: adding logic for list parts.
ShreyasSinha Oct 22, 2025
48e1a2f
chore: adding missing final statement
ShreyasSinha Oct 22, 2025
21c5c94
chore: adding beta api annotation.
ShreyasSinha Oct 22, 2025
fffd81b
chore: added missing annotation
ShreyasSinha Oct 22, 2025
3757fed
chore: Removed merge conflicts.
ShreyasSinha Oct 23, 2025
08b084f
chore: Adding lint issues.
ShreyasSinha Oct 23, 2025
eaff90e
Merge branch 'mpu-feature-2' into mpu-feature-3
ShreyasSinha Oct 23, 2025
0b378af
added test
ShreyasSinha Oct 23, 2025
c8fe7fe
Merged feature 2 into feature 3
ShreyasSinha Oct 24, 2025
5c7c6ca
chore: merged feature 2 into feature 3
ShreyasSinha Oct 24, 2025
daaad24
choere: coverted Integer to int.
ShreyasSinha Oct 26, 2025
b261690
chore: fixed datatypes of few variables
ShreyasSinha Oct 26, 2025
6acb2a2
chore:adding handling for storage class nad owner
ShreyasSinha Oct 26, 2025
ddfa9fd
chore: Removed unwanted library decleration
ShreyasSinha Oct 26, 2025
337f972
chore: Adding seperate test for all use case of List parts.
ShreyasSinha Oct 26, 2025
c7549b1
chore: adding beta api annotation.
ShreyasSinha Oct 28, 2025
3869259
chore: added review comments
ShreyasSinha Oct 30, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions google-cloud-storage/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,18 @@
<groupId>com.fasterxml.jackson.dataformat</groupId>
<artifactId>jackson-dataformat-xml</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-jsr310</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-annotations</artifactId>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
Expand Down
27 changes: 26 additions & 1 deletion google-cloud-storage/src/main/java/com/google/cloud/storage/MultipartUploadClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@
import com.google.api.core.InternalExtensionOnly;
import com.google.cloud.storage.multipartupload.model.CreateMultipartUploadRequest;
import com.google.cloud.storage.multipartupload.model.CreateMultipartUploadResponse;
import com.google.cloud.storage.multipartupload.model.ListPartsRequest;
import com.google.cloud.storage.multipartupload.model.ListPartsResponse;
import java.io.IOException;
import java.net.URI;

Expand All @@ -37,16 +39,39 @@ public abstract class MultipartUploadClient {

MultipartUploadClient() {}

/**
* Creates a new multipart upload.
*
* @param request The request object containing the details for creating the multipart upload.
* @return A {@link CreateMultipartUploadResponse} object containing the upload ID.
* @throws IOException if an I/O error occurs.
*/
@BetaApi
public abstract CreateMultipartUploadResponse createMultipartUpload(
CreateMultipartUploadRequest request) throws IOException;

/**
* Lists the parts that have been uploaded for a specific multipart upload.
*
* @param listPartsRequest The request object containing the details for listing the parts.
* @return A {@link ListPartsResponse} object containing the list of parts.
*/
@BetaApi
public abstract ListPartsResponse listParts(ListPartsRequest listPartsRequest);

/**
* Creates a new instance of {@link MultipartUploadClient}.
*
* @param config The configuration for the client.
* @return A new {@link MultipartUploadClient} instance.
*/
@BetaApi
public static MultipartUploadClient create(MultipartUploadSettings config) {
HttpStorageOptions options = config.getOptions();
return new MultipartUploadClientImpl(
URI.create(options.getHost()),
options.createRetrier(),
MultipartUploadHttpRequestManager.createFrom(options));
MultipartUploadHttpRequestManager.createFrom(options),
options.getRetryAlgorithmManager());
}
}
19 changes: 18 additions & 1 deletion google-cloud-storage/src/main/java/com/google/cloud/storage/MultipartUploadClientImpl.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,12 @@
package com.google.cloud.storage;

import com.google.api.core.BetaApi;
import com.google.cloud.storage.Conversions.Decoder;
import com.google.cloud.storage.Retrying.Retrier;
import com.google.cloud.storage.multipartupload.model.CreateMultipartUploadRequest;
import com.google.cloud.storage.multipartupload.model.CreateMultipartUploadResponse;
import com.google.cloud.storage.multipartupload.model.ListPartsRequest;
import com.google.cloud.storage.multipartupload.model.ListPartsResponse;
import java.io.IOException;
import java.net.URI;

Expand All @@ -32,19 +35,33 @@ final class MultipartUploadClientImpl extends MultipartUploadClient {
private final MultipartUploadHttpRequestManager httpRequestManager;
private final Retrier retrier;
private final URI uri;
private final HttpRetryAlgorithmManager retryAlgorithmManager;

MultipartUploadClientImpl(
URI uri,
Retrier retrier,
MultipartUploadHttpRequestManager multipartUploadHttpRequestManager) {
MultipartUploadHttpRequestManager multipartUploadHttpRequestManager,
HttpRetryAlgorithmManager retryAlgorithmManager) {
this.httpRequestManager = multipartUploadHttpRequestManager;
this.retrier = retrier;
this.uri = uri;
this.retryAlgorithmManager = retryAlgorithmManager;
}

@Override
@BetaApi
public CreateMultipartUploadResponse createMultipartUpload(CreateMultipartUploadRequest request)
throws IOException {
return httpRequestManager.sendCreateMultipartUploadRequest(uri, request);
}

@Override
@BetaApi
public ListPartsResponse listParts(ListPartsRequest request) {

return retrier.run(
retryAlgorithmManager.idempotent(),
() -> httpRequestManager.sendListPartsRequest(uri, request),
Decoder.identity());
}
}
23 changes: 23 additions & 0 deletions ...oud-storage/src/main/java/com/google/cloud/storage/MultipartUploadHttpRequestManager.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@
import com.google.api.services.storage.Storage;
import com.google.cloud.storage.multipartupload.model.CreateMultipartUploadRequest;
import com.google.cloud.storage.multipartupload.model.CreateMultipartUploadResponse;
import com.google.cloud.storage.multipartupload.model.ListPartsRequest;
import com.google.cloud.storage.multipartupload.model.ListPartsResponse;
import com.google.common.base.StandardSystemProperty;
import com.google.common.collect.ImmutableMap;
import java.io.IOException;
Expand Down Expand Up @@ -72,6 +74,27 @@ CreateMultipartUploadResponse sendCreateMultipartUploadRequest(
return httpRequest.execute().parseAs(CreateMultipartUploadResponse.class);
}

ListPartsResponse sendListPartsRequest(URI uri, ListPartsRequest request) throws IOException {

String encodedBucket = urlEncode(request.bucket());
String encodedKey = urlEncode(request.key());
String resourcePath = "/" + encodedBucket + "/" + encodedKey;
String queryString = "?uploadId=" + urlEncode(request.uploadId());

if (request.getMaxParts() != null) {
queryString += "&max-parts=" + request.getMaxParts();
}
if (request.getPartNumberMarker() != null) {
queryString += "&part-number-marker=" + request.getPartNumberMarker();
}
String listUri = uri.toString() + resourcePath + queryString;
HttpRequest httpRequest = requestFactory.buildGetRequest(new GenericUrl(listUri));
httpRequest.getHeaders().putAll(headerProvider.getHeaders());
httpRequest.setParser(objectParser);
httpRequest.setThrowExceptionOnExecuteError(true);
return httpRequest.execute().parseAs(ListPartsResponse.class);
}

@SuppressWarnings("DataFlowIssue")
static MultipartUploadHttpRequestManager createFrom(HttpStorageOptions options) {
Storage storage = options.getStorageRpcV1().getStorage();
Expand Down
5 changes: 5 additions & 0 deletions google-cloud-storage/src/main/java/com/google/cloud/storage/StorageClass.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
*/
package com.google.cloud.storage;

import com.fasterxml.jackson.annotation.JsonCreator;
import com.google.api.core.ApiFunction;
import com.google.cloud.StringEnumType;
import com.google.cloud.StringEnumValue;
Expand Down Expand Up @@ -111,7 +112,11 @@ public static StorageClass valueOfStrict(String constant) {
}

/** Get the StorageClass for the given String constant, and allow unrecognized values. */
@JsonCreator
public static StorageClass valueOf(String constant) {
if (constant == null || constant.isEmpty()) {
return null;
}
Comment on lines +115 to +119
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this necessary?

If I remove it and run the following test, the test passes:

  private static final class TestXmlObject2 {
    @JacksonXmlProperty(localName = "storageClass")
    private StorageClass storageClass;
  }
  @Test
  public void testParseStringValueEnum() throws IOException {
    //language=xml
    String xml = "<TestXmlObject2>\n"
        + "  <storageClass>STANDARD</storageClass>"
        + "</TestXmlObject2>";
    InputStream in = new ByteArrayInputStream(xml.getBytes(StandardCharsets.UTF_8));
    TestXmlObject2 expected = new TestXmlObject2(StorageClass.STANDARD);
    TestXmlObject2 actual =
        xmlObjectParser.parseAndClose(in, StandardCharsets.UTF_8, TestXmlObject2.class);
    assertThat(actual).isEqualTo(expected);
  }

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is required incase we have a null string. Try to run test https://paste.googleplex.com/5523315682836480 after removing it. You'll get https://paste.googleplex.com/5523315682836480 failure.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, but that is a deserialization issue not an enum resolution issue.

Enums can't have null values, and changing this method breaks that contract.
The exception explicitly states this fact:

Caused by: java.lang.IllegalArgumentException: Empty enum constants not allowed.
        at com.google.cloud.StringEnumType.valueOf(StringEnumType.java:66)
        at com.google.cloud.storage.StorageClass.valueOf(StorageClass.java:116)

Here is a PR that makes enum parsing work for xml without modifying any behavior or adding additional annotations: #3377

return type.valueOf(constant);
}

Expand Down
2 changes: 2 additions & 0 deletions google-cloud-storage/src/main/java/com/google/cloud/storage/XmlObjectParser.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
package com.google.cloud.storage;

import com.fasterxml.jackson.dataformat.xml.XmlMapper;
import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule;
import com.google.api.client.util.ObjectParser;
import com.google.common.annotations.VisibleForTesting;
import java.io.IOException;
Expand All @@ -31,6 +32,7 @@ final class XmlObjectParser implements ObjectParser {
@VisibleForTesting
public XmlObjectParser(XmlMapper xmlMapper) {
this.xmlMapper = xmlMapper;
this.xmlMapper.registerModule(new JavaTimeModule());
}

@Override
Expand Down
4 changes: 2 additions & 2 deletions ...in/java/com/google/cloud/storage/multipartupload/model/CreateMultipartUploadResponse.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,14 +39,14 @@ public final class CreateMultipartUploadResponse {
@JacksonXmlProperty(localName = "UploadId")
private String uploadId;

private CreateMultipartUploadResponse() {}

private CreateMultipartUploadResponse(Builder builder) {
this.bucket = builder.bucket;
this.key = builder.key;
this.uploadId = builder.uploadId;
}

private CreateMultipartUploadResponse() {}

/**
* Returns the name of the bucket where the multipart upload was initiated.
*
Expand Down
Loading