Skip to content
This repository has been archived by the owner on May 16, 2023. It is now read-only.

switch from s3 ListObjectsV2 to ListObjects #1821

Merged
merged 10 commits into from
May 4, 2022
Merged

switch from s3 ListObjectsV2 to ListObjects #1821

merged 10 commits into from
May 4, 2022

Conversation

hilmarf
Copy link
Member

@hilmarf hilmarf commented May 3, 2022 •
edited
Loading

Object Store doesn't support V2 API with token... :-(

When DCC Revocation is running, we need to query ALL objects from OBS buckets and delete the same, incase the ETag did change and we need to republish everything.

But in case of regular 'distribution' run, we should ignore all S3 objects with the delimiter /dcc-rl/, to speed up things.

@hilmarf hilmarf requested review from a team, T-Bendrath, AliceBilc and ioangut May 3, 2022 13:03
@hilmarf hilmarf added this to the 2.22 milestone May 3, 2022
@hilmarf hilmarf enabled auto-merge (squash) May 3, 2022 13:03
@hilmarf hilmarf disabled auto-merge May 3, 2022 13:31
ioangut
ioangut previously approved these changes May 3, 2022
View reviewed changes
@hilmarf hilmarf mentioned this pull request May 3, 2022
Merged
@hilmarf hilmarf requested a review from ioangut May 3, 2022 15:26
@hilmarf hilmarf enabled auto-merge (squash) May 3, 2022 15:35
@sonarqubecloud
Copy link

sonarqubecloud bot commented May 4, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

80.9% 80.9% Coverage
0.0% 0.0% Duplication

@hilmarf hilmarf merged commit b03357a into release/2.22 May 4, 2022
@hilmarf hilmarf deleted the HF/2.22.2 branch May 4, 2022 11:58
hilmarf added a commit that referenced this pull request May 4, 2022
@hilmarf @dependabot @ioangut
Merge/2.22 into main (#1819)
604b124 
* release 2.22

* Merge/main into 2.22 (#1808)

* Bump spring-boot.version from 2.6.6 to 2.6.7 (#1804)

Bumps `spring-boot.version` from 2.6.6 to 2.6.7.

Updates `spring-boot-dependencies` from 2.6.6 to 2.6.7
- [Release notes](https://github.com/spring-projects/spring-boot/releases)
- [Commits](spring-projects/spring-boot@v2.6.6...v2.6.7)

Updates `spring-boot-starter` from 2.6.6 to 2.6.7
- [Release notes](https://github.com/spring-projects/spring-boot/releases)
- [Commits](spring-projects/spring-boot@v2.6.6...v2.6.7)

Updates `spring-boot-starter-log4j2` from 2.6.6 to 2.6.7
- [Release notes](https://github.com/spring-projects/spring-boot/releases)
- [Commits](spring-projects/spring-boot@v2.6.6...v2.6.7)

Updates `spring-boot-starter-web` from 2.6.6 to 2.6.7
- [Release notes](https://github.com/spring-projects/spring-boot/releases)
- [Commits](spring-projects/spring-boot@v2.6.6...v2.6.7)

Updates `spring-boot-starter-test` from 2.6.6 to 2.6.7
- [Release notes](https://github.com/spring-projects/spring-boot/releases)
- [Commits](spring-projects/spring-boot@v2.6.6...v2.6.7)

Updates `spring-boot-maven-plugin` from 2.6.6 to 2.6.7
- [Release notes](https://github.com/spring-projects/spring-boot/releases)
- [Commits](spring-projects/spring-boot@v2.6.6...v2.6.7)

---
updated-dependencies:
- dependency-name: org.springframework.boot:spring-boot-dependencies
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.springframework.boot:spring-boot-starter
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.springframework.boot:spring-boot-starter-log4j2
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.springframework.boot:spring-boot-starter-web
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.springframework.boot:spring-boot-starter-test
  dependency-type: direct:development
  update-type: version-update:semver-patch
- dependency-name: org.springframework.boot:spring-boot-maven-plugin
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump protobuf.version from 3.20.0 to 3.20.1 (#1805)

Bumps `protobuf.version` from 3.20.0 to 3.20.1.

Updates `protobuf-java` from 3.20.0 to 3.20.1
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/main/generate_changelog.py)
- [Commits](https://github.com/protocolbuffers/protobuf/commits)

Updates `protobuf-java-util` from 3.20.0 to 3.20.1
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/main/generate_changelog.py)
- [Commits](https://github.com/protocolbuffers/protobuf/commits)

---
updated-dependencies:
- dependency-name: com.google.protobuf:protobuf-java
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: com.google.protobuf:protobuf-java-util
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump checkstyle from 10.1 to 10.2 (#1807)

Bumps [checkstyle](https://github.com/checkstyle/checkstyle) from 10.1 to 10.2.
- [Release notes](https://github.com/checkstyle/checkstyle/releases)
- [Commits](checkstyle/checkstyle@checkstyle-10.1...checkstyle-10.2)

---
updated-dependencies:
- dependency-name: com.puppycrawl.tools:checkstyle
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Feature/revocation etag (#1806)

* rename package

* fetch etag via http HEAD

* code coverage

* update docs

* add ETag repository + service

* add ConstructorWithParams

* abort further processing if DCC etag exists already

* store DCC etag when revocation list was fetched

* keep latest DCC etag in memory to be able to store it after successful payload storage

* fix link to Zenko CloudServer

* test cover DccRevocationListService

* RetentionPolicyTestRevocation

* package private makes it easier to test

* more code coverage

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Payload Signature verification (#1811)

* fix NPE issue with http HEAD (#1809)

* feign can't handle empty/null reponse body :-(

* move DB truncation into the same transaction as the insert, otherwise we might run into inconsistency!

* RetentionPolicyDccRevocationTest

* add DccRevocationClientDelegatorTest

* Add handling of FetchDccListException in Retention runner (#1818)

* ensure content-type zip for all *dcc-rl* entries

* switch from s3 ListObjectsV2 to ListObjects (#1821)

* switch from s3 ListObjectsV2 to ListObjects

* Update tests accordingly with the approach used

* introduce 'delimiter' on s3 ListObjectsRequest

* fix javadoc

Co-authored-by: Gut Ioan <ioan.gut@sap.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: ioangut <67064882+ioangut@users.noreply.github.com>
Co-authored-by: Gut Ioan <ioan.gut@sap.com>
@dsarkar dsarkar mentioned this pull request May 11, 2022
Closed
3 tasks
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@mlenkeit mlenkeit mlenkeit left review comments

@ioangut ioangut ioangut approved these changes

@T-Bendrath T-Bendrath Awaiting requested review from T-Bendrath

@AliceBilc AliceBilc Awaiting requested review from AliceBilc

Assignees

@AliceBilc AliceBilc

@ioangut ioangut

Labels
None yet
Projects
None yet
Milestone
2.22
Development

Successfully merging this pull request may close these issues.
4 participants
@hilmarf @mlenkeit @ioangut @AliceBilc