Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pandaproxy/sr: Improve schema lookup #25080

Merged

Conversation

IoannisRP
Copy link
Contributor

@IoannisRP IoannisRP commented Feb 13, 2025

Change lookup strategy to always apply active transformation to existing schemas, instead of only applying it when normalisation is turned on.

Motivation:

This allows a specific workflow performed by SR clients following this pattern to lookup schema IDs:

  • POST /subject/sub/versions?normalize=true -d {schema}
  • GET /subject/sub/versions/latest -> this returns the schema and latest version but NOT IDs
  • POST /subject/sub -d {schema-from-get} -> this returns ID

In our SR, a call to POST /subject/sub without normalize=true will result in this schema being sanitized which may change the content of the schema. With this change, both input schema and lookup schema go through the same operation and still be deemed equal.

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v24.3.x
  • v24.2.x
  • v24.1.x

Release Notes

  • none

Change lookup strategy to always apply active transformation to
existing schemas, instead of only applying it when normalisation is
turned on.
@IoannisRP IoannisRP requested a review from BenPope February 13, 2025 16:56
@IoannisRP IoannisRP requested a review from a team February 13, 2025 16:56
@vbotbuildovich
Copy link
Collaborator

CI test results

test results on build#61843
test_id test_kind job_url test_status passed
rptest.tests.compaction_recovery_test.CompactionRecoveryTest.test_index_recovery ducktape https://buildkite.com/redpanda/redpanda/builds/61843#01950084-c44a-4035-a07a-1f9b982d678c FLAKY 1/2
rptest.tests.compaction_recovery_test.CompactionRecoveryTest.test_index_recovery ducktape https://buildkite.com/redpanda/redpanda/builds/61843#019500a1-0b2c-46b3-8096-8bf14f420977 FLAKY 1/2
rptest.tests.log_compaction_test.LogCompactionTest.compaction_stress_test.cleanup_policy=compact.key_set_cardinality=1000.storage_compaction_key_map_memory_kb=10 ducktape https://buildkite.com/redpanda/redpanda/builds/61843#019500a1-0b2b-453d-8433-deb9178592fe FLAKY 1/2
rptest.tests.scaling_up_test.ScalingUpTest.test_scaling_up_with_recovered_topic ducktape https://buildkite.com/redpanda/redpanda/builds/61843#019500a1-0b2c-46b3-8096-8bf14f420977 FLAKY 1/2

@IoannisRP IoannisRP merged commit 1e5241f into redpanda-data:dev Feb 13, 2025
18 checks passed
@vbotbuildovich
Copy link
Collaborator

/backport v24.3.x

@vbotbuildovich
Copy link
Collaborator

/backport v24.2.x

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants