Skip to content

GCS visibility archival filters match values in unrelated filename fields #12506

Description

@Garyouki

Expected Behavior

GCS visibility archival queries using WorkflowId, RunId, or WorkflowType should return only records whose corresponding field matches the requested value.

Actual Behavior

The filename preconditions in common/archiver/gcloud/util.go search the whole object name for the requested hash. After splitting the name, they check whether the original string contains a piece extracted from that same string, which is always true. Thus a hash in a different field or object-path prefix can satisfy a filter. Substring matches also pass.

Additionally, visibilityArchiver.queryPrefix uses newWorkflowIDPrecondition for both RunId and WorkflowType filters instead of their field-specific preconditions.

Steps to Reproduce the Problem

  1. Use three archived visibility records with the same close time. Put the string target in the WorkflowType of the first, WorkflowId of the second, and RunId of the third. Give their other fields different values.
  2. Query WorkflowId = 'target' AND CloseTime = '2020-02-05T12:00:00Z' AND SearchPrecision = 'Day'.
  3. Observe that all three records pass the generated filters, although only the second should match.
  4. The equivalent queries for RunId and WorkflowType also incorrectly return all three records. This does not require a hash collision: the same input string hashes identically across fields.

I reproduced this through the real visibility archiver Query method and parser, with mocked storage applying the actual generated preconditions. All three query regression cases return three executions instead of one before the fix. Separate precondition tests reproduce cross-field, substring, path-prefix, and malformed-filename matches. The reproduction does not require a live GCS bucket.

Command with the regression tests added:

go test -tags test_dep ./common/archiver/gcloud -run '^TestVisibility(FieldPreconditions|QueryFieldFilters)$' -count=1

Specifications

  • Version: source commit f6279f9c0 (current main when reproduced)
  • Platform: WSL Linux amd64, Go 1.27.0
  • Backend: GCS visibility archival

This is separate from the pagination completion issue #12493.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions