Skip to content

chore(refactor): parsing logic into a reusable utility functions #665

@github-actions

Description

@github-actions

Problem Statement
The current VerifyIcebergSync test function contains tightly coupled query parsing, schema checks, and partition validation logic. This makes the function large, difficult to maintain. Refactoring is required to isolate parsing and validation into reusable utility functions.

Solution Summary
Extract the query execution, schema comparison, and partition parsing logic into dedicated utility functions. Keep VerifyIcebergSync focused only on orchestration and assertions. This reduces duplication, improves readability, and enables reuse across other Iceberg sync tests.

// TODO: Refactor parsing logic into a reusable utility functions
// verifyIcebergSync verifies that data was correctly synchronized to Iceberg
func VerifyIcebergSync(t *testing.T, tableName, icebergDB string, datatypeSchema map[string]string, schema map[string]interface{}, opSymbol, partitionRegex, driver string) {
t.Helper()
ctx := context.Background()
spark, err := sql.NewSessionBuilder().Remote(sparkConnectAddress).Build(ctx)

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions