[destination-snowflake] SnowflakeSQLException: SQL compilation error

[talnidam]

Connector Name

destination-snowflake

Connector Version

3.9.0

What step the error happened?

During the sync

Relevant information

This is a query that Airbyte executed in my Snowflake env in the latest snowflake version:
PUT file:///tmp/2f680376-2d30-445f-bcbe-xxxx.csv.gz @"airbyte_internal"."clientx_raw__stream_postgres_table"xxx-FCB5-4B1F-B425-7EED5EE5F2D4/ PARALLEL = 128;
Error:
SQL compilation error: invalid value [128] for parameter 'PARALLEL'

Relevant log output

Stack Trace: java.lang.RuntimeException: Exceptions thrown while uploading records into stage: net.snowflake.client.jdbc.SnowflakeSQLException: SQL compilation error:
invalid value [128] for parameter 'PARALLEL'
net.snowflake.client.jdbc.SnowflakeSQLException: SQL compilation error:
invalid value [128] for parameter 'PARALLEL'
net.snowflake.client.jdbc.SnowflakeSQLException: SQL compilation error:
invalid value [128] for parameter 'PARALLEL'
        at io.airbyte.integrations.destination.snowflake.operation.SnowflakeStagingClient.uploadRecordsToStage(SnowflakeStagingClient.kt:46)
        at io.airbyte.integrations.destination.snowflake.operation.SnowflakeStorageOperation.writeToStage(SnowflakeStorageOperation.kt:66)
        at io.airbyte.integrations.destination.snowflake.operation.SnowflakeStorageOperation.writeToStage(SnowflakeStorageOperation.kt:27)
        at io.airbyte.cdk.integrations.destination.staging.operation.StagingStreamOperations.writeRecords(StagingStreamOperations.kt:52)
        at io.airbyte.integrations.destination.snowflake.SnowflakeDestination.check(SnowflakeDestination.kt:153)
        at io.airbyte.cdk.integrations.base.IntegrationRunner.runInternal(IntegrationRunner.kt:160)
        at io.airbyte.cdk.integrations.base.IntegrationRunner.run(IntegrationRunner.kt:116)
        at io.airbyte.cdk.integrations.base.adaptive.AdaptiveDestinationRunner$Runner.run(AdaptiveDestinationRunner.kt:68)
        at io.airbyte.integrations.destination.snowflake.SnowflakeDestinationKt.main(SnowflakeDestination.kt:348)

Contribute

• Yes, I want to contribute

Tasks

Beta Give feedback

No tasks being tracked yet.

[talnidam]

Path: airbyte-integrations/connectors/destination-snowflake/src/main/kotlin/io/airbyte/integrations/destination/snowflake/operation/SnowflakeStagingClient.kt

Possible solution:

Line: 81

internal fun getPutQuery(stageName: String, stagingPath: String, filePath: String): String {
    val availableProcessors = Runtime.getRuntime().availableProcessors()
    val parallelValue = minOf(availableProcessors, 99)
    return String.format(
        PUT_FILE_QUERY,
        filePath,
        stageName,
        stagingPath,
        parallelValue
    )
}

[marcosmarxm]

I raised the issue to the connector team investigate @talnidam thanks for sharing.

[gisripa]

Fixed in #39135