Skip to content

Add the short type conversion #164

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
May 5, 2025
Merged

Conversation

s-vitaliy
Copy link
Contributor

@s-vitaliy s-vitaliy commented May 2, 2025

Resolves #163

We convert all short integers to Int since it seems that Iceberg does not support Short integer types.

Copy link

github-actions bot commented May 5, 2025

File Coverage
All files 73%
logging/ZIOLogAnnotations.scala 74%
models/ArcaneSchema.scala 92%
models/DataCell.scala 0%
models/DataCell.scala 0%
models/ArcaneSchema.scala 0%
models/DataCell.scala 71%
models/app/StreamContext.scala 0%
models/cdm/SimpleCdmModel.scala 88%
models/cdm/CdmParser.scala 89%
models/cdm/CdmParser.scala 93%
models/querygen/MergeQuerySegment.scala 88%
models/querygen/MergeQuerySegment.scala 85%
models/querygen/MergeQuery.scala 85%
models/querygen/InitializeQuery.scala 0%
models/settings/TablePropertiesSettings.scala 98%
services/app/StreamRunnerServiceImpl.scala 0%
services/app/PosixStreamLifetimeService.scala 0%
services/consumers/SqlServerChangeTracking.scala 86%
services/consumers/SqlServerChangeTracking.scala 90%
services/filters/FieldsFilteringService.scala 91%
services/filters/ColumnSummaryFieldsFilteringService.scala 88%
services/hooks/manager/EmptyHookManager.scala 0%
services/hooks/manager/EmptyHookManager.scala 0%
services/lakehouse/IcebergS3CatalogWriter.scala 90%
services/lakehouse/SchemaConversions.scala 28%
services/lakehouse/IcebergCatalogCredential.scala 60%
services/merging/JdbcMergeServiceClient.scala 36%
services/merging/JdbcMergeServiceClient.scala 94%
services/merging/MutableSchemaCache.scala 97%
services/mssql/MsSqlDataProvider.scala 62%
services/mssql/MsSqlBackfillOverwriteBatchFactory.scala 0%
services/mssql/MsSqlDataProvider.scala 68%
services/mssql/MsSqlStreamingDataProvider.scala 61%
services/mssql/QueryProvider.scala 93%
services/mssql/MsSqlConnection.scala 92%
services/mssql/SqlSchema.scala 83%
services/mssql/MsSqlHookManager.scala 0%
services/mssql/query/ScalarQueryResult.scala 66%
services/mssql/query/LazyQueryResult.scala 80%
services/storage/models/amazon/AmazonS3StoragePath.scala 80%
services/storage/models/azure/AzureModelConversions.scala 98%
services/storage/models/azure/AdlsStoragePath.scala 86%
services/storage/models/base/StoredBlob.scala 0%
services/storage/services/AzureBlobStorageReader.scala 71%
services/streaming/base/StreamGraphBuilder.scala 0%
services/streaming/graph_builders/GenericStreamingGraphBuilder.scala 48%
services/streaming/graph_builders/backfill/GenericBackfillMergeGraphBuilder.scala 83%
services/streaming/graph_builders/backfill/GenericBackfillOverwriteGraphBuilder.scala 53%
services/streaming/processors/batch_processors/backfill/BackfillDisposeBatchProcessor.scala 0%
services/streaming/processors/batch_processors/backfill/BackfillApplyBatchProcessor.scala 87%
services/streaming/processors/batch_processors/streaming/DisposeBatchProcessor.scala 68%
services/streaming/processors/transformers/StagingProcessor.scala 87%
services/synapse/SynapseLinkStreamingDataProvider.scala 72%
services/synapse/SynapseEntitySchemaProvider.scala 84%
services/synapse/SynapseBackfillOverwriteBatchFactory.scala 0%
services/synapse/SynapseHookManager.scala 95%
services/synapse/base/SynapseLinkReader.scala 60%
services/synapse/base/SynapseLinkDataProvider.scala 52%
utils/SqlUtils.scala 55%

Minimum allowed coverage is 40%

Generated by 🐒 cobertura-action against 96a9213

@s-vitaliy s-vitaliy merged commit 979b0b8 into main May 5, 2025
3 checks passed
@s-vitaliy s-vitaliy deleted the short-type-in-iceberg-support branch May 5, 2025 09:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

SQL Short not supported in IcebergWriter
2 participants