Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 Snowflake Destination: use better file size with S3 staging files #9920

Merged
merged 21 commits into from
Feb 15, 2022
Merged
Changes from 1 commit
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
6eb7333
split s3 staging files to files by 100 Mb and removed legacyS3StreamC…
andriikorotkov Jan 31, 2022
bebeef8
split s3 staging files to files by 100 Mb and removed legacyS3StreamC…
andriikorotkov Jan 31, 2022
c3f4dc8
Merge branch 'master' of github.com:airbytehq/airbyte into akorotkov/…
andriikorotkov Jan 31, 2022
1811359
Merge branch 'master' of github.com:airbytehq/airbyte into akorotkov/…
andriikorotkov Feb 1, 2022
6dd1c4b
updated code style
andriikorotkov Feb 1, 2022
d764804
fix remarks
andriikorotkov Feb 2, 2022
1641f46
fix remarks
andriikorotkov Feb 2, 2022
3569bfc
fix code style
andriikorotkov Feb 3, 2022
2e4480b
fix remarks
andriikorotkov Feb 3, 2022
9890715
Merge branch 'master' of github.com:airbytehq/airbyte into akorotkov/…
andriikorotkov Feb 3, 2022
1f2bc7d
Merge branch 'master' of github.com:airbytehq/airbyte into akorotkov/…
andriikorotkov Feb 7, 2022
a6004f4
fix remarks
andriikorotkov Feb 7, 2022
ff04702
Merge branch 'master' of github.com:airbytehq/airbyte into akorotkov/…
andriikorotkov Feb 8, 2022
18a915f
fix remarks
andriikorotkov Feb 8, 2022
219f59f
Merge branch 'master' of github.com:airbytehq/airbyte into akorotkov/…
andriikorotkov Feb 8, 2022
644d4f5
Merge branch 'master' of github.com:airbytehq/airbyte into akorotkov/…
andriikorotkov Feb 9, 2022
b54ab2f
Merge branch 'master' of github.com:airbytehq/airbyte into akorotkov/…
andriikorotkov Feb 11, 2022
2991552
updated documentations and images versions
andriikorotkov Feb 11, 2022
cb6db6e
Merge branch 'master' of github.com:airbytehq/airbyte into akorotkov/…
andriikorotkov Feb 14, 2022
956f066
Merge branch 'master' of github.com:airbytehq/airbyte into akorotkov/…
andriikorotkov Feb 15, 2022
08840f9
updated documentation
andriikorotkov Feb 15, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Merge branch 'master' of github.com:airbytehq/airbyte into akorotkov/…
…6517-snowflake

� Conflicts:
�	airbyte-integrations/connectors/destination-jdbc/src/main/java/io/airbyte/integrations/destination/jdbc/copy/s3/S3StreamCopier.java
  • Loading branch information
andriikorotkov committed Jan 31, 2022
commit c3f4dc8664af1eb09365efd704fc4c5fe6c5da14
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ public abstract class S3StreamCopier implements StreamCopier {
private final ConfiguredAirbyteStream configuredAirbyteStream;
private final Timestamp uploadTime;
protected final String stagingFolder;
protected final Map<String, S3Writer> stagingWritersByFile = new HashMap<>();
protected final Map<String, DestinationFileWriter> stagingWritersByFile = new HashMap<>();
protected final Set<String> activeStagingWriterFileNames = new HashSet<>();
protected final Set<String> stagingFileNames = new LinkedHashSet<>();
private final boolean purgeStagingData;
Expand Down
You are viewing a condensed version of this merge commit. You can view the full changes here.