Skip to content

Conversation

@dependabot
Copy link
Contributor

@dependabot dependabot bot commented on behalf of github Oct 6, 2024

Bumps parquet from 1.13.1 to 1.14.3.
Updates org.apache.parquet:parquet-avro from 1.13.1 to 1.14.3

Release notes

Sourced from org.apache.parquet:parquet-avro's releases.

Apache Parquet Java 1.14.3

What's Changed

  • GH-3007: Ensure version specific Jackson classes are shaded
  • GH-3013: Fix potential ClassCastException at reading DELTA_BYTE_ARRAY encoding
  • GH-3021: Upgrade Avro dependency to 1.11.4

Apache Parquet Java 1.14.3 RC2

What's Changed

  • GH-3007: Ensure version specific Jackson classes are shaded
  • GH-3013: Fix potential ClassCastException at reading DELTA_BYTE_ARRAY encoding
  • GH-3021: Upgrade Avro dependency to 1.11.4

Apache Parquet Java 1.14.2

What's Changed

  • GH-2948: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • GH-2956: Use avro SchemaBuilder API to convert record
  • GH-2935: Avoid double close of ParquetFileWriter
  • GH-2992: Gate LocalTimestamp references in AvroSchemaConverter
  • PARQUET-1126: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • PARQUET-1126: Write unencrypted Parquet files without Hadoop
  • PARQUET-2472: Close in finally block in ParquetFileWriter#end

Apache Parquet Java 1.14.2 RC2

What's Changed

  • GH-2948: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • GH-2956: Use avro SchemaBuilder API to convert record
  • GH-2935: Avoid double close of ParquetFileWriter
  • GH-2992: Gate LocalTimestamp references in AvroSchemaConverter
  • PARQUET-1126: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • PARQUET-1126: Write unencrypted Parquet files without Hadoop
  • PARQUET-2472: Close in finally block in ParquetFileWriter#end

Apache Parquet Java 1.14.2 RC1

What's Changed

  • GH-2948: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • GH-2956: Use avro SchemaBuilder API to convert record
  • GH-2935: Avoid double close of ParquetFileWriter
  • PARQUET-1126: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • PARQUET-1126: Write unencrypted Parquet files without Hadoop
  • PARQUET-2472: Close in finally block in ParquetFileWriter#end

Apache Parquet 1.14.1

What's Changed

... (truncated)

Changelog

Sourced from org.apache.parquet:parquet-avro's changelog.

Parquet

Version 1.14.1

Release Notes - Parquet - Version 1.14.1

Bug

  • PARQUET-2468 - ParquetMetadata.toPrettyJSON throws exception on file read when LOG.isDebugEnabled()
  • PARQUET-2498 - Hadoop vector IO API doesn't handle empty list of ranges

Version 1.14.0

Release Notes - Parquet - Version 1.14.0

Bug

  • PARQUET-2260 - Bloom filter bytes size shouldn't be larger than maxBytes size in the configuration
  • PARQUET-2266 - Fix support for files without ColumnIndexes
  • PARQUET-2276 - ParquetReader reads do not work with Hadoop version 2.8.5
  • PARQUET-2300 - Update jackson-core 2.13.4 to a version without CVE PRISMA-2023-0067
  • PARQUET-2325 - Fix parquet-cli's dictionary subcommand to work with FIXED_LEN_BYTE_ARRAY
  • PARQUET-2329 - Fix wrong help messages of parquet-cli subcommands
  • PARQUET-2330 - Fix convert-csv to show the correct position of the invalid record
  • PARQUET-2332 - Fix unexpectedly disabled tests to be executed
  • PARQUET-2336 - Add caching key to CodecFactory
  • PARQUET-2342 - Parquet writer produced a corrupted file due to page value count overflow
  • PARQUET-2343 - Fixes NPE when rewriting file with multiple rowgroups
  • PARQUET-2348 - Recompression/Re-encrypt should rewrite bloomfilter
  • PARQUET-2354 - Apparent race condition in CharsetValidator
  • PARQUET-2363 - ParquetRewriter should encrypt the V2 page header

... (truncated)

Commits
  • b5e376a [maven-release-plugin] prepare release apache-parquet-1.14.3-rc2
  • 7d970b7 GH-3021: Upgrade Avro dependency (#3022)
  • 4257337 [maven-release-plugin] prepare for next development iteration
  • cf1efcc [maven-release-plugin] prepare release apache-parquet-1.14.3-rc0
  • b1475a7 GH-3013: Fix potential ClassCastException at reading DELTA_BYTE_ARRAY encodin...
  • aec24e7 MINOR: Don't run all the tests on a release (#2999)
  • 2734728 GH-3007: Ensure version specific Jackson classes are shaded (#3017)
  • 7b6753d MINOR: fix version of parquet-plugins
  • dde627c Prepare for next development iteration
  • 2245b30 [maven-release-plugin] prepare for next development iteration
  • Additional commits viewable in compare view

Updates org.apache.parquet:parquet-column from 1.13.1 to 1.14.3

Release notes

Sourced from org.apache.parquet:parquet-column's releases.

Apache Parquet Java 1.14.3

What's Changed

  • GH-3007: Ensure version specific Jackson classes are shaded
  • GH-3013: Fix potential ClassCastException at reading DELTA_BYTE_ARRAY encoding
  • GH-3021: Upgrade Avro dependency to 1.11.4

Apache Parquet Java 1.14.3 RC2

What's Changed

  • GH-3007: Ensure version specific Jackson classes are shaded
  • GH-3013: Fix potential ClassCastException at reading DELTA_BYTE_ARRAY encoding
  • GH-3021: Upgrade Avro dependency to 1.11.4

Apache Parquet Java 1.14.2

What's Changed

  • GH-2948: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • GH-2956: Use avro SchemaBuilder API to convert record
  • GH-2935: Avoid double close of ParquetFileWriter
  • GH-2992: Gate LocalTimestamp references in AvroSchemaConverter
  • PARQUET-1126: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • PARQUET-1126: Write unencrypted Parquet files without Hadoop
  • PARQUET-2472: Close in finally block in ParquetFileWriter#end

Apache Parquet Java 1.14.2 RC2

What's Changed

  • GH-2948: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • GH-2956: Use avro SchemaBuilder API to convert record
  • GH-2935: Avoid double close of ParquetFileWriter
  • GH-2992: Gate LocalTimestamp references in AvroSchemaConverter
  • PARQUET-1126: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • PARQUET-1126: Write unencrypted Parquet files without Hadoop
  • PARQUET-2472: Close in finally block in ParquetFileWriter#end

Apache Parquet Java 1.14.2 RC1

What's Changed

  • GH-2948: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • GH-2956: Use avro SchemaBuilder API to convert record
  • GH-2935: Avoid double close of ParquetFileWriter
  • PARQUET-1126: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • PARQUET-1126: Write unencrypted Parquet files without Hadoop
  • PARQUET-2472: Close in finally block in ParquetFileWriter#end

Apache Parquet 1.14.1

What's Changed

... (truncated)

Changelog

Sourced from org.apache.parquet:parquet-column's changelog.

Parquet

Version 1.14.1

Release Notes - Parquet - Version 1.14.1

Bug

  • PARQUET-2468 - ParquetMetadata.toPrettyJSON throws exception on file read when LOG.isDebugEnabled()
  • PARQUET-2498 - Hadoop vector IO API doesn't handle empty list of ranges

Version 1.14.0

Release Notes - Parquet - Version 1.14.0

Bug

  • PARQUET-2260 - Bloom filter bytes size shouldn't be larger than maxBytes size in the configuration
  • PARQUET-2266 - Fix support for files without ColumnIndexes
  • PARQUET-2276 - ParquetReader reads do not work with Hadoop version 2.8.5
  • PARQUET-2300 - Update jackson-core 2.13.4 to a version without CVE PRISMA-2023-0067
  • PARQUET-2325 - Fix parquet-cli's dictionary subcommand to work with FIXED_LEN_BYTE_ARRAY
  • PARQUET-2329 - Fix wrong help messages of parquet-cli subcommands
  • PARQUET-2330 - Fix convert-csv to show the correct position of the invalid record
  • PARQUET-2332 - Fix unexpectedly disabled tests to be executed
  • PARQUET-2336 - Add caching key to CodecFactory
  • PARQUET-2342 - Parquet writer produced a corrupted file due to page value count overflow
  • PARQUET-2343 - Fixes NPE when rewriting file with multiple rowgroups
  • PARQUET-2348 - Recompression/Re-encrypt should rewrite bloomfilter
  • PARQUET-2354 - Apparent race condition in CharsetValidator
  • PARQUET-2363 - ParquetRewriter should encrypt the V2 page header

... (truncated)

Commits
  • b5e376a [maven-release-plugin] prepare release apache-parquet-1.14.3-rc2
  • 7d970b7 GH-3021: Upgrade Avro dependency (#3022)
  • 4257337 [maven-release-plugin] prepare for next development iteration
  • cf1efcc [maven-release-plugin] prepare release apache-parquet-1.14.3-rc0
  • b1475a7 GH-3013: Fix potential ClassCastException at reading DELTA_BYTE_ARRAY encodin...
  • aec24e7 MINOR: Don't run all the tests on a release (#2999)
  • 2734728 GH-3007: Ensure version specific Jackson classes are shaded (#3017)
  • 7b6753d MINOR: fix version of parquet-plugins
  • dde627c Prepare for next development iteration
  • 2245b30 [maven-release-plugin] prepare for next development iteration
  • Additional commits viewable in compare view

Updates org.apache.parquet:parquet-hadoop from 1.13.1 to 1.14.3

Release notes

Sourced from org.apache.parquet:parquet-hadoop's releases.

Apache Parquet Java 1.14.3

What's Changed

  • GH-3007: Ensure version specific Jackson classes are shaded
  • GH-3013: Fix potential ClassCastException at reading DELTA_BYTE_ARRAY encoding
  • GH-3021: Upgrade Avro dependency to 1.11.4

Apache Parquet Java 1.14.3 RC2

What's Changed

  • GH-3007: Ensure version specific Jackson classes are shaded
  • GH-3013: Fix potential ClassCastException at reading DELTA_BYTE_ARRAY encoding
  • GH-3021: Upgrade Avro dependency to 1.11.4

Apache Parquet Java 1.14.2

What's Changed

  • GH-2948: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • GH-2956: Use avro SchemaBuilder API to convert record
  • GH-2935: Avoid double close of ParquetFileWriter
  • GH-2992: Gate LocalTimestamp references in AvroSchemaConverter
  • PARQUET-1126: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • PARQUET-1126: Write unencrypted Parquet files without Hadoop
  • PARQUET-2472: Close in finally block in ParquetFileWriter#end

Apache Parquet Java 1.14.2 RC2

What's Changed

  • GH-2948: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • GH-2956: Use avro SchemaBuilder API to convert record
  • GH-2935: Avoid double close of ParquetFileWriter
  • GH-2992: Gate LocalTimestamp references in AvroSchemaConverter
  • PARQUET-1126: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • PARQUET-1126: Write unencrypted Parquet files without Hadoop
  • PARQUET-2472: Close in finally block in ParquetFileWriter#end

Apache Parquet Java 1.14.2 RC1

What's Changed

  • GH-2948: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • GH-2956: Use avro SchemaBuilder API to convert record
  • GH-2935: Avoid double close of ParquetFileWriter
  • PARQUET-1126: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • PARQUET-1126: Write unencrypted Parquet files without Hadoop
  • PARQUET-2472: Close in finally block in ParquetFileWriter#end

Apache Parquet 1.14.1

What's Changed

... (truncated)

Changelog

Sourced from org.apache.parquet:parquet-hadoop's changelog.

Parquet

Version 1.14.1

Release Notes - Parquet - Version 1.14.1

Bug

  • PARQUET-2468 - ParquetMetadata.toPrettyJSON throws exception on file read when LOG.isDebugEnabled()
  • PARQUET-2498 - Hadoop vector IO API doesn't handle empty list of ranges

Version 1.14.0

Release Notes - Parquet - Version 1.14.0

Bug

  • PARQUET-2260 - Bloom filter bytes size shouldn't be larger than maxBytes size in the configuration
  • PARQUET-2266 - Fix support for files without ColumnIndexes
  • PARQUET-2276 - ParquetReader reads do not work with Hadoop version 2.8.5
  • PARQUET-2300 - Update jackson-core 2.13.4 to a version without CVE PRISMA-2023-0067
  • PARQUET-2325 - Fix parquet-cli's dictionary subcommand to work with FIXED_LEN_BYTE_ARRAY
  • PARQUET-2329 - Fix wrong help messages of parquet-cli subcommands
  • PARQUET-2330 - Fix convert-csv to show the correct position of the invalid record
  • PARQUET-2332 - Fix unexpectedly disabled tests to be executed
  • PARQUET-2336 - Add caching key to CodecFactory
  • PARQUET-2342 - Parquet writer produced a corrupted file due to page value count overflow
  • PARQUET-2343 - Fixes NPE when rewriting file with multiple rowgroups
  • PARQUET-2348 - Recompression/Re-encrypt should rewrite bloomfilter
  • PARQUET-2354 - Apparent race condition in CharsetValidator
  • PARQUET-2363 - ParquetRewriter should encrypt the V2 page header

... (truncated)

Commits
  • b5e376a [maven-release-plugin] prepare release apache-parquet-1.14.3-rc2
  • 7d970b7 GH-3021: Upgrade Avro dependency (#3022)
  • 4257337 [maven-release-plugin] prepare for next development iteration
  • cf1efcc [maven-release-plugin] prepare release apache-parquet-1.14.3-rc0
  • b1475a7 GH-3013: Fix potential ClassCastException at reading DELTA_BYTE_ARRAY encodin...
  • aec24e7 MINOR: Don't run all the tests on a release (#2999)
  • 2734728 GH-3007: Ensure version specific Jackson classes are shaded (#3017)
  • 7b6753d MINOR: fix version of parquet-plugins
  • dde627c Prepare for next development iteration
  • 2245b30 [maven-release-plugin] prepare for next development iteration
  • Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot dependabot bot added dependencies Pull requests that update a dependency file java Pull requests that update Java code labels Oct 6, 2024
@findepi
Copy link
Member

findepi commented Oct 12, 2024

@dependabot rebase

@dependabot dependabot bot force-pushed the dependabot/gradle/parquet-1.14.3 branch from b9f6809 to 1d3d3e5 Compare October 12, 2024 19:13
@nastra nastra closed this Oct 18, 2024
@nastra nastra force-pushed the dependabot/gradle/parquet-1.14.3 branch from 1d3d3e5 to 9d58865 Compare October 18, 2024 07:16
@dependabot dependabot bot deleted the dependabot/gradle/parquet-1.14.3 branch October 18, 2024 07:16
@nastra nastra restored the dependabot/gradle/parquet-1.14.3 branch October 18, 2024 07:18
@nastra nastra deleted the dependabot/gradle/parquet-1.14.3 branch October 18, 2024 07:20
@nastra nastra reopened this Oct 18, 2024
@github-actions github-actions bot added the flink label Oct 18, 2024
Row binaryCol =
Row.of(
52L,
55L,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like column sizes got slightly larger:

Parquet 1.13.1

% parquet-cli/1.14.3/bin/parquet column-size old-version.parquet
binaryCol-> Size In Bytes: 52 Size In Ratio: 0.08695652
intCol-> Size In Bytes: 71 Size In Ratio: 0.1187291
decimalCol-> Size In Bytes: 85 Size In Ratio: 0.14214046
fixedCol-> Size In Bytes: 44 Size In Ratio: 0.073578596
booleanCol-> Size In Bytes: 32 Size In Ratio: 0.053511705
stringCol-> Size In Bytes: 79 Size In Ratio: 0.13210702
floatCol-> Size In Bytes: 71 Size In Ratio: 0.1187291
longCol-> Size In Bytes: 79 Size In Ratio: 0.13210702
doubleCol-> Size In Bytes: 85 Size In Ratio: 0.14214046

Parquet 1.14.3

% parquet-cli/1.14.3/bin/parquet column-size new-version.parquet
binaryCol-> Size In Bytes: 55 Size In Ratio: 0.085403726
intCol-> Size In Bytes: 77 Size In Ratio: 0.11956522
decimalCol-> Size In Bytes: 91 Size In Ratio: 0.14130434
fixedCol-> Size In Bytes: 47 Size In Ratio: 0.072981365
booleanCol-> Size In Bytes: 36 Size In Ratio: 0.055900622
stringCol-> Size In Bytes: 85 Size In Ratio: 0.13198757
floatCol-> Size In Bytes: 77 Size In Ratio: 0.11956522
longCol-> Size In Bytes: 85 Size In Ratio: 0.13198757
doubleCol-> Size In Bytes: 91 Size In Ratio: 0.14130434

@nastra nastra requested a review from findepi October 18, 2024 08:59
@findepi
Copy link
Member

findepi commented Oct 18, 2024

thank you @nastra !

@nastra nastra merged commit b8c2b20 into main Oct 21, 2024
51 of 89 checks passed
@Fokko Fokko mentioned this pull request Oct 30, 2024
RussellSpitzer added a commit to RussellSpitzer/iceberg that referenced this pull request Nov 4, 2024
This reverts commit b8c2b20.

apache/parquet-java#3040
Was discovered by @pan3793 in Parquet 1.14.(0,1,2,3).
RussellSpitzer added a commit that referenced this pull request Nov 4, 2024
RussellSpitzer added a commit to RussellSpitzer/iceberg that referenced this pull request Nov 4, 2024
RussellSpitzer added a commit that referenced this pull request Nov 4, 2024
Fokko added a commit to Fokko/iceberg that referenced this pull request Nov 8, 2024
Fokko added a commit that referenced this pull request Nov 20, 2024
* Revert "Revert "Build: Bump parquet from 1.13.1 to 1.14.3 (#11264)" (#11462)"

This reverts commit 7cc16fa.

* Bump to Parquet 1.14.4

* Lookup sizes instead

* Update build.gradle
zachdisc pushed a commit to zachdisc/iceberg that referenced this pull request Dec 23, 2024
Co-authored-by: Eduard Tudenhoefner <etudenhoefner@gmail.com>
zachdisc pushed a commit to zachdisc/iceberg that referenced this pull request Dec 23, 2024
zachdisc pushed a commit to zachdisc/iceberg that referenced this pull request Dec 23, 2024
* Revert "Revert "Build: Bump parquet from 1.13.1 to 1.14.3 (apache#11264)" (apache#11462)"

This reverts commit 7cc16fa.

* Bump to Parquet 1.14.4

* Lookup sizes instead

* Update build.gradle
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file flink java Pull requests that update Java code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants