HIVE-26829: Upgrade avro to 1.11.0

[Aggarwal-Raghav]

What changes were proposed in this pull request?

To upgrade avro version to 1.11.0

Why are the changes needed?

To address the CVE-2021-43045

Does this PR introduce any user-facing change?

No

How was this patch tested?

On local machine

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information

[cnauroth]

+1 (non-binding)

I applied the patch locally. I confirmed the new Avro version in mvn dependency:tree. I ran a full build with -Pdist -Piceberg and confirmed that the new version was getting bundled in the distro.

Thank you for the patch, @Aggarwal-Raghav .

[Aggarwal-Raghav]

How I tested this:
When on avro 1.8.2: Created a avro table and inserted some data to it.

Replaced avro jars with 1.11.0: Restarted the services and was able to read and insert data into the previously created avro table.

[brahmareddybattula]

@Aggarwal-Raghav thanks for reporting and working on this.

@cnauroth / @Aggarwal-Raghav , I have one query.?
How compatible is this with 1.9.2.? Did we already checked the hadoop change hence we removed the classifier.?
we need to check with hadoop as hadoop move to thirdparty
apache/hadoop-thirdparty#21 .?

[cnauroth]

Hello @brahmareddybattula . I reviewed Avro release notes between 1.9.2 and 1.11.0. I didn't find any mention of backward-incompatible changes, though I don't know if that's a definitive answer.

Hadoop noticed a potential incompatibility in the upgrade from 1.8.2. to 1.9.0, documented in HADOOP-13386. I'm wondering if the testing mentioned by @Aggarwal-Raghav is sufficient to show the upgrade is safe?

While researching, I noticed that Avro has released 1.11.1, so if we can confirm the upgrade is safe, then perhaps we want to jump to that.

I wasn't clear on what we should check with Hadoop, but if you want to start a conversation on the Hadoop dev list or tag other Hadoop contributors here, please do so.