Skip to content

ParquetWriter is incompatible with JDK25 #3328

@hugoncosta

Description

@hugoncosta

Describe the bug, including details regarding any error messages, version, and platform.

Hello, I'm trying to upgrade my code base to JDK25 and I'm having this issue on my tests

     java.lang.UnsupportedOperationException: getSubject is not supported
        at java.base/javax.security.auth.Subject.getSubject(Subject.java:277)
        at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:577)
        at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:3852)
        at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:3842)
        at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3630)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:557)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:290)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:541)
        at org.apache.hadoop.fs.Path.getFileSystem(Path.java:365)
        at org.apache.parquet.hadoop.util.HadoopOutputFile.fromPath(HadoopOutputFile.java:58)
        at com.amazon.networkvalidator.parquet.ParquetWriter.write(ParquetWriter.kt:75)
        at com.amazon.networkvalidator.parquet.ParquetWriterTest$test writing to parquet$1.invokeSuspend(ParquetWriterTest.kt:88)
        at com.amazon.networkvalidator.parquet.ParquetWriterTest$test writing to parquet$1.invoke(ParquetWriterTest.kt)
        at com.amazon.networkvalidator.parquet.ParquetWriterTest$test writing to parquet$1.invoke(ParquetWriterTest.kt)
        at kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt$runTest$2$1$1.invokeSuspend(TestBuilders.kt:318)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:101)
        at kotlinx.coroutines.test.TestDispatcher.processEvent$kotlinx_coroutines_test(TestDispatcher.kt:24)
        at kotlinx.coroutines.test.TestCoroutineScheduler.tryRunNextTaskUnless$kotlinx_coroutines_test(TestCoroutineScheduler.kt:99)
        at kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt$runTest$2$1$workRunner$1.invokeSuspend(TestBuilders.kt:327)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:101)
        at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:263)
        at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:95)
        at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:69)
        at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
        at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:47)
        at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
        at kotlinx.coroutines.test.TestBuildersJvmKt.createTestResult(TestBuildersJvm.kt:10)
        at kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt.runTest-8Mi8wO0(TestBuilders.kt:310)
        at kotlinx.coroutines.test.TestBuildersKt.runTest-8Mi8wO0(Unknown Source)
        at kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt.runTest-8Mi8wO0(TestBuilders.kt:168)
        at kotlinx.coroutines.test.TestBuildersKt.runTest-8Mi8wO0(Unknown Source)
        at kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt.runTest-8Mi8wO0$default(TestBuilders.kt:160)
        at kotlinx.coroutines.test.TestBuildersKt.runTest-8Mi8wO0$default(Unknown Source)
        at com.amazon.networkvalidator.parquet.ParquetWriterTest.test writing to parquet(ParquetWriterTest.kt:76) 

It appears this is an issue within the Hadoop libraries upon which Parquet Java depends on

Component(s)

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions