[BUG] docker build fails when ECLIPSELINK arg is set to true

[manojmukkamala]

Is this a possible security vulnerability?

• This is NOT a possible security vulnerability

Describe the bug

Context: I am trying to use postgres as backend.

docker build fails when ECLIPSELINK arg is set to true.

I've added persistence-new.xml under polaris-service/src/main/resources/META-INF/persistence-new.xml and updated polaris-server.yml as follows:

metaStoreManager:
  # type: in-memory
  type: eclipse-link # uncomment to use eclipse-link as metastore
  conf-file: META-INF/persistence-new.xml
  persistence-unit: polaris-dev

When I remove the build-arg, the build was successful but the container cannot start with error:

io.dropwizard.configuration.ConfigurationParsingException: polaris-server.yml has an error:
 * Failed to parse configuration at: metaStoreManager; Could not resolve type id 'eclipse-link' as a subtype of `org.apache.polaris.core.persistence.MetaStoreManagerFactory`: known type ids = [in-memory] (for POJO property 'metaStoreManager')
at [Source: UNKNOWN; byte offset: #UNKNOWN] (through reference chain: org.apache.polaris.service.config.PolarisApplicationConfig["metaStoreManager"])

To Reproduce

docker build . -t polaris_local_metastore --build-arg ECLIPSELINK=true

Actual Behavior

77.81 All projects evaluated.
77.81 
77.81 FAILURE: Build failed with an exception.
77.81 
77.81 * What went wrong:
77.81 10 actionable tasks: 10 executed
77.81 Circular dependency between the following tasks:
77.81 :polaris-service:jar
77.81 \--- :polaris-service:shadowJar
77.81      \--- :polaris-service:jar (*)
77.81 
77.81 (*) - details omitted (listed previously)
77.81 
77.81 
77.81 * Try:
77.81 > Run with --stacktrace option to get the stack trace.
77.81 > Run with --debug option to get more log output.
77.81 > Run with --scan to get full insights.
77.81 > Get more help at https://help.gradle.org.
77.81 
77.81 BUILD FAILED in 1m 17s
77.87 Some of the file system contents retained in the virtual file system are on file systems that Gradle doesn't support watching. The relevant state was discarded to ensure changes to these locations are properly detected. You can override this by explicitly enabling file system watching.
------

 1 warning found (use docker --debug to expand):
 - FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 23)
Dockerfile:34
--------------------
  32 |     
  33 |     # Build the rest catalog
  34 | >>> RUN ./gradlew --no-daemon --info -PeclipseLink=$ECLIPSELINK clean prepareDockerDist
  35 |     
  36 |     FROM registry.access.redhat.com/ubi9/openjdk-21-runtime:1.20-2.1721752928
--------------------
ERROR: failed to solve: process "/bin/sh -c ./gradlew --no-daemon --info -PeclipseLink=$ECLIPSELINK clean prepareDockerDist" did not complete successfully: exit code: 1

Expected Behavior

A successful build.

Additional context

NA

System information

Host: Mac OS
Docker Desktop: v4.34.0

[eric-maynard]

I think #227 might be relevant here?

[manojmukkamala]

I see the PR is still open so I took the code from run.sh in that PR and copied it to my local. Tried running the build again but ran into same error. I apologize if that's not how it should be done, I am novice in Java.

[MonkeyCanCode]

@manojmukkamala will check later today and see what is causing it.

[loicalleyne]

I also tried to work around it and it seems like some changes in the gradle build files is causing a circular dependency, I wasn't able to isolate it though.

[MonkeyCanCode]

So 99d8eb7 is the one that caused this issue.

You can use the following to do a build:

git checkout -b new_branch d3bfa05
docker build . -t polaris_local_metastore --build-arg ECLIPSELINK=true

I can look into where the circular dependencies is happening later tonight.

[manojmukkamala]

that worked, thanks a lot @MonkeyCanCode
can you help how to perform a bootstrap when running polaris as a container?

[MonkeyCanCode]

@manojmukkamala sure thing. So I raised a PR for helm in the past which I am using job to perform bootstrap: https://github.com/apache/polaris/pull/135/files#diff-922229e69535a3fb282010bf5c16aa261f558539c874ac0cf8205b5762790cf4

Now back to your question, the container will be running in server mode by default. Then with persisted backend, your backend won't be bootstrapped out of box. To bootstrap it, you will need to ssh into the container and run the following wrapper command (which is what I am using in the helm chart above):

/app/bin/polaris-service bootstrap polaris-server.yml

Alternative, you can use the following java -jar command as well:

java -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml

[manojmukkamala]

thanks for the inputs. I entered the container docker exec -it -u 0 <<container-name>> bash and ran /app/bin/polaris-service bootstrap polaris-server.yml but it threw Internal Exception: java.sql.SQLException: No suitable driver found for jdbc:postgres://postgres_metastore:5433/polaris

I tried downloading the postgres jar from https://repo1.maven.org/maven2/org/postgresql/postgresql/42.7.4/postgresql-42.7.4.jar into /app/lib but no luck. Is that the right location to place the jar file?

[MonkeyCanCode]

thanks for the inputs. I entered the container docker exec -it -u 0 <<container-name>> bash and ran /app/bin/polaris-service bootstrap polaris-server.yml but it threw Internal Exception: java.sql.SQLException: No suitable driver found for jdbc:postgres://postgres_metastore:5433/polaris

I tried downloading the postgres jar from https://repo1.maven.org/maven2/org/postgresql/postgresql/42.7.4/postgresql-42.7.4.jar into /app/lib but no luck. Is that the right location to place the jar file?

Hello, so this error is expected as Polaris itself doesn't package any jdbc driver which is actually needed when using a jdbc backend via eclipse link. This needs to be added as part of the EclipseLink build file instead.

See more details in #230.

[manojmukkamala]

ok so that means I have to wait a bit before I can start using the container based deployment with PG backend?

[MonkeyCanCode]

ok so that means I have to wait a bit before I can start using the container based deployment with PG backend?

Not really, here is what you can do:

# update extension/persistence/eclipselink/build.gradle.kts
## add the following within dependencies block
implementation("org.postgresql:postgresql:42.7.4")

Then build the image again. By doing so, your local image will then include the needed driver (in this case psql driver).

[MonkeyCanCode]

Also, I think I found where the circular dependency is coming from. Let me run couple more validations before I raised a PR to ensure nothing break in between.

[manojmukkamala]

cool! appreciate your prompt responses.
It worked this time btw, huge thanks for all the help!

[MonkeyCanCode]

@eric-maynard here is the PR for fixing this issue: #284

[MonkeyCanCode]

cool! appreciate your prompt responses. It worked this time btw, huge thanks for all the help!

Anytime. Here is a sample PR to avoid manual edit as additional dependencies are needed when using ecliselink: https://github.com/apache/polaris/pull/285/files