Unable to build a native image with a custom application directory

[stefferro]

Hi,

I am working on a spring boot project with maven and try to build a native image. The spring boot maven plugin uses buildpacks to generate its OCI images, that the reason why I am raising the issue here.
The build process ends fine and the native image works as expected. But when I specify a different application directory (the spring boot maven plugin has this option), the build process fails at the very end.

Expected Behavior

Changing the application directory should not make the build process failing.

Current Behavior

Changing the application directory makes the build process failing with the following message

[INFO]     [creator]     Finished generating 'com.example.demo.DemoApplication' in 1m 48s.
[INFO]     [creator]       Removing bytecode
[INFO]     [creator]     unable to invoke layer creator
[INFO]     [creator]     unable to remove /myapp/BOOT-INF
[INFO]     [creator]     unlinkat /myapp/BOOT-INF: permission denied
[INFO]     [creator]     ERROR: failed to build: exit status 1

Steps to Reproduce

• Generate a spring boot project using Spring Initializr; You can use this link
• Modify the spring-boot-maven-plugin configuration from the pom.xml file as below. It specifies an application directory other than /workspace and enables the native build.

<plugin>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-maven-plugin</artifactId>
        <configuration>
	    <image>
	        <applicationDirectory>/myapp</applicationDirectory>
		<env>
		    <BP_NATIVE_IMAGE>true</BP_NATIVE_IMAGE>
		</env>
	    </image>
	</configuration>
</plugin>

• Run the maven command to generate the image

./mvnw spring-boot:build-image -Pnative

Motivations

Since we have packaged our applications into docker images, we store the binaries at a specific location. We would like to deploy our native images using the same binary location.

[dmikusa]

[INFO]     [creator]     unable to remove /myapp/BOOT-INF
[INFO]     [creator]     unlinkat /myapp/BOOT-INF: permission denied

This sounds like it might be a permission issue. The build is complete, but when it goes to clean up after and remove stuff from the final image, it fails and can't delete the files. The file referenced that it can't delete would come directly from your input JAR, i.e. what gets built on your local machine.

It will inherit permissions from there, so if you have something restrictive that could cause an issue.

Does it build correctly for you without the custom app directory?

[dmikusa]

You might also try a pack build --workspace /myapp --path target/your-app.jar -e BP_NATIVE_IMAGE=true <app> and see if that works for you. That would help to break the problem down and see if it is an issue with Spring Boot build tools, or something in the buildpack.

[stefferro]

Does it build correctly for you without the custom app directory?

Yes, it does.

I have made the test you asked with pack and the same error occurs. In order to be consistent with what the spring-boot maven plugin does, I have provided the builder-jammy-java-tiny builder

pack build --workspace /myapp --path target/demo-0.0.1-SNAPSHOT.jar -e BP_NATIVE_IMAGE=true --builder paketobuildpacks/builder-jammy-java-tiny:latest demo-pack

The last lines of the build process are :

Produced artifacts:
 /layers/paketo-buildpacks_native-image/native-image/com.example.demo.DemoApplication (executable)
================================================================================
Finished generating 'com.example.demo.DemoApplication' in 2m 10s.
  Removing bytecode
unable to invoke layer creator
unable to remove /myapp/BOOT-INF
unlinkat /myapp/BOOT-INF: permission denied
ERROR: failed to build: exit status 1
ERROR: failed to build: executing lifecycle: failed with status code: 51

And if I remove the workspaceflag or provide its default value /workspace, the build ends fine

[exporter] Saving demo-pack...
[exporter] *** Images (2dc35b5933bb):
[exporter]       demo-pack
[exporter] Adding cache layer 'paketo-buildpacks/bellsoft-liberica:native-image-svm'
[exporter] Adding cache layer 'paketo-buildpacks/syft:syft'
[exporter] Adding cache layer 'paketo-buildpacks/native-image:native-image'
[exporter] Adding cache layer 'buildpacksio/lifecycle:cache.sbom'
Successfully built image demo-pack

[dmikusa]

OK, thanks for trying that. Something definitely seems off. A couple last questions. Is this happening locally or in CI, what is your OS, and how are you running docker or similar software? Docker Desktop, Podman, Rancher, etc... Thanks

[stefferro]

Hi @dmikusa, I have experienced this issue locally and in CI which use different OS (respectively Windows and Linux).
I occasionally develop on WSL, so I had a try and reproduced the same issue. I run docker images using Docker Desktop

[dmikusa]

So I was doing some builds the other night and I’m seeing this issue with Spring Boot 3.4 when trying to do a native image build. If I do the build with Spring Boot build tools, it fails with this error:

    [creator]     --------------------------------------------------------------------------------
    [creator]     Build artifacts:
    [creator]      /layers/paketo-buildpacks_native-image/native-image/com.mikusa.cgi.ContainersCgroupInfoApplication (executable)
    [creator]     ================================================================================
    [creator]     Finished generating 'com.mikusa.cgi.ContainersCgroupInfoApplication' in 1m 30s.
    [creator]       Removing bytecode
    [creator]     unable to invoke layer creator
    [creator]     unable to remove /workspace/BOOT-INF
    [creator]     unlinkat /workspace/BOOT-INF: permission denied
    [creator]     ERROR: failed to build: exit status 1

If I do the build with pack, then it succeeds. I need to look into this more and see what's happening, this might need a bug filed upstream with Spring.

I was running on MacOS with the latest Podman.