Error while creating mount source path (after update to 1.5.0) - No permission to create tempfile

[VladoDemcak]

Hi we have problems with creating testcontainers tmp file due permission denied.

I was not able to figure out how to change directory where this tmp file should be created. The best option for us would be target folder.

I dont have this problem in 1.4.3v (I know this file is created in next to pom.xml e.g) but problem is in 1.5.0. We tried to run as admin but we have some policy and we are not able to work with tmp directory.

We are running testcontainers on Windows 10.

Is there some option to change this directory?

[2017-12-13 18:42:47,680] [ERROR] [dockerjava-netty-1-6] [com.github.dockerjava.core.async.ResultCallbackTemplate:57] Error during callback
com.github.dockerjava.api.exception.InternalServerErrorException: {"message":"error while creating mount source path '/C/Users/XXXX/AppData/Local/Temp/.testcontainers-tmp-2562011949546442767': mkdir /C/Users/XXXX/AppData: permission denied"}

	at com.github.dockerjava.netty.handler.HttpResponseHandler.channelRead0(HttpResponseHandler.java:109)
	at com.github.dockerjava.netty.handler.HttpResponseHandler.channelRead0(HttpResponseHandler.java:33)
	at org.testcontainers.shaded.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at org.testcontainers.shaded.io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:241)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at org.testcontainers.shaded.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438)
	at org.testcontainers.shaded.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310)
	at org.testcontainers.shaded.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284)
	at org.testcontainers.shaded.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at org.testcontainers.shaded.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at org.testcontainers.shaded.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
	at org.testcontainers.shaded.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134)
	at org.testcontainers.shaded.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644)
	at org.testcontainers.shaded.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:579)
	at org.testcontainers.shaded.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:496)
	at org.testcontainers.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458)
	at org.testcontainers.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
	at org.testcontainers.shaded.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
	at java.lang.Thread.run(Thread.java:745)

java.lang.ExceptionInInitializerError
	at sun.misc.Unsafe.ensureClassInitialized(Native Method)
	at sun.reflect.UnsafeFieldAccessorFactory.newFieldAccessor(UnsafeFieldAccessorFactory.java:43)
	at sun.reflect.ReflectionFactory.newFieldAccessor(ReflectionFactory.java:142)
	at java.lang.reflect.Field.acquireFieldAccessor(Field.java:1088)
	at java.lang.reflect.Field.getFieldAccessor(Field.java:1069)
	at java.lang.reflect.Field.get(Field.java:393)
	at org.junit.runners.model.FrameworkField.get(FrameworkField.java:73)
	at org.junit.runners.model.TestClass.getAnnotatedFieldValues(TestClass.java:230)
	at org.junit.runners.ParentRunner.classRules(ParentRunner.java:255)
	at org.junit.runners.ParentRunner.withClassRules(ParentRunner.java:244)
	at org.junit.runners.ParentRunner.classBlock(ParentRunner.java:194)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:362)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:191)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: java.lang.reflect.UndeclaredThrowableException
	at com.sun.proxy.$Proxy62.exec(Unknown Source)
	at org.testcontainers.DockerClientFactory.runInsideDocker(DockerClientFactory.java:225)
	at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:114)
	at org.testcontainers.containers.GenericContainer.<init>(GenericContainer.java:124)
	at org.testcontainers.containers.FixedHostPortGenericContainer.<init>(FixedHostPortGenericContainer.java:16)
	at xxxxxxxxxxxxxTest.<clinit>(xxxxxxxxxxxxxTestTest.java:67)
	... 18 more
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testcontainers.dockerclient.AuditLoggingDockerClient.lambda$wrappedCommand$14(AuditLoggingDockerClient.java:98)
	... 24 more
Caused by: com.github.dockerjava.api.exception.InternalServerErrorException: {"message":"error while creating mount source path '/C/Users/XXXX/AppData/Local/Temp/.testcontainers-tmp-2562011949546442767': mkdir /C/Users/XXXX/AppData: permission denied"}

	at com.github.dockerjava.netty.handler.HttpResponseHandler.channelRead0(HttpResponseHandler.java:109)
	at com.github.dockerjava.netty.handler.HttpResponseHandler.channelRead0(HttpResponseHandler.java:33)
	at org.testcontainers.shaded.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at org.testcontainers.shaded.io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:241)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at org.testcontainers.shaded.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438)
	at org.testcontainers.shaded.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310)
	at org.testcontainers.shaded.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284)
	at org.testcontainers.shaded.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at org.testcontainers.shaded.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at org.testcontainers.shaded.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
	at org.testcontainers.shaded.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134)
	at org.testcontainers.shaded.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644)
	at org.testcontainers.shaded.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:579)
	at org.testcontainers.shaded.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:496)
	at org.testcontainers.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458)
	at org.testcontainers.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
	at org.testcontainers.shaded.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
	at java.lang.Thread.run(Thread.java:745)

[kiview]

This is indeed a change in 1.5.0, see #423 and #450.
Since we use the JDK api for this (see File) can you please try out this workaround:
Environment variable to control java.io.tmpdir

Btw. I wonder why Docker isn't allowed to create the file in /C/Users/XXXX/AppData/Local/Temp/. How are your Docker shared drives settings configured?

[VladoDemcak]

We have checked C drive in a "Shared Drives" page in docker settings.

I tried with workaround java.io.tmpdir and worked fine - created .testcontainers-tmp-421391171823811977 in folder I set with java.io.tmpdir.

 Server Version: 17.09.0-ce
  API Version: 1.32
  Operating System: Alpine Linux v3.5
  Total Memory: 9958 MB
        ℹ︎ Checking the system...
        ✔ Docker version should be at least 1.6.0
        ✔ Docker environment should have more than 2GB free disk space
        ✔ File should be mountable
        ✔ A port exposed by a docker container should be accessible

BTW is this file really needed? I would like to get rid of it. Also similar issue mentioned #423
If so there should be definitely some kind of flexibility to change that folder (add settings to GenericContainer, consider maven file structure ...)

[bsideup]

@VladoDemcak you can disable the checks as described here:
https://www.testcontainers.org/usage/properties.html

Please make sure to re-enable them if something goes wrong before reporting the issue - most of the time they help to understand what's broken

[VladoDemcak]

thanks @bsideup. I disabled check with testcontainers.properties and it works.

But I still consider this as an issue - or a restriction at least.

I think some users will not be able to use testcontainers 1.5.0 due permission denied for writing to directory where they do not have acces - in our case default temp dir.

[kiview]

I still don't understand why you don't have permissions to write into the temp directory, this should be a safe default.

We might be able use currentDir as a fallback solution, but I'm unsure if this doesn't make troubleshooting more confusing.

[VladoDemcak]

@kiview it's some restriction on working laptop - permission denied for accessing temp folder from IDE. in console I am able to write to directory but from IDE I am not.

I can work with the solution with testcontainers.properties but the check would be good to have anyway. I would keep the solution with temp (currentDir was more confusing when we used v1.4.3). The best option would be to create file in target as someone pointed out in commend I attached in the previous comment. But I understand it would be tricky.

I would keep this issue as 'minor'/'good to have' .

[kiview]

@VladoDemcak Ah, good to know that it's a known restriction on your system, else I would have been confused 😉

I don't think using target would work in a sane way, since we would need to guess the build system and configuration in this case (for example, Gradle uses build as the default directory).

So I see the following options:

• Fallback to workdir
• Make tempdir configurable in testcontainers.properties
• Leave as is (tempdir is already configurable using java.io.tmpdir JVM parameter), maybe add further documentation

[VladoDemcak]

@kiview
sure target is too specific ... totally agree...
and if it's possible to make tempdir configurable right now (in testcontainers.properties), it would be definitely cool to mention it in a documentation.

In my opinion as the most flexible way I see option num2 and by default use java.io.tmpdir.

[rupert-madden-abbott]

This also problematic for people affected by this issue i.e. AAD joined Windows 10 machines. Volume sharing is broken for AAD accounts so a local user must be setup to authenticate the share. However, that user must then be manually granted access to those directories which need access.

The workaround should be to grant this user access to my TEMP directory e.g. C:\Users\username\AppData\Local\Temp. However, the error states this:

error while creating mount source path '/host_mnt/c/Users/username/AppData/Local/Temp/.testcontainers-tmp-6074610156504796562': mkdir /host_mnt/c/Users/username/AppData: permission denied

I've tried granting access to just C:\Users\username\AppData\Local\Temp (My temp directory) but for some reason this is not sufficient and I had to grant access to the entirety of C:\Users\username\AppData to resolve this problem.