Unreadable forms in certain eval-in-project calls

[technomancy]

Sometimes eval-in-project calls will result in forms that contain unreadable #object... elements.

I have only been able to reproduce the problem with lein eastwood but others report problems with built-in tasks or with lein ring.

In the instance with eastwood, somehow the entire project map is being attached as metadata to the form being evaluated, and the metadata contains merge directives like reduce-dep-step which is a function.

[ccann]

I'm not sure if it's the same issue, but both eastwood and bikeshed fail for me with the same Caused by: java.lang.RuntimeException: No reader function for tag ... when compiling the target directory of the project.

I also get this line:
Java HotSpot(TM) 64-Bit Server VM warning: Unable to open cgroup memory limit file /sys/fs/cgroup/memory/memory.limit_in_bytes (No such file or directory)

[pmonks]

<digression warning>
The Java HotSpot(TM) 64-Bit Server VM warning: Unable to open cgroup memory limit file /sys/fs/cgroup/memory/memory.limit_in_bytes (No such file or directory) warning is unrelated - it's coming from the JVM, and is a side effect of the fix for #2304. While quite annoying, my gut tells me that that fix is "right", and that instead the JVM should not be complaining so loudly when it can't read the cgroups info.

[technomancy]

@pmonks if that path is predictable we could check for its existence before adding the flag to the :jvm-opts.

[jafingerhut]

I have done a tiny bit of investigation, and at least so far it appears that the reduce-dep-step things with #object tags are there both with Leiningen 2.7.1, and also with Leiningen 2.8.0. If it would be helpful, I can release to Clojars a version of Eastwood that enables dumping the project map that it passes to eval-in-project, showing metadata as well as the normal data.

[technomancy]

@jafingerhut thanks; I can just locally install a snapshot version which has what I need. I'm just confused as to why certain uses of eval-in-project trigger this and others don't; eastwood doesn't appear to be doing anything unusual with the form being evaluated.

[jafingerhut]

OK. Just in case it may be helpful, I went ahead and released Eastwood version 0.2.6-beta2 to clojars. If you run "lein eastwood lein-project" with earlier Eastwood versions, it pretty-prints the Leiningen project map, but without the metadata shown. With 0.2.6-beta2 it also shows the metadata. Running with that command line stops immediately after showing that map, without doing linting like Eastwood normally would.

You can also run:

lein eastwood '{:debug [:options]}'

and it will print out debug info, including the Leiningen project map with metadata, and then continue linting as normal.

[pmonks]

@technomancy wrote:

if that path is predictable we could check for its existence before adding the flag to the :jvm-opts.

Yeah that'd be handy, though sadly I don't think it is that predictable. My skim reading of JDK-8146115 leads me to suspect that the JVM checks a couple of different things to try to determine what the cgroups limits are.

Perhaps the best path is to raise an issue against Java, asking them to shush the JVM up a bit when -XX:+UseCGroupMemoryLimitForHeap doesn't find a cgroups limit? Though silently failing also has problems... 😢

[technomancy]

I tracked this down to a change which was applied in #2306; before that we did not print metadata when writing out code for eval-in-project's :subprocess method. I think that change needs to be reverted; it was only applied in order to avoid a reflection warning, and this problem is much more serious.