Failed to start an environment with obs = env.reset() on macos

[1931540]

import gym
import minerl
env = gym.make('MineRLBasaltFindCave-v0')

above code has no error

but got error with env.reset()

obs = env.reset()

I got the following error, and I cannot get minerl to work.

/Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/runpy.py:127: RuntimeWarning: 'minerl.utils.process_watcher' found in sys.modules after import of package 'minerl.utils', but prior to execution of 'minerl.utils.process_watcher'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/site-packages/gym/wrappers/time_limit.py", line 27, in reset
    return self.env.reset(**kwargs)
  File "/Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/site-packages/minerl/herobraine/env_specs/basalt_specs.py", line 78, in reset
    return self.env.reset()
  File "/Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/site-packages/minerl/herobraine/env_specs/basalt_specs.py", line 57, in reset
    return super().reset()
  File "/Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/site-packages/gym/core.py", line 251, in reset
    return self.env.reset(**kwargs)
  File "/Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/site-packages/minerl/env/_singleagent.py", line 22, in reset
    multi_obs = super().reset()
  File "/Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/site-packages/minerl/env/_multiagent.py", line 436, in reset
    self._setup_instances()
  File "/Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/site-packages/minerl/env/_multiagent.py", line 539, in _setup_instances
    self.instances.extend([f.result() for f in instance_futures])
  File "/Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/site-packages/minerl/env/_multiagent.py", line 539, in <listcomp>
    self.instances.extend([f.result() for f in instance_futures])
  File "/Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/concurrent/futures/_base.py", line 437, in result
    return self.__get_result()
  File "/Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/site-packages/minerl/env/_multiagent.py", line 805, in _get_new_instance
    instance.launch(replaceable=self._is_fault_tolerant)
  File "/Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/site-packages/minerl/env/malmo.py", line 427, in launch
    raise EOFError(
EOFError: /Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/site-packages/minerl/env/../MCP-Reborn
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[15:58:21] [Render thread/INFO]: Environment: authHost='https://authserver.mojang.com', accountsHost='https://api.mojang.com', sessionHost='https://sessionserver.mojang.com', servicesHost='https://api.minecraftservices.com', name='PROD'
[15:58:22] [Render thread/ERROR]: Failed to verify authentication!
[15:58:22] [Render thread/INFO]: Setting user: Player657
[15:58:22] [Render thread/INFO]: Backend library: LWJGL version 3.2.1 SNAPSHOT
---- Minecraft Crash Report ----
// Why did you do that?

Time: 22-8-9 下午3:58
Description: Initializing game

java.lang.ExceptionInInitializerError
	at org.lwjgl.glfw.GLFW.glfwCreateWindow(GLFW.java:1842)
	at net.minecraft.client.MainWindow.<init>(MainWindow.java:91)
	at net.minecraft.client.renderer.VirtualScreen.create(VirtualScreen.java:21)
	at net.minecraft.client.Minecraft.<init>(Minecraft.java:408)
	at net.minecraft.client.main.Main.main(Main.java:154)
Caused by: java.lang.IllegalStateException: GLFW windows may only be created on the main thread and that thread must be the first thread in the process. Please run the JVM with -XstartOnFirstThread. For offscreen rendering, make sure another window toolkit (e.g. AWT or JavaFX) is initialized before GLFW.
	at org.lwjgl.glfw.EventLoop$OffScreen.<clinit>(EventLoop.java:39)
	... 5 more


A detailed walkthrough of the error, its code path and all known details is as follows:
---------------------------------------------------------------------------------------

-- Head --
Thread: Render thread
Stacktrace:
	at org.lwjgl.glfw.GLFW.glfwCreateWindow(GLFW.java:1842)
	at net.minecraft.client.MainWindow.<init>(MainWindow.java:91)
	at net.minecraft.client.renderer.VirtualScreen.create(VirtualScreen.java:21)
	at net.minecraft.client.Minecraft.<init>(Minecraft.java:408)

-- Initialization --
Details:
Stacktrace:
	at net.minecraft.client.main.Main.main(Main.java:154)

-- System Details --
Details:
	Minecraft Version: 1.16.5
	Minecraft Version ID: 1.16.5
	Operating System: Mac OS X (x86_64) version 10.16
	Java Version: 1.8.0_312, Azul Systems, Inc.
	Java VM Version: OpenJDK 64-Bit Server VM (mixed mode), Azul Systems, Inc.
	Memory: 534386904 bytes (509 MB) / 1507852288 bytes (1438 MB) up to 3817865216 bytes (3641 MB)
	CPUs: 8
	JVM Flags: 1 total; -Xmx4G
	Launched Version: ~~NULL~~
	Backend library: LWJGL version 3.2.1 SNAPSHOT
	Backend API: NO CONTEXT
	GL Caps:
	Using VBOs: Yes
	Is Modded: Very likely; Jar signature invalidated
	Type: Client (map_client.txt)
	CPU: <unknown>
#@!@# Game crashed! Crash report saved to: #@!@# /Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/site-packages/minerl/MCP-Reborn/./crash-reports/crash-2022-08-09_15.58.23-client.txt


Minecraft process finished unexpectedly. There was an error with Malmo.

[1931540]

here's my conda env setting

name: minerl2
channels:
  - conda-forge
  - defaults
dependencies:
  - python=3.8
  - pip=22.1
  - openjdk=8.0.312
  - pip:
    - git+https://github.com/minerllabs/minerl

[Miffyli]

Hey! Are you running on a headless machine without display? MineRL/Minecraft requires a valid display buffer to be able to run. You can use xvfb for this with xvfb-run -a python [your-code] (on ubuntu installation is simple sudo apt install xvfb).

[lewisxy]

I got the exact same error on my MacBook. Any update to this? I don't think xvfb-run is available on MacOS.

[Miffyli]

Unfortunately MineRL v1.0 does not run on Mac at the moment, and no-one has had time to look into :(

[lewisxy]

OK, after hours of debug, I made it work on an MacBook Pro (with Apple Silicon M1 Pro) running MacOS 12.6.1. This solution SHOULD work on any Apple Silicon based Mac. Here is how I did it.

TLDR version

1. Download or clone the repository. This should create minerl directory.

git clone https://github.com/minerllabs/minerl.git

2. Modify the patch file by replacing all 3.2.1 to 3.3.1. This will update the version of lwjgl to 3.3.1.
3. Install the package using pip. Inside minerl directory, run the following command

pip install .

If installation failed, it could be the problem of JDK. The JDK I found working is the Oracle JDK 1.8, you will need to sign-in to download it. This JDK is for x86_64, so you will need Rosette to run it. Other people also has some success with OpenJDK.

4. If the previous step is successful, it should generate a directory called MCP-Reborn. Modify the launchClient.sh by adding -XstartOnFirstThread in the line that runs java ...... (somewhere close to the bottom of the file), so the new line becomes.

java -Xmx$maxMem -XstartOnFirstThread -jar $fatjar --envPort=$port

Note: you can also change the memory for JVM by changing the 4G defined in this file.

5. Modify the MCP-Reborn/src/main/java/net/minecraft/client/MainWindow.java. Comment out or remove the line

GLFW.glfwSetWindowIcon(this.handle, buffer);

Also comment out or remove the entire method body of checkGlfwError

public static void checkGlfwError(BiConsumer<Integer, String> glfwErrorConsumer) {
    // remove or comment out anything here
    // DO NOT remove this method declaration
}

6. Rebuild MCP-Reborn. Inside MCP-Reborn directory, run the following command

./gradlew clean build shadowJar

7. Overwrite the content inside MCP-Reborn directory in the install location with the content of MCP-Reborn we just modified. The exact install location depends on your python setup. It should be something like somepath/python3.x/site-packages/minerl/MCP-Reborn.
8. Now you should be able to run the example provided in official documentation. DO NOT run python inside the minerl directory we cloned, as it will mess up the python working directory.

Technical Details

• lwjgl starts to support Apple Silicon after version 3.3.0. The latest version at the time of writing is 3.3.1. See their release page for detail.
• -XstartOnFirstThread JVM arguments is hinted by the error message. It just took some time for me to figure out where is java called to start the minecraft instance.
• This blog explains in detail why the modifications in MainWindow.java are necessary.

It's appreciated if someone can verify this works. As I tried so many things in attempt to fix this, I might forgot to write everything down.

[Miffyli]

@lewisxy Oooo this is amazing! @trigaten any chance you could try this? I could also try on my M1 Mac but atm quite busy. It would be awesome to have this stuff merged into code :). It does not look like we are going to sacrifice any functionality. We just need to make sure these changes would still work on Windows and Linux.

[richwomanbtc]

I executed the command according to #659 (comment) and confirmed that it works. Thanks @lewisxy!
Environment M1 Mac + python 3.9 + poetry
Commands

• git clone https://github.com/minerllabs/minerl.git
• sed -i .bak 's/3\.2\.1/3.3.1/' ./minerl/scripts/mcp_patch.diff
• cd minerl && poetry run python setup.py
• sed -i .bak s/'java -Xmx\$maxMem'/'java -Xmx\$maxMem -XstartOnFirstThread'/ ./minerl/MCP-Reborn/launchClient.sh
• sed -i .bak /'GLFW.glfwSetWindowIcon(this.handle, buffer);'/d ./minerl/MCP-Reborn/src/main/java/net/minecraft/client/MainWindow.java
• sed -i .bak '125,136s/^/\/\//' ./minerl/MCP-Reborn/src/main/java/net/minecraft/client/MainWindow.java
• cd minerl/MCP-Reborn && ./gradlew clean build shadowJar
• cd ../../../ && poetry add "git+https://github.com/minerllabs/minerl"
• cp -rf minerrl/minerl/MCP-Reborn/* .venv/lib/python3.9/site-packages/minerl/MCP-Reborn/

[jhsul]

This worked and I was able to install minerl on my m2 mac (woot). However, python is having trouble finding the minerl.data module:

Traceback (most recent call last):
  File "...", line 2, in <module>
    import minerl.data
ModuleNotFoundError: No module named 'minerl.data'

Did you encounter this error at all?

[lewisxy]

I believe minerl.data only works for older version (<=0.4.x), and it is removed in newer version (>=1.0.x).

[jhsul]

Oh, I see. That'll do it. Thanks!