vnc for openAI gym

[AwokeKnowing]

openAI gym is working in the pytorch image, but to actually run the atari environments, it fails due to lack of x11. What's the best way to get it to where we can see the environments. I tried adding a desktop with vnc but it messed up the gpu support.

in my attempts strangely enough the atari environments would run when I run the container with docker, but when I run the container with nvidia-docker (gpu enabled), the gpu is definately working (torch.cuda.is_available()) but the atari games error out on

File "/usr/local/lib/python3.5/site-packages/pyglet/gl/glx_info.py", line 83, in have_version
    raise GLXInfoException('pyglet requires an X server with GLX')
pyglet.gl.glx_info.GLXInfoException: pyglet requires an X server with GLX
 

to be clear, I have an Nvidia GPU. my goal is to work with pytorch and openai gym. your pytorch image is perfect, except that I need to see the atari environments. I don't know how to see them but have been trying for 16 hours. I can follow instructions, but I can't seem to find instructions.

[houqp]

Could you send us the floydhub job URL so I can try to reproduce the error on my side?

[AwokeKnowing]

Hi I'm actually running this locally. My goal is to be able to seemlessly transition from docker local to docker on floydhub or elsewhere.

I have seen some people say that the trick is to install my nvidia drivers with option --no-opengl-files
https://davidsanwald.github.io/2016/11/13/building-tensorflow-with-gpu-support.html

So again, all I'm doing is essentially nvidia-docker run -it -p5959:5900 floydhub/pytorch:0.1.11-gpu-py3.6 bash . There I do see my two GPU's when running nvidia-smi. But when running a simple OpenAI gym (pong) environment with xvfb I get the above error.

Note, I build this entire machine from scratch precisely for working within Docker locally on OpenAi gym and pytorch, so I'm willing to do whatever up to including formatting the whole machine. I just need a reliable setup and floydhub/pytorch has everything except actually a way to run the environments. Again note that the environment do actually run with I just run docker floydhub/pytorch (vs nvidia-docker) but no GPU is found

[houqp]

Could you create a minimal reproducible example on Floydhub? That can help reduce the time for us to trace down the root cause if we can reliably reproduce it on our side.

[AwokeKnowing]

@houqp yesterday I re-installed the ubuntu from scratch, updated packages, blacklisted nouveau, installed build-essentials, and installed the nvidia runfile driver (375.86), with --no-opengl-files and saying "no" to any offers to modify xconfig. Then I installed docker and nvidia-docker and ran floydhub/pytorch, and the nvidia drivers are loaded and no errors when running against xvfb.

So the error part of this issue can be pinned on ubuntu ppa nvidia driver (not having option to install without messing up opengl).

However, now that everything is working correctly, the title of this issue comes into play. Since openAI gym is a graphical environment, what is the recommended way to "see" the environments rendered when running floydhub/pytorch locally (ie with a Monitor). Does it have vnc installed? How to run it? How to connect from host OS?

Obviously I can make my own image on top of this, but is there already a vnc setup in the image?

What I suggest (but I defer to your experience) is to have a script similar to your run_jupyter.sh which would be run_vnc.sh and expose 5999 or whatever. It's not necessary to install any "desktop" or x server, xvfb will do fine. (at least for gym) so just a vncserver to optionally "see" what's going on.

You know what would be really great is if someone figured out how to render live atari games within jupyter.../offtopic

[AwokeKnowing]

Note: here I posted a question describing the exact setup (just 2 commands on top of floydhub/pytorch)
https://askubuntu.com/questions/942768/how-can-i-properly-run-openai-gym-with-nvidia-docker-and-see-the-environments

You can see that the rendering is wrong. Which underscores the need for floydhub to have that "already set up" in the image.

I would truly appreciate if someone with experience in installing all the packages included this.

I suggest a new image floydhub/pytorch-gui
FROM: floydhub/pytorch
[stuff to run ai Gym, matplotlib etc over vnc]

Then we could just use the image, connect over vnc and get right to work, as floydhub's goal. It's crazy that I've spent 12 hours just trying to get a visual of openAI gym (and still not working), and it's sad that each developer would need to do the same, before starting working on deep learning code. Not everything is web-based, or console-based, so vnc is needed

[houqp]

I think it would be useful to add vnc streaming support to base image so all envrionmens will have access to it. I noticed you also installed fluxbox, is that a hard dependency?

Sorry that we are working on releasing a major update to the platform, so I won't have enough time to work on this in the upcoming weeks. That said, you are welcome to submit a patch to the dockerfile at https://github.com/floydhub/dockerfiles/blob/master/dl/dl-base/dl-base-1.x.x.jinja. I can help test and build the new image for you.

[AwokeKnowing]

@houqp
fluxbox is not a hard dependency. I just found some example that used it. I have posted on several forums including askubuntu and devtalk.nvidia.com regarding it, but have not yet found the best way. I supposed a very lightweight desktop in the base might be the best way.

[houqp]

I have a feeling that there is still space for optimization, we can probably get away without a desktop environment.

[AwokeKnowing]

@houqp So I got sidetracked from this project doing the Coursera DeepLearning Specialization, but when that's done I really want to nail the stack for local, docker-based OpenAI gym development (especially atari).

You're probably right that a full desktop is overkill. We just need working VNC of the gym environment(s) and any plot windows as well. I'm not familiar enough to know the best-practices way to do that.

[houqp]

Sounds good, looking forward to work with you on that when you finish the class. This will benefit a lot of people :)

[AwokeKnowing]

So based on the info at openAI for universe, I did the following:

 nvidia-docker run --privileged --rm -it \
 -e DOCKER_NET_HOST=172.17.0.1 \
 -v /var/run/docker.sock:/var/run/docker.sock \
 floydhub/pytorch:0.1.11-gpu-py3.6 bash

it seems to be important to map the docker socket in there for universe to start it's own docker containers.

With this setup I can run the environments and they appear in vnc localhost:5900 or web localhost:15900 (they have a built in web-based vnc client).

as an example, when running the above container, then I can write the below example code in a python script and run it, and view it from my machine at that web address.

import gym
import universe  # register the universe environments

env = gym.make('gym-core.Pong-v3')
env.configure(remotes=1)  # automatically creates a local docker container
observation_n = env.reset()

while True:
  action_n = [[('KeyEvent', 'ArrowUp', True)] for ob in observation_n]  # your agent here
  observation_n, reward_n, done_n, info = env.step(action_n)
  env.render()

However, this is all for the 'universe' version of the atari environments, not the native gym one. The difference is that the universe ones are wrapped in docker and the observation comes over vnc, so the frame rate is capped and there is network overhead and so training is slower, and the reward signal has lag. So I still would like to be able to do the same with the native gym atari environments, which are also already installed in the floydhub/pytorch image.

But this is a good start. Pytorch + atari in one floydhub image and simple command line, without installing anything else.

[AwokeKnowing]

Ok, so I now believe the best way for local development will be x forwarding. No need to install anything else in the image.

However, I believe something in the floydhub stack has corrupted the normal rendering. Some other environments work (the flash ones) but all the atari ones render with corrupted display.

To reproduce, run:

docker run -it --user=$(id -u) --env="DISPLAY" --workdir="/home/$USER" \
--volume="/tmp/.X11-unix:/tmp/.X11-unix:rw" floydhub/pytorch:0.1.11-gpu-py3.6 bash

Now in the image, type python and then the following:

import gym
gym.make('Pong-v0').render()

That should open up an x-forwarded window on your machine, but the display is corrupt (at least for me)

Above I actually used SpaceInvaders-v0

So, the point is that there is no need to install a 'desktop' or vnc or anything. x-forwarding can allow local development viewing of the windows just fine. However, something with openGL or something is messed up in the floydhub/pytorch image.

Please try the above and let me know if you can reproduce it. If it works fine for you then I guess something is wrong with my drivers.

[houqp]

This is awesome, I am going to look into it today.

[houqp]

I am able to reproduce the garbled screen on my laptop. One thing I noticed though is CartPole-v0 is working fine. So this is not happening to all the games.

EDIT: Sorry, I missed a detail in your previous comment. You were trying to get atari game working. Looks like all atari games are not working properly.

[houqp]

On the other hand, i am able to save content of screen as image locally with correct content:

env = gym.make('Pong-v0') 
env.reset()
env.env.ale.saveScreenPNG('test_image.png')