[R] Installing the GPU version on Windows

[Lodewic]

Having spent the last couple of hours trying to install the GPU version of mxnet in R, I decided to look for clarification here.
I downloaded the gpu version from here: https://github.com/dmlc/mxnet/releases/download/weekly_binary_build/20151209_win64_gpu.7z

And I have downloaded cuDNN v3, unpacked it into 3rdparty\cudnn and then ran setupenv.cmd . Now what do I do to install this in R? The installation guide mainly focuses on installing it in Python.

(I have no experience with Linux, or installing a package this way in R, so the Installation guide seems to go over my head.
I've also tried Building the package in Windows, using Cmake and whatnot but I get stuck at making a tar.gz file that I should be able to install.)

[thirdwing]

Hi, @Lodewic

Sorry about all the troubles. Compiling on Windows is a little annoying and I am afraid there is something you have to do manually. I list them blow:

(1) create the folder R-package/inst/libs/x64. We only support 64-bit operating system now, so you need x64 foler;

(2) put dll files in R-package/inst/libs/x64;

(3) create the folder R-package/inst/include and copy all header files into it. Just like what we do in https://github.com/dmlc/mxnet/blob/master/Makefile#L190-L192

(4) R CMD INSTALL R-package

BTW, you can find the pre-built package from mxnet_0.5.zip. You can unzip it and look throught it.

[Lodewic]

Thank you very much for your swift response @thirdwing . I do feel like I'm getting further with your help now, but I'm not quite there yet. I think I'm lacking some basic knowledge of installing packages this way. I have described in detail what I have done so far, and note that some of the information may be completely redundant but I am unsure as to what I am doing wrong.

Create R-package/inst/libs/x64 folder - SUCCESS
(1) I have done what you said and created a folder called R-package/inst/libs/x64 (in some arbitrary location I assumed)

Copy .dll files into R-package/inst/libs/x64
(2) I have copied a bunch of .dll files into the R-package/inst/libs/x64 folder. I'm unsure whether I have chosen the correct ones.
Compared to the mxnet_0.5.zip file, and the folder MXNET/libs/x64:

• I have the same .dll files. (Noted that the libmxnet.dll I copied is twice the size of the pre-built package file, because it is now the GPU version I'm guessing.)
• I have also copied 4 extra .dll files from the cudart and cudnn folders, namely,
  • cublas64_75.dll
  • cudart64_75.dll
  • cudnn64_4.dll
  • curand64_75.dll

I simply assumed I needed those .dll files as well, I hope I'm right about that.

Create/build R-package/inst/include folder
(3) I have created the R-package/inst/include folder and the folders in mxnet/nocudnn/include. Remember, I used the files from here https://github.com/dmlc/mxnet/releases/download/weekly_binary_build/20151209_win64_gpu.7z

So now, R-package/inst/include contains 3 subfolders, namely:

• dmlc
• mshadow (unsure about whether I need this one)
• mxnet

Attempt to use R CMD INSTALL
If I run R CMD INSTALL R-package at this point, I get Warning: invalid package 'R-package.

I figured, I first need to run R CMD build --no-build-vignettes R-package, as in https://github.com/dmlc/mxnet/blob/master/Makefile#L199
(Unsure about the --no-build-vignettes part though, I'm unsure of what it does and also tried the command without that argument.)

If I do try to run R CMD build --no-build-vignettes R-package now I get the Error that there is no DESCRIPTION file. Specifically I get:

• checking for file 'R-package/DESCRIPTION' ... NO

The ways I tried to fix it
So, naturally I copied the DESCRIPTION, INDEX, LICENSE etc. files from mxnet_0.5.zip into R-package/ and tried running the command again.

Now, it does seem like if I re-run R CMD build --no-build-vignettes R-package that this made some progress. This is the output:
* checking for file 'R-package/DESCRIPTION' ... OK
* preparing 'mxnet':
* checking DESCRIPTION meta-information ... OK
ERROR
computing Rd index failed

The R warnings it returned were: Warning in file(con, "r"): cannot open file 'man'
I wouldn't know where I should've added a file called 'man', so I am lost here.

I feel like I missed something trivial and much of the information I gave above may be redundant.

EDIT: Some extra output info

I tried running R CMD INSTALL R-package without the whole R CMD build step. After I added the DESCRIPTION, INDEX, LICENSE etc. files like I described above, I get the following output:

• installing to library 'D:/Program files/R-3.1.2/li
• installing binary package 'mxnet' ...
  cp: unknown option -- )
  Try `/usr/bin/cp --help' for more information.
  Warning: running command 'cp -R . "D:/Program files/R-3.1.2/library/mxnet" || (tar cd - .| (cd "D:/Program files/R-3.1.2/library/mxnet && tar -xf -))' had status 1
  ERROR: installing binary package failed
• removing 'D:/Program files/R-3.1.2/library/mxnet'

This does not help me personally, but it might be a known error to someone else.

[Lodewic]

Clearly, what I did before was the wrong way to do it. My post above can be disregarded but may be helpful for others that don't have a clue of what they're doing with the installation :')

So I've started over again.

This time I realised I first have to get everything from https://github.com/dmlc/mxnet , in which the R-package folder exists.

Following instructions from @thirdwing

(1) I created the folder 'R-package/inst/libs/x64'

(2) In this folder, I copied all .dll files I think I need from 20151228_win64_gpu.7z

To be precise, from the file I linked I copied all .dll files from:

• nocudnn/3rdparty/cudart
• nocudnn/3rdparty/cudnn/bin (cudnn64_70.dll)
• nocudnn/3rdparty/openblas/bin
• nocudnn/3rdparty/vc
• nocudnn/3rdparty/lib

(3) "create the folder R-package/inst/include and copy all header files into it. Just like what we do in https://github.com/dmlc/mxnet/blob/master/Makefile#L190-L192"

I did what you said here- and I end up with 3 subfolders in R-package/inst/nclude/, namely /dmlc, /mshadow and mxnet.

(4) Now I run, R CMD INSTALL R-package

Result from installing

(Note that I realised I at least had to update R since my last post)
When I run R CMD INSTALL R-package, I get this output:

* installing to library 'D:/Program files/R-3.2.3/library/mxnet'
* installing *source* package 'mxnet' ...
**libs

*** arch -i386
make: Nothing to be done forall'. installing to D:/Program files/R-3.2.3/library/mxnet/libs/i386`

*** arch -x64
make: Nothing to be done for 'all'.
installing to D:/Program files/R-3.2.3/library/mxnet/libs/x64

** R
** demo
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded
*** arch - i386
Error : .onLoad failed in loadNamespace() for 'mxnet', details:
call: library.dynam("libmxnet", pkgname, libname, local = FALSE)
error: DLL 'libmxnet' not found: maybe not installed for this architecture?
Error: loading failed
Execution halted

*** arch - x64
Error: loading failed for 'i386'
* removing 'D:/Program files/R-3.2.3/library/mxnet'

What I tried to do to fix it

So I googled a bit based on the error I got:
Error : .onLoad failed in loadNamespace() for 'mxnet', details:
call: library.dynam("libmxnet", pkgname, libname, local = FALSE)
error: DLL 'libmxnet' not found: maybe not installed for this architecture?

And I checked if I already had an installation of mxnet , which I don't. (I uninstalled the CPU-version)
And, I checked if the libmxnet.dll file was present in `R-package/inst/libs/x64', and it is.

Also, I am wondering why it tries to do anything with 'i386' in the first place..

I hope somebody can help me solve the last few steps, because I do think I am close to getting it.

[thirdwing]

@Lodewic Please try to use R CMD INSTALL --no-multiarch

[Lodewic]

@thirdwing You did it! I am very thankful, thank you for sticking around and replying so quickly

I learned a lot from this and look forward to using the mxnet package

[thirdwing]

Congratulations! I am closing this. You can reopen it if you met new issues.

[thirdwing]

@Lodewic Can you add what you have done into the documentation?

You can add a paragraph in https://github.com/dmlc/mxnet/blob/master/doc/build.md#r-package-installation and send a PR.

Thanks.

[raylite]

Hello @Lodewic and @thirdwing , I hope am not too late given that this thread is about 3 months old. ANyway I amstruggling with installing MxNet on windows with R. I have been using the cpu version but will like to migrate to thr gpu version.
Firstly, I have tried to follow the instructions on [http://myungjun-youn-demo.readthedocs.org/en/latest/build.html] but with no complete success yet. I have installed VS12 and the VS compiler.
I have managed to install it with python using the pre-built and I ran the test code which works. That is as far as I tested it for now. But all these was done at command prompt with python27. When I tried to import mxnet from anaconda it did not work neither was I able to run the setup.py file through Anaconda.

First question is, how can integrate mxnet with Anaconda given that Anaconda came pre-packed with its python and wasn't using the same package folder as python27.

My second question is that I am experiencing same as @Lodewic, I followed all his steps but on what platform will I run the command R CMD INSTALL --no-multiarch I am a bit lost there.

[raylite]

I figured out my mistakes and now I ran the command on promt. I got the following error:
`C:\Users\raylite>R CMD INSTALL --no-multiarch R-package

• installing to library 'C:/Users/raylite/Documents/R/win-library/3.2'
• installing source package 'mxnet' ...
  ** libs
  Warning: running command 'make -f "Makevars.win" -f "C:/PROGRA1/R/R-321.4/etc/
  x64/Makeconf" -f "C:/PROGRA1/R/R-321.4/share/make/winshlib.mk" SHLIB_LDFLAGS='
  $(SHLIB_CXXLDFLAGS)' SHLIB_LD='$(SHLIB_CXXLD)' SHLIB="mxnet.dll" WIN=64 TCLBIN=6
  4 OBJECTS="executor.o export.o io.o kvstore.o mxnet.o ndarray.o symbol.o"' had s
  tatus 127
  ERROR: compilation failed for package 'mxnet'
• removing 'C:/Users/raylite/Documents/R/win-library/3.2/mxnet'`
  Any suggestions what am doing wrongly?

I now have the nocudnn folder and the R-package folder with the right files in them. SHould I cut the R-package folder from its mxnet parent folder or leave it in there. WHich directory should I put the folders for compilation?

[thirdwing]

If you are using Rtools 3.3, please downgrade to 3.2 and try again.