Skip to content

Commit

Permalink
Adding Mac quick install script for mxnet with Python (apache#4642)
Browse files Browse the repository at this point in the history
  • Loading branch information
sandeep-krishnamurthy authored and Rahul Ravu committed Jan 21, 2017
1 parent 1ea1078 commit 44bee23
Show file tree
Hide file tree
Showing 3 changed files with 201 additions and 128 deletions.
147 changes: 67 additions & 80 deletions docs/get_started/osx_setup.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,63 @@
# Installing MXNet on OS X (Mac)
MXNet currently supports Python, R, Julia, and Scala. For users of Python on Mac, MXNet provides a set of Git Bash scripts that installs all of the required MXNet dependencies and the MXNet library.

## Prepare Environment for GPU Installation

This section is optional. Skip to next section if you don't plan to use GPUs. If you plan to build with GPU, you need to set up environment for CUDA and cuDNN.

First download and install [CUDA 8 toolkit](https://developer.nvidia.com/cuda-toolkit).

Once you have the CUDA Toolkit installed you will need to setup the required environment variables by adding the following to your ~/.bash_profile file:

```bash
export CUDA_HOME=/usr/local/cuda
export DYLD_LIBRARY_PATH="$CUDA_HOME/lib:$DYLD_LIBRARY_PATH"
export PATH="$CUDA_HOME/bin:$PATH"
```

Reload ~/.bash_profile file and install dependecies:
```bash
. ~/.bash_profile
brew install coreutils
brew tap caskroom/cask
```

Then download [cuDNN 5](https://developer.nvidia.com/cudnn).

Unzip the file and change to cudnn root directory. Move the header files and libraries to your local CUDA Toolkit folder:

```bash
$ sudo mv include/cudnn.h /Developer/NVIDIA/CUDA-8.0/include/
$ sudo mv lib/libcudnn* /Developer/NVIDIA/CUDA-8.0/lib
$ sudo ln -s /Developer/NVIDIA/CUDA-8.0/lib/libcudnn* /usr/local/cuda/lib/
```

Now we can start to build MXNet.

## Quick Installation
### Install MXNet for Python
Clone the MXNet source code repository to your computer and run the installation script. In addition to installing MXNet, the script installs all MXNet dependencies: ```Numpy```, ```LibBLAS``` and ```OpenCV```.

It takes around 5 to 10 minutes to complete the installation.

```bash
# Clone mxnet repository. In terminal, run the commands WITHOUT "sudo"
git clone https://github.com/dmlc/mxnet.git ~/mxnet --recursive

# If building with GPU, add configurations to config.mk file:
cd ~/mxnet
cp make/config.mk .
echo "USE_CUDA=1" >>config.mk
echo "USE_CUDA_PATH=/usr/local/cuda" >>config.mk
echo "USE_CUDNN=1" >>config.mk

# Install MXNet for Python with all required dependencies
cd ~/mxnet/setup-utils
bash install-mxnet-osx-python.sh
```
You can view the installation script we just used to install MXNet for Python [here](https://raw.githubusercontent.com/dmlc/mxnet/master/setup-utils/install-mxnet-osx-python.sh).

## Standard installation

Installing MXNet is a two-step process:

Expand All @@ -7,9 +66,9 @@ Installing MXNet is a two-step process:

**Note:** To change the compilation options for your build, edit the ```make/config.mk``` file and submit a build request with the ```make``` command.

## Build the Shared Library
### Build the Shared Library

### Install MXNet dependencies
#### Install MXNet dependencies
Install the dependencies, required for MXNet, with the following commands:
- [Homebrew](http://brew.sh/)
- OpenBLAS and homebrew/science (for linear algebraic operations)
Expand Down Expand Up @@ -38,40 +97,7 @@ Install the dependencies, required for MXNet, with the following commands:
pip install jupyter
```

### Prepare Environment for GPU Installation

If you plan to build with GPU, you need to set up environment for CUDA and cuDNN.

First download and install [CUDA 8 toolkit](https://developer.nvidia.com/cuda-toolkit).

Once you have the CUDA Toolkit installed you will need to setup the required environment variables by adding the following to your ~/.bash_profile file:

```bash
export CUDA_HOME=/usr/local/cuda
export DYLD_LIBRARY_PATH="$CUDA_HOME/lib:$DYLD_LIBRARY_PATH"
export PATH="$CUDA_HOME/bin:$PATH"
```

Reload ~/.bash_profile file and install dependecies:
```bash
. ~/.bash_profile
brew install coreutils
brew tap caskroom/cask
```

Then download [cuDNN 5](https://developer.nvidia.com/cudnn).

Unzip the file and change to cudnn root directory. Move the header files and libraries to your local CUDA Toolkit folder:

```bash
$ sudo mv include/cudnn.h /Developer/NVIDIA/CUDA-8.0/include/
$ sudo mv lib/libcudnn* /Developer/NVIDIA/CUDA-8.0/lib
$ sudo ln -s /Developer/NVIDIA/CUDA-8.0/lib/libcudnn* /usr/local/cuda/lib/
```

Now we can start to build MXNet.

### Build MXNet Shared Library
#### Build MXNet Shared Library
After you have installed the dependencies, pull the MXNet source code from Git and build MXNet to produce a MXNet library called ```libmxnet.so```.

The file called ```osx.mk``` has the configuration required for building MXNet on OS X. First copy ```make/osx.mk``` into ```config.mk```, which is used by the ```make``` command:
Expand Down Expand Up @@ -100,56 +126,17 @@ If building with ```GPU``` support, add the following configuration to config.mk
 

We have installed MXNet core library. Next, we will install MXNet interface package for the programming language of your choice:
- [Python](#install-the-mxnet-package-for-python)
- [R](#install-the-mxnet-package-for-r)
- [Julia](#install-the-mxnet-package-for-julia)
- [Scala](#install-the-mxnet-package-for-scala)

## Install the MXNet Package for Python

You need the following dependencies:

- Python version 2.7 or later.
- NumPy (to provide scientific computing operations).

To check if Python is already installed run below command and you should be able to see which version of Python is installed on your machine.

```bash
# Check if Python is already installed.
python --version
# Install Python if not already installed.
brew install python
# Install Numpy
brew install numpy
```

Next, we install Python package interface for MXNet. You can find the Python interface package for [MXNet on GitHub](https://github.com/dmlc/mxnet/tree/master/python/mxnet).

```bash
# Assuming you are in root mxnet source code folder
cd python
sudo python setup.py install
```
Done! We have installed MXNet with Python interface. Run below commands to verify our installation is successful.
```bash
# Open Python terminal
python

# You should be able to import mxnet library without any issues.
>>> import mxnet as mx;
>>> a = mx.nd.ones((2, 3));
>>> print ((a*2).asnumpy());
[[ 2. 2. 2.]
[ 2. 2. 2.]]
```
We actually did a small tensor computation using MXNet! You are all set with MXNet on your Mac.

## Install the MXNet Package for R
### Install the MXNet Package for R
You have 2 options:
1. Building MXNet with the Prebuilt Binary Package
2. Building MXNet from Source Code

### Building MXNet with the Prebuilt Binary Package
#### Building MXNet with the Prebuilt Binary Package

For OS X (Mac) users, MXNet provides a prebuilt binary package for CPUs. The prebuilt package is updated weekly. You can install the package directly in the R console using the following commands:

Expand All @@ -159,7 +146,7 @@ For OS X (Mac) users, MXNet provides a prebuilt binary package for CPUs. The pre
install.packages("mxnet")
```

### Building MXNet from Source Code
#### Building MXNet from Source Code

Run the following commands to install the MXNet dependencies and build the MXNet R package.

Expand All @@ -181,7 +168,7 @@ These commands create the MXNet R package as a tar.gz file that you can install
R CMD INSTALL mxnet_current_r.tar.gz
```

## Install the MXNet Package for Julia
### Install the MXNet Package for Julia
The MXNet package for Julia is hosted in a separate repository, MXNet.jl, which is available on [GitHub](https://github.com/dmlc/MXNet.jl). To use Julia binding it with an existing libmxnet installation, set the ```MXNET_HOME``` environment variable by running the following command:

```bash
Expand All @@ -202,7 +189,7 @@ You might want to add this command to your ```~/.bashrc``` file. If you do, you

For more details about installing and using MXNet with Julia, see the [MXNet Julia documentation](http://dmlc.ml/MXNet.jl/latest/user-guide/install/).

## Install the MXNet Package for Scala
### Install the MXNet Package for Scala
Before you build MXNet for Scala from source code, you must complete [building the shared library](#build-the-shared-library). After you build the shared library, run the following command from the MXNet source root directory to build the MXNet Scala package:

```bash
Expand Down
134 changes: 134 additions & 0 deletions setup-utils/install-mxnet-osx-python.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
#!/bin/bash
#
# This scripts installs the dependencies, downloads MXNet source
# and compiles it.
#
# The script also installs the MXNet package for Python.
#

#set -ex

export MXNET_HOME=`pwd`/mxnet
export MXNET_LOG=${MXNET_HOME}/buildMXNet_mac.log
# Insert the Homebrew directory at the top of your PATH environment variable
export PATH=/usr/local/bin:/usr/local/sbin:$PATH
LINE="########################################################################"

echo $LINE
echo " "
echo "This script installs MXNet on MacOS."
echo "It has been tested to work successfully on MacOS El Capitan and Sierra"
echo "and is expected to work fine on other versions as well."
echo " "
echo $LINE
sleep 2

#
# Install dependencies for MXNet
#

# Install Homebrew
yes '' | /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

brew_pkg_install () {
pkg=$1
brew ls --versions $pkg > /dev/null
ret=$?
if [[ ${ret} != 0 ]]; then
echo "brew install $pkg"
brew install $pkg
else
echo "$pkg already installed"
fi
}

runme() {
cmd=$*
echo "$cmd"
$cmd
ret=$?
if [[ ${ret} != 0 ]]; then
echo " "
echo "ERROR: Return value non-zero for: $cmd"
echo " "
exit 1
fi
}

runme brew update
runme brew_pkg_install pkg-config
runme brew_pkg_install python
runme brew_pkg_install openblas
runme brew_pkg_install opencv
# Needed for /usr/local/lib/graphviz to be created
runme brew_pkg_install graphviz
runme brew_pkg_install numpy

runme brew tap homebrew/science

runme pip install graphviz
runme pip install jupyter
runme pip install cython

#
# Fetch MXNet source and compile it
#
if [ -f ${MXNET_HOME} ]; then
ls -al ${MXNET_HOME}
echo " "
echo "ERROR: ${MXNET_HOME} directory already present"
echo "To correct the problem, please rename the directory or remove it."
echo " "
fi

runme git clone --recursive https://github.com/dmlc/mxnet
cd ${MXNET_HOME}
runme cp make/osx.mk ./config.mk
runme echo "USE_BLAS = openblas" >> ./config.mk
runme echo "ADD_CFLAGS += -I/usr/local/opt/openblas/include" >> ./config.mk
runme echo "ADD_LDFLAGS += -L/usr/local/opt/openblas/lib" >> ./config.mk
runme echo "ADD_LDFLAGS += -L/usr/local/lib/graphviz/" >> ./config.mk
echo " "
echo "Running Make"
echo " "
runme make -j$(sysctl -n hw.ncpu)

#
# Install MXNet package for Python
#
echo "Installing MXNet package for Python..."
runme cd ${MXNET_HOME}/python
runme sudo python setup.py install

#
# Test MXNet
#
echo "Testing MXNet now..."
python << END > mxnet_test.log
import mxnet as mx
a = mx.nd.ones((2, 3));
print ((a*2).asnumpy());
END
cat << END > mxnet_test.expected
[[ 2. 2. 2.]
[ 2. 2. 2.]]
END
diff mxnet_test.log mxnet_test.expected
if [[ $? = 0 ]]; then
echo $LINE
echo " "
echo "SUCCESS: MXNet test passed"
echo "SUCCESS: MXNet is successfully installed and works fine!"
echo ":-)" | banner -w 40
echo " "
echo $LINE
exit 0
else
echo $LINE
echo " "
echo "ERROR: MXNet test failed"
echo ":-(" | banner -w 40
echo " "
echo $LINE
exit 1
fi
48 changes: 0 additions & 48 deletions setup-utils/install-mxnet-osx.sh

This file was deleted.

0 comments on commit 44bee23

Please sign in to comment.