Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REVIEW] Initial version of devel images with BlazingSQL added #118

Closed
wants to merge 16 commits into from
Closed

[REVIEW] Initial version of devel images with BlazingSQL added #118

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
e28cd9d
Initial version of devel images with BlazingSQL.
rlratzel Jul 7, 2020
5f0235d
Merge remote-tracking branch 'upstream/branch-0.15' into branch-0.15-…
rlratzel Jul 7, 2020
9a6ba51
Merge remote-tracking branch 'upstream/branch-0.15' into branch-0.15-…
rlratzel Jul 7, 2020
3b202da
Commiting generated files from upstream for easier merge.
rlratzel Jul 10, 2020
6150f8e
Merge remote-tracking branch 'upstream/branch-0.15' into branch-0.15-…
rlratzel Jul 10, 2020
af05c8c
Updated install step to use gpuci_conda_retry.
rlratzel Jul 13, 2020
9b493cb
Merge remote-tracking branch 'upstream/branch-0.15' into branch-0.15-…
rlratzel Jul 13, 2020
88f7c29
Merge remote-tracking branch 'upstream/branch-0.15' into branch-0.15-…
rlratzel Jul 29, 2020
2d1d3c9
Update generate_dockerfiles.py
ajschmidt8 Aug 5, 2020
dd5580b
Merge remote-tracking branch 'upstream/branch-0.15' into branch-0.15-…
rlratzel Aug 5, 2020
6b1ebcd
Merge remote-tracking branch 'upstream/branch-0.15' into branch-0.15-…
rlratzel Aug 12, 2020
531fd3d
Updated docstring based on review feedback
rlratzel Aug 12, 2020
3152a0f
Merge remote-tracking branch 'origin/branch-0.15-blazingdevel' into b…
rlratzel Aug 12, 2020
a5a643d
Fixed typo
rlratzel Aug 13, 2020
2276794
Updates to support Blazing: removed special libm for gcc7 that preven…
rlratzel Aug 19, 2020
430ec8e
Merge remote-tracking branch 'upstream/branch-0.15' into branch-0.15-…
rlratzel Aug 19, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Commiting generated files from upstream for easier merge.
  • Loading branch information
@rlratzel
rlratzel committed Jul 10, 2020
commit 3b202da4b75facaffb0576b27d3ac74100915e25
4 changes: 2 additions & 2 deletions generated-dockerfiles/centos7-base.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@
#
# Copyright (c) 2020, NVIDIA CORPORATION.

ARG CUDA_VER=10.0
ARG CUDA_VER=10.1
ARG LINUX_VER=centos7
ARG PYTHON_VER=3.6
ARG PYTHON_VER=3.7
ARG RAPIDS_VER=0.15
ARG FROM_IMAGE=gpuci/rapidsai

Expand Down
61 changes: 31 additions & 30 deletions generated-dockerfiles/centos7-devel.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,38 @@
#
# Copyright (c) 2020, NVIDIA CORPORATION.

ARG CUDA_VER=10.0
ARG CUDA_VER=10.1
ARG LINUX_VER=centos7
ARG PYTHON_VER=3.6
ARG PYTHON_VER=3.7
ARG RAPIDS_VER=0.15
ARG FROM_IMAGE=gpuci/rapidsai

FROM ${FROM_IMAGE}:${RAPIDS_VER}-cuda${CUDA_VER}-devel-${LINUX_VER}-py${PYTHON_VER}

RUN git clone https://github.com/ccache/ccache.git /tmp/ccache && cd /tmp/ccache \
&& git checkout -b rapids-compose-tmp b1fcfbca224b2af5b6499794edd8615dbc3dc7b5 \
&& ./autogen.sh \
&& ./configure --disable-man --with-libb2-from-internet --with-libzstd-from-internet\
&& make install -j \
&& cd / \
&& rm -rf /tmp/ccache* \
&& mkdir -p /ccache

ENV CCACHE_NOHASHDIR=
ENV CCACHE_DIR="/ccache"
ENV CCACHE_COMPILERCHECK="%compiler% --version"

ENV CC="/usr/local/bin/gcc"
ENV CXX="/usr/local/bin/g++"
ENV NVCC="/usr/local/bin/nvcc"
ENV CUDAHOSTCXX="/usr/local/bin/g++"
RUN ln -s "$(which ccache)" "/usr/local/bin/gcc" \
&& ln -s "$(which ccache)" "/usr/local/bin/g++" \
&& ln -s "$(which ccache)" "/usr/local/bin/nvcc"

COPY ccache /ccache
RUN ccache -s

ARG PARALLEL_LEVEL=16
ARG RAPIDS_VER=0.15*

Expand All @@ -32,7 +56,7 @@ RUN source activate rapids \
&& conda info \
&& conda config --show-sources \
&& conda list --show-channel-urls
RUN gpuci_conda_retry install -y -n rapids \
RUN gpuci_retry conda install -y -n rapids \
rapids-build-env=${RAPIDS_VER} \
rapids-doc-env=${RAPIDS_VER} \
&& conda remove -y -n rapids --force-remove \
Expand Down Expand Up @@ -178,34 +202,11 @@ RUN cd ${RAPIDS_DIR}/dask-cuda && \

ENV LD_LIBRARY_PATH=${LD_LIBRARY_PATH_PREBUILD}

ENV BLAZING_DIR=/blazing
RUN ccache -s \
&& ccache -c \
&& chmod -R ugo+w /ccache \
&& ccache -s

RUN source activate rapids \
&& conda install -y \
google-cloud-cpp \
ninja \
gtest \
gmock \
cppzmq \
openjdk=8.0 \
maven \
thrift=0.13.0 \
jpype1 \
netifaces \
pyhive

ENV CUDF_HOME=/rapids/cudf

RUN mkdir -p ${BLAZING_DIR} \
&& cd ${BLAZING_DIR} \
&& git clone https://github.com/BlazingDB/blazingsql.git

RUN source activate rapids \
&& cd ${BLAZING_DIR}/blazingsql \
&& ./build.sh
RUN mkdir -p ${BLAZING_DIR} \
&& cd ${BLAZING_DIR} \
&& git clone https://github.com/BlazingDB/Welcome_to_BlazingSQL_Notebooks.git

RUN conda clean -afy \
&& chmod -R ugo+w /opt/conda ${RAPIDS_DIR}
Expand Down
4 changes: 2 additions & 2 deletions generated-dockerfiles/centos7-quick.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@
#
# Copyright (c) 2020, NVIDIA CORPORATION.

ARG CUDA_VER=10.0
ARG CUDA_VER=10.1
ARG LINUX_VER=centos7
ARG PYTHON_VER=3.6
ARG PYTHON_VER=3.7
ARG RAPIDS_VER=0.15
ARG FROM_IMAGE=rapidsaistaging/rapidsai-dev-nightly-staging

Expand Down
4 changes: 2 additions & 2 deletions generated-dockerfiles/centos7-runtime.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@
#
# Copyright (c) 2020, NVIDIA CORPORATION.

ARG CUDA_VER=10.0
ARG CUDA_VER=10.1
ARG LINUX_VER=centos7
ARG PYTHON_VER=3.6
ARG PYTHON_VER=3.7
ARG RAPIDS_VER=0.15
ARG FROM_IMAGE=gpuci/rapidsai

Expand Down
4 changes: 2 additions & 2 deletions generated-dockerfiles/ubuntu18.04-base.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@
#
# Copyright (c) 2020, NVIDIA CORPORATION.

ARG CUDA_VER=10.0
ARG CUDA_VER=10.1
ARG LINUX_VER=ubuntu18.04
ARG PYTHON_VER=3.6
ARG PYTHON_VER=3.7
ARG RAPIDS_VER=0.15
ARG FROM_IMAGE=gpuci/rapidsai

Expand Down
61 changes: 31 additions & 30 deletions generated-dockerfiles/ubuntu18.04-devel.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,38 @@
#
# Copyright (c) 2020, NVIDIA CORPORATION.

ARG CUDA_VER=10.0
ARG CUDA_VER=10.1
ARG LINUX_VER=ubuntu18.04
ARG PYTHON_VER=3.6
ARG PYTHON_VER=3.7
ARG RAPIDS_VER=0.15
ARG FROM_IMAGE=gpuci/rapidsai

FROM ${FROM_IMAGE}:${RAPIDS_VER}-cuda${CUDA_VER}-devel-${LINUX_VER}-py${PYTHON_VER}

RUN git clone https://github.com/ccache/ccache.git /tmp/ccache && cd /tmp/ccache \
&& git checkout -b rapids-compose-tmp b1fcfbca224b2af5b6499794edd8615dbc3dc7b5 \
&& ./autogen.sh \
&& ./configure --disable-man --with-libb2-from-internet --with-libzstd-from-internet\
&& make install -j \
&& cd / \
&& rm -rf /tmp/ccache* \
&& mkdir -p /ccache

ENV CCACHE_NOHASHDIR=
ENV CCACHE_DIR="/ccache"
ENV CCACHE_COMPILERCHECK="%compiler% --version"

ENV CC="/usr/local/bin/gcc"
ENV CXX="/usr/local/bin/g++"
ENV NVCC="/usr/local/bin/nvcc"
ENV CUDAHOSTCXX="/usr/local/bin/g++"
RUN ln -s "$(which ccache)" "/usr/local/bin/gcc" \
&& ln -s "$(which ccache)" "/usr/local/bin/g++" \
&& ln -s "$(which ccache)" "/usr/local/bin/nvcc"

COPY ccache /ccache
RUN ccache -s

ARG PARALLEL_LEVEL=16
ARG RAPIDS_VER=0.15*

Expand All @@ -35,7 +59,7 @@ RUN source activate rapids \
&& conda info \
&& conda config --show-sources \
&& conda list --show-channel-urls
RUN gpuci_conda_retry install -y -n rapids \
RUN gpuci_retry conda install -y -n rapids \
rapids-build-env=${RAPIDS_VER} \
rapids-doc-env=${RAPIDS_VER} \
&& conda remove -y -n rapids --force-remove \
Expand Down Expand Up @@ -175,34 +199,11 @@ RUN cd ${RAPIDS_DIR}/dask-cuda && \



ENV BLAZING_DIR=/blazing
RUN ccache -s \
&& ccache -c \
&& chmod -R ugo+w /ccache \
&& ccache -s

RUN source activate rapids \
&& conda install -y \
google-cloud-cpp \
ninja \
gtest \
gmock \
cppzmq \
openjdk=8.0 \
maven \
thrift=0.13.0 \
jpype1 \
netifaces \
pyhive

ENV CUDF_HOME=/rapids/cudf

RUN mkdir -p ${BLAZING_DIR} \
&& cd ${BLAZING_DIR} \
&& git clone https://github.com/BlazingDB/blazingsql.git

RUN source activate rapids \
&& cd ${BLAZING_DIR}/blazingsql \
&& ./build.sh
RUN mkdir -p ${BLAZING_DIR} \
&& cd ${BLAZING_DIR} \
&& git clone https://github.com/BlazingDB/Welcome_to_BlazingSQL_Notebooks.git

RUN conda clean -afy \
&& chmod -R ugo+w /opt/conda ${RAPIDS_DIR}
Expand Down
4 changes: 2 additions & 2 deletions generated-dockerfiles/ubuntu18.04-quick.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@
#
# Copyright (c) 2020, NVIDIA CORPORATION.

ARG CUDA_VER=10.0
ARG CUDA_VER=10.1
ARG LINUX_VER=ubuntu18.04
ARG PYTHON_VER=3.6
ARG PYTHON_VER=3.7
ARG RAPIDS_VER=0.15
ARG FROM_IMAGE=rapidsaistaging/rapidsai-dev-nightly-staging

Expand Down
4 changes: 2 additions & 2 deletions generated-dockerfiles/ubuntu18.04-runtime.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@
#
# Copyright (c) 2020, NVIDIA CORPORATION.

ARG CUDA_VER=10.0
ARG CUDA_VER=10.1
ARG LINUX_VER=ubuntu18.04
ARG PYTHON_VER=3.6
ARG PYTHON_VER=3.7
ARG RAPIDS_VER=0.15
ARG FROM_IMAGE=gpuci/rapidsai

Expand Down