Skip to content

Commit 74bbd0b

Browse files
authored
Implement run-hooks as a separate script (#1979)
* Implement run-hooks as a separate script * Add more tests * Add more docs
1 parent c2bf3c6 commit 74bbd0b

File tree

10 files changed

+163
-34
lines changed

10 files changed

+163
-34
lines changed

docs/using/common.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -86,7 +86,7 @@ You do so by passing arguments to the `docker run` command.
8686

8787
```{note}
8888
`NB_UMASK` when set only applies to the Jupyter process itself -
89-
you cannot use it to set a `umask` for additional files created during run-hooks.
89+
you cannot use it to set a `umask` for additional files created during `run-hooks.sh`.
9090
For example, via `pip` or `conda`.
9191
If you need to set a `umask` for these, you **must** set the `umask` value for each command.
9292
```
@@ -135,7 +135,7 @@ or executables (`chmod +x`) to be run to the paths below:
135135
- `/usr/local/bin/before-notebook.d/` - handled **after** all the standard options noted above are applied
136136
and ran right before the Server launches
137137

138-
See the `run-hooks` function in the [`jupyter/docker-stacks-foundation start.sh`](https://github.com/jupyter/docker-stacks/blob/main/images/docker-stacks-foundation/start.sh)
138+
See the `run-hooks.sh` script [here](https://github.com/jupyter/docker-stacks/blob/main/images/docker-stacks-foundation/run-hooks.sh) and how it's used in the [`start.sh`](https://github.com/jupyter/docker-stacks/blob/main/images/docker-stacks-foundation/start.sh)
139139
script for execution details.
140140

141141
## SSL Certificates

docs/using/selecting.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ It contains:
3636
with ownership over the `/home/jovyan` and `/opt/conda` paths
3737
- `tini` as the container entry point
3838
- A `start.sh` script as the default command - useful for running alternative commands in the container as applications are added (e.g. `ipython`, `jupyter kernelgateway`, `jupyter lab`)
39+
- A `run-hooks.sh` script, which can source/run files in a given directory
3940
- Options for a passwordless sudo
4041
- Common system libraries like `bzip2`, `ca-certificates`, `locales`
4142
- `wget` to download external files

images/docker-stacks-foundation/Dockerfile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -127,7 +127,7 @@ ENTRYPOINT ["tini", "-g", "--"]
127127
CMD ["start.sh"]
128128

129129
# Copy local files as late as possible to avoid cache busting
130-
COPY start.sh /usr/local/bin/
130+
COPY run-hooks.sh start.sh /usr/local/bin/
131131

132132
USER root
133133

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
#!/bin/bash
2+
# Copyright (c) Jupyter Development Team.
3+
# Distributed under the terms of the Modified BSD License.
4+
5+
# The run-hooks.sh script looks for *.sh scripts to source
6+
# and executable files to run within a passed directory
7+
8+
if [ "$#" -ne 1 ]; then
9+
echo "Should pass exactly one directory"
10+
return 1
11+
fi
12+
13+
if [[ ! -d "${1}" ]] ; then
14+
echo "Directory ${1} doesn't exist or is not a directory"
15+
return 1
16+
fi
17+
18+
echo "Running hooks in: ${1} as uid: $(id -u) gid: $(id -g)"
19+
for f in "${1}/"*; do
20+
# Hadling a case when the directory is empty
21+
[ -e "${f}" ] || continue
22+
case "${f}" in
23+
*.sh)
24+
echo "Sourcing shell script: ${f}"
25+
# shellcheck disable=SC1090
26+
source "${f}"
27+
;;
28+
*)
29+
if [ -x "${f}" ] ; then
30+
echo "Running executable: ${f}"
31+
"${f}"
32+
else
33+
echo "Ignoring non-executable: ${f}"
34+
fi
35+
;;
36+
esac
37+
done
38+
echo "Done running hooks in: ${1}"

images/docker-stacks-foundation/start.sh

Lines changed: 6 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -14,33 +14,6 @@ _log () {
1414
}
1515
_log "Entered start.sh with args:" "$@"
1616

17-
# The run-hooks function looks for .sh scripts to source and executable files to
18-
# run within a passed directory.
19-
run-hooks () {
20-
if [[ ! -d "${1}" ]] ; then
21-
return
22-
fi
23-
_log "${0}: running hooks in: ${1} as uid: $(id -u) gid: $(id -g)"
24-
for f in "${1}/"*; do
25-
case "${f}" in
26-
*.sh)
27-
_log "${0}: sourcing shell script: ${f}"
28-
# shellcheck disable=SC1090
29-
source "${f}"
30-
;;
31-
*)
32-
if [[ -x "${f}" ]] ; then
33-
_log "${0}: running executable: ${f}"
34-
"${f}"
35-
else
36-
_log "${0}: ignoring non-executable: ${f}"
37-
fi
38-
;;
39-
esac
40-
done
41-
_log "${0}: done running hooks in: ${1}"
42-
}
43-
4417
# A helper function to unset env vars listed in the value of the env var
4518
# JUPYTER_ENV_VARS_TO_UNSET.
4619
unset_explicit_env_vars () {
@@ -62,7 +35,8 @@ else
6235
fi
6336

6437
# NOTE: This hook will run as the user the container was started with!
65-
run-hooks /usr/local/bin/start-notebook.d
38+
# shellcheck disable=SC1091
39+
source /usr/local/bin/run-hooks.sh /usr/local/bin/start-notebook.d
6640

6741
# If the container started as the root user, then we have permission to refit
6842
# the jovyan user, and ensure file permissions, grant sudo rights, and such
@@ -160,7 +134,8 @@ if [ "$(id -u)" == 0 ] ; then
160134
fi
161135

162136
# NOTE: This hook is run as the root user!
163-
run-hooks /usr/local/bin/before-notebook.d
137+
# shellcheck disable=SC1091
138+
source /usr/local/bin/run-hooks.sh /usr/local/bin/before-notebook.d
164139

165140
unset_explicit_env_vars
166141
_log "Running as ${NB_USER}:" "${cmd[@]}"
@@ -255,7 +230,8 @@ else
255230
fi
256231

257232
# NOTE: This hook is run as the user we started the container as!
258-
run-hooks /usr/local/bin/before-notebook.d
233+
# shellcheck disable=SC1091
234+
source /usr/local/bin/run-hooks.sh /usr/local/bin/before-notebook.d
259235
unset_explicit_env_vars
260236
_log "Executing the command:" "${cmd[@]}"
261237
exec "${cmd[@]}"

tests/conftest.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -108,6 +108,7 @@ def run_and_wait(
108108
timeout: int,
109109
no_warnings: bool = True,
110110
no_errors: bool = True,
111+
no_failure: bool = True,
111112
**kwargs: Any,
112113
) -> str:
113114
running_container = self.run_detached(**kwargs)
@@ -119,7 +120,10 @@ def run_and_wait(
119120
assert not self.get_warnings(logs)
120121
if no_errors:
121122
assert not self.get_errors(logs)
122-
assert rv == 0 or rv["StatusCode"] == 0
123+
if no_failure:
124+
assert rv == 0 or rv["StatusCode"] == 0
125+
else:
126+
assert rv != 0 and rv["StatusCode"] != 0
123127
return logs
124128

125129
@staticmethod
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
#!/usr/bin/env python3
2+
# Copyright (c) Jupyter Development Team.
3+
# Distributed under the terms of the Modified BSD License.
4+
5+
print("Executable python file was successfully run")
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
#!/usr/bin/env python3
2+
# Copyright (c) Jupyter Development Team.
3+
# Distributed under the terms of the Modified BSD License.
4+
5+
assert False
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
#!/bin/bash
2+
# Copyright (c) Jupyter Development Team.
3+
# Distributed under the terms of the Modified BSD License.
4+
5+
export SOME_VAR=123
Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
# Copyright (c) Jupyter Development Team.
2+
# Distributed under the terms of the Modified BSD License.
3+
import logging
4+
from pathlib import Path
5+
6+
from tests.conftest import TrackedContainer
7+
8+
LOGGER = logging.getLogger(__name__)
9+
THIS_DIR = Path(__file__).parent.resolve()
10+
11+
12+
def test_run_hooks_zero_args(container: TrackedContainer) -> None:
13+
logs = container.run_and_wait(
14+
timeout=5,
15+
tty=True,
16+
no_failure=False,
17+
command=["bash", "-c", "source /usr/local/bin/run-hooks.sh"],
18+
)
19+
assert "Should pass exactly one directory" in logs
20+
21+
22+
def test_run_hooks_two_args(container: TrackedContainer) -> None:
23+
logs = container.run_and_wait(
24+
timeout=5,
25+
tty=True,
26+
no_failure=False,
27+
command=[
28+
"bash",
29+
"-c",
30+
"source /usr/local/bin/run-hooks.sh first-arg second-arg",
31+
],
32+
)
33+
assert "Should pass exactly one directory" in logs
34+
35+
36+
def test_run_hooks_missing_dir(container: TrackedContainer) -> None:
37+
logs = container.run_and_wait(
38+
timeout=5,
39+
tty=True,
40+
no_failure=False,
41+
command=[
42+
"bash",
43+
"-c",
44+
"source /usr/local/bin/run-hooks.sh /tmp/missing-dir/",
45+
],
46+
)
47+
assert "Directory /tmp/missing-dir/ doesn't exist or is not a directory" in logs
48+
49+
50+
def test_run_hooks_dir_is_file(container: TrackedContainer) -> None:
51+
logs = container.run_and_wait(
52+
timeout=5,
53+
tty=True,
54+
no_failure=False,
55+
command=[
56+
"bash",
57+
"-c",
58+
"touch /tmp/some-file && source /usr/local/bin/run-hooks.sh /tmp/some-file",
59+
],
60+
)
61+
assert "Directory /tmp/some-file doesn't exist or is not a directory" in logs
62+
63+
64+
def test_run_hooks_empty_dir(container: TrackedContainer) -> None:
65+
container.run_and_wait(
66+
timeout=5,
67+
tty=True,
68+
command=[
69+
"bash",
70+
"-c",
71+
"mkdir /tmp/empty-dir && source /usr/local/bin/run-hooks.sh /tmp/empty-dir/",
72+
],
73+
)
74+
75+
76+
def test_run_hooks_with_files(container: TrackedContainer) -> None:
77+
host_data_dir = THIS_DIR / "run-hooks-data"
78+
cont_data_dir = "/home/jovyan/data"
79+
# https://forums.docker.com/t/all-files-appear-as-executable-in-file-paths-using-bind-mount/99921
80+
# Unfortunately, Docker treats all files in mounter dir as executable files
81+
# So we make a copy of mounted dir inside a container
82+
command = (
83+
"cp -r /home/jovyan/data/ /home/jovyan/data-copy/ &&"
84+
"source /usr/local/bin/run-hooks.sh /home/jovyan/data-copy/ &&"
85+
"echo SOME_VAR is ${SOME_VAR}"
86+
)
87+
logs = container.run_and_wait(
88+
timeout=5,
89+
volumes={str(host_data_dir): {"bind": cont_data_dir, "mode": "ro"}},
90+
tty=True,
91+
command=["bash", "-c", command],
92+
)
93+
assert "Executable python file was successfully" in logs
94+
assert "Ignoring non-executable: /home/jovyan/data-copy//non_executable.py" in logs
95+
assert "SOME_VAR is 123" in logs

0 commit comments

Comments
 (0)