Skip to content

Examples in the docs: base PR for cleaning up notebook workflow #303

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 70 commits into from
Jun 30, 2021
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
1b1a560
copy make.jl from st/examples
st-- Jun 16, 2021
7cabf13
add general docs dependencies
st-- Jun 16, 2021
d2cefec
clean up
st-- Jun 16, 2021
60acddb
auto-collect examples as suggested
st-- Jun 16, 2021
1831892
add docs dependencies for notebooks
st-- Jun 16, 2021
a594008
fix direct notebook bug
st-- Jun 16, 2021
271d3d7
add blacklist as pairwise() is broken
st-- Jun 16, 2021
ef894a4
comment out examples as it breaks for non-existing directory (everyth…
st-- Jun 16, 2021
beb4adb
add docs README
st-- Jun 16, 2021
00ccba9
Apply suggestions from code review
st-- Jun 16, 2021
feda9e0
remove duplicate doc entries
st-- Jun 16, 2021
e30169f
Merge branch 'st/examples-base' of github.com:JuliaGaussianProcesses/…
st-- Jun 16, 2021
901b5d0
revert deletion
st-- Jun 16, 2021
a1718a3
separate examples into individual diretories with own Project.toml
st-- Jun 17, 2021
66706c0
handle empty examples directory
st-- Jun 17, 2021
bddbea8
update README for non-unix systems
st-- Jun 17, 2021
c4a36f8
Apply suggestions from code review
st-- Jun 17, 2021
384ac9c
Apply suggestions from code review
st-- Jun 17, 2021
80deae7
revert removed doc
st-- Jun 17, 2021
004d45e
Merge branch 'st/examples-base' of github.com:JuliaGaussianProcesses/…
st-- Jun 17, 2021
cd0bc2b
Apply suggestions from code review
st-- Jun 21, 2021
75db97d
remove unused dependencies from svm example
st-- Jun 21, 2021
8c657bc
fix bug in svm notebook
st-- Jun 21, 2021
7ac29c6
Merge branch 'st/examples-base' of github.com:JuliaGaussianProcesses/…
st-- Jun 21, 2021
001dfc3
Apply suggestions from code review
st-- Jun 21, 2021
a60e230
fix examples scripts compose
st-- Jun 21, 2021
b2faceb
Merge branch 'st/examples-base' of github.com:JuliaGaussianProcesses/…
st-- Jun 21, 2021
14ad37d
fix script output name
st-- Jun 21, 2021
715e0a5
bugfix
st-- Jun 21, 2021
09990f8
include preprocessor for Documenter@setup
st-- Jun 21, 2021
4f34f37
Apply suggestions from code review
st-- Jun 21, 2021
f2db1ea
Correct file_path for environment
theogf Jun 21, 2021
c8298ec
Adding the necessary header for the @setup replacement
theogf Jun 21, 2021
754a8db
correct headers for remaining examples scripts
st-- Jun 22, 2021
8641c78
fix KRR notebook and swap it with SVM in blacklist
st-- Jun 22, 2021
407d6f0
do not execute notebook
st-- Jun 22, 2021
3e452ae
update .gitignore
st-- Jun 22, 2021
262ceef
Update docs/make.jl
st-- Jun 22, 2021
495bf19
update docs and examples README and ignore examples/README in docs/ma…
st-- Jun 23, 2021
34c97fa
fix plotting
st-- Jun 23, 2021
87da659
Update examples/kernel-ridge-regression/script.jl
st-- Jun 23, 2021
e851307
add under-construction warnings to examples
st-- Jun 23, 2021
b3c8aa6
copy over fixes from KRR example
st-- Jun 23, 2021
6fd8175
Apply suggestions from code review
st-- Jun 23, 2021
16ba774
Merge branch 'master' of github.com:JuliaGaussianProcesses/KernelFunc…
st-- Jun 28, 2021
11119c2
Merge remote-tracking branch 'origin' into st/examples-base
st-- Jun 29, 2021
f034c63
commit examples/*/Manifest.toml
st-- Jun 29, 2021
2fa2f1b
fix optional package loading
st-- Jun 29, 2021
6b3238b
doc fixes
st-- Jun 29, 2021
1ea5c6c
explicit Pkg.add of Literate
st-- Jun 29, 2021
241cf62
Apply suggestions from code review
st-- Jun 29, 2021
df1c526
Update examples/README.md
st-- Jun 29, 2021
cac9e7d
update README
st-- Jun 29, 2021
f2d8dcc
Update docs/Project.toml
st-- Jun 29, 2021
d060958
Merge branch 'master' into st/examples-base
st-- Jun 29, 2021
2b1627b
update examples *.toml
st-- Jun 29, 2021
a91edc9
use cleaner f form
st-- Jun 29, 2021
1daf1c3
fix errors
st-- Jun 29, 2021
b262ac0
fix example Manifest.tomls
st-- Jun 29, 2021
4cce78e
Update examples/README.md
st-- Jun 29, 2021
64d5ef7
better error messages?
st-- Jun 29, 2021
bf47391
Merge branch 'st/examples-base' of github.com:JuliaGaussianProcesses/…
st-- Jun 29, 2021
853b437
Revert "better error messages?"
st-- Jun 29, 2021
eff7c11
give responsibility for adding Literate dependency back to examples
st-- Jun 29, 2021
a44d48f
preprocess header in italics
st-- Jun 29, 2021
fa0033b
Apply suggestions from code review
st-- Jun 30, 2021
1fd657d
Update examples/README.md
st-- Jun 30, 2021
735b437
compat for Kronecker in docs
st-- Jun 30, 2021
03a6840
add [compat]
st-- Jun 30, 2021
7aebf83
Apply suggestions from code review
st-- Jun 30, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/Project.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,12 @@
[deps]
Distributions = "31c24e10-a181-5473-b8eb-7969acd0382f"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be cleaner and probably easier to maintain (less conflicting dependencies etc) if docs/Project.toml only contains the dependencies for the actual documentation and each example uses a separate project environment with its own pinned dependencies.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have played around with this in https://github.com/devmotion/CalibrationErrors.jl.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I agree - I was wondering how you would make that work but the Pkg.activate() context manager is really neat, so I'll try to add that :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mhm, that would mean having to run the Pkg.develop in every single example subdirectory if you want to work on them though, and would that not make it harder to check the examples run consistently with any future changes to KernelFunctions (in a PR)?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mhm, that would mean having to run the Pkg.develop in every single example subdirectory if you want to work on them

Yes, the main point is to have independent environments. In this way you can avoid that dependencies of different examples are conflicting and it is clear which packages and which versions were used to run the individual examples.

They are re-run every time the documentation is built so one would just check that the build passes and the webpage looks fine.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, for the examples to actually pull in the in-development version of the package I think your make.jl would need to also include a Pkg.develop(path=joinpath(@__DIR__, "..")), but with that I think it should work:)

I think it'd still be good to provide a Makefile or some other way of making it easy for contributors to locally run tests, formatter, etc..

Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4"
Flux = "587475ba-b771-5e3f-ad9e-33799f191a9c"
KernelFunctions = "ec8451be-7e33-11e9-00cf-bbf324bd1392"
Literate = "98b081ad-f1c9-55d3-8b20-4c87d4299306"
MLDataUtils = "cc2ba9b6-d476-5e6d-8eaf-a92d5412d41d"
PDMats = "90014a1f-27ba-587c-ab20-58faa44d9150"
Zygote = "e88e6eb3-aa80-5325-afca-941959d7151f"

[compat]
Documenter = "0.27"
Expand Down
35 changes: 35 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# How to build the docs locally
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to get improvements to the workflow. :)


Assuming you are in this (docs/) directory:

Install docs dependencies:
```bash
julia --project=. -e 'using Pkg; Pkg.instantiate()'
```
This will use the last released KernelFunctions.jl for building the docs.

If instead you want to reflect the local state of the Git repository in your built docs, make sure to change the dependency to the development version:
```bash
julia --project=. -e "using Pkg; Pkg.develop(path=\"$(readlink -f ..)\")"
```
To undo the pinning to the local path by the previous command, run
```bash
julia --project=. -e 'using Pkg; Pkg.free("KernelFunctions")'
```

To actually build the docs, run
```bash
julia --project=. make.jl
```
The built docs will be underneath build/

# How to contribute to the docs

## To add additional docs dependencies

```bash
julia --project=. -e 'using Pkg; Pkg.add("NewDependency")'
```
and commit the changes to Project.toml


26 changes: 22 additions & 4 deletions docs/make.jl
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
using Documenter
using Literate

# Print `@debug` statements (https://github.com/JuliaDocs/Documenter.jl/issues/955)
if haskey(ENV, "GITHUB_ACTIONS")
Expand All @@ -7,6 +8,21 @@ end

using KernelFunctions

const EXAMPLES_SRC = joinpath(@__DIR__, "..", "examples")
const EXAMPLES_OUT = joinpath(@__DIR__, "src", "examples")
const BLACKLIST = ["deepkernellearning", "kernelridgeregression", "svm"]

if ispath(EXAMPLES_OUT)
rm(EXAMPLES_OUT; recursive=true)
end

for filepath in readdir(EXAMPLES_SRC; join=true)
endswith(filepath, ".jl") || continue
any([occursin(blacklistname, filepath) for blacklistname in BLACKLIST]) && continue
Literate.markdown(filepath, EXAMPLES_OUT; documenter=true)
Literate.notebook(filepath, EXAMPLES_OUT; documenter=true)
end

DocMeta.setdocmeta!(
KernelFunctions,
:DocTestSetup,
Expand All @@ -20,9 +36,9 @@ DocMeta.setdocmeta!(
)

makedocs(;
sitename="KernelFunctions",
format=Documenter.HTML(),
modules=[KernelFunctions],
sitename="KernelFunctions.jl",
format=Documenter.HTML(),
pages=[
"Home" => "index.md",
"userguide.md",
Expand All @@ -32,8 +48,10 @@ makedocs(;
"theory.md",
"create_kernel.md",
"API" => "api.md",
"Examples" => "example.md",
"Design" => "design.md",
# "Examples" =>
# joinpath.(
# "examples", filter(filename -> endswith(filename, ".md"), readdir(EXAMPLES_OUT))
# ),
],
strict=true,
checkdocs=:exports,
Expand Down
4 changes: 2 additions & 2 deletions examples/svm.jl
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ function f(x, k, λ)
inv(kernelmatrix(k, X; obsdim=1) + exp(λ[1]) * I) *
y
end # Optimal prediction f
svmloss(y, ŷ) = ŷ -> sum(maximum.(0.0, 1 - y * ŷ)) - λ * norm(ŷ)(f(X, k, λ)) # Total svm loss with regularisation
pred = f(Xgrid, k, λ) #Compute prediction on a grid
svmloss(y, ŷ, λ) = ŷ -> sum(maximum.(0.0, 1 - y * ŷ)) - λ * norm(ŷ)(f(X, k, λ)) # Total svm loss with regularisation
pred = f(Xgrid, k, 0.1) #Compute prediction on a grid
contourf(xgrid, xgrid, pred)
scatter!(eachcol(X)...; color=y, lab="data")