Haddock+Hoogle for Reflex

[dfordivam]

I think it would be good to have haddock hosted on github and may be updated nightly, so that we have an easy access to the latest haddock.

I intend to link haddock to the docs, so need a proper place to have the latest version.

[mightybyte]

Fantastic idea!

[ryantrinkle]

Agreed.

[3noch]

Up-to-date haddocks anywhere is a great idea. :P

[3noch]

That's my way of saying 👍

[ryantrinkle]

@3noch They're in the try-reflex shell :D But on the internet would also be awesome.

[dfordivam]

@ryantrinkle can this be generated along with the nix cache ... ?
Like just automatic pushing of the generated html in a sub-directory of this repo...

[ElvishJerricco]

Haddocks are generated in the Nix builds. I know you can use ghcWitihHoogle to access them. And incidentally, both ./try-reflex and shell.nix use that.

$ cd reflex-platform
$ nix-shell --run "hoogle serve --local -p 8080"

Having a site on reflex-frp.org that just shows this hoogle server would probably be an easy and useful way to provide docs.

[dfordivam]

Yeah good idea!
But I am not sure whether it can work or not by simply starting a hoogle server instance... The local hoogle server points to the generated haddock html in /nix/...
For an external server it needs to be on some "static" directory...
or may be there is some way I am not aware of...

[ElvishJerricco]

The --local argument doesn't serve file:// links so it should work fine, but the recommended way is probably the --cdn argument.

[ryantrinkle]

Yes, it'd be great to have current haddocks hosted. I always run ./cache before pushing to develop, so if someone can figure out what needs to be added there, we can make it a policy.

…

On Jun 29, 2017 03:08, "Will Fancher" ***@***.***> wrote: The --local argument doesn't serve file:// links so it should work fine, but the recommended way is probably the --cdn argument. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#9 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABGlYFk36HDrGNrqgmqab5fYvwI4S-a1ks5sI01fgaJpZM4OBy7y> .

[dfordivam]

Though the "--local" argument to hoogle will be able to serve the files directly from /nix/store path...

I think the better way will be to copy the html files to a separate server (which might not even have the nix) and serve the files directly from some static dir.
I tried to implement this using the script in #11 and it seems to work fine.

And another benefit is that we can have absolute paths to the haddocks for linking from docs. With --local option the paths will be like
(http://localhost:8080/file/nix/store/mny4j8ikbyj7n9nf9v41ilzxlbym33ny-reflex-0.5.0/share/doc/x86_64-linux-ghc-8.0.2/reflex-0.5.0/html/index.html)

[ElvishJerricco]

Isn't that what the --cdn argument would solve?

[dfordivam]

Yeah but --cdn argument will require the haddocs to be hosted somewhere. So we do need to host the haddocks separately...

--cdn is just a another way to do what I did in the script. I used the @url option...

[ElvishJerricco]

Oh I see. Using --cdn still requires --local to properly serve the haddocks. That's quite a silly deficiency on Hoogle's part.

[ElvishJerricco]

Though I just did a test and it seems Hoogle gives a 404 for any attempt to fetch arbitrary non-haddock files from the nix store. My mistake. Just doesn't list arbitrary directories. You can download arbitrary files.

Not to mention, if the server this runs on is already a binary cache, then it's already arbitrarily serving those files anyway.

[dfordivam]

Though it is possible to do it..

I think it would be better to get reflex on Hackage and use it from there. If we put links to an alternate haddock in the documentation, we will create one more dependency to manage.

So I will close this ..

ref: reflex-frp/reflex#75

[srid]

See also: reflex-frp/reflex-platform#234 (comment)

[lspitzner]

Not aware of this issue, I recently created a simple script to host up-to-date API docs on my server. I agree that docs around reflex are too spread out, so I will post the script to enable hosting the same in some better location.

• This is based on very basic tools only - cabal and haddock. No nix and no hoogle. Not sure if there'd be any advantage to using those instead. To fix the absolute links in the generated html I resorted to sed. The resulting docs should be fully relocatable as all links are relative.
• One thing this script does not do yet is properly mention commit hashes in the index.html. Trivial to fix.
• It does not link to hackage docs for packages that are available, but instead simply hosts all deps.
• The static size of the docs is ~700MB, compressed ~30MB. I did not purge the non-html stuff (i think there are still the haddock databases in there) but the html is the large majority anyway.

Behold my bash skills! (hah, do you even know what ${BOOTLIBLOC//\//\\\/} does? yeah those sed lines are meant to be written once and not to be maintained.)

#!/bin/bash

set -ex

############################
# system-dependent variables
# adapt as necessary
############################

GHCSHARELOC="REPLACEME" # /usr/share/ghc or whatever that is on your system
BOOTLIBLOC="$GHCSHARELOC/doc/ghc-8.0.2/html/libraries"
COMPILERVERSIONPART="x86_64-linux-ghc-8.0.2"

####


OUTPUTDIR=$(pwd)
TEMPDIR=$(mktemp -d -t haddock-gen-XXXX)
echo "using dir $TEMPDIR"
trap 'rm -r $TEMPDIR' EXIT

TEMPDIRDOCS="$TEMPDIR/doc"
mkdir "$TEMPDIRDOCS" || true

cd "$TEMPDIR"

# build stuff, generate docs
####

git clone git@github.com:reflex-frp/reflex.git
git clone git@github.com:reflex-frp/reflex-dom.git
git clone git@github.com:bennofs/reflex-host.git

cabal sandbox init
cabal install --enable-documentation --haddock-options='--hyperlinked-source'\
  reflex/reflex.cabal\
  reflex-dom/reflex-dom-core/reflex-dom-core.cabal\
  reflex-dom/reflex-dom/reflex-dom.cabal\
  reflex-host/reflex-host.cabal\

# copy bootlib and newly generated docs into one directory; fix absolute html
# references
####

cp -r -t "$TEMPDIRDOCS" -- "$BOOTLIBLOC"/*
cp -r -t "$TEMPDIRDOCS" -- "$TEMPDIR/.cabal-sandbox/share/doc/$COMPILERVERSIONPART/"/*

find "$TEMPDIRDOCS" -name "*.html" | xargs sed -i -e "s/file:\/\/${BOOTLIBLOC//\//\\\/}/..\/../g;s/file:\/\/${TEMPDIR//\//\\\/}\/.cabal-sandbox\/share\/doc\/$COMPILERVERSIONPART/..\/../g"

# also copy over our custom index.html
####

cp "$OUTPUTDIR/index.html" "$TEMPDIRDOCS"

# pack it up (switch to directory to have flat locations in the tar)
####

cd "$TEMPDIRDOCS"
tar -c -v -J -f "$OUTPUTDIR/doc.tar.xz" -- *

# todo: upload tar to server etc.
####