Installing executables (was: yesod-bin as "extra build tool")

[chreekat]

Is yesod-bin an "extra build tool"?

From the FAQ:

How do I get extra build tools?

stack will automatically install build tools required by your packages [emphasis mine] or their dependencies, in particular alex and happy.

yesod-bin is not technically required, as anyone can use Yesod libraries without bothering with yesod-bin. However, I want it to be available to my project's contributors with a minimum of fuss. Should stack solve this problem for me?

Self-answer: Yes. I can add yesod-bin as an extra-dep, and then 'stack exec yesod' works. Still, the difference between yesod-bin and e.g. happy are a bit magical to me. Do you just have a list of packages known to need a particular tool? What distinguishes them from tool XYZ, needed to build XYZmabob? Will I still need another build tool on top of stack to solve the general problem? Why don't I have to add 'happy' as an extra-dep?

The following issue seems sufficiently related:

yesod-bin as extra-dep

After I add yesod-bin to extra-deps and rebuild, the following occurs:

$ stack exec yesod devel
Yesod devel server. Press ENTER to quit
Application can be accessed at:

http://127.0.0.1:3000
https://127.0.0.1:3443
If you wish to test https capabilities, you should set the following variable:
  export APPROOT=https://127.0.0.1:3443

WARNING: the following source files are not listed in exposed-modules or other-modules:
./Handler/About.hs
./SnowdriftEmailDaemon.hs
./SnowdriftSendmail.hs
yesod: cabal: createProcess: runInteractiveProcess: exec: does not exist (No such file or directory)

[snoyberg]

It's not an extra build tool in the sense of the others, since it's not used during the build process in order to compile your code. alex and happy are dependencies of the build, whereas yesod-bin is a user-facing tool. I can add that distinction to the README if it's clear (or, if you'd like, please modify the README yourself for clarity).

Now actually using yesod devel with stack's sandboxes would require some changes to yesod-bin, since yesod-bin depends on cabal to do the build. That's a change I'll happily make in the future :)

[chreekat]

I felt a twinge writing this ticket because it wasn't very focused, so I'm glad we agree on the definition of a build tool. This might not be the place to talk about it, but I guess the underlying question is, is it within stack's scope to set up (Haskell) binaries where a user can get to them? I want to say no, because the two tasks seem very different, yet the two tasks share many identical pain points that stack is designed to solve.

[snoyberg]

So firstly, we have part of a solution with the stack deps command. Running stack deps yesod-bin should build the executable. The question is how do you get stack to copy that executable to somewhere you can use it?

I was talking about the lack of an install command with @feuerbach yesterday. Ideally, stack install yesod-bin would build yesod-bin and then copy the resulting executable to some "global"-ish install place. There are two problems:

1. Where would that location be? We could do the cabal thing and copy to ~/.stack/bin, which is reasonable, but then we'd need to tell users to modify their PATH to include that directory (which, again, may be perfectly reasonable).
2. install as a command comes with a lot of mental baggage from the cabal world, where install means "grab dependencies, install them to a global place, build the local package, and copy it over." We do dependencies building automatically in stack, but keep it all isolated. There's no concept of "installing" libraries, only executables. I'm concerned that people will be very confused with an install command.

Perhaps I'm overthinking this and we should just provide install, or perhaps we should come up with some other command. I'm open to thoughts. In any event, for now, I've been using the following incantation while working on stack:

stack build && cp $(stack exec which stack) ~/.cabal/bin

[chreekat]

I would say you use a completely different name precisely because so much of the confusion around cabal stems from the fact that 'cabal install' and 'cabal build' are such different commands. Better to avoid that legacy.

The tradeoff, of course, would be having two separate commands.

That would be acceptable to me, but if it's too much, perhaps one could use 'install-tool' instead of 'install'?

As for where to install, I would suggest ~/.local/bin. It's a can of worms, but npm uses it, and apparently python too. Fedora supports it out of the box, but I had to add it to my PATH manually on Ubuntu.

Using ~/.local/bin would be a nice shortcut for me because I already sandbox-build tools and copy them there manually. :)

[snoyberg]

So to summarize, I'm seeing the following work items coming out of this:

1. Add a new install-tool command (name to be bikeshedded to death) that will do the same thing as build and then copy any executables generated by it to the user bin directory. We'll use ~/.local/bin on POSIX systems, and figure something out for Windows. We should probably warn the user if that directory is not on his/her PATH.
2. Update the README to include information on this new install-tool command and explain how it's different from cabal install.

Does that sound reasonable?

[UnkindPartition]

In addition to copying, install-tool should also do stripping. That can reduce the size of a Haskell executable by ~40%.

[snoyberg]

👍

[mboes]

That said, with the new DWARF support in GHC 7.10, the extra debug info that strip removes become very useful (e.g. for performance profiling). There's something to be said about being able to quick and dirty perf analysis by default for an haskell executable. Is the executable size an issue?

[snoyberg]

This is now implemented. Some details:

• Running stack install-exe will install executables for the "wanted" targets, following the same rules as stack test, for example
• It gives a warning when the destination directory is not on the PATH
• On Windows, this cannot be used to upgrade stack yet, since we need to do some fancy footwork about moving the exe around (TODO added)

Now that I've implemented this, I'm beginning to think I made a mistake about saying we should give it a name besides install. Just because cabal has weird nomenclature doesn't mean we should do. What does everyone think about renaming the command to install instead?

[snoyberg]

Fixing the Windows upgrade problem was easy, and I've implemented it. But I forgot to mention: there's some discussion warranted around the path on Windows. Currently, it's using directories like C:\Users\Michael\AppData\Roaming\local\bin. Any better ideas?

[UnkindPartition]

I have no problem with install.

Does it work for non-local targets? E.g. I may want to just say stack install pandoc.

[snoyberg]

Yes, it should. One thing we could consider is figuring out good behavior for running stack install outside of a project directory, presumably working the same way stack exec will end up working.

[chreekat]

My real preference is to have two commands with the same distinction as
exists between 'apt-get install' and 'make install'. The latter makes no
sense in a non-project directory, and the former doesn't care about $PWD at
all.

If a command is to do both, it may as well be called install. :P Maybe
we'll advance to a bright future where I can just apt-get install the
latest binaries from LTS Stackage anyway...

On Wed, Jun 10, 2015 at 4:47 AM, Michael Snoyman notifications@github.com
wrote:

Yes, it should. One thing we could consider is figuring out good behavior
for running stack install outside of a project directory, presumably
working the same way stack exec will end up working.

—
Reply to this email directly or view it on GitHub
#153 (comment)
.

[snoyberg]

Right now, if you run stack install pandoc outside of a project directory, you'll be guaranteed to get the version from upstream, which may be pretty close to what you're looking for

[chreekat]

Aye, but if I run stack install pandoc inside a project, it.... oh. Hm! It does a smart thing. I am satisfied.

Well, except for the following:

Couldn't find executable make-pandoc-man-pages in directory /home/b/.stack/snapshots/x86_64-linux/lts-2.9/7.8.4/bin/
Copying from /home/b/.stack/snapshots/x86_64-linux/lts-2.9/7.8.4/bin/pandoc to /home/b/.local/bin/pandoc
Couldn't find executable trypandoc in directory /home/b/.stack/snapshots/x86_64-linux/lts-2.9/7.8.4/bin/

[snoyberg]

That might be that the buildable flag isn't being tracked properly right
now. There's a todo in the codebase to track that.

On Wed, Jun 10, 2015, 5:44 PM Bryan Richter notifications@github.com
wrote:

Aye, but if I run stack install pandoc inside a project, it.... oh. Hm!
It does a smart thing. I am satisfied.

Well, except for the following:

Couldn't find executable make-pandoc-man-pages in directory /home/b/.stack/snapshots/x86_64-linux/lts-2.9/7.8.4/bin/
Copying from /home/b/.stack/snapshots/x86_64-linux/lts-2.9/7.8.4/bin/pandoc to /home/b/.local/bin/pandoc
Couldn't find executable trypandoc in directory /home/b/.stack/snapshots/x86_64-linux/lts-2.9/7.8.4/bin/

—
Reply to this email directly or view it on GitHub
#153 (comment)
.