drill_up fails (maybe due to Docker Toolkit?)

[espinielli]

I am eager to try out what described in the blog post about 0.8.0 version, but after having installed as instructed

devtools::install_git("https://git.sr.ht/~hrbrmstr/sergeant", ref="0.8.0")

and then running the code below, I got a failure:

> library(sergeant)
> library(tidyverse)
── Attaching packages ───────────────────────────────────────────────────────── tidyverse 1.2.1 ──
✔ ggplot2 3.1.0     ✔ purrr   0.2.5
✔ tibble  2.0.1     ✔ dplyr   0.7.8
✔ tidyr   0.8.2     ✔ stringr 1.3.1
✔ readr   1.3.1     ✔ forcats 0.3.0
── Conflicts ──────────────────────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag()    masks stats::lag()
> # start Docker in terminal, then open 
> dr <- drill_up(data_dir = here::here("data-raw"))
Drill container started. Waiting for the service to become active (this may take up to 30s).
Error in drill_up(data_dir = here::here("data-raw")) : 
  Could not connect to Drill container.
> 

I have been able to have drill running manually and mapping the local file system to my directory of interest.
On the Docker side I have an old MBP, so I have Docker Toolkit.
(That maybe is the reason for the failure.)
If I run drill image from the command line I get the following:

> docker run drill/apache-drill                                                                                         (master|✚2✭)
Jan 27, 2019 2:25:49 PM org.jline.utils.Log logr
WARNING: Unable to create a system terminal, creating a dumb terminal (enable debug logging for more information)
Apache Drill 1.15.0
"The only truly happy people are children, the creative minority and Drill users."
0: jdbc:drill:zk=local> Closing: org.apache.drill.jdbc.impl.DrillConnectionImpl

I tried to hello_world Docker example using stevedore package and it works...so somewhat Docker (at least the easy bit) seems to work.

I then decided to dig into drill_up implementation and executed line by line to debug.

> docker <- stevedore::docker_client()
> (drill <- docker$container$run(image = "drill/apache-drill:1.15.0", name = "drill", cmd = "/usr/local/bin/bash", volumes = sprintf("%s:/data", here::here("data-raw"))))
E> Jan 27, 2019 3:01:03 PM org.jline.utils.Log logr
E> WARNING: Unable to create a system terminal, creating a dumb terminal (enable debug logging for more information)
E> Apache Drill 1.15.0
E> "Say hello to my little Drill."
E> Closing: org.apache.drill.jdbc.impl.DrillConnectionImpl
O> 0: jdbc:drill:zk=local> <docker_run_output>
  $container:
    <docker_container>
      id: 2db3d96f27f55d35017dd0361531b37e25c80fbbe9e01cf731654c15adb1a773
      name: drill

  $logs:
    Jan 27, 2019 3:01:03 PM org.jline.utils.Log logr
    WARNING: Unable to create a system terminal, creating a dumb terminal (enable debug logging for more information)
    Apache Drill 1.15.0
    "Say hello to my little Drill."
    Closing: org.apache.drill.jdbc.impl.DrillConnectionImpl
    0: jdbc:drill:zk=local> 
> 

Then the 30 attempts of drill_active fail...(here I just show one manual try :-) )

> (drill_con <- drill_connection("localhost"))
<Drill REST API Direct Connection to localhost:8047>
> drill_active(drill_con)
[1] FALSE

I am out of ideas to try to make it work...any directions is welcome.

[hrbrmstr]

Thanks for the amazing problem write-up! Can you dump the session info on that system? I may be able to replicate your environment (I have a few old macs around with various macOSes on them).

[espinielli]

Here is the relevant info out of a restarted session:

> library(sergeant)
> library(tidyverse)
── Attaching packages ───────────────────────────────────────────────────────── tidyverse 1.2.1 ──
✔ ggplot2 3.1.0     ✔ purrr   0.2.5
✔ tibble  2.0.1     ✔ dplyr   0.7.8
✔ tidyr   0.8.2     ✔ stringr 1.3.1
✔ readr   1.3.1     ✔ forcats 0.3.0
── Conflicts ──────────────────────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag()    masks stats::lag()
> # start Docker in terminal, then open 
> dr <- drill_up(data_dir = here::here("data-raw"))
Drill container started. Waiting for the service to become active (this may take up to 30s).
Error in drill_up(data_dir = here::here("data-raw")) : 
  Could not connect to Drill container.
> devtools::session_info()
─ Session info ─────────────────────────────────────────────────────────────────────────────────
 setting  value                       
 version  R version 3.5.2 (2018-12-20)
 os       OS X El Capitan 10.11.6     
 system   x86_64, darwin15.6.0        
 ui       RStudio                     
 language (EN)                        
 collate  en_US.UTF-8                 
 ctype    en_US.UTF-8                 
 tz       Europe/Brussels             
 date     2019-01-27                  

─ Packages ─────────────────────────────────────────────────────────────────────────────────────
 package     * version    date       lib source                         
 askpass       1.1        2019-01-13 [1] CRAN (R 3.5.2)                 
 assertthat    0.2.0      2017-04-11 [1] CRAN (R 3.5.0)                 
 backports     1.1.3      2018-12-14 [1] CRAN (R 3.5.1)                 
 bindr         0.1.1      2018-03-13 [1] CRAN (R 3.5.0)                 
 bindrcpp      0.2.2      2018-03-29 [1] CRAN (R 3.5.0)                 
 bit           1.1-14     2018-05-29 [1] CRAN (R 3.5.0)                 
 bit64         0.9-7      2017-05-08 [1] CRAN (R 3.5.0)                 
 broom         0.5.1      2018-12-05 [1] CRAN (R 3.5.0)                 
 callr         3.1.1      2018-12-21 [1] CRAN (R 3.5.1)                 
 cellranger    1.1.0      2016-07-27 [1] CRAN (R 3.5.0)                 
 cli           1.0.1      2018-09-25 [1] CRAN (R 3.5.0)                 
 colorspace    1.4-0      2019-01-13 [1] CRAN (R 3.5.2)                 
 crayon        1.3.4      2017-09-16 [1] CRAN (R 3.5.0)                 
 curl          3.3        2019-01-10 [1] CRAN (R 3.5.2)                 
 DBI           1.0.0.9001 2019-01-27 [1] Github (r-dbi/DBI@be695e2)     
 dbplyr        1.3.0      2019-01-09 [1] CRAN (R 3.5.2)                 
 desc          1.2.0      2018-05-01 [1] CRAN (R 3.5.0)                 
 devtools      2.0.1      2018-10-26 [1] CRAN (R 3.5.2)                 
 digest        0.6.18     2018-10-10 [1] CRAN (R 3.5.0)                 
 dplyr       * 0.7.8      2018-11-10 [1] CRAN (R 3.5.0)                 
 forcats     * 0.3.0      2018-02-19 [1] CRAN (R 3.5.0)                 
 fs            1.2.6      2018-08-23 [1] CRAN (R 3.5.0)                 
 generics      0.0.2      2018-11-29 [1] CRAN (R 3.5.0)                 
 ggplot2     * 3.1.0      2018-10-25 [1] CRAN (R 3.5.0)                 
 glue          1.3.0.9000 2019-01-27 [1] Github (tidyverse/glue@8188cea)
 gtable        0.2.0      2016-02-26 [1] CRAN (R 3.5.0)                 
 haven         2.0.0      2018-11-22 [1] CRAN (R 3.5.0)                 
 here          0.1        2017-05-28 [1] CRAN (R 3.5.0)                 
 hms           0.4.2      2018-03-10 [1] CRAN (R 3.5.0)                 
 htmltools     0.3.6      2017-04-28 [1] CRAN (R 3.5.0)                 
 httr          1.4.0      2018-12-11 [1] CRAN (R 3.5.0)                 
 jsonlite      1.6        2018-12-07 [1] CRAN (R 3.5.0)                 
 lattice       0.20-38    2018-11-04 [1] CRAN (R 3.5.0)                 
 lazyeval      0.2.1      2017-10-29 [1] CRAN (R 3.5.0)                 
 lubridate     1.7.4      2018-04-11 [1] CRAN (R 3.5.0)                 
 magrittr      1.5        2014-11-22 [1] CRAN (R 3.5.0)                 
 memoise       1.1.0      2017-04-21 [1] CRAN (R 3.5.0)                 
 modelr        0.1.2      2018-05-11 [1] CRAN (R 3.5.0)                 
 munsell       0.5.0      2018-06-12 [1] CRAN (R 3.5.0)                 
 nlme          3.1-137    2018-04-07 [2] CRAN (R 3.5.2)                 
 openssl       1.2.1      2019-01-17 [1] CRAN (R 3.5.2)                 
 pillar        1.3.1      2018-12-15 [1] CRAN (R 3.5.0)                 
 pkgbuild      1.0.2      2018-10-16 [1] CRAN (R 3.5.0)                 
 pkgconfig     2.0.2      2018-08-16 [1] CRAN (R 3.5.0)                 
 pkgload       1.0.2      2018-10-29 [1] CRAN (R 3.5.0)                 
 plyr          1.8.4      2016-06-08 [1] CRAN (R 3.5.0)                 
 prettyunits   1.0.2      2015-07-13 [1] CRAN (R 3.5.0)                 
 processx      3.2.1      2018-12-05 [1] CRAN (R 3.5.0)                 
 ps            1.3.0      2018-12-21 [1] CRAN (R 3.5.0)                 
 purrr       * 0.2.5      2018-05-29 [1] CRAN (R 3.5.0)                 
 R6            2.3.0      2018-10-04 [1] CRAN (R 3.5.0)                 
 Rcpp          1.0.0      2018-11-07 [1] CRAN (R 3.5.0)                 
 readr       * 1.3.1      2018-12-21 [1] CRAN (R 3.5.1)                 
 readxl        1.2.0      2018-12-19 [1] CRAN (R 3.5.0)                 
 remotes       2.0.2      2018-10-30 [1] CRAN (R 3.5.0)                 
 rlang         0.3.1      2019-01-08 [1] CRAN (R 3.5.1)                 
 rprojroot     1.3-2      2018-01-03 [1] CRAN (R 3.5.0)                 
 rstudioapi    0.9.0      2019-01-09 [1] CRAN (R 3.5.1)                 
 rvest         0.3.2      2016-06-17 [1] CRAN (R 3.5.0)                 
 scales        1.0.0      2018-08-09 [1] CRAN (R 3.5.0)                 
 sergeant    * 0.8.0      2019-01-27 [1] git2r (@814195c)               
 sessioninfo   1.1.1      2018-11-05 [1] CRAN (R 3.5.0)                 
 stevedore     0.9.1      2019-01-02 [1] CRAN (R 3.5.2)                 
 stringi       1.2.4      2018-07-20 [1] CRAN (R 3.5.0)                 
 stringr     * 1.3.1      2018-05-10 [1] CRAN (R 3.5.0)                 
 testthat      2.0.1      2018-10-13 [1] CRAN (R 3.5.0)                 
 tibble      * 2.0.1      2019-01-12 [1] CRAN (R 3.5.2)                 
 tidyr       * 0.8.2      2018-10-28 [1] CRAN (R 3.5.0)                 
 tidyselect    0.2.5      2018-10-11 [1] CRAN (R 3.5.0)                 
 tidyverse   * 1.2.1      2017-11-14 [1] CRAN (R 3.5.0)                 
 usethis       1.4.0      2018-08-14 [1] CRAN (R 3.5.0)                 
 withr         2.1.2      2018-03-15 [1] CRAN (R 3.5.0)                 
 xml2          1.2.0      2018-01-24 [1] CRAN (R 3.5.0)                 
 yaml          2.2.0      2018-07-25 [1] CRAN (R 3.5.0)                 

[1] /Users/espin/r.library
[2] /Library/Frameworks/R.framework/Versions/3.5/Resources/library

Thanks!