[R] Enable package to be made available on CRAN

[asfimport]

As of ARROW-1325, the project contains a minimal working R package that takes advantage of R's built-in support for calling C++ code and uses Rcpp for added support.

Though the exact public interface of the package hasn't been developed yet (so this issue may be running before we walk), one major feature in its development will be release to CRAN, the official package repository for the R language.

Completing this story would mean:

• Getting source code to state where  R CMD CHECK passes with 0 ERRORS, 0 WARNINGS, and 0 NOTES

• Setting up build process for source and precompiled binary packages (see  data.table and XGBoost as examples of packages that do this)

• Submission to CRAN and acceptance of the first release

   

  Distribution via CRAN would be a much more natural fit for most R users' workflows than the current build-from-source workflow, and comes with all the other benefits of package managers (e.g. version pegging, easy distribution of platform-specific binaries).

Reporter: James Lamb / @jameslamb
Assignee: Neal Richardson / @nealrichardson

Related issues:

• Arrow recipe removed from homebrew (Blocked)
• [C++] Fix build for 32-bit MSYS2 (relates to)
• [C++] Restore apache-arrow Homebrew recipe and define process for maintaining and updating for releases (depends upon)
• [R][CI] Build and test on Windows in Appveyor (depends upon)
• [R] Support for building package for Windows (depends upon)
• [R] Make sure winbuilder tests pass for package (depends upon)
• [Website] Blog post announcing R package release (is depended upon by)

PRs and other links:

• GitHub Pull Request #4908

_{Note: This issue was originally created as ARROW-3204. Please see the migration documentation for further details.}

[asfimport]

Wes McKinney / @wesm:
I agree that this software should be made available on CRAN. Because of GPL dependencies we (the Arrow PMC) may have to leave the actual distribution of binaries for this software to the R community to do as it wishes. We can develop the tools to assist with that here

[asfimport]

Hyukjin Kwon:
+100! I can't wait to use Arrow R API!

[asfimport]

Javier Luraschi / @javierluraschi:
A few folks working on R and Arrow got together today, we agree that the R Arrow package should be in CRAN as soon as possible. However, this is likely to take a few months since we have to first:

1. Have an appropriate build process for Windows which Jeroen is leading, see [rwinlib/arrow|[https://github.com/rwinlib/arrow].]

2. Proper appveyor automation. The UrsaLabs team is planning to hire a build engineer to help, in the meantime, Kirill might be able to help get this going to make sure Jeroen's work does not regress as Arrow 0.13 gets developed.

3. Have an appropriate build process for OS X.

4. The Arrow libraries need to get distributed in the official Debian repository to be allowed in CRAN. Wes mentioned that, Kouhei (hoping I got this right), might be able to help the project navigate this in the next few months.

   So, while we completely agree with you that is imperative that Arrow gets in CRAN, in the meantime, we could still make use of Arrow in OS X and Linux using the remotes package (or devtools) by installing the Arrow runtime first, followed by:

   remotes::install_github("apache/arrow", subdir = "r")

[asfimport]

Jeroen / @jeroen:
The Homebrew lead maintainer has dropped arrow from homebrew because it isn't building: Homebrew/homebrew-core@6cd7f61 . So this further complicates things for R bindings on MacOS.

I am guessing that the build broke because the 0.11 tarball has been removed from the apache website, even though 0.12 hasn't even been announced yet? https://www.apache.org/dist/arrow/

 

[asfimport]

Wes McKinney / @wesm:
AFAIK no one from this community has been maintaining the Homebrew recipe. So that will obviously have to change so that updating the Homebrew recipe is part of the release management process. No one did anything wrong here

I'm creating a sub-task about Homebrew and attaching it as a child of this issue

EDIT: opened ARROW-4300

[asfimport]

Wes McKinney / @wesm:
I'll keep attaching child issues to this issue that are required (or related, like setting up CI for Windows) as part of the testing and packaging process. If there is anything necessary that is not in the issue tracker already please create new issues

[asfimport]

Javier Luraschi / @javierluraschi:
I don't see an issue for distributing into Debian/Fedora so I'll open one and link here, will also request help from dev@arrow.apache.org.

[asfimport]

Wes McKinney / @wesm:
Let this be closed when the package is finally on CRAN; it can't really be resolved until after the next Apache release has taken place anyway

[asfimport]

Wes McKinney / @wesm:
Issue resolved by pull request 4908
#4908