This post has been updated:
- 30. March 2019: changed library path to
3.7 - 28. August 2018: added more required software
- 1. March 2018: Removed `bash ./tools/rsync-recommended` and `bash ./tools/link-recommended`. They should not be needed anymore.
- 15. November 2017: Removed installing of recommended packages from build script. Those packages are now simply installed in the local library path which is also added to the `LIB_SITE`. This also allows that checked packages depend on recommended packages and those will also be updated with
update.packages(). - 1. July 2016: Updates thanks to comment of Frederik Eaton
- 2. April 2016: Included downloading and building recommended package (I couldn’t check packages requiring recommended packages without that).
- 29. March 2016: Updated
r-devel.shscript as it stopped working (Thanks to Matt Flor for noticing that).
Before submitting an R package to CRAN the CRAN policies require that a package runs R CMD check --as-cran without any warnings or notes. More specifically, the policies demand that “[t]his should be done with the current version of R-devel”. Previously I have done this on Windows, but even with the latest R-devel there are always issues that just do not pop up there (e.g., checking of URLs). Consequently, I have recently (finally) switched to Linux. But as a total Linux noob setting up R-devel was not trivial. This post shows how to set up R-devel next to the latest regular R and how to use it to test packages.
To install R-devel you first need to make sure you are able to install R, R-packages, and have a working LaTeX installation (e.g., texlive). Please follow the instructions here: https://cran.rstudio.com/bin/linux/ubuntu/README
Alternatively, simply run:
sudo apt-get install texlive-base sudo apt-get install texinfo sudo apt-get install texlive-latex-base sudo apt-get install texlive-latex-extra sudo apt-get install texlive-fonts-extra sudo apt-get install texlive-fonts-recommended
Then you will also need subversion (for getting the latest source), I also needed to install xorg (in case you get some unexpected errors it may be necessary to also install the suggestions given here), and some more software listed below:
sudo apt-get install subversion ccache sudo apt-get install xorg-dev sudo apt-get install libcurl4-openssl-dev
Now you need to get a current version of the source code via svn (i.e., subversion). For this, first create a folder svn in your home directory, change to this directory, and then checkout the latest version of the source code.
cd ~ mkdir svn cd svn svn co https://svn.r-project.org/R/trunk r-devel/R
You also need to download and link the recommended packages for using them in R CMD check as described in the R installation manual (this needs to happen in the correct folder).
cd r-devel/R/
Following a comment from Martin Maechler we do not build R in the svn directory, but rather in a newly created build directory:
mkdir ~/svn/R-devel-build
Next you need to build R-devel, preferably in a different location then your usual R installation. We do this with a script courtesy of Dirk Eddelbuettel (which I changed so it uses the correct path). Simply create a new file build-R-devel with the following content. It makes a lot of sense to put this file in a location that is in the $PATH, I recommend /usr/local/bin (which requires root/sudo). At the end of this, we automatically run make install which we also add to the script (which is in contrast to the recommendations of Martin Macheler from above, but it works just fine).
## ~/bin/build-R-devel #!/bin/sh cd ~/svn/R-devel-build # R_PAPERSIZE=letter \ # R_BATCHSAVE="--no-save --no-restore" \ # R_BROWSER=xdg-open \ # PAGER=/usr/bin/pager \ # PERL=/usr/bin/perl \ # R_UNZIPCMD=/usr/bin/unzip \ # R_ZIPCMD=/usr/bin/zip \ # R_PRINTCMD=/usr/bin/lpr \ # LIBnn=lib \ # AWK=/usr/bin/awk \ # CC="ccache gcc" \ # CFLAGS="-ggdb -pipe -std=gnu99 -Wall -pedantic -DTESTING_WRITE_BARRIER" \ # CXX="ccache g++" \ # CXXFLAGS="-ggdb -std=c++0x -pipe -Wall -pedantic" \ # FC="ccache gfortran" \ # FCFLAGS="-ggdb -pipe -Wall -pedantic" \ # F77="ccache gfortran" \ # FFLAGS="-ggdb -pipe -Wall -pedantic" \ # MAKE="make -j4" \ # ./configure \ # --prefix=/usr/local/lib/R-devel \ # --enable-R-shlib \ # --enable-strict-barrier \ # --with-blas \ # --with-lapack \ # --with-readline \ # --without-recommended-packages R_PAPERSIZE=letter \ R_BATCHSAVE="--no-save --no-restore" \ R_BROWSER=xdg-open \ PAGER=/usr/bin/pager \ PERL=/usr/bin/perl \ R_UNZIPCMD=/usr/bin/unzip \ R_ZIPCMD=/usr/bin/zip \ R_PRINTCMD=/usr/bin/lpr \ LIBnn=lib \ AWK=/usr/bin/awk \ CC="ccache gcc" \ CFLAGS="-ggdb -pipe -std=gnu99 -Wall -pedantic" \ CXX="ccache g++" \ CXXFLAGS="-ggdb -pipe -Wall -pedantic" \ FC="ccache gfortran" \ F77="ccache gfortran" \ MAKE="make -j4" \ ../r-devel/R/configure \ --prefix=/usr/local/lib/R-devel \ --enable-R-shlib \ --with-blas \ --with-lapack \ --with-readline \ --without-recommended-packages #CC="clang -O3" \ #CXX="clang++ -03" \ #make svnonly make echo "*** Done -- now run 'make install'" make install echo "*** All Done -- start R-devel with 'bash R-devel'"
Next, you need to install R-devel using this script which automatically changes to the correct directory. Note that this step requires sudo.
sudo bash build-R-devel
Now we only need a custom run script to actually run R-devel. Again this script is based on one written by Dirk Eddelbuettel and it should also be in a folder which is in the path such as /usr/local/bin. Note that we need to export the local library path for this installation as well. Below this is ~/R/x86_64-pc-linux-gnu-library/3.7, but this might need updating depending on your setup.
## ~/bin/R-devel.sh
#!/bin/bash
export R_LIBS_SITE=${R_LIBS_SITE-'/usr/local/lib/R-devel/lib/R/library:~/R/x86_64-pc-linux-gnu-library/3.7'}
export PATH="/usr/local/lib/R-devel/bin:$PATH"
R "$@"
To finally run R-devel simply call it as a script: bash R-devel
It should show you that it is the current development version: R Under development (unstable) (CURRENT DATE)
As this is a vanilla version of R, you will have to install all the packages you need.
The R-devel script can also be used to build or check your package:
bash R-devel CMD build your-package-directory bash R-devel CMD check --as-cran your-package.tar.gz
Once all this is set up, the steps of updating your R-devel installation to the latest version only involves calling the scripts in the right order:
cd ~/svn/r-devel/R svn update sudo bash build-R-devel
After doing so, it is probably a good idea to update the packages:
bash R-devel update.packages(ask=FALSE, checkBuilt = TRUE)
I have used the following sources to compile this post:
- https://stat.ethz.ch/pipermail/r-sig-debian/2012-August/001937.html
- http://sites.psu.edu/theubunturblog/2012/08/09/installing-the-development-version-of-r-on-ubuntu-alongside-the-current-version-of-r/
- http://stackoverflow.com/q/17473547/289572
- http://serverfault.com/q/139451
- https://stat.ethz.ch/pipermail/r-devel/2016-May/072777.html
Thanks for that fantastic guide! That went very smoothly.
However, I encountered a weird error when installing packages. Some went fine, others threw a
object 'checkCompilerOptions' not founderror, like here: http://stackoverflow.com/questions/35503633/r-devel-object-checkcompileroptions-not-found-when-installing-magrittrSo, to anyone also seeing this: I finally managed to solve the issue after reading https://github.com/rocker-org/drd/issues/3
All I had to do was removing the
/usr/lib/R/librarypath inR-devel.sh:Matt, thanks for your comment. I also needed to make this change to get it to work now. I have updated the guide accordingly.
Thanks for this manual! It all worked pretty well
Interesting post, thank you!
you configure –without-recommended-packages, but you first do ./tools/rsync-recommended, why?
you configure with –prefix=/usr/local/lib/R-devel, but the script to run R points to /usr/lib/R-devel, why?
Martin Maechler suggested using a “build directory”, see https://stat.ethz.ch/pipermail/r-devel/2016-May/072777.html
Hi Frederik,
Thanks a lot for your comments. I have finally found the time to update the guide accordingly. I also have removed the step to call
make installseparately and added it to the script.Thanks and let me know if you see other problems.
Henrik
Hi Henrik,
the rsync stage does not wor on my ubuntu 16.04 distribution. I did the command
bash ./tools/rsync-recommended
There is the shell output:
rsync: failed to connect to cran.r-project.org (137.208.57.37): No route to host (113)
rsync error: error in socket IO (code 10) at clientserver.c(128) [Receiver=3.1.1]
*** rsync failed to update Recommended files ***
The error message suggests it is actually a problem with your internet connection. Perhaps it was temporary down or the SVN server unavailable. It still works on my machine.
On the r-help ML, I was told there were issues with the Vienna servers. However, I still got the same output. Can it be a firewall? I am connecting from my university.
Hi Henrik,
Thank you so much for this tutorial. Here are some issues that I got and how I solved them. It might not be the optimal solution but it worked. I am on ubuntu 14.04 LTS.
I) Problem of ‘connection time out’ when doing bash ./tools/rsync-recommended
I have executed these commands instead:
bash -x ./tools/rsync-recommended
I took the the version info on the ‘rsync’ line which should output at the end.
I did: wget -r -l1 –no-parent -A*.gz -nd -P src/library/Recommended https://CRAN.R-project.org/src/contrib/3.5.0/Recommended/
II) sudo bash build-R-devel: got error ‘/usr/bin/install: cannot stat ‘NEWS.pdf’: No such file or directory’
going to ~/svn/R-devel-build and typing ‘sudo make’
got error message: ‘pdflatex’ is needed to make vignettes but is missing on your system.’
do: ‘sudo apt-get install texlive-latex-base’
do: ‘sudo apt-get install texlive-latex-extra’
do: ‘sudo apt-get install texlive-fonts-extra’
do: ‘sudo apt-get build-dep r-base’
do: sudo make
do: sudo make install
bash R-devel worked.
Hope this can help.
(PS: Did you use an online solution to build your website? thanks)
Hi Nicolas,
Thanks for your comment. Regarding (I), I haven’t yet encountered this. Good to know there is a solution. Regarding (II), I should probably add
texliveto the requirements. Thanks for catching that.My website is a simple wordpress website on a commercial server running the pitch premium theme from Site Origin (which is not available any more).
Thanks so much for this post! Works smoothly up to when I run lines 51-58. I get “configure error: in ‘/root/svn/R-devel-build’: configure: error: C compiler cannot create executables’. Do you have any recommendations? I am new to Linux (apparently, my package is not running with Debian) and fairly lost…
Ah I also had to run ‘sudo apt-get build-dep r-base’ – seems to work now 🙂
Hi Henrik,
I tried to update the R-devel and I am running through several errors. I do not manage to understand what to look for. I think that the version of R that is loaded is not the correct one but I am not sure. Below are the commands with errors. any idea?
> sudo bash build-R-devel[..........]
*** No rule to make target `../../../r-devel/R/src/nmath/expm1.c', needed by `Makefile'. Stop.
> bash R-devel
Error: package or namespace load failed for ‘methods’:
package ‘methods’ was installed by an R version with different internals; it needs to be reinstalled for use with this R version
... "and other similar messages"
Hi Nicolas,
The error is that the recommended packages you have installed were compiled with a different version of R-devel (e.g., 3.4 when it is now 3.5). The code now suggests to not install them anymore directly with R. I am not really sure how to solve this elegantly. But I think simply reinstalling R-devel (make sure to use the up-to-date code) should work.
Make sure to delete the version of R-devel you are having from disk before reinstalling!
it worked. thank you so much!
Hi, thank you very much for this awesome guide. It helps a lot.
I have a very quick question: you write ” Note that we need to export the local library path for this installation as well. Below this is ~/R/x86_64-pc-linux-gnu-library/3.6″ and the script executes “export R_LIBS_SITE=${R_LIBS_SITE-‘/usr/local/lib/R-devel/lib/R/library:~/R/x86_64-pc-linux-gnu-library/3.6’}”.
Could you please explain what this does exactly and why we do it? As I understood the packages installed for R and R-devel are separate anyhow.
Thank you very much!
I am sure there is a better way for this, but we need to tell
R-develwhich library path to use. And here I simply use the one for the upcoming R version (whichR-develwill turn into, which currently is 3.6). I think if I did not do it, both installed R versions used the same library. And in case, the built for them was different that led to problems. Without this, I definitely had some problems at some point. But if you have a better solution, I would be happy to hear about it. I am also not an expert on these matters.Hey, thanks for this clear and very helpful guide.
Everything ran as expected, just when I finally started R-devel and ran .libPaths(), I get
> .libPaths()
[1] "/home/steffi/R/x86_64-pc-linux-gnu-library/3.5"
[2] "/home/steffi/R/x86_64-pc-linux-gnu-library/3.6"
[3] "/usr/local/lib/R-devel/lib/R/library"
So it will still look for packages in the library for the release version.
Any idea why this is still there despite defining the paths in R-devel.sh ?
Thanks!
Unfortunately, I cannot replicate that. Under the “normal”
RI get:.libPaths()[1] "/home/henrik/R/x86_64-pc-linux-gnu-library/3.5"
[2] "/usr/local/lib/R/site-library"
[3] "/usr/lib/R/site-library"
[4] "/usr/lib/R/library"
Under
R-develI get:> .libPaths()[1] "/home/henrik/R/x86_64-pc-linux-gnu-library/3.6"
[2] "/usr/local/lib/R-devel/lib/R/library"
So I guess something needs to be different in your configuration compared to mine.
And sorry for the slow reply. I only use my Linux for development, so needed some days to boot into it again.
One possibility for the difference is that the current development version is now
3.7(as3.6will be released in the next few weeks). I just noticed this as well and changed the library path inR-devel.shaccordingly.