Compiling packages reporting missing headers in windows

For anyone else who hits this and doesn’t want to wait for someone to put the compiled package into CRAN

Chris Evans https://www.psyctc.org/R_blog/ (PSYCTC.org)https://www.psyctc.org/psyctc/
2021-11-12
Show code
library(ggplot2)
library(tidyverse)
as_tibble(list(x = 1,
               y = 1)) -> tibDat

ggplot(data = tibDat,
       aes(x = x, y = y)) +
  geom_text(label = "Rtools, Bash and pacman:\npackages for packages!",
            size = 12,
            colour = "red",
            angle = 30) +
  xlab("") +
  ylab("") +
  theme_bw() +
  theme(panel.grid.major = element_blank(),
        panel.grid.minor = element_blank(),
        panel.border = element_blank(),
        panel.background = element_blank(),
        axis.line = element_blank(),
        axis.text = element_blank(),
        axis.ticks = element_blank()) 

This really is pretty geeky and may be as much for my benefit when, as I suspect I will, I hit it in the future and have forgotten these details.

Geeky background

In case you don’t know, most software is written in “source code” and then compiled to create an executable: the program itself that can be run within an operating system if it compiles correctly. (Actually, some code is “interpreted” and run directly, line by line; R and most forms of BASIC on which many people learned to program a few decades ago are such interpreted languages.)

R packages for Windoze exist in two forms: executable/binary packages which run directly within R if installed with

install.packages("packagename"))

and source packages which are source code, perhaps including source code in multiple other compiled languages, most often C, C++ or FORTRAN. Source code packages need to be compiled to executable/binary packages to run within R.

The issue

Recently I found that a number of R packages were reporting that there were upgrades available but only available as source packages, i.e. requiring me to compile them on my own machine if I wanted the latest upgrade. That happens regularly for me but I think that in the default R setup in Windoze R won’t waste your time telling you about the upgrade being available because the default R setup assumes you’re not going to want to do that and doesn’t install all the extra software necessary to compile source packages.

To compile R packages from source in Windoze you need that extra software which is provided by the Rtools toolset. See https://cran.r-project.org/bin/windows/Rtools/. Rtools it’s not an R package, it’s a collection of programs that run under Windoze (and a lot of other support materials). This means that you have install it into Windoze like any other Windoze program, that confused me some years ago. As well as installing Rtools, for it to work from R to compile source packages you must also add the location of Rtools to the Windoze path so R knows where to find the tools: https://github.com/r-windows/docs/blob/master/rtools40.md explains this. (In my VM Windoze I found that I had to put the location of Rtools in the system path not the user path for the Rtools shell to find Rtools, we’ll come to that below.)

Rtools, as the name suggests, gives you all the tools necessary to compile many packages. All the tools are open source so there is no charge for Rtools. Furthermore, it ensures that compiling R packages is entirely automated: usually all you have to do if you have installed Rtools is to say “yes” when asked if you want local compiling of an R package where the source package is more up to date than the compiled version. Then R crunches through the compiling with a fascinating cascade of messages from the various stages and then you get the usual messages that installation has worked.

So if you have installed Rtools then if you use the menu option to update packages R will will give you the option to compile locally (i.e. on your own machine) if the source package is more up to date than the executable package. However, occasionally the compiling will fail even though you have Rtools installed. When that happens I find that usually I only have to wait a few days and the compiled package, or a new source package that has fixed whatever failed, appears on CRAN and your package updating works for that package again. Even more occasionally you wait some days and still the issue doesn’t go away and you start to wonder if there is something wrong with your system!

This happened for me with three packages: gsl, igraph and nloptr. This is where I discovered that sometimes you don’t just need Rtools to compile source packages locally but you may also need some packages for packages.

What was happening was that instead of the cascade of compilation messages (and the occasional warning) scooting past and ending up with a message that the package had been installed each was giving the message:

   **********************************************
   WARNING: this package has a configure script
         It probably needs manual configuration
   **********************************************

And things like this:

*** arch - i386
"C:/rtools40/mingw32/bin/"gcc  -I"C:/PROGRA~1/R/R-41~1.2/include" -DNDEBUG -I/include         -O2 -Wall  -std=gnu99 -mfpmath=sse -msse2 -mstackrealign  -c airy.c -o airy.o
airy.c:1:10: fatal error: gsl/gsl_sf_airy.h: No such file or directory
 #include <gsl/gsl_sf_airy.h>
          ^~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [C:/PROGRA~1/R/R-41~1.2/etc/i386/Makeconf:238: airy.o] Error 1
ERROR: compilation failed for package 'gsl'

I know enough about computers and compiling source code to know that message is telling me that the compiler couldn’t find a “header” file, in this case gsl_sf_airy.h. (After all, that’s what it says!!) However, searching the interweb for that didn’t come up with anything recent about anyone having this problem under windows (beware things on the interweb with problems and solutions more than a year old: too often they’ve been superceeded by subsequent developments).

I was also puzzled by all three giving the message:

   **********************************************
   WARNING: this package has a configure script
         It probably needs manual configuration
   **********************************************

Again, I wasn’t finding answers about this. After a while I decided that the fact I had been seeing this for a week or so and on two really rather different Windoze systems (one sitting directly on an old laptop, the other running in a VirtualBox virtual machine under Ubuntu) meant I ought to try harder to work out what was wrong. I took a punt and Emailed Jeroen Ooms the maintainer of Rtools. I got a fast response pointing me back into https://github.com/r-windows/docs/blob/master/rtools40.md, specifically to https://github.com/r-windows/docs/blob/master/rtools40.md#example-installing-a-library-with-pacman and this time I persevered trying to understand it and got there in the end. Hence I’m writing this for others who might, like me, not be sufficiently immersed in these things as people like Jeroen and struggle to understand not only the instructions but also that you have to be a bit lateral thinking and search a bit more if things aren’t quite as easy as the example he has there.

“SOLVED”, “howto”, solution!

Firstly, I think the “this package has a configure script” is a bit of red herring as I don’t think any of these packages or their supporting facilities actually have a configure script or need manual configuration. What they do need is installation of the necessary header (and no doubt other) files and actually that’s pretty easy and in Jeroen’s page on github. However, I needed to unpack it.

My unpacking of what I had to do

That’s it! Problem cracked. Huge thanks to Jeoem Ooms for a nearly instant response, for pushing me in the right direction but above all for his work on Rtools and the other open source projects he supports … and thanks to the package maintainers for the three packages and really everyone contributing to R.

Citation

For attribution, please cite this work as

Evans (2021, Nov. 12). Chris (Evans) R SAFAQ: Compiling packages reporting missing headers in windows. Retrieved from https://www.psyctc.org/R_blog/posts/2021-11-12-compiling-packages-reporting-missing-headers-in-windows/

BibTeX citation

@misc{evans2021compiling,
  author = {Evans, Chris},
  title = {Chris (Evans) R SAFAQ: Compiling packages reporting missing headers in windows},
  url = {https://www.psyctc.org/R_blog/posts/2021-11-12-compiling-packages-reporting-missing-headers-in-windows/},
  year = {2021}
}