What is Shiny

Shiny is a web application framework for R. More specifically it is an R package from RStudio that makes it easy to build interactive web applications with R. With only a few lines of code a user interface can be built.

Shiny Server

Shiny Server is a server program that makes Shiny applications available over the web.

System

  • Sever: canopus
  • CentOS: 6.6
  • Python: 2.6.6
  • GCC/G++: 4.4.7
  • git: 1.7.1
  • root: not available
  • gfortran: not available
  • cmake: not available

This is the setup I used when setting up the Shiny Server on Uberspace. Keep that in mind, as results may vary with other setups!


1. Install R


Preparations

Building R from source requires a Fortran compiler. Unfortunately, a Fortran compiler is not available on Uberspace, so we have to install it ourselves:

# create a temporary installation folder
mkdir /home/${USER}/install
cd /home/${USER}/install 

# use yumdownloader to download a fortran compile that is required
# to build R from source
yumdownloader gcc-gfortran libgfortran.x86_64 

# create a new folder and enter directory
mkdir -p /home/${USER}/fortran
cd /home/${USER}/fortran

# convert rpm to cpio and extract files
rpm2cpio ../install/gcc-gfortran-4.4.6-4.el6.x86_64.rpm | cpio -id
rpm2cpio ../install/libgfortran-4.4.6-4.el6.x86_64.rpm | cpio -id

In early 2015 the R-Project decided to remove, amongst others, the zlib, bzip2, and pcre libraries so that system versions are now required. We will install or update these libraries first by using toast, a neat little utility available on Uberspace that automatically installs and links the required software.

# move to installation folder
cd /home/${USER}/install

# zlib' library
wget http://zlib.net/zlib-1.2.8.tar.gz
toast arm zlib-1.2.8.tar.gz

# 'bzip2' library
wget http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz
toast arm bzip2-1.0.6.tar.gz
export CFLAGS="-I$HOME/.toast/armed/include -L$HOME/.toast/armed/lib"

# 'xz' library
wget http://tukaani.org/xz/xz-5.2.1.tar.gz
toast arm xz-5.2.1.tar.gz

# 'curl' library
wget http://curl.haxx.se/download/curl-7.43.0.tar.gz
toast arm curl-7.43.0.tar.gz

We will further need the PCRE library, but this one is required to be configured with UTF support. Hence, we need to compile this manually.

# download any version of PCRE between >= 8.10 and < 10.0
wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.36.tar.gz
tar xf pcre-8.36.tar.gz
cd pcre-8.36
export C_INCLUDE_PATH=${C_INCLUDE_PATH}:/home/${USER}/pcre/include

# create a destination folder and make install
mkdir /home/${USER}/pcre
./configure --prefix=/home/${USER}/pcre/ --enable-utf8
make
make install

Last but not least, we also need a Java interpreter. Unfortunately, we are not able to download the file via shell as Oracle wants us to accept the License Agreement first. Go here, download the latest 64-bit Linux source and finally move this file to your Uberspace via any SFTP manager.

# we will again use 'toast' to install java
toast build <jdk.tar.gz>
toast arm jdk
export JAVA_HOME=/home/${USER}/.toast/armed/


Build R

Due to the lack of root access we have to install R the “complicated” way by building it from source. Since we also have to compile several R packages we will need the development version of R, which is available CRAN. The development version contains additional C header files that are required for compiling some R packages (e.g., Rcpp or httr). When compiling R we have to make sure that it builds the shared/dynamic library by setting --enable-R-shlib="yes".

# create new folder for R
mkdir -p /home/${USER}/R
cd /home/${USER}/install

# download the latest development build of R (R-devel)
wget ftp://ftp.stat.math.ethz.ch/Software/R/R-devel_2015-08-10.tar.gz

# extract all files and enter the new directory
tar xf R-devel_2015-08-10.tar.gz
cd R-devel

# we need to tell gcc and g++ about the include/ folder of R-devel
# as some R packages require those header files when build from source
export C_INCLUDE_PATH=/home/${USER}/install/R-devel/include
export CPLUS_INCLUDE_PATH=/home/${USER}/install/R-devel/include

# check whether the include folder was successfully added
`gcc -print-prog-name=cc1` -v
`g++ -print-prog-name=cc1` -v

# configure, make, make install
# note that configure requires to --enable-R-shlib flag to 
# build the shared/dynamic library ('no' by default)
# the --prefix flag controls where R is installed and mandatory
# for installing R on Uberspace
export PATH=${PATH}:/home/${USER}/fortran/usr/bin/
export LDFLAGS=-L/home/${USER}/fortran/usr/lib64
./configure --prefix=/home/${USER}/R --with-x="no" --enable-R-shlib="yes"
make
make install

Once this is done (might take a while) you should be able to run R.

cd /home/${USER}/R/bin
. R

It is probably a good idea to add R to $PATH so it is globally accessible. Open your .bash_profile in /home/${USER}/ and add the following line PATH=$PATH:$HOME/R/bin.

Finally, we need to check whether we are able to plot data in R to a PNG file. Since we run on a headless server this might fail as R wants to access the X11 device.

# start R from shell
R

# in R we will open a file connection to see if it works
png("test.png")

# if R throws the error "Error in png("test.png") : X11 is not available" then we need to run the following
options(bitmapType='cairo')

# we should now be able to try this again
png("test.png")
plot(1)
dev.off()


2. Install shiny package in R

Before we install the Shiny Server we will install the shiny package first. Since we want the latest development build of shiny, we use the devtools package to install the package from GitHub.

# start R from Linux shell
R

# in R we install the devtools package first
install.packages("devtools")

# with devtool we then install the latest development build of shiny from GitHub
devtools::install_github("rstudio/shiny")

R should install all necessary dependencies, of which there are a lot if shiny is installed for the first time. If some packages fail to be build from source and warning that ‘R.h’, ‘Rinternals.h’ or other C header files are not available, make sure that gcc and g++ know about the (correct) include paths (see step 1). You may need to add the include path for every new session.


3. Install Shiny-Server

As is the case for R, Shiny-Server also requires root access. However, by building from source and doing some modifications on the code before compiling we can circumvent the need for root. Building Shiny Server from source requires python 2.6/2.7, gcc, g++, git, R-devel and cmake. Fortunately, most of these prerequisites are already available on Uberspace. R-devel was already installed in step 1. We are only missing cmake, so we will install that first:

# building shiny-server from source requires cmake, which is not available on uberspace
# therefore we have to compile cmake ourselves
mkdir ~/.bin && cd ~/.bin
curl -O http://www.cmake.org/files/v3.0/cmake-3.0.1.tar.gz
tar -zxvf cmake-3.0.1.tar.gz && rm cmake-3.0.1.tar.gz && cd cmake-3.0.1
./bootstrap && make

# add cmake to PATH
export PATH=$HOME/.bin/cmake-3.0.1/bin:$PATH

In what follows is basically the installation guide by RStudio for building Shiny Server from source (click here). In contrast to the original guide we will, however, tell cmake to use another -DCMAKE_INSTALL_PREFIX with a folder we actually have access to (instead of /usr/local/).

# Clone the repository from GitHub
git clone https://github.com/rstudio/shiny-server.git

# Get into a temporary directory in which we'll build the project
cd shiny-server
mkdir tmp
cd tmp

# Add the bin directory to the path so we can reference node
DIR=`pwd`
PATH=$PATH:$DIR/../bin/

# See the "Python" section below if your default python version is not 2.6 or 2.7. 
PYTHON=`which python`

# Check the version of Python. If it's not 2.6.x or 2.7.x, see the Python section below.
$PYTHON --version

# Use cmake to prepare the make step. Modify the "--DCMAKE_INSTALL_PREFIX"
# if you wish the install the software at a different location.
cmake -DCMAKE_INSTALL_PREFIX=/home/${USER}/shiny-server-local -DPYTHON="$PYTHON" ../
# Get an error here? Check the "How do I set the cmake Python version?" question below

Before compiling, we have to do an important change the source of Shiny Server first. By default, Shiny Server will look for its configuration file in /etc/shiny-server. But due to the missing root access it cannot write the config file to that folder. We change the lookup path, which is hardcoded in the Main.js file, to a directory in our home folder:

# location of the shiny-server.config is hardcoded and needs to be changed
# before compiling
cd
cd shiny-server/lib/
nano main.js

# in main.js look for the following line:
var configFilePath = '/etc/shiny-server/shiny-server.conf';
# and change to
var configFilePath = '/home/YOUR_YOUSERNAME/shiny-server/shiny-server.conf';

We can now follow RStudios official guide again and build Shiny Server from source:

# Recompile the npm modules included in the project
make
mkdir ../build
(cd .. && bin/npm --python="$PYTHON" rebuild)
# Need to rebuild our gyp bindings since 'npm rebuild' won't run gyp for us.
(cd .. && ext/node/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js 
  --python="$PYTHON" rebuild)

# Install the software at the predefined location
make install

Since we changed the lookup directory for the server settings file, we need to create such a file in the specified folder. Fortunately, Shiny Server comes with a bunch of different config files which we can copy to our specified folder. The expected file name is shiny-server.conf, so we also rename the file. Make sure to edit the config file with a text editor and change the port to a port that is actually reachable from outside. Note that you will have to write an E-Mail to hallo@uberspace.de first and make sure to read this first: https://wiki.uberspace.de/system:ports. In the config file we also have to change the default directories where Shiny Apps and the log files are expected.

# we need to copy a default server-config file to the directory we specified 
# manually in the main.js file. A predefined can be found in ~/shiny-server/
cd
cd shiny-server/config/
cp default.config $HOME/shiny-server

# go to shiny-server root directory and rename file
cd ..
mv default.config shiny-server.conf

# in shiny-server.conf we need to change the default port and the paths 
# to the app and log directories
mkdir apps
mkdir log
nano shiny-server.conf


4. Shiny-Server daemon

When everything went fine, you should be able to start Shiny server by just entering shiny-server in the console. But then the server will only be up and available as long as your SSH session lasts. Of course, we want the server to be available ‘24/7’, so we need a daemon that keeps the process alive. Shiny Server comes with prebuild upstart, init.d and systemd scripts, but which are not of use for us. Instead, we make use of daemontools that is already available on Uberspace by default. Before entering the following code please make sure to read Uberspaces’ wiki first: https://wiki.uberspace.de/system:daemontools.

# creat person ~/service folder (needs to be run only once per account)
test -d ~/service || uberspace-setup-svscan

# make a folder for daemon scripts
mkdir daemon
cd daemon

# create a shell script with nano.
# the file should contain the following lines:
# #!/bin/bash
# exec /home/YOUR_USERNAME/shiny-server/bin/shiny-server 2>&1
nano
# quit nano and save file as 'run'

# chmod the new file
chmod +x $HOME/daemon/run

# create a Symlink to the ~/service folder. The daemon will start almost
# immediately and hence your server
ln -s $HOME/daemon ~/service

Congratulations, your server should now be up-and-running! You can access your Shiny Server by entering your Uberspace domain followed by the port specified in the shiny-server.conf file. In my case, this is http://canopus.uberspace.de:64230.