Installation of Frome

Granting davidf rights to manage software and services:

(root)$ visudo

and adding:


Preparing for source install:

(root)$ cd /usr/local/src/
(root)$ mkdir hisparc
(root)$ chown davidf.hisparc hisparc/
$ chmod g+w hisparc/

In /etc/ new file usrlocal.conf, to let ldconfig find libraries of locally installed software:




$ cd /usr/local/src/hisparc
$ wget
$ tar xvzf Python-2.6.4.tgz
$ cd Python-2.6.4
$ ./configure --enable-shared
$ make
(root)$ make install

Then, run:

(root)$ ldconfig

Now, the python libraries are registered.

Python Setuptools

From egg:

$ cd /usr/local/src/hisparc
$ wget
(root)$ sh setuptools-0.6c11-py2.6.egg

IPython, an interactive Python shell

Download and install IPython:

(root)$ easy_install ipython

Web server

Install apache development libraries:

$ sudo yum install httpd-devel

 Package             Arch        Version                 Repository        Size
 httpd-devel         i386        2.2.3-31.sl5.2          sl-security      147 k
 httpd-devel         x86_64      2.2.3-31.sl5.2          sl-security      147 k
Installing for dependencies:
 apr                 x86_64      1.2.7-11.el5_3.1        sl-security      118 k
 apr-devel           x86_64      1.2.7-11.el5_3.1        sl-security      237 k
 apr-util            x86_64      1.2.7-7.el5_3.2         sl-security       74 k
 apr-util-devel      x86_64      1.2.7-7.el5_3.2         sl-security       53 k
 httpd               x86_64      2.2.3-31.sl5.2          sl-security      1.2 M

Change configuration in /etc/httpd/conf/httpd.conf. Patch:

--- httpd.conf.orig     2009-12-04 14:35:39.000000000 +0100
+++ httpd.conf  2009-12-04 14:35:50.000000000 +0100
@@ -228,8 +228,8 @@
 #  when the value of (unsigned)Group is above 60000;
 #  don't use Group #-1 on these systems!
-User apache
-Group apache
+User www
+Group www

 ### Section 2: 'Main' server configuration

Enabling httpd on startup:

$ sudo /sbin/chkconfig --add httpd
$ sudo /sbin/chkconfig --levels 35 httpd on

Starting httpd now:

$ sudo /sbin/service httpd start

For mod_wsgi:

$ cd /usr/local/src/hisparc
$ wget
$ tar xvzf mod_wsgi-3.1.tar.gz
$ cd mod_wsgi-3.1
$ ./configure
$ make
(root)$ make install

Change configuration in /etc/httpd/conf/httpd.conf. Patch:

--- httpd.conf.orig     2009-12-04 15:19:01.000000000 +0100
+++ httpd.conf  2009-12-04 15:34:30.000000000 +0100
@@ -197,6 +197,7 @@
 LoadModule mem_cache_module modules/
 LoadModule cgi_module modules/
 LoadModule version_module modules/
+LoadModule wsgi_module modules/

 # The following modules are not loaded by default:

Restarting apache:

$ sudo /sbin/service httpd restart

Version control

Install git from source:

$ cd /usr/local/src/hisparc
$ wget
$ tar xvzf git-
$ cd git-
    $ make prefix=/usr/local all
    (root)$ sudo make prefix=/usr/local install

Datastore web application

The datastore application is driving our central data storage solution. It is a pure Python implementation under complete version control.


The datastore application uses PyTables and the underlying HDF5 library to store binary data files. PyTables depends heavily on NumPy.:

(root)$ easy_install numpy

Now install the HDF5 library:

$ cd /usr/local/src/hisparc
$ wget
$ tar xvzf hdf5-1.8.3.tar.gz
$ cd hdf5-1.8.3
$ ./configure --prefix=/usr/local
$ make
(root)$ make install
(root)$ ldconfig

And, finally, install PyTables itself:

(root)$ easy_install tables

Setting up datastore

In summary:

  • Created a /var/www/wsgi-bin directory from which to run the wsgi applications
  • Created a subdirectory owned by davidf.hisparc inside this wsgi-bin
  • Did a checkout of the datastore sources inside the subdirectory
  • Made a local copy of the application into the parent (wsgi-bin) and edited to set the correct local full path
  • Added the wsgi application to the Apache configuration

Here we go:

(root)$ cd /var/www
(root)$ mkdir wsgi-bin
(root)$ cd wsgi-bin
(root)$ mkdir datastore
(root)$ chown davidf.hisparc datastore
(root)$ chmod g+w datastore
$ git clone /var/www/wsgi-bin/datastore

Copy the application.wsgi and config.ini from the examples directory:

(root)$ cd /var/www/wsgi-bin
(root)$ cp datastore/examples/application.wsgi datastore.wsgi
(root)$ cp datastore/examples/config.ini datastore/
(root)$ chown davidf.hisparc datastore.wsgi datastore/config.ini
(root)$ chmod g+w datastore.wsgi datastore/config.ini

Edited /var/www/wsgi-bin/datastore.wsgi and set the correct paths:

configfile = ('/var/www/wsgi-bin/datastore/config.ini')

The config.ini now reads:



I had to create the appropriate directory in /var/log and grant rights:

(root)$ cd /var/log
(root)$ mkdir hisparc
(root)$ chown www.hisparc hisparc
(root)$ chmod g+w hisparc

Then, added datastore to the Apache configuration:

(root)$ cd /etc/httpd/conf.d/
(root)$ touch hisparc.conf
(root)$ chown davidf.hisparc hisparc.conf
(root)$ chmod g+w hisparc.conf

And edited hisparc.conf to contain:

WSGIScriptAlias /hisparc/upload /var/www/wsgi-bin/datastore.wsgi

Reload Apache configuration:

$ sudo /sbin/service httpd reload


Write a wrapper for the writer:

$ vim /var/www/wsgi-bin/datastore/
"""Wrapper for the writer application"""

import sys


import writer

configfile = ('/var/www/wsgi-bin/datastore/config.ini')

Start the writer app

(root)$ screen
(root)$ sudo -u www python /var/www/wsgi-bin/datastore/

This process will process incoming data and write them into the datastore


Run the script to receive configuration updates from the Public Database Based on the file: publicdb/scripts/

$ runuser -l hisparc -c 'hisparc-datastore'

(Maybe) Not relevant

install: yum-utils easy_install paramiko easy_install dozer easy_install pil (requirement of dozer) easy_install mysql-python (for migration) install: gcc-gfortran easy_install virtualenvwrapper install: blas-devel lapack-devel (for scipy)