Granting davidf rights to manage software and services:
(root)$ visudo
and adding:
davidf ALL = SOFTWARE, SERVICES
Preparing for source install:
(root)$ cd /usr/local/src/
(root)$ mkdir hisparc
(root)$ chown davidf.hisparc hisparc/
$ chmod g+w hisparc/
In /etc/ld.so.conf.d new file usrlocal.conf, to let ldconfig find libraries of locally installed software:
/usr/local/lib
Python:
$ cd /usr/local/src/hisparc
$ wget http://www.python.org/ftp/python/2.6.4/Python-2.6.4.tgz
$ tar xvzf Python-2.6.4.tgz
$ cd Python-2.6.4
$ ./configure --enable-shared
$ make
(root)$ make install
Then, run:
(root)$ ldconfig
Now, the python libraries are registered.
From egg:
$ cd /usr/local/src/hisparc
$ wget http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c11-py2.6.egg#md5=bfa92100bd772d5a213eedd356d64086
(root)$ sh setuptools-0.6c11-py2.6.egg
Install apache development libraries:
$ sudo yum install httpd-devel
================================================================================
Package Arch Version Repository Size
================================================================================
Installing:
httpd-devel i386 2.2.3-31.sl5.2 sl-security 147 k
httpd-devel x86_64 2.2.3-31.sl5.2 sl-security 147 k
Installing for dependencies:
apr x86_64 1.2.7-11.el5_3.1 sl-security 118 k
apr-devel x86_64 1.2.7-11.el5_3.1 sl-security 237 k
apr-util x86_64 1.2.7-7.el5_3.2 sl-security 74 k
apr-util-devel x86_64 1.2.7-7.el5_3.2 sl-security 53 k
httpd x86_64 2.2.3-31.sl5.2 sl-security 1.2 M
Change configuration in /etc/httpd/conf/httpd.conf. Patch:
--- httpd.conf.orig 2009-12-04 14:35:39.000000000 +0100
+++ httpd.conf 2009-12-04 14:35:50.000000000 +0100
@@ -228,8 +228,8 @@
# when the value of (unsigned)Group is above 60000;
# don't use Group #-1 on these systems!
#
-User apache
-Group apache
+User www
+Group www
### Section 2: 'Main' server configuration
#
Enabling httpd on startup:
$ sudo /sbin/chkconfig --add httpd
$ sudo /sbin/chkconfig --levels 35 httpd on
Starting httpd now:
$ sudo /sbin/service httpd start
For mod_wsgi:
$ cd /usr/local/src/hisparc
$ wget http://modwsgi.googlecode.com/files/mod_wsgi-3.1.tar.gz
$ tar xvzf mod_wsgi-3.1.tar.gz
$ cd mod_wsgi-3.1
$ ./configure
$ make
(root)$ make install
Change configuration in /etc/httpd/conf/httpd.conf. Patch:
--- httpd.conf.orig 2009-12-04 15:19:01.000000000 +0100
+++ httpd.conf 2009-12-04 15:34:30.000000000 +0100
@@ -197,6 +197,7 @@
LoadModule mem_cache_module modules/mod_mem_cache.so
LoadModule cgi_module modules/mod_cgi.so
LoadModule version_module modules/mod_version.so
+LoadModule wsgi_module modules/mod_wsgi.so
#
# The following modules are not loaded by default:
Restarting apache:
$ sudo /sbin/service httpd restart
Install git from source:
$ cd /usr/local/src/hisparc
$ wget https://git-core.googlecode.com/files/git-1.8.4.3.tar.gz
$ tar xvzf git-1.8.4.3.tar.gz
$ cd git-1.8.4.3.tar.gz
$ make prefix=/usr/local all
(root)$ sudo make prefix=/usr/local install
The datastore application is driving our central data storage solution. It is a pure Python implementation under complete version control.
The datastore application uses PyTables and the underlying HDF5 library to store binary data files. PyTables depends heavily on NumPy.:
(root)$ easy_install numpy
Now install the HDF5 library:
$ cd /usr/local/src/hisparc
$ wget http://www.hdfgroup.org/ftp/HDF5/prev-releases/hdf5-1.8.3/src/hdf5-1.8.3.tar.gz
$ tar xvzf hdf5-1.8.3.tar.gz
$ cd hdf5-1.8.3
$ ./configure --prefix=/usr/local
$ make
(root)$ make install
(root)$ ldconfig
And, finally, install PyTables itself:
(root)$ easy_install tables
In summary:
- Created a /var/www/wsgi-bin directory from which to run the wsgi applications
- Created a subdirectory owned by davidf.hisparc inside this wsgi-bin
- Did a checkout of the datastore sources inside the subdirectory
- Made a local copy of the application into the parent (wsgi-bin) and edited to set the correct local full path
- Added the wsgi application to the Apache configuration
Here we go:
(root)$ cd /var/www
(root)$ mkdir wsgi-bin
(root)$ cd wsgi-bin
(root)$ mkdir datastore
(root)$ chown davidf.hisparc datastore
(root)$ chmod g+w datastore
$ git clone https://github.com/HiSPARC/datastore.git /var/www/wsgi-bin/datastore
Copy the application.wsgi and config.ini from the examples directory:
(root)$ cd /var/www/wsgi-bin
(root)$ cp datastore/examples/application.wsgi datastore.wsgi
(root)$ cp datastore/examples/config.ini datastore/
(root)$ chown davidf.hisparc datastore.wsgi datastore/config.ini
(root)$ chmod g+w datastore.wsgi datastore/config.ini
Edited /var/www/wsgi-bin/datastore.wsgi and set the correct paths:
sys.path.append('/var/www/wsgi-bin/datastore/wsgi')
configfile = ('/var/www/wsgi-bin/datastore/config.ini')
The config.ini now reads:
[General]
log=/var/log/hisparc/hisparc.log
loglevel=debug
station_list=/databases/frome/station_list.csv
data_dir=/databases/frome
[Writer]
sleep=1
I had to create the appropriate directory in /var/log and grant rights:
(root)$ cd /var/log
(root)$ mkdir hisparc
(root)$ chown www.hisparc hisparc
(root)$ chmod g+w hisparc
Then, added datastore to the Apache configuration:
(root)$ cd /etc/httpd/conf.d/
(root)$ touch hisparc.conf
(root)$ chown davidf.hisparc hisparc.conf
(root)$ chmod g+w hisparc.conf
And edited hisparc.conf to contain:
WSGIScriptAlias /hisparc/upload /var/www/wsgi-bin/datastore.wsgi
Reload Apache configuration:
$ sudo /sbin/service httpd reload
Write a wrapper for the writer:
$ vim /var/www/wsgi-bin/datastore/writer_app.py
"""Wrapper for the writer application"""
import sys
sys.path.append('/var/www/wsgi-bin/datastore/writer')
import writer
configfile = ('/var/www/wsgi-bin/datastore/config.ini')
writer.writer(configfile)
Start the writer app
(root)$ screen
(root)$ sudo -u www python /var/www/wsgi-bin/datastore/writer_app.py
This process will process incoming data and write them into the datastore
Run the script to receive configuration updates from the Public Database Based on the file: publicdb/scripts/fake-datastore-xmlrpc-server.py
$ runuser -l hisparc -c 'hisparc-datastore'
install: yum-utils easy_install paramiko easy_install dozer easy_install pil (requirement of dozer) easy_install mysql-python (for migration) install: gcc-gfortran easy_install virtualenvwrapper install: blas-devel lapack-devel (for scipy)