wiki:clearinghouse

How to setup Sensibility Testbed Clearinghouse

This is an adaptation of the Seattle Clearinghouse setup.

Setting up a non-privileged account

First of all, on the server or VM you use for the clearinghouse, we recommend to set up an account specific to the Clearinghouse instance you are going to set up. This ensures all of the code, config files, etc. remain isolated from that of other services on the same machine.

The account should not be granted interactive login for security reasons. Use sudo -i -u theusername instead to work in the account directory. Needless to say, the user should not have root privileges or be able to acquire them.

Any user name will be fine. We will use ch in the instructions.

Install dependencies

Clearinghouse requires at least the following software to be installed (note that different from the ch account, the following installation requires root privileges):

  • Python in version 2.6, or 2.7 -- the language the Sensibility Testbed Clearinghouse is written in
  • Python MySQLDB -- the python mysql interface
  • MySQL -- the database
  • Apache 2.4 -- the web server
  • mod_wsgi -- necessary for interfacing with Django code
  • OpenSSL -- necessary for https:// support
  • Django in version 1.6.x -- necessary to run Django code
  • Django Social Auth -- for OpenID and Oauth support.

Most of these can be installed through a package manager. For example, on a Debian-based system:

$ sudo apt-get install apache2 libapache2-mod-wsgi
$ sudo apt-get install mysql-server mysql-client python-mysqldb
$ # MySQL will prompt you to set a root database password, which you should do.
$ sudo apt-get install ntp
$ sudo apt-get install openssl

Django, the web framework the clearinghouse uses, is available at https://www.djangoproject.com/download/ or through a package manager.

$ pip install django

Depending on your actual OS and setup, this command might require sudo privileges. After that, the most up-to-date django will be installed.

Note that even if you don't enable OpenID and OAuth, Clearinghouse requires the specific Django package installed:

$ easy_install django-social-auth

Optional: Setup OpenID and OAuth

If you would like your Clearinghouse to support login not only through user accounts it manages itself, but ID/authentication services like OpenID and OAuth, or web services like Google, Facebook, or GitHub, take a look at the social auth support instructions page.

Create MySQL databases

You need two mysql databases and seperate users with access to each.

  • First database name: clearinghouse
  • Second database name: keydb

For simplicity, make the database usernames correspond to the database names.

Here's an example of creating a database and a user:

$ mysql -u root -p
$ # This requires entering the database root password set during install!

mysql> create database clearinghouse;

mysql> GRANT ALL PRIVILEGES 
ON clearinghouse.* 
TO 'clearinghouse'@'localhost'
IDENTIFIED BY 'desired password for clearinghouse';

mysql> create database keydb;

mysql> GRANT ALL PRIVILEGES 
ON keydb.* 
TO 'keydb'@'localhost'
IDENTIFIED BY 'desired password for keydb';

where you would replace the password strings with suitable ones. Afterwards, type \q to leave the MySQL prompt:

mysql> \q
Bye!
$ 

Deploying and running Clearinghouse

In this section, we will deploy and run a copy of the Clearinghouse from your current user account in a temporary directory. This is mainly useful for testing.

Initial preparation

  • Change to the clearinghouse user account: sudo -i -u ch.
  • Clone the Clearinghouse repository into ch's home directory, and let the initialize script fetch dependencies:
$ cd ~
$ git clone https://github.com/kaushik1091/clearinghouse.git
$ cd clearinghouse/scripts
$ python initialize.py
  • Deploy all necessary files to a directory of your choice. We'll use ~/deployment (a directory called deployment under the clearinghouse user's account) in these instructions. You'll need to give two arguments to the deployment script: The parent directory of the clearinghouse repo you checked out, and a directory you want to deploy to. (In case the latter exists, you will be asked if you want a backup to be created.) For example:
$ python ~/clearinghouse/deploymentscripts/deploy_clearinghouse.py ~ ~/deployment

Deploy Repy runtime

Create a seattle directory within deployment (where the deployed clearinghouse dir already exists), and run the build script.

$ mkdir ~/deployment/seattle
$ cd ~/clearinghouse/scripts
$ python build.py ~/deployment/seattle

Generating keys

The Seattle backend scripts require a set of public keys (called state keys) to work. From the seattle runtime directory created and populated earlier, make runnable and then run the script that will generate state keys for you:

$ cd ~/deployment/seattle
$ chmod +x generate_state_keys.sh
$ mkdir ../clearinghouse/node_state_transitions/statekeys   # Work around https://github.com/SeattleTestbed/clearinghouse/issues/149
$ ./generate_state_keys.sh ../clearinghouse/node_state_transitions/statekeys

Note that only the *.publickey files are required for the clearinghouse. You can safely remove the *.privatekeys from ~/deployment/clearinghouse/node_state_transitions/statekeys.

Setup the website and start a development version

  • Be sure you've already created a MySQL database for the clearinghouse (e.g. called clearinghouse).
  • Edit the configuration file for the clearinghouse Django project, ~/deployment/clearinghouse/website/settings.py:
    • The clearinghouse database name and credentials in the DATABASES dict.
      # Database
      # https://docs.djangoproject.com/en/1.6/ref/settings/#databases
      DATABASES = {
          'default': {
              # you can use django.db.backends.sqlite3 instead of mysql. If you
              # decide to do so, you can leave the other fields empty
              'ENGINE': 'django.db.backends.mysql',
              'NAME': 'clearinghouse',
              'USER': 'clearinghouse',
              'PASSWORD': 'the user\'s password',
              'HOST': '',
              'PORT': '',
          }
      }
      
    • Set SECRET_KEY to a long, random string.
    • Make sure there is a reference to 'clearinghouse.website.control' under INSTALLED_APPS in settings.py
      INSTALLED_APPS = (
        'django.contrib.admin',
        'django.contrib.auth',
        'django.contrib.contenttypes',
        'django.contrib.sessions',
        'django.contrib.sites',
        'django.contrib.messages',
        'django.contrib.staticfiles',
        'social_auth',
        'clearinghouse.website.control',)
      
    • If this is a production launch, also set DEBUG to False, and uncomment and change the fields for sending ADMINS email.
    • If your clearinghouse is supposed to provide installers other than the stock Seattle ones, you need to set up a Custom Installer Builder and point SEATTLECLEARINGHOUSE_INSTALLER_BUILDER_XMLRPC to that URL.
    • You can also adapt the clearinghouse TIME_ZONE.
    • To further customize the appearance of your setup, modify the TESTBED name and URL, term you would like to use for CLEARINGHOUSE, and project mailing lists.
  • Add these lines to the clearinghouse user's .bashrc. (The changes will become effective the next time the user logs in, so you might want to log out and log in again following the edit.)
    # In ~/.bashrc
    export PYTHONPATH=$PYTHONPATH:/home/ch/deployment:/home/ch/deployment/seattle:/home/ch/clearinghouse
    export DJANGO_SETTINGS_MODULE='clearinghouse.website.settings'
    

Note: the /home/ch/deployment path entry is to make available the two packages clearinghouse and seattle which the deployment script created in the target directory. The Seattle path item is to ensure that the Repy runtime works.

  • Create the database structure. You may want to create a superuser account when asked. Note that this user will be able to log in over the web using the Django admin page. Use a strong password, and update it frequently! (The password can be changed on the command line using manage.py changepassword followed by the user account name).
    cd ~/deployment/clearinghouse
    python website/manage.py makemigrations control
    python website/manage.py migrate
    
Last modified 2 years ago Last modified on Oct 17, 2015 5:35:21 PM