Merge pull request #77 from apel/release-2.3.0

Release 2.3.0 to master

CHANGELOG

@@ -1,5 +1,11 @@
 Changelog for ssm
 =================
+* Wed Aug 16 2018 Adrian Coveney <adrian.coveney@stfc.ac.uk> - 2.3.0-1
+ - Added support for stomp.py versions from 3.1.6 onwards which allows for
+   builds on Ubuntu Trusty and should enable IPv6 support.
+ - Added script for creating Ubuntu (.deb) builds.
+ - Added script for creating Docker container builds.
+
 * Mon May 14 2018 Adrian Coveney <adrian.coveney@stfc.ac.uk> - 2.2.1-1
  - Added a check that the server certificate used for encryption hasn't expired
    so that a sending SSM won't start with an out of date server certificate.

Dockerfile

@@ -0,0 +1,29 @@
+FROM centos:7
+MAINTAINER APEL Administrators <apel-admins@stfc.ac.uk>
+
+# Copy the SSM Git repository to /tmp/ssm
+COPY . /tmp/ssm
+# Then set /tmp/ssm as the working directory
+WORKDIR /tmp/ssm
+
+# Add the EPEL repo so we can get pip
+RUN yum -y install epel-release
+# Then get pip
+RUN yum -y install python-pip
+
+# Install the system requirements of python-ldap
+RUN yum -y install gcc python-devel openldap-devel
+
+# Install the system requirements of ssm
+RUN yum -y install openssl
+
+# Install the python requirements of SSM
+RUN pip install -r requirements.txt
+# Then install the SSM
+RUN python setup.py install
+
+# Set the working directory back to /
+WORKDIR /
+# Then delete the temporary copy of the SSM Git repository
+# as there is no need for it after the image has been built.
+RUN rm -rf /tmp/ssm

README.md

@@ -22,10 +22,10 @@ The EPEL repository must be enabled.  This can be done by installing
 the RPM for your version of SL, which is available on this page:
 http://fedoraproject.org/wiki/EPEL
 
-The python stomp library (N.B. version 4 is currently not supported)
+The python stomp library (N.B. versions 3.1.1 and above are currently supported)
 * `yum install stomppy`
 
-The python daemon library
+The python daemon library (N.B. only versions below 2.2.0 are currently supported)
 * `yum install python-daemon`
 
 The python ldap library
@@ -75,6 +75,31 @@ The RPM carries out a number of steps to run the SSM in a specific way.
 6. It creates the pidfile directory /var/run/apel/
 7. It installs a service script in /etc/init.d/
 
+## Installing the DEB
+
+### Installation
+Install APEL SSM:
+* `dpkg -i apel-ssm_<version>_all.deb`
+
+Install any missing system packages needed for the SSM:
+* `apt-get -f install`
+
+Install any missing python requirements that don't have system packages:
+* `pip install "stomp.py>=3.1.1" dirq`
+
+If you wish to run the SSM as a receiver, you will also need to install the python-daemon system package:
+* `apt-get install python-daemon`
+
+### What the DEB does
+
+The DEB carries out a number of steps to run the SSM in a specific way.
+
+1. It installs the core files in the python libraries directory
+2. It installs scripts in /usr/bin
+3. It installs configuration files in /etc/apel
+4. It creates the messages directory /var/spool/apel/
+5. It creates the log directory /var/log/apel/
+6. It creates the pidfile directory /var/run/apel/
 
 ## Configuring the SSM
 
@@ -121,13 +146,44 @@ add your messages.
  * Run 'ssmsend'
  * SSM will pick up any messages and send them to the configured
    queue on the configured broker
-
+
+### Sender (container)
+ * Download the example [configuration file](conf/sender.cfg)
+ * Edit the downloaded sender.cfg file to configure the queue and broker
+ * Run the following docker command to send
+ ```
+ docker run \
+     -d --entrypoint ssmsend \
+     -v /path/to/downloaded/config/sender.cfg:/etc/apel/sender.cfg \
+     -v /path/to/read/messages:/var/spool/apel/outgoing \
+     -v /etc/grid-security:/etc/grid-security \
+     -v /path/to/persistently/log:/var/log/apel \
+     stfc/ssm
+ ```
+ * The line `-v /path/to/persistently/log:/var/log/apel \` is only required if you want to access the sender log as a file. If `console: true` is set in your `sender.cfg`, the container will also log to stdout/stderr.
+
 ### Receiver (service)
 
  * Run `service apelssm start`
  * If this fails, check /var/log/apel/ssmreceive.log for details
  * To stop, run `service apelssm stop`
 
+### Receiver (container)
+ * Download the example [configuration file](conf/receiver.cfg)
+ * Edit the downloaded receiver.cfg file to configure the queue and broker
+ * Run the following docker command to launch containerized receiver process
+ ```
+ docker run \
+     -d --entrypoint ssmreceive \
+     -v /path/to/downloaded/config/sender.cfg:/etc/apel/sender.cfg \
+     -v /path/to/read/messages:/var/spool/apel/outgoing \
+     -v /path/to/dns/file:/etc/apel/dns \
+     -v /etc/grid-security:/etc/grid-security \
+     -v /path/to/persistently/log:/var/log/apel \
+     stfc/ssm
+ ```
+  * The line `-v /path/to/persistently/log:/var/log/apel \` is only required if you want to access the receiver log as a file. If `console: true` is set in your `receiver.cfg`, the container will also log to stdout/stderr.
+
 ### Receiver (manual)
 
  * Run 'ssmreceive'

apel-ssm.spec

@@ -4,7 +4,7 @@
 %endif
 
 Name:           apel-ssm
-Version:        2.2.1
+Version:        2.3.0
 %define releasenumber 1
 Release:        %{releasenumber}%{?dist}
 Summary:        Secure stomp messenger
@@ -21,7 +21,7 @@ BuildArch:      noarch
 BuildRequires:  python-devel
 %endif
 
-Requires:       stomppy < 4.0.0, python-daemon, python-dirq, python-ldap
+Requires:       stomppy >= 3.1.1, python-daemon < 2.2.0, python-dirq, python-ldap
 Requires(pre):  shadow-utils
 
 %define ssmconf %_sysconfdir/apel
@@ -100,6 +100,12 @@ rm -rf $RPM_BUILD_ROOT
 %doc %_defaultdocdir/%{name}
 
 %changelog
+* Wed Aug 16 2018 Adrian Coveney <adrian.coveney@stfc.ac.uk> - 2.3.0-1
+ - Added support for stomp.py versions from 3.1.6 onwards which allows for
+   builds on Ubuntu Trusty and should enable IPv6 support.
+ - Added script for creating Ubuntu (.deb) builds.
+ - Added script for creating Docker container builds.
+
 * Mon May 14 2018 Adrian Coveney <adrian.coveney@stfc.ac.uk> - 2.2.1-1
  - Added a check that the server certificate used for encryption hasn't expired
    so that a sending SSM won't start with an out of date server certificate.

requirements.txt

@@ -1,5 +1,5 @@
-stomp.py<=3.1.6
-python-daemon
+stomp.py>=3.1.1
+python-daemon<2.2.0
 python-ldap
 dirq
 # Dependencies for testing

scripts/ssm-build-deb.sh

@@ -0,0 +1,85 @@
+#!/bin/bash
+
+# Execute the following as root to install lintian and fpm:
+# apt-get install lintian
+# apt-get install ruby ruby-dev build-essential
+# gem install --no-ri --no-rdoc fpm
+
+# Then run this file, as any user, altering the
+# version number in the TAG variable.
+# This file will create two versions of the deb file:
+# - apel-ssm_<tag>_all.deb contains all the files necessary to run a
+#   the SSM as a sender.
+# - apel-ssm-service_<tag>_all.deb will install service daemon files
+#   necessary to run the SSM as a receiver as a service.
+# After building apel-ssm_<tag>_all.deb, this script will run it
+# against lintian to highlight potential issues to the builder.
+
+set -eu
+
+TAG=2.3.0-1
+
+SOURCE_DIR=~/debbuild/source
+BUILD_DIR=~/debbuild/build
+
+# Where to install the python lib files
+PYTHON_INSTALL_LIB=/usr/lib/python2.7/dist-packages
+
+# Split the tag into version and package number
+# so they can be passed to fpm separately.
+# This will work with tags of the form <version_number>-<iteration>
+VERSION=$(echo "$TAG" | cut -d - -f 1)
+ITERATION=$(echo "$TAG" | cut -d - -f 2)
+
+# Create SSM and DEB dir (if not present)
+mkdir -p $SOURCE_DIR
+mkdir -p $BUILD_DIR
+
+# Clean up any previous build
+rm -rf $SOURCE_DIR/*
+rm -rf $BUILD_DIR/*
+
+# Get and extract the source
+TAR_FILE=${TAG}.tar.gz
+TAR_URL=https://github.com/apel/ssm/archive/$TAR_FILE
+wget --no-check-certificate $TAR_URL -O $TAR_FILE
+tar xvf $TAR_FILE -C $SOURCE_DIR
+rm -f $TAR_FILE
+
+fpm -s python -t deb \
+-n apel-ssm \
+-v $VERSION \
+--iteration $ITERATION \
+-m "Apel Administrators <apel-admins@stfc.ac.uk>" \
+--description "Secure Stomp Messenger (SSM)." \
+--no-auto-depends \
+--depends python2.7 \
+--depends python-pip \
+--depends python-ldap \
+--depends libssl-dev \
+--depends libsasl2-dev \
+--deb-changelog $SOURCE_DIR/ssm-$TAG/CHANGELOG \
+--python-install-bin /usr/bin \
+--python-install-lib $PYTHON_INSTALL_LIB \
+--exclude *.pyc \
+--package $BUILD_DIR \
+$SOURCE_DIR/ssm-$TAG/setup.py
+
+fpm -s pleaserun -t deb \
+-n apel-ssm-service \
+-v $VERSION \
+--iteration $ITERATION \
+-m "Apel Administrators <apel-admins@stfc.ac.uk>" \
+--description "Secure Stomp Messenger (SSM) Service Daemon files." \
+--architecture all \
+--no-auto-depends \
+--depends apel-ssm \
+--package $BUILD_DIR \
+/usr/bin/ssmreceive
+
+# Clean up files generated by script
+rm -rf apel_ssm.egg-info/ build/
+
+# Check the resultant (non service) deb for 'lint'
+echo "Possible Issues to Fix:"
+lintian $BUILD_DIR/apel-ssm_${TAG}_all.deb

scripts/ssm-build-rpm.sh

@@ -10,7 +10,7 @@
 rpmdev-setuptree
 
 RPMDIR=/home/rpmb/rpmbuild
-VERSION=2.2.1-1
+VERSION=2.3.0-1
 SSMDIR=apel-ssm-$VERSION
 
 # Remove old sources and RPMS

setup.py

@@ -2,8 +2,11 @@
 
 This script installs the APEL SSM library, sender and reciever. This
 should be similar to installing the RPM apel-ssm, although there
-may be some differences. A known difference is the RPM installs pyc
-and pyo files, whereas this script does not.
+may be some differences.
+
+Known differences are:
+- the RPM installs pyc and pyo files, whereas this script does not.
+- this script will not install system specific init style files.
 
 Usage: 'python setup.py install'
 
@@ -28,15 +31,6 @@ def main():
         copyfile('scripts/apel-ssm.logrotate', 'conf/apel-ssm')
         copyfile('README.md', 'apel-ssm')
 
-        if not path.exists('/var/log/apel'):
-            makedirs('/var/log/apel')
-
-        if not path.exists('/var/run/apel'):
-            makedirs('/var/run/apel')
-
-        if not path.exists('/var/spool/apel'):
-            makedirs('/var/spool/apel')
-
     # conf_files will later be copied to conf_dir
     conf_dir = '/etc/apel/'
     conf_files = ['conf/receiver.cfg',
@@ -56,16 +50,19 @@ def main():
           url='http://apel.github.io/',
           download_url='https://github.com/apel/ssm/releases',
           license='Apache License, Version 2.0',
-          install_requires=['stomp.py<=3.1.6', 'python-ldap', 'dirq'],
+          install_requires=['stomp.py>=3.1.1', 'python-ldap', 'dirq'],
           extras_require={
-              'python-daemon': ['python-daemon'],
+              'python-daemon': ['python-daemon<2.2.0'],
           },
-          packages=find_packages(exclude=['bin']),
+          packages=find_packages(exclude=['bin', 'test']),
           scripts=['bin/ssmreceive', 'bin/ssmsend'],
           data_files=[(conf_dir, conf_files),
                       ('/etc/logrotate.d', ['conf/apel-ssm']),
-                      ('/etc/init.d', ['bin/apel-ssm']),
-                      ('/usr/share/doc', ['apel-ssm'])],
+                      ('/usr/share/doc/apel-ssm', ['apel-ssm']),
+                      # Create empty directories
+                      ('/var/log/apel', []),
+                      ('/var/run/apel', []),
+                      ('/var/spool/apel', [])],
           # zip_safe allows setuptools to install the project
           # as a zipfile, for maximum performance!
           # We have disabled this feature so installing via the setup

ssm/__init__.py

@@ -19,7 +19,7 @@
 import logging
 import sys
 
-__version__ = (2, 2, 1)
+__version__ = (2, 3, 0)
 
 LOG_BREAK = '========================================'