2015

Probes into Plone and Zope

Introduction

On all teams, we need to check if Plone turns well. We need some probes to be sure our Plone site runs well !

With Jean-François Roche, we started to have a look on Products.ZNagios. This product allow you to have some probes from Zope, you can ask your instance (live):

  • Number of unresolved conflict on Zope
  • CPU usage
  • DB sizes
  • Memory percent
  • Uptime of Zope
  • ...

You can access to the probes with a thread which listen on Zope  on port 8888 (in this conf). You just have to add zope-conf-additional in your buildout like this:

[instance]
...
zope-conf-additional =
<product-config five.z2monitor>
  bind 0.0.0.0:8888
  </product-config>

If you want more information on this, you can see documentation of five.z2monitor package.

collective.monitor

I created this package for adding some probes into Plone. I created probes as Products.ZNagios. We used a zope interface for registering all probes (zc.z3monitor.interfaces.IZ3MonitorPlugin). In this package, I added these probes:

  • count users
  • count valid users (user logged during 3 last months)
  • check if smtp is set up
  • last login time of a user
  • last time a plone or zope object was modified

How use it

Adding collective.monitor in your buildout in eggs and zcml instance section

[instance]
...
eggs +=
...
collective.monitor
zcml +=
...
collective.monitor

And also adding zope-conf-additional as explain above.

After this little config, you can access to probes with different way

1. bin/instance

After starting instance (bin/instance fg) you can access to probes with

./bin/instance monitor dbinfo main
./bin/instance monitor objectcount
./bin/instance monitor stats./bin/instance monitor help

2. netcat

After starting instance (bin/instance fg) you can access to probes with

echo 'dbinfo main' | nc -i 1 127.0.0.1 8888

3. telnet

After starting instance (bin/instance fg) you can access to probes with

$ telnet 127.0.0.1 8888
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
last_modified_zope_object_time
2015/08/11 11:49:48.540729 GMT+2
Connection closed by foreign host.

Conclusion

With this package, you can make stats on your instance.

We use diamond to collect and put informations from probes on graphite.

It's very helpful for having state of our infrastructure.

 


Geonode, Geoserver, Postgis with Docker

Intro

We have some clients which needs a framework for maps creation. We took a look on market of open source solutions for this kind of feature and we became fan of geonode project.

We begin to be familiar with Docker so we decided to create Docker images for Geonode. We would like to separate geoserver and geonode. The goal is to be able to move geoserver or geonode on distinct server if the load increase. So we create different images for Geonode, Geoserver. We also use Nginx image for creation of link between Geonode and Geoserver (and a Postgis image for development).

For this project, we customise geonode. We use django template for project creation as explain on documentation ( http://docs.geonode.org/en/latest/tutorials/devel/projects/setup.html).

$ django-admin startproject imio_geonode --template=https://github.com/GeoNode/geonode-project/archive/master.zip -epy,rst

 

docker-compose

https://docs.docker.com/compose

We use 2 differents docker-compose.yml files, one for production and one for development. 

Differences are :

  • entrypoint and command options are define in Dockerfile for prod and in docker-compose.yml for dev (we overide options on dev).
  • image vs build : build is used in dev. For production, we build images and use image option.
  • postgis : An image of postgis is used in dev and no image on prod, we use a specific database cluster.
This is development docker-compose.yml
 
postgis:
build: Dockerfiles/postgis/
hostname: postgis
volumes:
- ./postgres_data:/var/lib/postgresql

geoserver:
build: Dockerfiles/geoserver/
hostname: geoserver
links:
- postgis
ports:
- 8080:8080
volumes:
- ./geoserver_data:/opt/geoserver/data_dir

geonode:
build: .
hostname: geonode
links:
- postgis
ports:
- 8000:8000
volumes:
- .:/opt/geonode/
entrypoint:
- /usr/bin/python
command: manage.py runserver 0.0.0.0:8000

nginx:
image: nginx:latest
ports:
- 80:80
links:
- geonode
- geoserver
- postgis
volumes:
- nginx-default.conf:/etc/nginx/conf.d/default.conf

 

Nginx

We need and nginx images to make the link between geoserver and geonode. With Docker >= 1.8 and docker-compose >= 1.4, a new 'network' option arrived and seems to depreced this nginx utility.

Nginx default image (https://registry.hub.docker.com/_/nginx/) is used with this config:

upstream geonode {
    server geonode:8000;
}
upstream geoserver {
    server geoserver:8080;
}

server {
        listen   80;
        client_max_body_size 128m;

        location / {
            proxy_pass         http://geonode;
            proxy_set_header   Host $http_host;
            proxy_set_header   X-Real-IP       $remote_addr;
            proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
        }

        location /geoserver {
            proxy_pass         http://geoserver;
            proxy_set_header   Host $http_host;
            proxy_set_header   X-Real-IP       $remote_addr;
            proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
        }

        error_log /var/log/nginx/error.log warn;
        access_log /var/log/nginx/access.log combined;
}

Nginx is useful for connection login from geoserver to geonode and for upload layers, ... from geonode to geoserver.

For production, we add static and upload folder rules :

location /static {
alias /opt/geonode/static;
} location /uploaded {
alias /opt/geonode/uploaded;
}
 

 

Postgis

We use postgis docker image for development. Indead, we have a dedicated server for our databases. For dev, we use this Dockerfile :

FROM postgres:9.4
RUN apt-get update && apt-get install -y postgresql-9.4-postgis-2.1
RUN mkdir /docker-entrypoint-initdb.d COPY ./initdb-postgis.sh /docker-entrypoint-initdb.d/initdb-postgis.sh

From postgres images, all sh files in docker-entrypoint-initdb.d folder are run during postgres initialisation. And db init script for geonode looks like :

#!/bin/sh
POSTGRES="gosu postgres"
$POSTGRES postgres --single -E <<EOSQL
CREATE ROLE geonode ENCRYPTED PASSWORD 'geonode' LOGIN;
EOSQL
$POSTGRES postgres --single -E <<EOSQL
CREATE DATABASE geonode OWNER geonode ;
CREATE DATABASE "geonode-imports" OWNER geonode ;
EOSQL
$POSTGRES pg_ctl -w start
$POSTGRES psql -d geonode-imports -c 'CREATE EXTENSION postgis;'
$POSTGRES psql -d geonode-imports -c 'GRANT ALL ON geometry_columns TO PUBLIC;'
$POSTGRES psql -d geonode-imports -c 'GRANT ALL ON spatial_ref_sys TO PUBLIC;'

 

Geoserver

I create an Docker image from 'tomcat:8-jre7' image, and install geoserver from http://build.geonode.org/geoserver/latest/geoserver.war.

Dockerfile looks like :

FROM tomcat:8-jre7

RUN apt-get update && apt-get install wget
RUN wget -O /usr/local/tomcat/webapps/geoserver.war http://build.geonode.org/geoserver/latest/geoserver.war
RUN apt-get remove -y wget ENV GEOSERVER_DATA_DIR /opt/geoserver/data_dir

 

Geonode

Production

We use gunicorn for production (https://pypi.python.org/pypi/gunicorn/)

Development

We use 'python manage.py runserver' for development. As you see in docker-compose.yml file, source code is added into a docker image with a volume, thus when you change code on your local computer, it's directly update on docker image.

Dockerfile :

FROM ubuntu:14.04
RUN \
apt-get update && \
apt-get install -y build-essential && \
apt-get install -y libxml2-dev libxslt1-dev libjpeg-dev gettext git python-dev python-pip libgdal1-dev && \
apt-get install -y python-pillow python-lxml python-psycopg2 python-django python-bs4 python-multipartposthandler transifex-client python-paver python-nose python-django-nose python-gdal python-django-pagination python-django-jsonfield python-django-extensions python-django-taggit python-httplib2 RUN mkdir -p /opt/geonode WORKDIR /opt/geonode
ADD requirements.txt /opt/geonode/
RUN pip install -r requirements.txt
ADD . /opt/geonode

COPY ./docker-entrypoint.sh /
ENTRYPOINT ["/docker-entrypoint.sh"]

Settings and local_settings

You have to update your local_settings. Settings have to match with settings into Dockerfiles and docker-compose.yml. Here is an exemple of local_settings.py for OGC_SERVER and DATABASES : 

SITENAME = 'GeoNode'
SITEURL = 'http://localhost'
GEOSERVER_URL = SITEURL + '/geoserver/'
GEOSERVER_BASE_URL = GEOSERVER_URL
# OGC (WMS/WFS/WCS) Server Settings
OGC_SERVER = {
'default': {
'BACKEND': 'geonode.geoserver',
'LOCATION': 'http://172.17.42.1:80/geoserver/', # Docker IP
'PUBLIC_LOCATION': GEOSERVER_URL,
'USER': 'admin',
'PASSWORD': 'admin',
'MAPFISH_PRINT_ENABLED': True,
'PRINT_NG_ENABLED': True,
'GEONODE_SECURITY_ENABLED': True,
'GEOGIT_ENABLED': False,
'WMST_ENABLED': False,
'BACKEND_WRITE_ENABLED': True,
'WPS_ENABLED': True,
# Set to name of database in DATABASES dictionary to enable
'DATASTORE': 'datastore',
}
}

DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'geonode',
'USER': 'geonode',
'PASSWORD': 'geonode',
'HOST': 'postgis',
'PORT': 5432,
},
'datastore': {
'ENGINE': 'django.contrib.gis.db.backends.postgis',
'NAME': 'geonode-imports',
'USER': 'geonode',
'PASSWORD': 'geonode',
'HOST': 'postgis',
'PORT': 5432,
}
}

You also have to set link between Geoserver and Geonode login. Use Docker IP for settings that (geoserver_data/security/auth/geonodeAuthProvider/config.xml) :

<org.geonode.security.GeoNodeAuthProviderConfig>
<id>-0000000000000</id>
<name>geonodeAuthProvider</name>
<className>org.geonode.security.GeoNodeAuthenticationProvider</className>
<baseUrl>http://172.17.42.1:80/</baseUrl>
</org.geonode.security.GeoNodeAuthProviderConfig>

 

Conclusion

You can now start you geonode project with a simple :

$ docker-compose up

And make Django syncdb of your database :

$ docker-compose run --rm --entrypoint='/usr/bin/python' geonode manage.py syncdb


New recipe, collective.recipe.buildoutcache

Introduction

This recipe generate a buildout-cache archive. We use pre-generated buildout-cache folder for speed up buildout duration. The archive contains one single buildout-cache folder. In this folder, there are 2 folders:

  • eggs: contains all eggs use by your buildout except eggs which have to be compiled.
  • downloads: contains zip eggs which must be compiled (as AccessControl, lxml, Pillow, ZODB, ...)

Before starting a buildout, we download and extract buildout-cache and use it on our buildout. We add eggs-directory and download-cache parameters on buildout section like this:

[buildout]

eggs-directory = buildout-cache/eggs download-cache = buildout-cache/downloads

 

Use case

In our organization, we have a Jenkins server. We created a Jenkins job which generate buildout-cache.tar.gz2 every night and push it into a file server.

We also use Docker, our Dockerfiles download and untar buildout-cache before starting buildout, so creation of docker image became very faster !

 

How it works

Simply, you have to add an parts with this recipe on your buildout project.

Like this :

[buildout]

parts = ... makebuildoutcachetargz [makebuildoutcachetargz] recipe = collective.recipe.buildoutcache

You can use some parameters for changing name of your archive, use another working directory than ./tmp or use another buildout file than buildout.cfg for eggs downloads, See https://github.com/collective/collective.recipe.buildoutcache.

For recipe installation you can make this command line:

./bin/buildout install makebuildoutcachetargz

And start recipe script:

./bin/makebuildoutcachetargz

 

Conclusion

Use collective.recipe.buildoutcache and decrease time lost with your buildout ;)


How I made my wedding site

I decide to make a website for my wedding with a list of gift, my honneymoon, presentation of my witnesses and so on.

I was looking for a litlle CMS with a "list of gift" (an online shopping) which can be installed on cheap and reliable hosting (An it's when I loose Plone)

Pyramid vs Django

I started looking on Pyramid (because I'm a Plone/Zope dev). I thought Kotti, but I didn't find a way to make easily gift, and I thougt project looks cool, but it'was maybe a little young for my kind of requirements. I didn't find good solution on pyramid for a wedding list. 

Such as I have some exprience in Django, And in my daily work, we started intereset on Geonode for GIS project.

-> I started looking on Django !

Django CMS vs Mezzanine

Django CMS and Django CMS e-commerce plugin. But it seems this project is a almost dead ? Last commit on github make me septic.

With little search, I found Mezzanine and Cartridge. I try it and It seems perfect for my porject, So I choose it !

Hosting

My first choose was OVH, because it's very cheap (5€ / month). But with little search, it is almost impossible to create a complex Django site (by complex, I mean a "Mezzanine" Django site, and it's not very complex). I pursued my searching... And I found Webfaction. They have local pythons, postgres, 600Go data for 10 € / month. It looks perfect for me, except they do not manage domain name directly. So I host my wedding site on webfaction and my domain name on OVH.

Maybe I could made an heroku Django website, but I was little affraid about complexity.

 

Next step is to create an online shop with Kotti or with Pyramid !


Plone and Docker

Introduction

In this post, I will try to explain how we put Plone sites into production in our organization (IMIO, Belgium).


For this process, we used some software as Puppet, Jenkins, but the process we use should be agnostic from these softwares.

Short story: when a push is made on Github, Jenkins builds a new image for Docker, pushes this image into a private Docker registry and updates the Docker image on server.

Docker images

We create Docker images with packer. We build .deb files with buildout, mr.bob and Jenkins. We create “debian” folder used to .deb files creation with a mr.bob template. We create 3 deb files:

  • plone-core-${version}.deb: contains eggs
  • plone-zeoserver-${version}.deb: contains zeoserver config files
  • plone-instance-${version}.deb: contains instance config files

After creation of deb files, packer uses (and installs) those deb files to create 2 Docker images:

  • docker.private-registry.be/plone.zeoserver.be:latest
  • docker.private-registry.be/plone.instance.be:latest

We think this is a good way to have good isolation.

Both images are based on a "base IMIO image". Our base image is based on ubuntu image from docker hub. Each image has a size of +/- 530 MB because we have a lot of plone eggs in our buildout/plone site.

You could also create a simple Dockerfile which pulls a github repo and runs buildout to create your Docker image.

Once Packer has built the Docker images, Jenkins pushes them into a private Docker registry.

Private registry

For this post, I imagine we have a private docker registry in this url : docker.private-registry.be.
We use private registry to store our images.
Our images are created with tag latest and YYYYMMDD-JENKINS_JOB_NUMBER (20150127-97)
We use a private registry for each environment, (staging, production, …) and we copy images between environments. Actually, we automatically update dev and staging environments and when we see there are no problem, we copy images on production.

Update production

We use fig to orchestrate our docker containers (zeo server must be started before zeo clients). We use a script to update our docker images. This script checks if the currently running docker containers use the latest image. If not, the script downloads the latest image, stops the docker containers which are running, remove them and restart containers from new images (we use upstart scripts to starting docker daemon).

cd /fig/directory;fig pulll > /dev/null 2>&1
NAME='plone'
REGISTRY_URL='docker.private-registry.be'
INSTANCE="instance"
ZEO="zeo"
INSTANCE_NAME="instance_1"
INSTANCE_IMAGE="$REGISTRY_URL/$INSTANCE"
ZEO_IMAGE="$REGISTRY_URL/$ZEO"

LATEST_INSTANCE_IMAGE_ID=$(docker images | grep $INSTANCE_IMAGE | grep latest | awk '{print $3}')
LATEST_ZEO_IMAGE_ID=$(docker images | grep $ZEO_IMAGE | grep latest | awk '{print $3}')

TAG_INSTANCE_IMAGE_ID=$(docker images | grep $LATEST_INSTANCE_IMAGE_ID | grep -v latest | awk '{print $2}')
TAG_ZEO_IMAGE_ID=$(docker images | grep $LATEST_ZEO_IMAGE_ID | grep -v latest | awk '{print $2}')

if [ "$TAG_INSTANCE_IMAGE_ID" != "$TAG_ZEO_IMAGE_ID" ];then
    echo "Error: instance and zeo images are no the same tag !" 1>&2
    exit 1
fi
RUNNING=$(docker ps | grep $INSTANCE_NAME | awk '{print $2}')
LATEST="$INSTANCE_IMAGE:$TAG_INSTANCE_IMAGE_ID"
if [ "$RUNNING" != "$LATEST" ];then
    echo "restarting $NAME"
    stop $NAME
    start $NAME
else
    echo "$NAME up to date"
fi

Storage and backup

We use Docker data containers (called storage in our case) for filestorage, blobstorage and backup folders. We start docker container with --volumes-from option. We have to be carefull to NEVER delete a storage container (maybe we have to improve docker for that).

We configure our buildouts to backup all data into var/backups folder and so, we launch docker with --volumes-from and -v options for backup and restore. Thanks to -v options, backups are stored on server and not in Docker. Later, backups are synced to our backup server.

With this zeo docker image, it’s easy to backup, pack and restore zodb. In the futur, we envision using relstorage instead of zeoserver. But currently, there is no DB admin in the company (hint to our boss ?).

Conclusion

Docker runs great in production !

I intend to follow docker machine, docker swarm and docker compose. 

Thank you to my colleagues Cédric de Wilde and Jean-François Roche for having worked with me to setup our production Plone into Docker.


Document Actions