majic-ansible-roles Changeset

Changeset - 67dd87d59abb

Parent rev.

Child rev.

[Not reviewed]

0 6 0

Branko Majic (branko) - 9 years ago 2016-11-28 22:39:12
branko@majic.rs

MAR-83: Added support to wsgi_website role for specifying headers that should be passed on to Gunicorn by Nginx. Updated hello.wsgi app to demonstrate the feature.

6 files changed with 24 insertions and 3 deletions:

docs/rolereference.rst

docs/usage.rst

roles/wsgi_website/defaults/main.yml

roles/wsgi_website/templates/nginx_site.j2

testsite/group_vars/web.yml

testsite/playbooks/roles/wsgihello/files/hello.wsgi

0 comments (0 inline, 0 general)

docs/rolereference.rst

➞

Show inline comments

 PHP Website
 -----------
 The ``php_website`` role can be used for setting-up a website powered by PHP on
 destination machine.
 This role is normally not supposed to be used directly, but should instead serve
 as the basis for writing website-specific roles. Therefore the role is written
 in quite generic way, allowing the integrator to write his/her own logic for
 deploying the necessary PHP applications, while still reusing a common base and
 reducing the workload.
 The role implements the following:
 * Creates a dedicated user/group for running the PHP scripts.
 * Creates a dedicated administrator user for maintaining the website.
 * Creates a base directory where the website-specific code and data should be
   stored at.
 * Adds nginx to website's group, so nginx could read the necessary files.
 * Adds website administrator to website's group, so administrator could manage
   the code and data.
 * Installs additional packages required for running the role (as configured).
 * Deploys the HTTPS TLS private key and certificate (for website vhost).
 * Configures PHP FPM and nginx to serve the website.
 The role is implemented with the following layout/logic in mind:
 * Website users are named after the ``FQDN`` (fully qualified domain name) of
   website, in format of ``web-ESCAPEDFQDN``, where ``ESCAPEDFQDN`` is equal to
   ``FQDN`` where dots have been replaced by underscores (for example,
   ``web-cloud_example_com``).
 * Website users are set-up via GECOS field to have their umask set to ``0007``
   (in combination with ``pam_umask``).
 * Administrator users are named after the ``FQDN`` (fully qualified domain name)
   of website, in format of ``admin-ESCAPEDFQDN``, where ``ESCAPEDFQDN`` is equal
   to ``FQDN`` where dots have been replaced by underscores (for example,
   ``admin-cloud_example_com``).
 * All websites reside within a dedicated sub-directory in ``/var/www``. The
   sub-directory name is equal to the ``FQDN`` used for accessing the
   website. Owner of the directory is set to be the application administrator,
   while group is set to be the website group. Additionally, ``SGID`` bit is set
   on the directory. This allows admin, with correct umask, to create necessary
   files and directories that should be readable (and eventually writeable) by
   the website user (running the PHP scripts) without having to become root.
 * All files placed in the website directory should be either created there
   directly, or copied to the directory in order to make sure the ``SGID`` gets
   honored. **Do not move the files, the permissions will not be set correctly.**
 * Within the website directory, nginx/php5-fpm will expect to find the relevant
   files within the htdocs sub-directory (this can be symlink too).
 * nginx communicates with PHP FPM over a dedicated Unix socket for each website.
 Role dependencies
 ~~~~~~~~~~~~~~~~~
 Depends on the following roles:
 * **common**
 * **web_server**
 Parameters
 ~~~~~~~~~~
 **additional_fpm_config** (dict, optional, ``{}``)
   Additional PHP FPM configuration options that should be included for PHP
   website's pool. Keys are parameter names, values are associated values. Don't
   forget to include quotes in the value itself if expected value type is string.
 **additional_nginx_config** (list, optional, ``[]``)
   List providing additional Nginx configuration options to include. This can be
   useful for specifying things like error pages. Options are applied inside of a
   **server** context of Nginx configuration file.
   Each item is a dictionary with the following options describing the extra
   configuration option:
   **comment** (string, mandatory)
     Comment describing the configuration option.
   **value** (string, mandatory)
     Configuration option.
 **admin_uid** (integer, optional, ``whatever OS picks``)
   UID of the dedicated website administrator user. The user will be member of
   website group.
 **deny_files_regex** (list, optional, ``[]``)
   List of regular expressions for matching files/locations to which the web
   server should deny access. This is useful to block access to any sensitive
   files that should not be served directly by the web server. The format must be
   compatible with regular expressions used by ``nginx`` for ``location ~``
   syntax.
 **enforce_https** (boolean, optional, ``True``)
   Specify if HTTPS should be enforced for the website or not. If enforced,
   clients connecting via plaintext will be redirected to HTTPS, and clients will
   be served with ``Strict-Transport-Security`` header with value of
   ``max-age=31536000; includeSubDomains``.
 **environment_indicator** (dictionary, optional, ``null``)
   Specify configuration for including environment indicator on all HTML
   pages. Indicator is a simple strip at bottom of a page with custom background
   colour, text colour, and text.
   Specifying environment indicator is useful for avoiding mistakes when testing
   by having better visibility what environment you are in
   (production/staging/test).
   The following keys need to be specified:
   **background_colour** (string, mandatory)
     Background colour to use for the strip at bottom. This should be value
     compatible with CSS ``background-color`` attribute.
   **text_colour** (string, mandatory
     Text colour to use for the strip at bottom. This should be value compatible
     with CSS ``color`` attribute.
   **text** (string, mandatory)
     Text to show in show in the strip at bottom.
 **fqdn** (string, mandatory)
   Fully-qualified domain name where the website is reachable. This value is used
   for calculating the user/group name for dedicated website user, as well as
   home directory of the website user (where data/code should be stored at).
 **index** (string, optional, ``index.php``)
   Space-separated list of files which should be treated as index files by the
   web server. The web server will attempt opening these index files, in
   succession, until the first match, or until it runs out of matches, when a
   client requests an URI pointing to directory.
 **https_tls_certificate** (string, optional, ``{{ lookup('file', tls_certificate_dir + '/' + fqdn + '_https.pem') }}``)
   X.509 certificate used for TLS for HTTPS service. The file will be stored in
   directory ``/etc/ssl/certs/`` under name ``{{ fqdn }}_https.pem``.
 **https_tls_key** (string, optional, ``{{ lookup('file', tls_private_key_dir + '/' + fqdn + '_https.key') }}``)
   Private key used for TLS for HTTPS service. The file will be stored in
   directory ``/etc/ssl/private/`` under name ``{{ fqdn }}_https.key``.
 **php_file_regex** (string, optional, ``\.php$``)
   Regular expression used for determining which file should be interepted via
   PHP.
 **php_rewrite_urls** (list, optional, ``[]``)
   A list of rewrite rules that are applied to incoming requests. These rewrite
   rules are specifically targetted at prettying-up the URLs used by the PHP
   scripts. Each element of the list should be a string value compatible with the
   format of ``nginx`` option ``rewrite``. The keyword ``rewrite`` itself should
   be omitted, as well as trailing semi-colon (``;``).
 **rewrites** (list, optional, ``[]``)
   A list of rewrite rules that are applied to incoming requests. Each element of
   the list should be a string value compatible with the format of ``nginx``
   option ``rewrite``. The keyword ``rewrite`` itself should be omitted, as well
   as trailing semi-colon (``;``).
 **packages** (list, optional, ``[]``)
   A list of additional packages to install for this particular PHP
   appliction. This is usually going to be different PHP extensions.
 **uid** (integer, optional, ``whatever OS picks``)
   UID/GID (they are set-up to be the same) of the dedicated website
   user/group.
 **website_mail_recipients** (string, optional, ``root``)
   Space-separated list of e-mails or local users to which the mails, sent to
   either the website admin or website user, should be forwarded to. Forwarding
   is configured via ``~/.forward`` configuration file.
 Examples
 ~~~~~~~~
 Here is an example configuration for setting-up two (base) PHP websites (for
 running *ownCloud* and *The Bug Genie* applications):
 .. code-block:: yaml
     - role: php_website
       fqdn: cloud.example.com
       uid: 2001
       php_file_regex: \.php($|/)
       rewrites:
         - ^/\.well-known/host-meta /public.php?service=host-meta
         - ^/\.well-known/host-meta\.json /public.php?service=host-meta-json
         - ^/\.well-known/carddav /remote.php/carddav/ redirect
         - ^/\.well-known/caldav /remote.php/caldav/ redirect
         - ^/apps/calendar/caldav\.php /remote.php/caldav/
         - ^/apps/contacts/carddav\.php /remote.php/carddav/
         - ^/remote/(.*) /remote.php
       deny_files_regex:
         - ^(\.|autotest|occ|issue|indie|db_|console|build/|tests/|config/|lib/|3rdparty/|templates/).*
       packages:
         # For ownCloud
         - php5-gd
         - php5-json
         - php5-mysql
         - php5-curl
       https_tls_key: "{{ lookup('file', inventory_dir + '/tls/cloud.example.com_https.key') }}"
       https_tls_certificate: "{{ lookup('file', inventory_dir + '/tls/cloud.example.com_https.pem') }}"
       additional_nginx_config:
         - comment: Use custom page for forbidden files.
           value: error_page 403 /core/templates/403.php;
         - comment: Use custom page for non-existing locations/files.
           value: error_page 404 /core/templates/404.php;
       additional_fpm_config:
         "env[PATH]": "\"/usr/local/bin:/usr/bin:/bin\""
       website_mail_recipients: "root john.doe@example.com"
       environment_indicator:
         background_colour: "green"
         text_colour: "black"
         text: "TEST ENVIRONMENT"
     - role: php_website
       deny_files_regex:
         - ^\..*
       php_rewrite_urls:
         - ^(.*) /index.php?url=$1
       fqdn: tbg.example.com
       uid: 2007
       https_tls_key: "{{ lookup('file', inventory_dir + '/tls/tbg.example.com_https.key') }}"
       https_tls_certificate: "{{ lookup('file', inventory_dir + '/tls/tbg.example.com_https.pem') }}"
 WSGI Website
 ------------
 The ``wsgi_website`` role can be used for setting-up a website powered by Python
 on destination machine. The website needs to use the WSGI specification for
 making the Python web application(s) available.
 This role is normally not supposed to be used directly, but should instead serve
 as the basis for writing website-specific roles. Therefore the role is written
 in quite generic way, allowing the integrator to write his/her own logic for
 deploying the necessary Python applications/packages, while still reusing a
 common base and reducing the workload.
 The role implements the following:
 * Creates a dedicated user/group for running the WSGI application.
 * Creates a dedicated administrator user for maintaining the website.
 * Creates a base directory where the website-specific code and data should be
   stored at.
 * Adds nginx to website's group, so nginx could read the necessary files.
 * Adds website administrator to website's group, so administrator could manage
   the code and data.
 * Installs additional packages required for running the role (as configured).
 * Sets-up a dedicated Python virtual environment for website.
 * Install ``futures`` package in Python virtual environment (required for
   Gunicorn in combination withg Python 2.7).
 * Install Gunicorn in Python virtual environment.
 * Installs additional packages required for running the role in Python virtual
   environment (as configured).
 * Configures systemd to run the website code (using Gunicorn)
 * Deploys the HTTPS TLS private key and certificate (for website vhost).
 * Configures nginx to serve the website (static files served directly, requests
   passed on to Gunicorn).
 The role is implemented with the following layout/logic in mind:
 * Website users are named after the ``FQDN`` (fully qualified domain name) of
   website, in format of ``web-ESCAPEDFQDN``, where ``ESCAPEDFQDN`` is equal to
   ``FQDN`` where dots have been replaced by underscores (for example,
   ``web-wiki_example_com``).
 * Website users are set-up via GECOS field to have their umask set to ``0007``
   (in combination with ``pam_umask``).
 * Administrator users are named after the ``FQDN`` (fully qualified domain name)
   of website, in format of ``admin-ESCAPEDFQDN``, where ``ESCAPEDFQDN`` is equal
   to ``FQDN`` where dots have been replaced by underscores (for example,
   ``admin-cloud_example_com``).
 * All websites reside within a dedicated sub-directory in ``/var/www``. The
   sub-directory name is equal to the ``FQDN`` used for accessing the
   website. Owner of the directory is set to be the application administrator,
   while group is set to be the website group. Additionally, ``SGID`` bit is set
   on the directory. This allows admin, with correct umask, to create necessary
   files and directories that should be readable (and eventually writeable) by
   the website user (running the WSGI application) without having to become root.
 * All files placed in the website directory should be either created there
   directly, or copied to the directory in order to make sure the ``SGID`` gets
   honored. **Do not move the files, the permissions will not be set correctly.**
 * Within the website directory, Python virtual environment can be found within
   the ``virtualenv`` sub-directory. Switching to administrator user via login
   shell will automatically activate the virtual environment.
 * Within the website directory, nginx will expect to find the static files
   within the ``htdocs`` sub-directory (this can be symlink too). Locations/aliases
   can be configured for static file serving.
 * Within the website directory, systemd service will expect to find the website
   code within the ``code`` sub-directory (this can be symlink too).
 * nginx communicates with WSGI server over a dedicated Unix socket for each
   website.
 Role dependencies
 ~~~~~~~~~~~~~~~~~
 Depends on the following roles:
 * **common**
 * **web_server**
 Parameters
 ~~~~~~~~~~
 **additional_nginx_config** (list, optional, ``[]``)
   List providing additional Nginx configuration options to include. This can be
   useful for specifying things like error pages. Options are applied inside of a
   **server** context of Nginx configuration file.
   Each item is a dictionary with the following options describing the extra
   configuration option:
   **comment** (string, mandatory)
     Comment describing the configuration option.
   **value** (string, mandatory)
     Configuration option.
 **admin_uid** (integer, optional, ``whatever OS picks``)
   UID of the dedicated website administrator user. The user will be member of
   website group.
 **enforce_https** (boolean, optional, ``True``)
   Specify if HTTPS should be enforced for the website or not. If enforced,
   clients connecting via plaintext will be redirected to HTTPS, and clients will
   be served with ``Strict-Transport-Security`` header with value of
   ``max-age=31536000; includeSubDomains``.
 **environment_indicator** (dictionary, optional, ``null``)
   Specify configuration for including environment indicator on all HTML
   pages. Indicator is a simple strip at bottom of a page with custom background
   colour, text colour, and text.
   Specifying environment indicator is useful for avoiding mistakes when testing
   by having better visibility what environment you are in
   (production/staging/test).
   The following keys need to be specified:
   **background_colour** (string, mandatory)
     Background colour to use for the strip at bottom. This should be value
     compatible with CSS ``background-color`` attribute.
   **text_colour** (string, mandatory
     Text colour to use for the strip at bottom. This should be value compatible
     with CSS ``color`` attribute.
   **text** (string, mandatory)
     Text to show in show in the strip at bottom.
 **environment_variables** (dict, optional, ``{}``)
   Specify additional environment variables that should be set for running the
   service. Environment variables will be set in both the systemd service and for
   the application's administrator user (when logged in as one).
 **fqdn** (string, mandatory)
   Fully-qualified domain name where the website is reachable. This value is used
   for calculating the user/group name for dedicated website user, as well as
   home directory of the website user (where data/code should be stored at).
 **futures_version** (string, optional, ``3.0.5``)
   Version of ``futures`` package to deploy in virtual environment. Required by
   Gunicorn when using Python 2.7. Default version is tested with the test site.
 **gunicorn_version** (string, optional, ``19.6.0``)
   Version of Gunicorn to deploy in virtual environment for running the WSGI
   application. Default version is tested with the test site.
 **https_tls_certificate** (string, optional, ``{{ lookup('file', tls_certificate_dir + '/' + fqdn + '_https.pem') }}``)
   X.509 certificate used for TLS for HTTPS service. The file will be stored in
   directory ``/etc/ssl/certs/`` under name ``{{ fqdn }}_https.pem``.
 **https_tls_key** (string, optional, ``{{ lookup('file', tls_private_key_dir + '/' + fqdn + '_https.key') }}``)
   Private key used for TLS for HTTPS service. The file will be stored in
   directory ``/etc/ssl/private/`` under name ``{{ fqdn }}_https.key``.
 **packages** (list, optional, ``[]``)
   A list of additional packages to install for this particular WSGI
   website. This is usually going to be development libraries for building Python
   packages.
 **proxy_headers** (dictionary, optional, ``{}``)
   Additional headers to set when proxying request to Gunicorn. Keys are header
   names, values are header values. Both should be compatible with Nginx
   ``proxy_set_header``. If you need to provide an empty value, use quotes (don't
   forget to surround them by another set of quotes for YAML syntax, for example
   ``"\"\""`` or ``'""'``).
 **rewrites** (list, optional, ``[]``)
   A list of rewrite rules that are applied to incoming requests. Each element of
   the list should be a string value compatible with the format of ``nginx``
   option ``rewrite``. The keyword ``rewrite`` itself should be omitted, as well
   as trailing semi-colon (``;``).
 **static_locations** (list, optional, ``[]``)
   List of locations that should be treated as static-only, and not processed by
   the WSGI application at all. This is normally used for designating serving of
   static/media files by Nginx (for example, in case of Django projects for
   ``/static/`` and ``/media/``).
 **uid** (integer, optional, ``whatever OS picks``)
   UID/GID (they are set-up to be the same) of the dedicated website
   user/group.
 **use_paste** (boolean, optional, ``False``)
   Tell Gunicorn to assume that the passed-in ``wsgi_application`` value is a
   filename of a Python Paste ``ini`` file instead of WSGI application.
 **virtuaelnv_packages** (list, optional, ``[]``)
   A list of additional packages to install for this particular WSGI appliction
   in its virtual environment using ``pip``.
 **website_mail_recipients** (string, optional, ``root``)
   Space-separated list of e-mails or local users to which the mails, sent to
   either the website admin or website user, should be forwarded to. Forwarding
   is configured via ``~/.forward`` configuration file.
 **wsgi_application** (string, mandatory)
   WSGI application that should be started by Gunicorn. The format should be
   conformant to what the ``gunicorn`` command-line tool accepts. If the
   ``use_paste`` option is enabled, the value should be equal to filename of the
   Python Paste ini file, located in the ``code`` sub-directory. It should be
   noted that in either case the value should be specsified relative to the
   ``code`` sub-directory. I.e. don't use full paths.
 Examples
 ~~~~~~~~
 Here is an example configuration for setting-up a (base) WSGI website (for
 running a bare Django project):
 .. code-block:: yaml
     - role: wsgi_website
       fqdn: django.example.com
       static_locations:
         - /static
         - /media
       uid: 2004
       virtualenv_packages:
         - django
       wsgi_application: django_example_com.wsgi:application
       environment_variables:
         DJANGO_SETTINGS_MODULE: "django_example_com.settings.production"
       https_tls_key: "{{ lookup('file', inventory_dir + '/tls/wsgi.example.com_https.key') }}"
       https_tls_certificate: "{{ lookup('file', inventory_dir + '/tls/wsgi.example.com_https.pem') }}"
       futures_version: 3.0.5
       gunicorn_version: 19.6.0
       additional_nginx_config:
         - comment: Use custom page for forbidden files.
           value: error_page 403 /static/403.html;
         - comment: Use custom page for non-existing locations/files.
           value: error_page 404 /static/404.html;
       website_mail_recipients: "root john.doe@example.com"
       environment_indicator:
         background_colour: "green"
         text_colour: "black"
         text: "TEST ENVIRONMENT"
       proxy_headers:
         Accept-Encoding: '""'
 Database Server
 ---------------
 The ``database_server`` role can be used for setting-up a MariaDB database
 server on destination machine.
 The role implements the following:
 * Installs MariaDB server and client.
 * Configures MariaDB server and client to use *UTF-8* encoding by default.
 * Sets password for the database root user.
 * Deploys MariaDB client configuration in location ``/root/.my.cnf`` that
   contains username and password for the root database user.
 Role dependencies
 ~~~~~~~~~~~~~~~~~
 Depends on the following roles:
 * **common**
 Parameters
 ~~~~~~~~~~
 **db_root_password** (string, mandatory)
   Password for the *root* database user.
 Examples
 ~~~~~~~~
 Here is an example configuration for setting-up the database server:
 .. code-block:: yaml
    ---
    db_root_password: root
 Database
 --------
 The ``database`` role can be used for creating a MariaDB database and
 accompanying user on destination machine.
 The role implements the following:
 * Creates MariaDB database.
 * Creates a dedicated user capable of performing any operation on the created
   database. Username is set to be same as the name of database.
 * Sets-up pre-backup task that creates database dump in location
   ``/srv/backup/mariadb/{{ db_name }}.sql``.
 Role dependencies
 ~~~~~~~~~~~~~~~~~
 Depends on the following roles:
 * **database_server**
 * **backup_client**
 Backups
 ~~~~~~~
 If the backup for this role has been enabled, the following paths are backed-up:
 **/srv/backup/maraidb/{{ db_name }}.sql**
   Dump of the database. Database dump is created every day at 01:45 in the
   morning.
 Parameters
 ~~~~~~~~~~
 **db_name** (string, mandatory)
   Name of the database that should be created.
 **db_password** (string, mandatory)
   Password for the database user.
 Examples
 ~~~~~~~~
 Here is an example configuration for creating a single database (for some
 website):
 .. code-block:: yaml
   - role: database
     db_name: phpinfo_example_com
     db_password: phpinfo_example_com
 Backup Server
 -------------
 The ``backup_server`` role can be used for setting-up a server to act as backup
 storage for the backup clients. Storage is made available to the clients
 exclusively via SFTP on a dedicated port and dedicated OpenSSH server
 instance. This instance is specifically configured and tailored for this
 purpose.
 The role is primarily aimed for use with `Duplicity
 <http://duplicity.nongnu.org/>`_, but should be also usable for generic SFTP
 uploads.
 The role implements the following:
 * Installs backup software (Duplicity, Duply).
 * Creates a dedicated directory structure for backups with the following structure:
   * ``/srv/backups/`` - main directory under which all the backups reside.
   * ``/srv/backups/SERVER_NAME/`` - home directory for the backup user, name
     after the server. Backup users are confined to their respective home
     directory via chroot. Backup users can't write to their own home directory,
     though.
   * ``/srv/backups/SERVER_NAME/duplicity/`` - directory where the Duplicity
     backups are stored at. This directory is writable by the respective backup
     user.
   * ``SERVER_NAME/.ssh/`` - directory where authorized keys are stored. Backup
     user is not allowed to make modifications to this directory and files
     contained within (i.e. backup users can't add more keys to the
     ``authorized_keys`` file).
 * Creates dedicated operating system users for backup clients. These users will
   be made members of the ``backup`` group as well (as an additional group).
 * Sets-up ``authorized_keys`` for the backup clients.
 * Makes sure the backup users can't log-in via regular OpenSSH server instance.
 * Sets-up dedicated OpenSSH server instances to be used exclusively by backup
   clients. The instance listens on TCP port ``2222``.
 * Updates firewall to allow incoming TCP connections to port
   ``2222``. Connections are allowed only from the configured IP addresses
   associated with backup clients.
 Role dependencies
 ~~~~~~~~~~~~~~~~~
 Depends on the following roles:
 * **common**
 Parameters
 ~~~~~~~~~~
 **backup_clients** (list, optional)
   List of backup clients that are connecting to the backup server. This is
   usually done on a per-server basis. Each item in the list is a dictionary
   describing the backup client. The following keys are available:
   **server** (string, mandatory)
     Name of the server that is backed up. It is highly recommended to use
     server's FQDN for this purpose. The dedicated operating system user created
     will have the name of format ``bak-ESCAPED_SERVER_NAME``, where
     ``ESCAPED_SERVER_NAME`` is calculated by taking the passed-in server name
     and replacing all dots (``.``) with undescores (``_``). For example,
     ``web.example.com`` will be turned into ``bak-web_example_com``.
   **uid** (integer, optional, ``whatever OS picks``)
     Uid for the operating system user. User's default group will have a GID
     identical to the user's UID if specified. Otherwise user's default group
     will have OS-determined GID.
   **ip** (IPv4 address, mandatory)
     IPv4 address from which the backup client server is connecting to the backup
     server. Used for introducing stricter firewall rules.
   **public_key** (string, mandatory)
     SSH public key used by backup client to connect to the backup server.
 **backup_host_ssh_private_keys** (dictionary, mandatory)
   Defines host keys used for the dedicated OpenSSH server instance for
   backup. Key values that must be provided are: **dsa**, **rsa**, **ed25519**,
   and **ecdsa**, with values for each one of them corresponding to a private key
   generated using the appropriate algorithm. Keys for this purpose can be easily
   created via commands::
     ssh-keygen -f backup_server_dsa_key -N '' -t dsa
     ssh-keygen -f backup_server_rsa_key -N '' -t rsa
     ssh-keygen -f backup_server_ed25519_key -N '' -t ed25519
     ssh-keygen -f backup_server_ecdsa_key -N '' -t ecdsa
 Examples
 ~~~~~~~~
 Here is an example configuration for setting-up the backup server role:
 .. code-block:: yaml
   - role: backup_server
     backup_clients:
       - server: web.example.com
         uid: 3000
         public_key: "{{ lookup('file', inventory_dir + '/ssh/web.example.com.pub') }}"
         ip: 10.32.64.18
       - server: mail.example.com
         public_key: "{{ lookup('file', inventory_dir + '/ssh/mail.example.com.pub') }}"
         ip: 10.32.64.15
     backup_host_ssh_private_keys:
       dsa: "{{ lookup('file', inventory_dir + '/ssh/backup_server_dsa_key') }}"
       rsa: "{{ lookup('file', inventory_dir + '/ssh/backup_server_rsa_key') }}"
       ed25519: "{{ lookup('file', inventory_dir + '/ssh/backup_server_ed25519_key') }}"
       ecdsa: "{{ lookup('file', inventory_dir + '/ssh/backup_server_ecdsa_key') }}"
 Backup Client
 -------------
 The ``backup_client`` role can be used for setting-up the server as a backup
 client so it can perform backups to the backup server.
 Backup clients utilise duplicity (via the duply convenience wrapper) for
 performing the backups to a backup server via *SFTP* protocol.
 The role itself will take care of deploying the necessary software,
 configuration files, and encryption/signing private key to the backup client in
 order to be able to perform backup.
 Files that should be backed-up are specified using the ``backup`` role.
 The role implements the following:
 * Installs backup software (Duplicity, Duply).
 * Sets-up Duply configuration under directory ``/etc/duply/main/``.
 * Deploys encryption/signing private key (usually host-specific), as well as
   additional encryption public keys to the server, and imports them into local
   GnuPG keyring used by backup software.
 * Deploys private SSH key for logging-in into the backup server over SFTP.
 * Deploys ``known_hosts`` file for SFTP fingerprint verification.
 * Sets-up a handler that runs scripts/binaries before the actual backup
   run. This is helpful for producing database backups. Such scripts/binaries
   should be deployed to directory ``/etc/duply/main/pre.d/``, and marked
   executable by the root user.
 Duply is configured as follows:
 * GnuPG keyring is stored under ``/etc/duply/main/gnupg/``. The keyring should
   not be managed manually.
 * SSH private key for logging-in into backup server is stored in location
   ``/etc/duply/main/ssh/identity``. Backup server SFTP fingerprint is stored in
   location ``/etc/duply/main/ssh/known_hosts``.
 * Base directory for back-ups is root (``/``), but *all* files are excluded by
   default to prevent huge back-ups. Ansible roles that want to utilise the
   backup client role can specify which patterns should be included in the backup
   when including the ``backup`` role. Include pattern file is assembled and
   stored in location ``/etc/duply/main/include``.
 * Backups are encrypted and signed with the specified encryption key.
 * Maximum age for old backups is set to 6 months.
 * Maximum age for full backups is set to 1 month.
 * Volume size is set to 1GB.
 * Pre-backup scripts are run via ``/etc/duply/main/pre`` handler that tries to
   execute scripts/binaries from directory ``/etc/duply/main/pre.d/``.
 .. note::
    Since at time of this writing there are no lookup plugins for extracting key
    material/information from GnuPG keyring, you may want to resort to extraction
    of keys on the controller machine via lookups similar to::
      lookup('pipe', 'gpg2 --homedir /path/to/your/keyring --armor --export some_identifier')
      lookup('pipe', 'gpg2 --homedir /path/to/your/keyring --armor --export-secret-keys some_identifier')
    This may not be the most elegant solution, but for now it offers better
    flexibility (theoretically, you could store all those keys etc as plaintext
    files instead).
 Parameters
 ~~~~~~~~~~
 **backup_additional_encryption_keys** (list, optional, ``[]``)
   List of additional public encryption keys used for backup operation. Each item
   in the list should be an ASCII armour-encoded public key exported from a GnuPG
   keyring. These additional public keys are useful in cases where the backups
   should be decryptable with some master key in addition to server-specific key.
 **backup_client_username** (string, optional, ``bak-{{ ansible_fqdn | replace('.', '_') }}``)
   Username for connecting to the backup server via SFTP.
 **backup_encryption_key** (string, mandatory)
   Private GnuPG key, encoded using ASCII armor, used for encryption and signing
   operations when running the backup on the client server. This *must* be a
   private key! This is normally host-specific encryption key that is distributed
   to destination server and that can be also used for the restore operations
   (for data decryption). The key must not be password-protected.
 **backup_server** (string, mandatory)
   Backup server to connect to.
 **backup_server_destination** (string, optional, ``//duplicity``)
   Target directory on the backup server where the backups are stored.
 **backup_server_host_ssh_public_keys** (list, mandatory)
   SSH public keys presented by the server during client authentication. These
   public keys are used for populating the known hosts on the backup client side
   for host verification purposes.
 **backup_server_port** (int, optional, ``2222``)
   Port on the backup server to connect to for accessing the SFTP service.
 **backup_ssh_key** (string, mandatory)
   SSH private key for logging-in into the backup server.
 Examples
 ~~~~~~~~
 Here is an example configuration for setting-up the role (take note that lookup
 plugin is quite useful here for fetching key values from some local directory):
 .. code-block:: yaml
   - role: backup_client
     backup_additional_encryption_keys: "-----BEGIN PGP PUBLIC KEY BLOCK-----\n...\n-----END PGP PUBLIC KEY BLOCK-----"
     backup_client_username: "user"
     backup_encryption_key: "-----BEGIN PGP PRIVATE KEY BLOCK-----\n...\n-----END PGP PRIVATE KEY BLOCK-----"
     backup_server: "backup.example.com"
     backup_server_destination: "//example/host"
     backup_server_host_ssh_public_keys:
       - "{{ lookup('file', inventory_dir + '/ssh/backup_server_dsa_key.pub') }}"
       - "{{ lookup('file', inventory_dir + '/ssh/backup_server_ecdsa_key.pub') }}"
       - "{{ lookup('file', inventory_dir + '/ssh/backup_server_ed25519_key.pub') }}"
       - "{{ lookup('file', inventory_dir + '/ssh/backup_server_rsa_key.pub') }}"
     backup_server_port: 22
     backup_ssh_key: "{{ lookup('file', inventory_dir + '/ssh/web.example.com') }}"
 Backup
 ------
 The ``backup`` role can be used to specify what files should be backed-up to the
 backup server.
 The role provides a convenient way to deploy a file containing file and
 directory patterns describing the file/directory paths that should be included
 in the back-up.
 The role implements the following:
 * Installs a file with provided patterns in directory
   ``/etc/duply/main/patterns/``.
 * Assembles/refresshes the main include pattern file at
   ``/etc/duply/main/include``.
 Role dependencies
 ~~~~~~~~~~~~~~~~~
 Depends on the following roles:
 * **backup_client**
 Parameters
 ~~~~~~~~~~
 **backup_patterns_filename** (string, mandatory)
   Name of the backup patterns file. The file is stored in directory
   ``/etc/duply/main/patterns/``. This should be a unique filename amongst all
   roles. If role can be included multiple times, make sure the filename is
   always unique when depending on the backup role.
 **backup_patterns** (list, optional, ``[]``)
   List of globbing patterns defining which files or directories should be
   backed-up.
 Examples
 ~~~~~~~~
 Here is an example configuration for setting-up the role:
 .. code-block:: yaml
   - role: backup
     backup_patterns_filename: myapp

docs/usage.rst

➞

Show inline comments

@@ @@ -1104,768 +1104,770 @@ Nginx. @@
           - ldap_client
           - mail_forwarder
           - web_server
 . You know the drill, role configuration comes up next. Actually... The web
    server role parameters are all optional, and they default to some ok-ish
    values. But let us spicen up things a bit nevertheless. No configuration has
    been deployed before for the web server, so we will be creating a new file.
    :file:`~/mysite/group_vars/web.yml`
    ::
       ---
       web_default_title: "Welcome to default page!"
       web_default_message: "Nothing to see here, move along..."
 . The only thing left now is to create the TLS private key/certificate pair
    that should be used for default virtual host.
 . Create new template for ``certtool``:
       :file:`~/mysite/tls/www.example.com_https.cfg`
       ::
          organization = "Example Inc."
          country = SE
          cn = "Exampe Inc. Web Server"
          expiration_days = 365
          dns_name = "www.example.com"
          tls_www_server
          signing_key
          encryption_key
 . Create the keys and certificates for default web server virtual host based
       on the template::
         certtool --sec-param normal --generate-privkey --outfile ~/mysite/tls/www.example.com_https.key
         certtool --generate-certificate --load-ca-privkey ~/mysite/tls/ca.key --load-ca-certificate ~/mysite/tls/ca.pem --template ~/mysite/tls/www.example.com_https.cfg --load-privkey ~/mysite/tls/www.example.com_https.key --outfile ~/mysite/tls/www.example.com_https.pem
 . Apply the changes::
      workon mysite && ansible-playbook playbooks/site.yml
 . If no errors have been reported, at this point you should have a default web
    page available and visible at https://www.example.com/ . By default plaintext
    connections are disabled, and trying to visit http://www.example.com/ should
    simply redirect you to the HTTPS address. Feel free to try it out with some
    browser. Keep in mind you will get a warning about the untrusted certificate!
 Adding the database server
 --------------------------
 Since both of the web applications we want to deploy need a database, we will
 proceed to set-up the database server role on the web server itself. *Majic
 Ansible Roles* in particular come with a role that will deploy MariaDB database
 server.
 . Update the playbook for web server to include the database server role.
     :file:`~/mysite/playbooks/web.yml`
     ::
       ---
       - hosts: web
         remote_user: ansible
         become: yes
         roles:
           - common
           - ldap_client
           - mail_forwarder
           - web_server
           - database_server
 . Now let's configure the role. This is rather simplistic, since we only need
    to set the database server root (admin) password.
    :file:`~/mysite/group_vars/web.yml`
    ::
       db_root_password: root
 . No TLS support has been implemented for this role (yet), so simply apply the
    changes::
      workon mysite && ansible-playbook playbooks/site.yml
 . If no errors have been reported, you should have a database server up and
    running on the web server. You should be able to log-in using password
    ``root`` by running the following command on the web server itself::
      mysql -uroot -p
    Of course, no database has been created for either of the web applications,
    but we will get to that one later (there is a dedicated ``database`` role
    which can be combined with web app roles for this purpose).
 Deploying a PHP web application (The Bug Genie)
 -----------------------------------------------
 We have some basic infrastructure up and running now on our web server, so we
 shall move on to setting-up a PHP web application on it. As mentioned before, we
 will take *The Bug Genie* as an example.
 For this we will create a local role in our site to take care of it. This role
 will in turn utilise two roles coming from *Majic Ansible Roles* that will make
 our life (a little) easier.
 To make the example a bit simpler, no parameters will be introducd for this role
 (not even the password for database, we'll hard-code everything).
 Before we start, here is a couple of useful pointers regarding the
 ``php_website`` role we'll be using for the PHP part:
 * The role is designed to execute every application via dedicated user and
   group. The user/group name is automatically derived from the FQDN of website,
   for example ``web-tbg_example_com``.
 * While running the application, application user's umask is set to ``0007``
   (letting the administrator user be able to manage any files created while the
   application is running).
 * An administrative user is created as well, and this user should be used when
   running maintenance and installation commands. Similar to application user,
   the name is also derived from the FQDN of website, for example
   ``admin-tbg_example_com``. Administrative user does not have a dedicated
   group, and instead belongs to same group as the application user.
 * PHP applications are executed via FastCGI, using the ``php5-fpm`` package.
 * If you ever need to set some additional PHP FPM settings, this can easily be
   done via the ``additional_fpm_config`` role parameter. This particular example
   does not set any, though.
 * Mails deliverd to local admin/application users are forwarded to ``root``
   account instead (this can be configured via ``website_mail_recipients`` role
   parameter.
 * If you ever find yourself mixing-up test and production websites, have a look
   at ``environment_indicator`` role parameter. It lets you insert small strip at
   bottom of each HTML page automatically.
 * Static content (non-PHP) is served directly by *Nginx*.
 * Each web application gets distinct sub-directory under ``/var/www``, named
   after the FQDN. All sub-directories created under there are created with
   ``2750`` permissions, with ownership set to admin user, and group set to the
   application's group. In other words, all directories will have ``SGID`` bit
   set-up, allowing you to create files/directories that will have their group
   automatically set to the group of the parent directory.
 * Files are served (both by *Nginx* and *php5-fpm*) from sub-directory called
   ``htdocs`` (located in website directory). For example
   ``/var/www/tbg.example.com/htdocs/``. Normally, this can be a symlink to some
   other sub-directory within the website directory (useful for having multiple
   versions for easier downgrades etc).
 * Combination of admin user membership in application group, ``SGID``
   permission, and the way ownership of sub-directories is set-up usually means
   that the administrator will be capable of managing application files, and
   application can be granted write permissions to a *minimum* of necessary
   files.
   .. warning::
      Just keep in mind that some file-management commands, like ``mv``, do *not*
      respect the ``SGID`` bit. In fact, I would recommend using ``cp`` when you
      deploy new files to the directory instead (don't simply move them from your
      home).
 . Start-off with creating the necessary directories for the new role::
      mkdir -p ~/mysite/roles/tbg/{tasks,meta,files}/
 . Let's set-up role dependencies, reusing some common roles to make our life
    easier.
    :file:`~/mysite/roles/tbg/meta/main.yml`
    ::
       ---
       dependencies:
          # Ok, so this role helps us set-up Nginx virtual host for serving our
          # app.
          - role: php_website
            # Our virtual host will for PHP website will respond to this name.
            fqdn: tbg.example.com
            # Some additional packages are required in order to deploy and use TBG.
            packages:
               - php5-gd
               - php5-curl
               - git
               - php5-mysql
               - expect
            # Set-up URL rewriting. This is based on public/.htaccess file from
            # TBG.
            php_rewrite_urls:
               - ^(.*)$ /index.php?url=$1
            # We don't necessarily need this, but in case you have a policy on
            # uid/gid usage, this is useful. Take note that below value is used
            # for both the dedicated uid and gid for application user.
            uid: 2000
            admin_uid: 3000
          # And this role sets up a new dedicated database for our web
          # application.
          - role: database
            # This is both the database name, _and_ name of the database user
            # that will be granted full privileges on the database.
            db_name: tbg
            # This will be the password of our user 'tbg' for accessing the
            # database. Take note the user can only login from localhost.
            db_password: tbg
 . Now for my favourite part again - creating private keys and certificates!
    Why?  Because the ``php_website`` role requires a private key/certificate
    pair to be deployed. So... Moving on:
 . Create new template for ``certtool``:
       :file:`~/mysite/tls/tbg.example.com_https.cfg`
       ::
          organization = "Example Inc."
          country = SE
          cn = "Exampe Inc. Issue Tracker"
          expiration_days = 365
          dns_name = "tbg.example.com"
          tls_www_server
          signing_key
          encryption_key
 . Create the keys and certificates for the application::
         certtool --sec-param normal --generate-privkey --outfile ~/mysite/tls/tbg.example.com_https.key
         certtool --generate-certificate --load-ca-privkey ~/mysite/tls/ca.key --load-ca-certificate ~/mysite/tls/ca.pem --template ~/mysite/tls/tbg.example.com_https.cfg --load-privkey ~/mysite/tls/tbg.example.com_https.key --outfile ~/mysite/tls/tbg.example.com_https.pem
 . Time to get our hands a bit more dirty... Up until now we didn't have to write
    custom tasks, but at this point we need to.
    :file:`~/mysite/roles/tbg/tasks/main.yml`
    ::
       ---
       - name: Define TBG version
         set_fact: tbg_version=4.1.0
       - name: Download the TBG archive
         get_url: url=https://github.com/thebuggenie/thebuggenie/archive/v{{ tbg_version }}.tar.gz
                  dest="/var/www/tbg.example.com/thebuggenie-{{ tbg_version }}.tar.gz"
                  sha256sum=0fd0a680ba281adc97d5d2c720e63b995225c99716a36eca6a198b8a5ebf8057
         become: yes
         become_user: admin-tbg_example_com
       - name: Download Composer
         get_url: url=https://getcomposer.org/download/1.0.0-alpha10/composer.phar
                  dest="/usr/local/bin/composer"
                  sha256sum=9f2c7d0364bc743bcde9cfe1fe84749e5ac38c46d47cf42966ce499135fd4628
                  owner=root group=root mode=755
       - name: Unpack TBG
         unarchive: src="/var/www/tbg.example.com/thebuggenie-{{ tbg_version }}.tar.gz"
                    dest="/var/www/tbg.example.com/" copy=no
                    creates="/var/www/tbg.example.com/thebuggenie-{{ tbg_version }}"
         become: yes
         become_user: admin-tbg_example_com
       - name: Create TBG cache directory
         file: path="/var/www/tbg.example.com/thebuggenie-{{ tbg_version }}/cache" state=directory mode=2770
         become: yes
         become_user: admin-tbg_example_com
       - name: Set-up the necessary write permissions for TBG directories
         file: path="{{ item }}" mode=g+w
         with_items:
            - /var/www/tbg.example.com/thebuggenie-{{ tbg_version }}/
            - /var/www/tbg.example.com/thebuggenie-{{ tbg_version }}/public/
            - /var/www/tbg.example.com/thebuggenie-{{ tbg_version }}/core/config/
       - name: Create symbolic link to TBG application
         file: src="/var/www/tbg.example.com/thebuggenie-{{ tbg_version }}/public"
               path="/var/www/tbg.example.com/htdocs"
               state=link
               owner="admin-tbg_example_com" group="web-tbg_example_com" mode=2750
         become: yes
         become_user: admin-tbg_example_com
       - name: Install TBG dependencies
         composer: command=install working_dir="/var/www/tbg.example.com/thebuggenie-{{ tbg_version }}"
         become: yes
         become_user: admin-tbg_example_com
       - name: Deploy database configuration file for TBG
         copy: src="b2db.yml" dest="/var/www/tbg.example.com/thebuggenie-{{ tbg_version }}/core/config/b2db.yml"
               mode=640 owner=admin-tbg_example_com group=web-tbg_example_com
       - name: Deploy expect script for installing TBG
         copy: src="tbg_expect_install" dest="/var/www/tbg.example.com/tbg_expect_install" mode=750
         become: yes
         become_user: admin-tbg_example_com
       - name: Run TBG installer via expect script
         command: /var/www/tbg.example.com/tbg_expect_install
                  chdir=/var/www/tbg.example.com/thebuggenie-{{ tbg_version }}
                  creates=/var/www/tbg.example.com/thebuggenie-{{ tbg_version }}/installed
         become: yes
         become_user: admin-tbg_example_com
 . Set-up the files that are deployed by our role.
    :file:`~/mysite/roles/tbg/files/b2db.yml`
    ::
       b2db:
           username: "tbg"
           password: "tbg"
           dsn: "mysql:host=localhost;dbname=tbg"
           tableprefix: ''
           cacheclass: '\thebuggenie\core\framework\Cache'
    :file:`~/mysite/roles/tbg/files/tbg_expect_install`
    ::
       #!/usr/bin/expect
       spawn ./tbg_cli install --accept_license=yes --url_subdir=/ --use_existing_db_info=yes --enable_all_modules=yes --setup_htaccess=yes
       expect "Press ENTER to continue with the installation: "
       send "\r"
       expect "Press ENTER to continue: "
       send "\r"
       interact
 . And... Let's add the new role to our web server.
    :file:`~/mysite/playbooks/web.yml`
    ::
       ---
       - hosts: web
         remote_user: ansible
         become: yes
         roles:
           - common
           - ldap_client
           - mail_forwarder
           - web_server
           - database_server
           - tbg
 . Apply the changes::
      workon mysite && ansible-playbook playbooks/site.yml
 . At this point *The Bug Genie* has been installed, and you should be able to
    open the URL https://tbg.example.com/ (if you open http://tbg.example.com/ ,
    you will be redirected to the HTTPS URL) and log-in into *The Bug Genie*
    with username ``administrator`` and password ``admin``.
 Deploying a WSGI application (Django Wiki)
 ------------------------------------------
 Next thing up will be to deploy a WSGI Python application.
 Similar to the PHP application deployment, we will use a couple of roles to make
 it easier to deploy it in a standardised manner, and we will not have any kind
 of parameters for configuring the role to keep things simple.
 Most of the notes on how a ``php_website`` role is deployed also stand for the
 ``wsgi_website`` role, but we will reiterate and clarify them a bit just to be
 on the safe side:
 * The role is designed to execute every application via dedicated user and
   group. The user/group name is automatically derived from the FQDN of website,
   for example ``web-wiki_example_com``.
 * While running the application, application user's umask is set to ``0007``
   (letting the administrator user be able to manage any files created while the
   application is running).
 * An administrative user is created as well, and this user should be used when
   running maintenance and installation commands. Similar to application user,
   the name is also derived from the FQDN of website, for example
   ``admin-wiki_example_com``. Administrative user does not have a dedicated
   group, and instead belongs to same group as the application user. As
   convenience, whenever you switch to this user the Python virtual environment
   will be automatically activated for you.
 * WSGI applications are executed via *Gunicorn*. The WSGI server listens on a
   Unix socket, making the socket accessible by *Nginx*.
 * If you ever need to set some environment variables, this can easily be done
   via the ``environment_variables`` role parameter. This particular example does
   not set any, though.
 * You can also specify headers to be passed on via Nginx ``proxy_set_header``
   directive to Gunicorn running the application.
 * Mails deliverd to local admin/application users are forwarded to ``root``
   account instead (this can be configured via ``website_mail_recipients`` role
   parameter.
 * If you ever find yourself mixing-up test and production websites, have a look
   at ``environment_indicator`` role parameter. It lets you insert small strip at
   bottom of each HTML page automatically.
 * Static content is served directly by *Nginx*.
 * Each web application gets distinct sub-directory under ``/var/www``, named
   after the FQDN. All sub-directories created under there are created with
   ``2750`` permissions, with ownership set to admin user, and group set to the
   application's group. In other words, all directories will have ``SGID`` bit
   set-up, allowing you to create files/directories that will have their group
   automatically set to the group of the parent directory.
 * Each WSGI website gets a dedicated virtual environment, stored in the
   sub-directory ``virtualenv`` of the website directory, for example
   ``/var/www/wiki.example.com/virtualenv``.
 * Static files are served from sub-directory ``htdocs`` in the website
   directory, for example ``/var/www/wiki.example.com/htdocs/``.
 * The base directory where your website/application code should be at is
   expected to be in sub-directory ``code`` in the website directory, for example
   ``/var/www/wiki.example.com/code/``.
 * Combination of admin user membership in application group, ``SGID``
   permission, and the way ownership of sub-directories is set-up usually means
   that the administrator will be capable of managing application files, and
   application can be granted write permissions to a *minimum* of necessary
   files.
   .. warning::
      Just keep in mind that some file-management commands, like ``mv``, do *not*
      respect the ``SGID`` bit. In fact, I would recommend using ``cp`` when you
      deploy new files to the directory instead (don't simply move them from your
      home).
 . Set-up the necessary directories first::
      mkdir -p ~/mysite/roles/wiki/{tasks,meta,files}/
 . Set-up some role dependencies, reusing the common role infrastructure.
    :file:`~/mysite/roles/wiki/meta/main.yml`
    ::
       ---
       dependencies:
          - role: wsgi_website
            fqdn: wiki.example.com
            # In many cases you need to have some development packages available
            # in order to build Python packages installed via pip
            packages:
               - build-essential
               - python-dev
               - libjpeg-dev
               - libzip-dev
               - libtiff-dev
               - libfreetype6-dev
               - liblcms2-dev
               - libwebp-dev
               - libopenjpeg-dev
               - libmariadb-client-lgpl-dev
               - libmariadb-client-lgpl-dev-compat
            # Here we specify that anything accessing our website with "/static/"
            # URL should be treated as request to a static file, to be served
            # directly by Nginx instead of the WSGI server.
            static_locations:
               - /static/
            # Again, not mandatory, but it is good to have some sort of policy
            # for assigning UIDs.
            uid: 2001
            admin_uid: 3001
            # These are additional packages that should be installed in the
            # virtual environment.
            virtualenv_packages:
              - pillow
              - django==1.8.13
              - wiki
              - MySQL-python
            # This is the name of the WSGI application to
            # serve. wiki_example_com.wsgi will be the Python "module" that is
            # accesed, while application is the object instantiated within it (the
            # application itself). The module is referenced relative to the code
            # directory (in our case /var/www/wiki.example.com/code/).
            wsgi_application: wiki_example_com.wsgi:application
          - role: database
            db_name: wiki
            db_password: wiki
 . Let's create a dedicated private key/certificate pair for the wiki website:
 . Create new template for ``certtool``:
       :file:`~/mysite/tls/wiki.example.com_https.cfg`
       ::
          organization = "Example Inc."
          country = SE
          cn = "Exampe Inc. Wiki"
          expiration_days = 365
          dns_name = "wiki.example.com"
          tls_www_server
          signing_key
          encryption_key
 . Create the keys and certificates for the application::
         certtool --sec-param normal --generate-privkey --outfile ~/mysite/tls/wiki.example.com_https.key
         certtool --generate-certificate --load-ca-privkey ~/mysite/tls/ca.key --load-ca-certificate ~/mysite/tls/ca.pem --template ~/mysite/tls/wiki.example.com_https.cfg --load-privkey ~/mysite/tls/wiki.example.com_https.key --outfile ~/mysite/tls/wiki.example.com_https.pem
 . At this point we have exhausted what we can do with the built-in roles. Time
    to add some custom tasks.
    :file:`~/mysite/roles/wiki/tasks/main.yml`
    ::
       ---
       - name: Create Django project directory
         file: dest="/var/www/wiki.example.com/code" state=directory
               owner=admin-wiki_example_com group=web-wiki_example_com
               mode=2750
       - name: Start Django project for the Wiki website
         command: /var/www/wiki.example.com/virtualenv/bin/exec django-admin.py startproject wiki_example_com /var/www/wiki.example.com/code
                  chdir=/var/www/wiki.example.com
                  creates=/var/www/wiki.example.com/code/wiki_example_com
         become: yes
         become_user: admin-wiki_example_com
       - name: Deploy settings for wiki website
         copy: src="{{ item }}" dest="/var/www/wiki.example.com/code/wiki_example_com/{{ item }}"
               mode=640 owner=admin group=web-wiki_example_com
         with_items:
            - settings.py
            - urls.py
         notify:
            - Restart website wiki.example.com
       - name: Deploy project database and deploy static files
         django_manage: command="{{ item }}"
                        app_path="/var/www/wiki.example.com/code/"
                        virtualenv="/var/www/wiki.example.com/virtualenv/"
         become: yes
         become_user: admin-wiki_example_com
         with_items:
            - syncdb
            - migrate
            - collectstatic
       - name: Deploy the superadmin creation script
         copy: src="create_superadmin.py" dest="/var/www/wiki.example.com/code/create_superadmin.py"
               owner=admin-wiki_example_com group=web-wiki_example_com mode=750
       - name: Create initial superuser
         command: /var/www/wiki.example.com/virtualenv/bin/exec ./create_superadmin.py
                  chdir=/var/www/wiki.example.com/code/
         become: yes
         become_user: admin-wiki_example_com
         register: wiki_superuser
         changed_when: wiki_superuser.stdout == "Created superuser."
 . There is a couple of files that we are deploying through the above
    tasks. Let's create them as well.
    :file:`~/mysite/roles/wiki/files/settings.py`
    ::
       """
       Django settings for wiki_example_com project.
       For more information on this file, see
       https://docs.djangoproject.com/en/1.6/topics/settings/
       For the full list of settings and their values, see
       https://docs.djangoproject.com/en/1.6/ref/settings/
       """
       # Build paths inside the project like this: os.path.join(BASE_DIR, ...)
       import os
       BASE_DIR = os.path.dirname(os.path.dirname(__file__))
       # Quick-start development settings - unsuitable for production
       # See https://docs.djangoproject.com/en/1.6/howto/deployment/checklist/
       # SECURITY WARNING: keep the secret key used in production secret!
       SECRET_KEY = 'l^q+t$7h$ebls)v34+w9m9v4$n+^(9guxqntu&#cc4m&lfd-6_'
       # SECURITY WARNING: don't run with debug turned on in production!
       DEBUG = False
       TEMPLATE_DEBUG = False
       ALLOWED_HOSTS = ["wiki.example.com", "localhost"]
       # Application definition
       INSTALLED_APPS = (
           'django.contrib.admin',
           'django.contrib.auth',
           'django.contrib.contenttypes',
           'django.contrib.sessions',
           'django.contrib.messages',
           'django.contrib.staticfiles',
           'django.contrib.sites',
           'django.contrib.humanize',
           'django_nyt',
           'mptt',
           'sekizai',
           'sorl.thumbnail',
           'wiki',
           'wiki.plugins.attachments',
           'wiki.plugins.notifications',
           'wiki.plugins.images',
           'wiki.plugins.macros',
+      )
       MIDDLEWARE_CLASSES = (
           'django.contrib.sessions.middleware.SessionMiddleware',
           'django.middleware.common.CommonMiddleware',
           'django.middleware.csrf.CsrfViewMiddleware',
           'django.contrib.auth.middleware.AuthenticationMiddleware',
           'django.contrib.messages.middleware.MessageMiddleware',
           'django.middleware.clickjacking.XFrameOptionsMiddleware',
+      )
       ROOT_URLCONF = 'wiki_example_com.urls'
       WSGI_APPLICATION = 'wiki_example_com.wsgi.application'
       # Database
       # https://docs.djangoproject.com/en/1.6/ref/settings/#databases
       DATABASES = {
           'default': {
               'ENGINE': 'django.db.backends.mysql',
               'NAME': 'wiki',
               'USER': 'wiki',
               'PASSWORD': 'wiki',
               'HOST': '127.0.0.1',
               'PORT': '3306',
+          }
+      }
       # Internationalization
       # https://docs.djangoproject.com/en/1.6/topics/i18n/
       LANGUAGE_CODE = 'en-us'
       TIME_ZONE = 'Europe/Stockholm'
       USE_I18N = True
       USE_L10N = True
       USE_TZ = True
       # Static files (CSS, JavaScript, Images)
       # https://docs.djangoproject.com/en/1.6/howto/static-files/
       STATIC_URL = '/static/'
       STATIC_ROOT = '/var/www/wiki.example.com/htdocs/static'
       from django.conf import settings
       TEMPLATE_CONTEXT_PROCESSORS = settings.TEMPLATE_CONTEXT_PROCESSORS + (
           "django.core.context_processors.debug",
           "django.core.context_processors.request",
           "sekizai.context_processors.sekizai",
+      )
       SITE_ID=1
    :file:`~/mysite/roles/wiki/files/urls.py`
    ::
       from django.conf.urls import patterns, include, url
       from wiki.urls import get_pattern as get_wiki_pattern
       from django_nyt.urls import get_pattern as get_nyt_pattern
       from django.contrib import admin
       admin.autodiscover()
       urlpatterns = patterns('',
           # Examples:
           # url(r'^$', 'wiki_example_com.views.home', name='home'),
           # url(r'^blog/', include('blog.urls')),
           url(r'^admin/', include(admin.site.urls)),
+      )
       urlpatterns += patterns('',
           (r'^notifications/', get_nyt_pattern()),
           (r'', get_wiki_pattern())
+      )
    :file:`~/mysite/roles/wiki/files/create_superadmin.py`
    ::
       #!/usr/bin/env python
       import os
       from django import setup
       os.environ['DJANGO_SETTINGS_MODULE']='wiki_example_com.settings'
       setup()
       from django.conf import settings
       from django.contrib.auth.models import User
       User.objects.all()
       if len(User.objects.filter(username="admin")) == 0:
           User.objects.create_superuser('admin', 'john.doe@example.com', 'admin')
           print("Created superuser.")
 . Time to add the new role to our web server.
    :file:`~/mysite/playbooks/web.yml`
    ::
       ---
       - hosts: web
         remote_user: ansible
         become: yes
         roles:
           - common
           - ldap_client
           - mail_forwarder
           - web_server
           - database_server
           - tbg
           - wiki
 . Apply the changes:
    .. warning::
       Due to `Debian Bug 766996
       <https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=766996>`_ Majic Ansible
       Roles try to detect if you install ``libmariadb-client-lgpl-dev-compat``
       package, and create symbolic link from ``/usr/local/bin/mysql_config`` to
       ``/usr/bin/mariadb_config`` automatically. Otherwise the MySQL-python
       package will fail to build due to missing ``mysql_config`` binary.
    ::
      workon mysite && ansible-playbook playbooks/site.yml
 . At this point Django Wiki has been installed, and you should be able to open
    the URL https://wiki.example.com/ (if you open http://wiki.example.com/ , you
    will be redirected to the HTTPS URL) and log-in into *Django Wiki* with
    username ``admin`` and password ``admin``.
 Backups, backups, backups!
 --------------------------
 As it is well known, everyone has backups of their important data. Right?
 Riiiiight?
 There are three Ansible roles that implement backup functionality -
 ``backup_server``, ``backup_client``, and ``backup``. Backup is based around the
 use of `Duplicity <http://duplicity.nongnu.org/>`_ and its convenience wrapper,
 `Duply <http://duply.net>`_. Due to this selection, it should be noted that the
 backup clients are the ones making connection to the backup server (not the
 other way around).
 Backups are encrypted and signed using GnuPG before being stored on the backup
 server. Private key used for encryption and signing is therefore stored on the
 client side. This key should not be encrypted in order to allow for unattended
 backups.
 Although not necessary, it is highly recommended to have separation between
 different backup clients and the keys used for encryption and
 signing. I.e. stick to a separate encryption/signing key for each backup
 client. It should be noted that it is also possible to specify additional
 *public* keys to encrypt with. This lets you have a backup decryptable with some
 other, "master" key.
 The backups are transferred to the backup server via SFTP - the
 ``backup_server`` role sets-up a dedicated OpenSSH server instance that limits
 the connecting clients to a SFTP chroot.

roles/wsgi_website/defaults/main.yml

➞

Show inline comments

 ---
 additional_nginx_config: {}
 enforce_https: True
 packages: []
 rewrites: []
 static_locations: []
 use_paste: False
 virtualenv_packages: []
 environment_variables: {}
 admin: "web-{{ fqdn | replace('.', '_') }}"
 https_tls_certificate: "{{ lookup('file', tls_certificate_dir + '/' + fqdn + '_https.pem') }}"
 https_tls_key: "{{ lookup('file', tls_private_key_dir + '/' + fqdn + '_https.key') }}"
 gunicorn_version: "19.6.0"
 futures_version: "3.0.5"
 website_mail_recipients: "root"
 environment_indicator: null
@@ \ No newline at end of file @@
 environment_indicator: null
 proxy_headers: {}

roles/wsgi_website/templates/nginx_site.j2

➞

Show inline comments

 {% if enforce_https -%}
 server {
     # HTTP (plaintext) configuration.
     listen 80;
     server_name {{ fqdn }};
     # Redirect plaintext connections to HTTPS
     return 301 https://$host$request_uri;
+}
 {% endif -%}
 server {
     # Base settings.
     root {{ home }}/htdocs/;
     server_name {{ fqdn }};
 {% if not enforce_https %}
     # HTTP (plaintext) configuration.
     listen 80;
 {% endif %}
     # HTTPS (TLS) configuration.
     listen 443 ssl;
     listen [::]:443 ssl;
     ssl_certificate_key /etc/ssl/private/{{ fqdn }}_https.key;
     ssl_certificate /etc/ssl/certs/{{ fqdn }}_https.pem;
 {% if default_enforce_https -%}
     # Set-up HSTS header for preventing downgrades for users that visited the
     # site via HTTPS at least once.
     add_header Strict-Transport-Security "max-age=31536000; includeSubDomains";
 {% endif -%}
     {% for config in additional_nginx_config -%}
     # {{ config.comment }}
     {{ config.value }}
     {% endfor -%}
     {% if rewrites -%}
     # Site rewrites.
     {% for rewrite in rewrites -%}
     rewrite {{ rewrite }};
     {% endfor -%}
     {% endif %}
     {% if static_locations -%}
     # Static locations
     {% for location in static_locations -%}
     location {{ location }} {
         try_files $uri $uri/ =404;
+    }
     {% endfor -%}
     {% endif %}
     # Pass remaining requests to the WSGI server.
     location / {
         try_files $uri @proxy_to_app;
+    }
     location @proxy_to_app {
         proxy_set_header X-Forwarded-Proto $scheme;
         proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
         proxy_set_header Host $http_host;
         proxy_redirect off;
     {% for header, value in proxy_headers.iteritems() -%}
     proxy_set_header {{ header }} {{ value }};
     {% endfor -%}
         proxy_pass http://unix:/run/wsgi/{{ fqdn }}.sock;
+    }
     {% if environment_indicator -%}
     # Show environment indicator on HTML pages.
     sub_filter_types text/html;
     sub_filter_once on;
     sub_filter "</body>" "<div id='website-environment' style='background-color: {{ environment_indicator.background_colour }}; width: 100%; text-align: center; position: fixed; bottom: 5px; color: {{ environment_indicator.text_colour }}; font-weight: bold;'>{{ environment_indicator.text }}</div></body>";
     {% endif -%}
     access_log /var/log/nginx/{{ fqdn }}-access.log;
     error_log /var/log/nginx/{{ fqdn }}-error.log;
+}

testsite/group_vars/web.yml

➞

Show inline comments

 ---
 local_mail_aliases:
   root: "root john.doe@{{ testsite_domain }}"
 smtp_relay_host: mail.{{ testsite_domain }}
 smtp_relay_truststore: "{{ lookup('file', inventory_dir + '/tls/ca.pem') }}"
 default_https_tls_key: "{{ lookup('file', inventory_dir + '/tls/web.' + testsite_domain + '_https.key') }}"
 default_https_tls_certificate: "{{ lookup('file', inventory_dir + '/tls/web.' + testsite_domain + '_https.pem') }}"
 web_default_title: "Welcome to Example Inc."
 web_default_message: "You are attempting to access the web server using a wrong name or an IP address. Please check your URL."
 db_root_password: "root"
 website_mail_recipients: "john.doe@example.com"
 environment_indicator:
   background_colour: "purple"
   text_colour: "white"
   text: "Majic Ansible Roles Test Site"
@@ \ No newline at end of file @@
   text: "Majic Ansible Roles Test Site"
 proxy_headers:
   Accept-Encoding: '"gzip"'

testsite/playbooks/roles/wsgihello/files/hello.wsgi

➞

Show inline comments

 #!/usr/bin/env python
 import os
 def application(environ, start_response):
     status = '200 OK'
     template = """<!DOCTYPE html>
 <html lang="en">
   <head>
     <meta charset="utf-8">
     <title>{title}</title>
   </head>
   <body>
     <h1>Hello, world!</h1>
     <p>I am website {title}</p>
     <p>Accept-Encoding header was set to {acceptencoding}</p>
   </body>
 </html>
 """
     output = template.format(title=os.environ.get("WEBSITE_NAME", "that nobody set a name for :("))
     output = template.format(title=os.environ.get("WEBSITE_NAME", "that nobody set a name for :("),
                              acceptencoding=environ.get("HTTP_ACCEPT_ENCODING"))
     response_headers = [('Content-type', 'text/html'),
                         ('Content-Length', str(len(output)))]
     start_response(status, response_headers)
     return [output]

0 comments (0 inline, 0 general)