*.pyc

*~

tmp/

docs/_build/

testsite/preseed_files/

# Ignore "temporary" files created with the playbook tls.yml (certs, keys, and

# host config files for GnuTLS - ca.cfg is versioned, though).

testsite/tls/*.pem

testsite/tls/*.key

testsite/tls/*.*_*.cfg

**https_tls_certificate** (string, optional, ``{{ tls_certificate_dir }}/{{ ansible_fqdn }}_https.pem``)

  Path to file on Ansible host that contains the X.509 certificate used for TLS

  for HTTPS service. The file will be copied to directory ``/etc/ssl/certs/``.

**web_default_title** (string, optional, ``Welcome``)

  Title for the default web page shown to users (if no other vhosts were matched).

**web_default_message** (string, optional, ``You are attempting to access the web server using a wrong name or an IP address. Please check your URL.``)

  Message for the default web page shown to users (if no other vhosts were

  matched).

Examples

~~~~~~~~

Here is an example configuration for setting-up web server:

.. code-block:: yaml

  ---

  https_tls_key: "{{ inventory_dir }}/tls/web.example.com_https.key"

  https_tls_certificate: "{{ inventory_dir }}/tls/web.example.com_https.pem"

  web_default_title: "Welcome to Example Inc."

  web_default_message: "You are attempting to access the web server using a wrong name or an IP address. Please check your URL."

PHP Website

-----------

The ``php_website`` role can be used for setting-up a website powered by PHP on

destination machine.

This role is normally not supposed to be used directly, but should instead serve

as the basis for writing website-specific roles. Therefore the role is written

in quite generic way, allowing the integrator to write his/her own logic for

deploying the necessary PHP applications, while still reusing a common base and

reducing the workload.

The role implements the following:

* Creates a dedicated user/group for running the PHP scripts.

* Creates a base directory where the website-specific code and data should be

  stored at.

* Adds nginx to website's group, so nginx could read the necessary files.

* Adds website administrator to website's group, so administrator could manage

  the code and data.

* Installs additional packages required for running the role (as configured).

* Deploys the HTTPS TLS private key and certificate (for website vhost).

* Configures PHP FPM and nginx to serve the website.

The role is implemented with the following layout/logic in mind:

* Website users are named after the ``FQDN`` (fully qualified domain name) of

  website, in format of ``web-ESCAPEDFQDN``, where ``ESCAPEDFQDN`` is equal to

  ``FQDN`` where dots have been replaced by underscores (for example,

  ``web-cloud_example_com``).

* All websites reside within a dedicated sub-directory in ``/var/www``. The

  sub-directory name is equal to the ``FQDN`` used for accessing the

  website. Owner of the directory is set to be the application administrator,

  while group is set to be the website group. Additionally, ``SGID`` bit is set

  on the directory. This allows admin, with correct umask, to create necessary

  files and directories that should be readable (and eventually writeable) by

  the website user (running the PHP scripts) without having to become root.

* All files placed in the website directory should be either created there

  directly, or copied to the directory in order to make sure the ``SGID`` gets

  honored. **Do not move the files, the permissions will not be set correctly.**

* Within the website directory, nginx/php5-fpm will expect to find the relevant

  files within the htdocs sub-directory (this can be symlink too).

* nginx communicates with PHP FPM over a dedicated Unix socket for each website.

Parameters

~~~~~~~~~~

**admin** (string, optional, ``web-{{ fqdn | replace('.', '_') }}``)

  Name of the operating system user in charge of maintaining the website. This

  user is capable of making modifications to website configuration and data

  stored within the website directory.

**deny_files_regex** (list, optional, ``[]``)

  List of regular expressions for matching files/locations to which the web

  server should deny access. This is useful to block access to any sensitive

  files that should not be served directly by the web server. The format must be

  compatible with regular expressions used by ``nginx`` for ``location ~``

  syntax.

**fqdn** (string, mandatory)

  Fully-qualified domain name where the website is reachable. This value is used

  for calculating the user/group name for dedicated website user, as well as

  home directory of the website user (where data/code should be stored at).

**index** (string, optional, ``index.php``)

  Space-separated list of files which should be treated as index files by the

  web server. The web server will attempt opening these index files, in

  succession, until the first match, or until it runs out of matches, when a

  client requests an URI pointing to directory.

**https_tls_certificate** (string, optional, ``{{ tls_certificate_dir }}/{{ fqdn }}_https.pem``)

  Path to file on Ansible host that contains the X.509 certificate used for TLS

  for HTTPS service. The file will be copied to directory ``/etc/ssl/certs/``.

**https_tls_key** (string, optional, ``{{ tls_private_key_dir }}/{{ fqdn }}_https.key``)

  Path to file on Ansible host that contains the private key used for TLS for

  HTTPS service. The file will be copied to directory ``/etc/ssl/private/``.

**php_file_regex** (string, optional, ``\.php$``)

  Regular expression used for determining which file should be interepted via

  PHP.

**php_rewrite_urls** (list, optional, ``[]``)

  A list of rewrite rules that are applied to incoming requests. These rewrite

  rules are specifically targetted at prettying-up the URLs used by the PHP

  scripts. Each element of the list should be a string value compatible with the

  format of ``nginx`` option ``rewrite``. The keyword ``rewrite`` itself should

  be omitted, as well as trailing semi-colon (``;``).

**rewrites** (list, optional, ``[]``)

  A list of rewrite rules that are applied to incoming requests. Each element of

  the list should be a string value compatible with the format of ``nginx``

  option ``rewrite``. The keyword ``rewrite`` itself should be omitted, as well

  as trailing semi-colon (``;``).

**packages** (list, optional, ``[]``)

  A list of additional packages to install for this particular PHP

  appliction. This is usually going to be different PHP extensions.

**uid** (integer, optional, ``whatever OS picks``)

  UID/GID (they are set-up to be the same) of the dedicated website

  user/group.

Examples

~~~~~~~~

Here is an example configuration for setting-up two (base) PHP websites (for

running *ownCloud* and *The Bug Genie* applications):

.. code-block:: yaml

    - role: php_website

      fqdn: cloud.example.com

      uid: 2001

      admin: admin

      php_file_regex: \.php($|/)

      rewrites:

        - ^/\.well-known/host-meta /public.php?service=host-meta

        - ^/\.well-known/host-meta\.json /public.php?service=host-meta-json

        - ^/\.well-known/carddav /remote.php/carddav/ redirect

        - ^/\.well-known/caldav /remote.php/caldav/ redirect

        - ^/apps/calendar/caldav\.php /remote.php/caldav/

        - ^/apps/contacts/carddav\.php /remote.php/carddav/

        - ^/remote/(.*) /remote.php

      deny_files_regex:

        - ^(\.|autotest|occ|issue|indie|db_|console|build/|tests/|config/|lib/|3rdparty/|templates/).*

      packages:

        # For ownCloud

        - php5-gd

        - php5-json

        - php5-mysql

        - php5-curl

      https_tls_key: "{{ inventory_dir }}/tls/cloud.example.com_https.key"

      https_tls_certificate: "{{ inventory_dir }}/tls/cloud.example.com_https.pem"

    - role: php_website

      admin: admin

      deny_files_regex:

        - ^\..*

      php_rewrite_urls:

        - ^(.*) /index.php?url=$1

      fqdn: tbg.example.com

      uid: 2007

      https_tls_key: "{{ inventory_dir }}/tls/tbg.example.com_https.key"

      https_tls_certificate: "{{ inventory_dir }}/tls/tbg.example.com_https.pem"

WSGI Website

------------

The ``wsgi_website`` role can be used for setting-up a website powered by Python

on destination machine. The website needs to use the WSGI specification for

making the Python web application(s) available.

This role is normally not supposed to be used directly, but should instead serve

as the basis for writing website-specific roles. Therefore the role is written

in quite generic way, allowing the integrator to write his/her own logic for

deploying the necessary Python applications/packages, while still reusing a

common base and reducing the workload.

The role implements the following:

* Creates a dedicated user/group for running the WSGI application.

* Creates a base directory where the website-specific code and data should be

  stored at.

* Adds nginx to website's group, so nginx could read the necessary files.

* Adds website administrator to website's group, so administrator could manage

  the code and data.

* Installs additional packages required for running the role (as configured).

* Sets-up a dedicated Python virtual environment for website.

* Install Gunicorn in Python virtual environment.

* Installs additional packages required for running the role in Python virtual

  environment (as configured).

* Configures systemd to run the website code (using Gunicorn)

* Deploys the HTTPS TLS private key and certificate (for website vhost).

* Configures nginx to serve the website (static files served directly, requests

  passed on to Gunicorn).

The role is implemented with the following layout/logic in mind:

* Website users are named after the ``FQDN`` (fully qualified domain name) of

  website, in format of ``web-ESCAPEDFQDN``, where ``ESCAPEDFQDN`` is equal to

  ``FQDN`` where dots have been replaced by underscores (for example,

  ``web-wiki_example_com``).

* All websites reside within a dedicated sub-directory in ``/var/www``. The

  sub-directory name is equal to the ``FQDN`` used for accessing the

  website. Owner of the directory is set to be the application administrator,

  while group is set to be the website group. Additionally, ``SGID`` bit is set

  on the directory. This allows admin, with correct umask, to create necessary

  files and directories that should be readable (and eventually writeable) by

  the website user (running the WSGI application) without having to become root.

* All files placed in the website directory should be either created there

  directly, or copied to the directory in order to make sure the ``SGID`` gets

  honored. **Do not move the files, the permissions will not be set correctly.**

* Within the website directory, Python virtual environment can be found within

  the ``virtualenv`` sub-directory. The virtual environment is also symlinked to

  website admin's ``~/.virtualenvs/`` directory for easier access (and

  auto-completion with virtualenvwrapper).

* Within the website directory, nginx will expect to find the static files

  within the ``htdocs`` sub-directory (this can be symlink too). Locations/aliases

  can be configured for static file serving.

* Within the website directory, systemd service will expect to find the website

  code within the ``code`` sub-directory (this can be symlink too).

* nginx communicates with WSGI server over a dedicated Unix socket for each

  website.

Parameters

~~~~~~~~~~

**admin** (string, optional, ``web-{{ fqdn | replace('.', '_') }}``)

  Name of the operating system user in charge of maintaining the website. This

  user is capable of making modifications to website configuration anda data

  stored within the website directory.

**fqdn** (string, mandatory)

  Fully-qualified domain name where the website is reachable. This value is used

  for calculating the user/group name for dedicated website user, as well as

  home directory of the website user (where data/code should be stored at).

**https_tls_certificate** (string, optional, ``{{ tls_certificate_dir }}/{{ fqdn }}_https.pem``)

  Path to file on Ansible host that contains the X.509 certificate used for TLS

  for HTTPS service. The file will be copied to directory ``/etc/ssl/certs/``.

**https_tls_key** (string, optional, ``{{ tls_private_key_dir }}/{{ fqdn }}_https.key``)

  Path to file on Ansible host that contains the private key used for TLS for

  HTTPS service. The file will be copied to directory ``/etc/ssl/private/``.

**packages** (list, optional, ``[]``)

  A list of additional packages to install for this particular WSGI

  website. This is usually going to be development libraries for building Python

  packages.

**rewrites** (list, optional, ``[]``)

  A list of rewrite rules that are applied to incoming requests. Each element of

  the list should be a string value compatible with the format of ``nginx``

  option ``rewrite``. The keyword ``rewrite`` itself should be omitted, as well

  as trailing semi-colon (``;``).

**static_locations** (list, optional, ``[]``)

  List of locations that should be treated as static-only, and not processed by

  the WSGI application at all. This is normally used for designating serving of

  static/media files by Nginx (for example, in case of Django projects for

  ``/static/`` and ``/media/``).

**uid** (integer, optional, ``whatever OS picks``)

  UID/GID (they are set-up to be the same) of the dedicated website

  user/group.

**use_paste** (boolean, optional, ``False``)

  Tell Gunicorn to assume that the passed-in ``wsgi_application`` value is a

  filename of a Python Paste ``ini`` file instead of WSGI application.

**virtuaelnv_packages** (list, optional, ``[]``)

  A list of additional packages to install for this particular WSGI appliction

  in its virtual environment using ``pip``.

**wsgi_application** (string, mandatory)

  WSGI application that should be started by Gunicorn. The format should be

  conformant to what the ``gunicorn`` command-line tool accepts. If the

  ``use_paste`` option is enabled, the value should be equal to filename of the

  Python Paste ini file, located in the ``code`` sub-directory.

Examples

~~~~~~~~

Here is an example configuration for setting-up a (base) WSGI website (for

running a bare Django project):

.. code-block:: yaml

    - role: wsgi_website

      admin: admin

      fqdn: django.example.com

      static_locations:

        - /static

        - /media

      uid: 2004

      virtualenv_packages:

        - django

      wsgi_application: django_example_com.wsgi:application

      https_tls_key: "{{ inventory_dir }}/tls/wsgi.example.com_https.key"

      https_tls_certificate: "{{ inventory_dir }}/tls/wsgi.example.com_https.pem"

Database Server

---------------

The ``database_server`` role can be used for setting-up a MariaDB database

server on destination machine.

The role implements the following:

* Installs MariaDB server and client.

* Configures MariaDB server and client to use *UTF-8* encoding by default.

* Sets password for the database root user.

* Deploys MariaDB client configuration in location ``/root/.my.cnf`` that

  contains username and password for the root database user.

Parameters

~~~~~~~~~~

**db_root_password** (string, mandatory)

  Password for the *root* database user.

Examples

~~~~~~~~

Here is an example configuration for setting-up the database server:

.. code-block:: yaml

   ---

   db_root_password: root

Database

--------

The ``database`` role can be used for creating a MariaDB database and

accompanying user on destination machine.

The role implements the following:

* Creates MariaDB database.

* Creates a dedicated user capable of performing any operation on the created

  database. Username is set to be same as the name of database.

Parameters

~~~~~~~~~~

**db_name** (string, mandatory)

  Name of the database that should be created.

**db_password** (string, mandatory)

  Password for the database user.

Examples

~~~~~~~~

Here is an example configuration for creating a single database (for some

website):

.. code-block:: yaml

  - role: database

    db_name: phpinfo_example_com

    db_password: phpinfo_example_com

.. _testsite:

Test Site

=========

*Majic Ansible Roles* comes with a small sample test site configuration which

demonstrates use of every role. This test site also serves as starting point for

developing new roles etc, and can be used for testing regressions/breakages.

The test site covers everything, starting from generating the Debian preseed

files, through bootstrap process for new nodes, and onto deployment of all

remaining roles.

By default, the test site uses domain ``example.com``, but it has been designed

so it is easy to set your own domain (see below in step-by-step

instructions). Some changes may be necessary to listed commands in that case

(i.e. replace every occurance of ``example.com`` with your own domain).

All example commands listed within this section should be ran from within the

``testsite`` directory in order to have proper environment available for

playbook runs.

A number of playbooks is provided out of the box:

bootstrap.yml (for bootstrapping fresh nodes)

  This playbook can be used for bootstrapping fresh nodes. By default, the

  entire test site will be included in the bootstrap. If you wish to limit

  bootstrap to a single server, just run the playbook with (for example):

  .. code-block:: shell

    ansible-playbook -l ldap.example.com playbooks/bootstrap.yml

ldap.yml

  This playbook sets-up the LDAP servers. It is included in ``site.yml``.

mail.yml

  This playbook sets-up the mail server. It is included in ``site.yml``.

preseed.yml

  This playbook sets-up the Debian preseed files. It is included in

  ``site.yml``.

site.yml

  This playbook sets-up all servers, including preseed files on local host.

web.yml

  This playbook sets-up the web server. It is included in ``site.yml``.

xmpp.yml

  This playbook sets-up the XMPP server. It is included in ``site.yml``.

In order to deploy the test site, the following steps would normally be taken:

1. As mentioned in introduction, default domain used by test site is

   ``example.com``. To change it, perform the following steps (otherwise, just

   skip to step 2):

   a. Update the file ``hosts``. Simply replace all occurances of

      ``example.com`` with your chosen domain.

   b. Update the file ``group_vars/all.yml``, changing the value of variable

      ``testsite_domain``. This value will then be used to calculate some of

      derived values, like LDAP base DN (which will be set to something along

      the lines of ``dc=example,dc=com`` or

      ``dc=your,dc=domain,dc=components``).

2. If you do not wish to have the hassle of creating the private keys and

   issuing certificates, there is a small playbook that can help you with

   this. Just run the ``tls.yml`` playbook, and skip to step 6 (otherwise follow

   steps 3 through 5):

   .. code-block:: shell

     ansible-playbook playbooks/tls.yml

3. Create TLS private keys (relative to top level directory), making sure to

   change domain in filenames if necessary:

   - ``testsite/tls/mail.example.com_imap.key``

   - ``testsite/tls/mail.example.com_smtp.key``

   - ``testsite/tls/xmpp.example.com_xmpp.key``

   - ``testsite/tls/ldap.example.com_ldap.key``

   - ``testsite/tls/web.example.com_https.key``

   - ``testsite/tls/phpfino.example.com_https.key``

   - ``testsite/tls/wsgi.example.com_https.key``

4. Issue TLS certificates corresponding to the generated TLS private keys

   (correct FQDN for DNS subject alternative name **must** be used), making sure

   to change domain in filenames if necessary:

   - ``testsite/tls/mail.example.com_imap.pem`` (subject alternative name should

     be ``mail.example.com``)

   - ``testsite/tls/mail.example.com_smtp.pem`` (subject alternative name should

     be ``mail.example.com``)

   - ``testsite/tls/xmpp.example.com_xmpp.pem`` (subject alternative name should

     be ``xmpp.example.com``)

   - ``testsite/tls/ldap.example.com_ldap.pem`` (subject alternative name should

     be ``ldap.example.com``)

   - ``testsite/tls/web.example.com_https.pem`` (subject alternative name should

     be ``web.example.com``)

   - ``testsite/tls/web.example.com_https.pem`` (subject alternative name should

     be ``web.example.com``)

   - ``testsite/tls/phpinfo.example.com_https.pem`` (subject alternative name

     should be ``phpinfo.example.com``)

   - ``testsite/tls/wsgi.example.com_https.pem`` (subject alternative name

     should be ``wsgi.example.com``)

5. Create ``PEM`` truststore file which contains all CA certificates that form

   CA chain for the issued end entity certificates from previous step at

   location ``testsite/tls/ca.pem``. It is very important to

   include the full CA chain used for LDAP server.

  .. code-block:: shell

    ansible-playbook playbooks/preseed.yml

   remove old entries if you are redoing the process). You can easily obtain all

   the necessary fingerprints with command (don't forget to modify domain if you

   need to):

   .. code-block:: shell

      ssh-keyscan -t ed25519 mail.example.com ldap.example.com xmpp.example.com web.example.com $(resolveip -s mail.example.com) $(resolveip -s ldap.example.com) $(resolveip -s xmpp.example.com) $(resolveip -s web.example.com)

  .. code-block:: shell

    ansible-playbook playbooks/bootstrap.yml

  .. code-block:: shell

    ansible-playbook playbooks/site.yml

The playbooks and configurations for test site make a couple of assumptions:

* Each server will be set-up with an operating system user ``admin``, capable of

  running the sudo commands.

* The password for operating system user ``admin`` is hard-coded to ``admin``.

* An SSH ``authorized_keys`` file is set-up for the operating system user

  ``admin``. The SSH key stored in it will be read from location

  ``~/.ssh/id_rsa.pub`` (i.e. from home directory of user running the Ansible

  commands).

For more details on how the playbooks and configuration have been implemented,

feel free to browse the test site files (in directory ``testsite``).