.. _development:

Development

===========

This section covers procedures and information related to development of *Majic

Ansible Roles*.

Preparing environment

---------------------

The easiest way to get going with role development is to set-up a separate

Python virtual environment with the necessary packages. This can be done by

performing the following steps:

1. Ensure that the following minimum set of packages are installed via

   distribution package manager:

   - `Git <https://git-scm.com/>`_

   - `libffi <https://sourceware.org/libffi/>`_ runtime and development package.

   - `OpenSSL <https://www.openssl.org/>`_ runtime and development package.

   - `pip <https://pypi.python.org/pypi/pip/>`_

   - `virtualenv <https://pypi.python.org/pypi/virtualenv>`_

   - `virtualenvwrapper <https://virtualenvwrapper.readthedocs.io/en/latest/>`_

   - Development packages for Python.

   On Debian this can be easily done with::

     apt-get install virtualenv virtualenvwrapper git python-pip python-dev \

     libffi-dev libssl-dev

2. In order to be able to run role tests, it is necessary to install `VirtualBox

   <https://www.virtualbox.org/>`_ and `Vagrant <https://www.vagrantup.com/>`_,

   using instructions outlined on their respective websites. It is recommended

   to use latest versions available. At time of this writing the role tests have

   been successfully run on *VirtualBox 5.2.12* and *Vagrant 2.0.4*.

3. In order to allow static IPv6 addresses to be allocated to virtual

   machines during testing, it is necessary to explicitly white-list

   the range used by the tests. Once the configuration file has been

   created, however, even the VirtualBox default allowed IPv4 subnet

   needs to be in the configuration explicitly as well.

   Update the VirtulBox configuration file (and make sure it can be

   read by the user running the tests):

   :file:`/etc/vbox/networks.conf`

   ::

     * 192.168.56.0/21

     * fd00::192:168:56:0/116

4. Clone the git repository::

     git clone https://code.majic.rs/majic-ansible-roles/ ~/projects/

5. Create a separate Python virtual environment::

     mkvirtualenv majic-ansible-roles -a ~/projects/majic-ansible-roles/

6. Make sure the virtual environment has been activated, and install `pip-tools

   <https://github.com/jazzband/pip-tools>`_::

     workon majic-ansible-roles

     pip install pip-tools

7. Synchronise Python virtual environment with requirements file using

   **pip-tools**::

     workon majic-ansible-roles

     pip-sync

Development conventions

-----------------------

In order to maintain consistency across different roles and documentation, this

section describes development conventions that should be followed while making

modifications.

Task specifications

~~~~~~~~~~~~~~~~~~~

When writing new and updating existing tasks, keep the following in mind:

- Quote sensibly. If specifying paths (for example ``src``, ``dest``, ``path``

  etc in various models), quote the string to make it stand-out better and to

  avoid breakages.

- Avoid usage of ``set_facts`` task when same functionality can be achieved via

  ``defaults/main.yml``.

- When specifying tasks, use the fully expanded form. Do not use single-line

  form with ``param=value``.

- When specifying ``command`` or ``shell`` tasks, in case a ``creates``

  parameter or such need to be used, specify them as part of task's ``args``

  parameter. E.g.::

    - name: Run command

      command: mycommand

      args:

        creates: "/etc/mycommand"

- When sepcifying tasks, keep the following ordering between different task

  parameters:

  - ``name``

  - Module and its parameters.

  - ``become``

  - ``become_user``

  - ``when``

  - ``with_items`` / ``with_dict`` / ``with_nested``

  - ``wait_for``

  - ``register``

  - ``changed_when``

  - ``failed_when``

  - ``no_log``

  - ``notify``

  - Task tags.

Running role tests directly

---------------------------

Role tests are implemented using `Molecule <https://molecule.readthedocs.io/>`_,

`Testinfra <https://testinfra.readthedocs.io/en/latest/>`_, `VirtualBox

<https://www.virtualbox.org/>`_ and `Vagrant

<https://www.vagrantup.com/>`_. *Molecule* and *Testinfra* are installed inside

of Pyhton virtual environment, while *VirtualBox* and *Vagrant* need to be

installed distribution-wide, following instructions outlined on their

corresponding websites.

Tests can be run directly for a single role, or for one or more roles using a

dedicated shell script (see below). The shell script can also be used for

generating reports in an automated environment.

In order to run tests for a specific role, perform the following steps:

1. Switch to Python virtual environment::

     workon majic-ansible-roles

2. Change directory::

     cd roles/ROLENAME/

3. Run the default test scenario (this will normally test against

   multiple Debian versions if supported)::

     molecule test

Running role tests via shell script

-----------------------------------

In order to make it easier to run tests for all roles, and eventually produce

reports of such runs, a dedicated shell script is provided for running the

tests.

In order to run tests, perform the following steps:

1. Switch to Python virtual environment::

     workon majic-ansible-roles

2. Make sure you are within the root directory of Git repository.

3. Run tests for all roles and generate report::

     ./scripts/run_tests.sh -r all

   .. note::

      Optionally you can run tests for a specific set of roles, or without

      generating the report, for example ``./scripts/run_tests.sh web_server

      common``

4. Check results either from script output, or within directory

   ``test_report-YYYY_MM_DD-hh_mm_ss``. For overview of what roles have failed,

   have a look at ``summary.txt``. For details have a look at each role's

   individual report.

.. _testsite:

Test Site

---------

*Majic Ansible Roles* comes with a small sample test site configuration which

demonstrates use of every role. This test site also serves as starting point for

developing new roles etc, and can be used for testing regressions/breakages.

The test site covers everything, starting from generating the Debian preseed

files, through bootstrap process for new nodes, and onto deployment of all

remaining roles.

By default, the test site uses domain ``example.com``, but it has been designed

so it is easy to set your own domain (see below in step-by-step

instructions). Some changes may be necessary to listed commands in that case

(i.e. replace every occurance of ``example.com`` with your own domain).

By default, the following hosts are used in test site (listed names

should be resolvable towards the servers):

- ``ldap.example.com``, acting as LDAP server

- ``xmpp.example.com``, acting as XMPP server

- ``mail.example.com``, acting as mail server

- ``web.example.com``, acting as web server

- ``backup.example.com``, acting as backup server

- ``ws01.example.com``, acting as "workstation" (mainly demonstrating

  use of mail forwarder role)

The web server is also set-up with one sample PHP application and two

WSGI applications deployed via roles:

- ``phpinfo``, used to serve information about the local PHP

  installation.

- ``wsgihello``, small hello world application.

- ``wsgihello2``, small hello world application using a couple of

  additional ``wsgi_website`` role parameters.

To make the web applications reachable, make sure the web server

(``web.example.com``) IP is also resolvable via names:

- ``phpinfo.example.com`` (``phpinfo`` application/role)

- ``wsgi.example.com`` (``wsgihello`` application/role)

- ``wsgireq.example.com`` (``wsgihello2`` application/role)

All example commands listed within this section should be ran from within the

``testsite`` directory in order to have proper environment available for

playbook runs.

A number of playbooks are provided out of the box:

bootstrap.yml (for bootstrapping fresh nodes)

  This playbook can be used for bootstrapping fresh nodes. By default, the

  entire test site will be included in the bootstrap. If you wish to limit

  bootstrap to a single server, just run the playbook with (for example):

  .. code-block:: shell

    ansible-playbook -l ldap.example.com playbooks/bootstrap.yml

ldap.yml

  This playbook sets-up the LDAP servers. It is included in ``site.yml``.

mail.yml

  This playbook sets-up the mail server. It is included in ``site.yml``.

preseed.yml

  This playbook sets-up the Debian preseed files. It is included in

  ``site.yml``.

site.yml

  This playbook sets-up all servers, including preseed files on local host.

web.yml

  This playbook sets-up the web server. It is included in ``site.yml``.

xmpp.yml

  This playbook sets-up the XMPP server. It is included in ``site.yml``.

backup.yml

  This playbook sets-up the backup server. It is included in ``site.yml``.

The playbooks and configurations for test site make a couple of assumptions: