About Majic Ansible Roles

=========================

Majic Ansible Roles is a collection of Ansible roles that are used on regular

basis for deployment and maintenance of Majic infrastructure.

The roles are kept as a separate project in hope of making them potentially

useful to wider audience, and for reference purposes.

Roles cover different aspects of infrastructure, such as mail servers, web

servers, web applications etc. The roles are mainly well-suited for smaller

installations.

Roles are written for use with *Debian GNU/Linux*. For more details on

supported releases, see :ref:`rolereference`.

At the moment, the roles have been written for and tested against **Ansible

2.9.x**.

The roles also utilise the ``ipv4/ipv6`` lookup plugins which require

Why were these roles created?

-----------------------------

For a long time I have had a couple of Internet-facing servers where I hosted

all the IT infrastructure I needed for my day-to-day life.

This started off with some basic services, like mail and XMPP server, and in

time extended to include a web server, code repository etc.

As the number of services I used grew, I found it more difficult to track

updates and upgrades, let alone test them in reliable way. The biggest problem

in particular was lack of time to properly document all the different things

I've set-up.

Being familiar with some Puppet-based deployments, I've started looking into the

possibility of using a configuration management system. Ansible emerged as

something that I thought would be easy to use, due to its agent-less nature.

Once I passed some basic tutorials and got to know the system a bit, I decided

to start my journey on implementing the different roles, in the way I want them,

that would let me easily set-up my servers (and reinstall them, amongst other

things).

The roles you see within this repository are the fruit of this labour. I hope

you find them useful.

Features

--------

*Majic Ansible Roles* have the following features:

.. warning::

   Of course, you may want to take some statements with a pinch of salt, and

   possibly attribute them to either delusions of grandeur, or bragging :)

* Emphasis on small, self-hosted deployments.

* Modular role design where possible and where necessary.

* A number of roles covering common set-up of servers, databases, web server,

  XMPP server, mail server, and LDAP server.

* Streamlined integration with LDAP server for most of the services.

* Well-documented, with role reference documentation, examples, and test/sample

  site.

* Balanced implementation allowing both configurability and ease of deployment.

* Free Software, released under liberal BSD license.

Available roles:

* ``bootstrap`` (for setting-up servers for Ansible use)

* ``common`` (for setting-up basic security, accounts, and configuration on

  servers)

* ``database`` (for creating databases to be used for applications)

* ``database_server`` (for deploying a database server, MariaDB)

* ``ldap_client`` (for setting-up LDAP client configuration)

* ``ldap_server`` (for deploying an LDAP server, OpenLDAP)

* ``mail_forwarder`` (for setting-up forwarding of local mails to smart host,

  Postfix)

* ``mail_server`` (for deploying Postfix, Dovecot, ClamAV)

* ``php_website`` (for deploying PHP websites)

* ``preseed`` (for preparing Debian preseed files)

* ``prosody`` (for deploying XMPP server, Prosody)

* ``web_server`` (for deploying web server, Nginx)

* ``wsgi_website`` (for deploying WSGI/Python applications)

Support

-------

In case of problems with the roles or provided code, please do not hestitate to

contact the author at **majic-ansible-roles (at) majic.rs**. The project can be

found at:

* https://code.majic.rs/majic-ansible-roles

* https://projects.majic.rs/majic-ansible-roles

License

-------

Majic Ansible Roles is released under terms of *BSD (3-Clause) License*::

  Copyright (c) 2018, Branko Majic

  All rights reserved.

  Redistribution and use in source and binary forms, with or without modification,

  are permitted provided that the following conditions are met:

    Redistributions of source code must retain the above copyright notice, this

    list of conditions and the following disclaimer.

    Redistributions in binary form must reproduce the above copyright notice, this

    list of conditions and the following disclaimer in the documentation and/or

    other materials provided with the distribution.

    Neither the name of Branko Majic nor the names of any other

    contributors may be used to endorse or promote products derived from

    this software without specific prior written permission.

  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND

  ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED

  WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE

  DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR

  ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES

  (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;

  LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON

  ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT

  (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS

  SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Majic Ansible Roles documentation

=================================

Majic Ansible Roles is a collection of Ansible roles that are used on regular

basis for deployment and maintenance of Majic infrastructure.

The roles are kept as a separate project in hope of making them potentially

useful to wider audience, and for reference purposes.

Roles cover different aspects of infrastructure, such as mail servers, web

servers, web applications etc. The roles are mainly well-suited for smaller

installations.

Roles are written for use with *Debian GNU/Linux*. For more details on

supported releases, see :ref:`rolereference`.

At the moment, the roles have been written for and tested against **Ansible

2.9.x**.

The roles also utilise the ``ipv4/ipv6`` lookup plugins which require

Contents

========

.. toctree::

   :maxdepth: 2

   about

   usage

   rolereference

   development

   releaseprocedures

   releasenotes

Indices and tables

==================

* :ref:`genindex`

* :ref:`modindex`

* :ref:`search`

Release notes

=============

x.y.z

-----

**Breaking changes:**

* All roles

  * Dropped support for Debian 11 (Bullseye).

**New features/improvements**

* ``backup_client`` role

  * Switched to using Paramiko + SFTP backend (instead of pexpect +

    SFTP), which should improve the backup performance.

**Bug fixes:**

* ``common`` role

  * Fixed permission errors with Python cache directories in the pip

    requirements upgrade checks virtual environment that can happen if

    the initial virtual environment set-up fails.

8.0.0

-----

Dropped support for Python 2.7 and Debian 10 Buster. Added support for

Debian 12 Bookworm. Numerous minor improvements and fixes.

**Breaking changes:**

* All roles

  * Dropped support for Debian 10 (Buster).

  * Added support for Debian 12 (Bookworm).

  * ``netaddr`` Python package is now required for using the roles.

  * ``dnspython`` Python package is no longer required for using the

    roles.

* ``backup_client`` role

  * Previously the backup would run even if pre-backup scripts would

    fail. This is no longer the case, and all pre-backup scripts must

    exit with non-zero exit code in order for backup process to

    kick-in.

  * Old backups are now automatically purged after successful

    backup. This could lead to longer runtimes for entire backup

    process, as well as higher CPU usage.

* ``common`` role

  * Dropped support for Python 2.7 pip requirements upgrade

    checks. Only Python 3 is supported now.

    Requirements (input) files for Python 3 are now put under the

    ``/etc/pip_check_requirements_upgrades`` directory instead of

    ``/etc/pip_check_requirements_upgrades-py3``.

    The ``pip_check_requirements_py3`` /

    ``pip_check_requirements_py3_in`` role parameters have been

    renamed to ``pip_check_requirements`` /

    ``pip_check_requirements_in``.

  * Parameter ``maintenance_allowed_hosts`` has been dropped and

    replaced with parameter ``maintenance_allowed_sources``. The new

    parameter expects a list of IPv4 and IPv6 addresses (or

    subnets). Resolvable names can no longer be specified.

  * NTP server configuration is now based on use of pools instead of

    servers. Parameter ``ntp_servers`` has been deprecated and

    replaced with parameter ``ntp_pools``.

* ``ldap_server`` role

  * Starting with Debian 12 Bookworm, the role no longer deploys

    *rsyslog* and *logrotate* configuration for writing and rotating

    the LDAP servers logs under ``/var/log/slapd.log``. Primary

    reason is that Debian 12 Bookworm no longer installs *rsyslog* by

    default, and it is considered to be deprecated at this point. The

    LDAP server logs can be read via ``journalctl -u slapd`` when

    necessary.

* ``mail_forwarder`` role

  * Firewall rules for incoming connections from the SMTP relay server

    are now based on relay's IPv4 and IPv6 addresses as resolved on

    managed machine during deployment time.

    In case the SMTP relay server's IP addresses change, the role

    needs to get reapplied against managed machines for those changes

    to take place.

    This change in behaviour was introduced to avoid firewall-related

    errors due to inability to resolve names via DNS servers during

    boot time.

* ``mail_server`` role

  * Parameter ``mail_server_tls_protocols`` has been dropped and

    replaced with parameter ``mail_server_minimum_tls_protocol``. Full

    list of TLS protocols can no longer be specified, only the minimum

    one.

* ``wsgi_website`` role

  * Dropped support for Python 2.7. Only Python 3 is supported now.

    The ``python_version`` role parameter has been dropped. The

    ``python_interpreter`` parameter is still available, but it

    defaults to Python 3 binary.

    Python (input) requirements files are now placed under the

    ``/etc/pip_check_requirements_upgrades`` path, in accordance to

    changes made in this release to the ``common`` role.

  * Dropped the ``proxy_headers`` parameter, and replaced it with the

    ``http_header_overrides`` parameter. The new parameter has similar

    function, but the values should no longer include double

    quotes. Main goal is ease of use and consistency between the PHP

    and WSGI website roles.

**New features/improvements**

* ``php_website`` role

  * Environment indicator can now be collapsed by clicking on the

    arrows on the left side.

  * Added parameter ``http_header_overrides`` which can be used to

    set/override request's HTTP headers before passing it on to the

    PHP application.

* ``mail_server`` role

  * Protection against forging of sender addresses has been

    implemented, preventing logged-in users from using arbitrary

    sender mail addresses, even if authenticated. Authenticated users

    can use either their own login as sender, or one of the aliases

    that are associated with their mail account.

* ``web_server`` role

  * Added parameter ``environment_indicator`` which is used on the

    default page to indicate what environment the web server belongs

    to. Useful for distinguishing between test, staging, and

    production environments (similar to what is already available in

    the ``php_website`` and ``wsgi_website`` roles). Indicator can be

    collapsed by clicking on the arrows on the left side.

* ``wsgi_website`` role

  * Environment indicator can now be collapsed by clicking on the

    arrows on the left side.

**Bug fixes:**

* ``common`` role

  * Fix deprecation warnings for Python requirements upgrade checks

    when using pip-tools 7.3.0. This would result in unnecessary

    notifications being sent out to server administrator.

7.1.0

-----

Added support for Debian 11 (Bullseye), with some smaller bug fixes.

**New features/improvements**

* All roles

  * Added support for Debian 11 (Bullseye).

**Bug fixes:**

* ``xmpp_server`` role

  * Make sure to take care of deprecation-related package and

    configuration removals prior to running the rest of the tasks to

    avoid errors related to deprecated elements being invalid (like

    repository URLs for Prosody).

7.0.0

-----

Dropped support for Debian 9 (Stretch), moved away from using

non-Debian project repositories (like Prosody ones).

**Breaking changes:**

* All roles

  * Dropped support for Debian 9 (Stretch).

* ``xmpp_server`` role

  * Parameter ``xmpp_prosody_package`` has been dropped.

.. _usage:

Usage

=====

Majic Ansible Roles are targeted at sysadmins who wish to deploy services for

their own, small-scale use. This chapter gives a simple tutorial-like set of

instructions for using all of the roles available.

.. contents:: :local:

Overview

--------

There is a number of different roles that can prove useful for setting-up a

small infrastructure of your own.

Some roles are suited for one-off operations during installation, like the

``preseed`` and ``bootstrap``, while some are better suited for periodic runs

for maintaining the users and integrity of the system.

By the end of following the instructions, you will have the following:

* Ansible server, used as controller for configuring and managing the

  remaining servers.

* Communications server, providing the LDAP, mail, and XMPP services.

* Web server, providing the web services.

* Backup server, used for storing all of the backups.

Pre-requisites

--------------

For the set-up outlined in this usage guide you'll need the following:

* One server where Ansible will be installed at. Debian Bookworm will

  be installed on top of this server. The server will be set-up

  manually (this is currently out of scope for the *Majic Ansible

  Roles* automated set-up).

* Three servers where the services will be set-up. All servers must be

  able to communicate over network with each-other, the Ansible

  servers, and with Internet. Debian Bookworm will be installed on top

  of these servers as part of the usage instructions.

* Debian Bookworm network installation CD.

* All servers should be on the same network.

* IP addresses for all servers should be known.

* Netmask for all servers should be known.

* Gateway for all servers should be known.

In case of the servers listed above, it might be safest to have them

as virtual machines - this is cheapest thing to do, and simplest (who

wants to deal with pesky hardware anyway?).

Usage instructions assume the following:

* Domain used for all servers is ``example.com``. If you wish to use a different

  domain, adjust the instructions accordingly.

* Server hostnames are ``ansible``, ``comms``, ``www``, and ``bak`` (for Ansible

  server, communications server, web server, and backup server, respectively).

Installing the OS on Ansible server

-----------------------------------

Start-off by installing the operating system on the Ansible server:

1. Fire-up the ``ansible`` server, and boot from the network installation CD.

2. Select the **Install** option.

3. Pick **English** as language.

4. Pick the country you are living in (or whatever else you want).

5. Pick the **en_US.UTF-8** locale.

6. Pick the **American English** keymap.

7. Configure the network if necessary.

8. Set the hostname to ``ansible``.

9. Set the domain to ``example.com``.

10. Set the root password.

11. Create a new user. For simplicity, call the user **Ansible user**, with

    username **ansible**.

12. Set-up partitioning in any way you want. You can go for **Guided - use

    entire disk** if you want to keep it simple and are just testing things.

13. Wait until the base system has been installed.

14. Pick whatever Debian archive mirror is closest to you.

15. If you have an HTTP proxy, provide its URL.

16. Pick if you want to participate in package survey or not.

17. Make sure that at least the **standard system utilities** and **SSH server**

    options are selected on task selection screen.

18. Wait for packages to be installed.

19. Install the GRUB boot loader on MBR.

20. Finalise the server install, and remove the installation media from server.

Installing required packages

----------------------------

With the operating system installed, it is necessary to install a couple of

packages, and to prepare the environment a bit on the Ansible server:

1. Install the necessary system packages (using the ``root`` account)::

     apt-get install -y virtualenv virtualenvwrapper git python3-pip python3-dev libffi-dev libssl-dev

2. Set-up loading of ``virtualenvwrapper`` via Bash completions (using the ``root`` account)::

     ln -s /usr/share/bash-completion/completions/virtualenvwrapper /etc/bash_completion.d/virtualenvwrapper

3. Set-up the virtual environment (using the ``ansible`` account):

   .. warning::

      If you are already logged-in as user ``ansible`` in the server, you will

      need to log-out and log-in again in order to be able to use

      ``virtualenvwrapper`` commands!

   ::

     mkdir ~/mysite/

     mkvirtualenv -a ~/mysite/ mysite

     pip install -U pip setuptools

.. warning::

   The ``netaddr`` package is needed for ``ipv4/ipv6`` lookup plugins

Cloning the *Majic Ansible Roles*

---------------------------------

With most of the software pieces in place, the only missing thing is the Majic

Ansible Roles:

1. Clone the git repository::

     git clone https://code.majic.rs/majic-ansible-roles ~/majic-ansible-roles

2. Checkout the correct version of the roles::

     cd ~/majic-ansible-roles/

     git checkout -b 8.0-dev 8.0-dev

Preparing the basic site configuration

--------------------------------------

Phew... Now that was a bit tedious and boring... But at least you are now ready

to set-up your own site :)

First of all, let's set-up some basic directory structure and configuration:

1. Create Ansible configuration file.

   .. warning::

      Since Ansible 2.x has introduced much stricter controls over security of

      deployed Python scripts, it is recommended (as in this example) to use the

      ``pipelining`` option (which should also improve performance). This is in

      particular necessary in cases where the SSH user connecting to remote

      machine is *not* ``root``, but there are tasks that use ``become`` with

      non-root ``become_user`` (which is the case in Majic Ansible Roles). See

      `official documentation

      <https://docs.ansible.com/ansible/latest/become.html#becoming-an-unprivileged-user>`_

      and other alternatives to this.

   :file:`~/mysite/ansible.cfg`

   ::

     [defaults]

     roles_path=/home/ansible/majic-ansible-roles/roles:/home/ansible/mysite/roles

     force_handlers = True

     inventory = /home/ansible/mysite/hosts

     interpreter_python = /usr/bin/python3

     [ssh_connection]

     pipelining = True

2. Create directory where retry files will be stored at (so they woudln't

   pollute your home directory)::

     mkdir ~/mysite/retry

3. Create the inventory file.

   :file:`~/mysite/hosts`

   ::

     [preseed]

     localhost ansible_connection=local

     [communications]

     comms.example.com

     [web]

     www.example.com

     [backup]

     bak.example.com

4. Create a number of directories for storing playbooks, group

   variables, SSH keys, X.509 artefacts (for TLS), and GnuPG keyring

   (we'll get to this later)::

     mkdir ~/mysite/playbooks/

     mkdir ~/mysite/group_vars/

     mkdir ~/mysite/ssh/

     mkdir ~/mysite/tls/

     mkdir ~/mysite/gnupg/

5. Create SSH private/public key pair that will be used by Ansible for

   connecting to destination servers, as well as for some roles::

     ssh-keygen -f ~/.ssh/id_rsa -N ''

Protecting communications using TLS

-----------------------------------

In order to protect the communications between users and servers, as

well as between servers themselves, it is important to set-up and

properly configure TLS for each role.

*Majic Ansible Roles* mandates use of TLS wherever possible. In other

words, *you must* have TLS private keys and certificates issued by

some CA for all servers in order to be able to use most of the

roles. The private keys and certificates are primarily meant to be

generated *per service*, and that is the approach we will pursue here

as well.

TLS private keys should be ideally generated locally and kept in a

safe environment (possibly encrypted until needed), while the X.509

certificates should be issued by a relevant certification

authority. You can choose to roll-out your own CA, use one of the

public CAs, or perhaps go for a mix of both.

For the purpose of this guide, we'll set-up a small simple local CA to

issue all the necessary certificates, and we'll generate the private

keys and issue server certificates on the go as needed, storing them

all under the ``~/mysite/tls/`` directory.

So, let us make a slight detour to create a CA of our own:

1. First off, install a couple more tools on the Ansible server. We

   will be using ``certtool`` for our improvised CA needs (run this as

   ``root``)::

     apt-get install -y gnutls-bin

2. Create a template for the ``certtool`` so it would know what

   extensions and content to have in the CA certificate:

   :file:`~/mysite/tls/ca.cfg`

   ::

      organization = "Example Inc."

      country = "SE"

      cn = "Example Inc. Test Site CA"

      expiration_days = 1825

      ca

      cert_signing_key

      crl_signing_key

3. Almost there... Now let us generate the CA private key and

   self-signed certificate::

     certtool --sec-param high --generate-privkey --outfile ~/mysite/tls/ca.key

     certtool --template ~/mysite/tls/ca.cfg --generate-self-signed --load-privkey ~/mysite/tls/ca.key --outfile ~/mysite/tls/ca.pem

4. And just one more small tweak - we need to provide a truststore PEM

   file containing all CA certificates in the chain for services to be

   able to connect to each-other (where necessary). In this particular

   case we have a super-simple hierarchy (root CA is also issuing the

   end entity certificates), so simply make a copy of the ``ca.pem``::

     cp ~/mysite/tls/ca.pem ~/mysite/tls/truststore.pem

.. note::

   A useful feature that all roles implement is a check to see if

   certificates will expire within the next 30 days. This check is

   performed via cronjob at midnight, and failing results will end-up

   being delivered to the ``root`` user on local server. Later on,

   once you have configured the mail server, you should be able to

   set-up the necessary aliases to have the mails delivered to

   non-local accounts too.

Preseed files

-------------

The ``preseed`` role is useful for generating Debian preseed files. Preseed

files can be used for automating the Debian installation process.

Preseed files are created on the Ansible controller, and then supplied

to Debian installer.

So, let's set this up for start:

1. First of all, create the playbook for generating the preseed files locally.

   :file:`~/mysite/playbooks/preseed.yml`

   ::

      ---

      - hosts: preseed

        roles:

          - preseed

2. Now we need to configure the role. Two parameters are mandatory -

   one that specifies where the preseed files are to be stored, and

   one that specifies the public key that should be used to

   pre-populate the SSH authorized keys for the ``root`` account. This

   is required for the initial bootstrap of servers because Debian

   GNU/Linux does not by default allow the ``root`` user to

# Ansible and role runtime.

ansible~=10.3

netaddr

python-ldap

# Development and testing.

ansible-lint

defusedxml

flake8

gimmecert

molecule[testinfra]~=24.8.0

molecule-plugins[vagrant]~=23.5.0

paramiko

# Documentation.

sphinx

sphinx-rtd-theme

# Python virtualenv management.

pip

pip-tools

setuptools