About Majic Ansible Roles

=========================

Majic Ansible Roles is a collection of Ansible roles that are used on regular

basis for deployment and maintenance of Majic infrastructure.

The roles are kept as a separate project in hope of making them potentially

useful to wider audience, and for reference purposes.

Roles cover different aspects of infrastructure, such as mail servers, web

servers, web applications etc. The roles are mainly well-suited for smaller

installations.

Roles are written for use with *Debian GNU/Linux*. For more details on

supported releases, see :ref:`rolereference`.

At the moment, the roles have been written for and tested against **Ansible

2.9.x**.

Why were these roles created?

-----------------------------

For a long time I have had a couple of Internet-facing servers where I hosted

all the IT infrastructure I needed for my day-to-day life.

This started off with some basic services, like mail and XMPP server, and in

time extended to include a web server, code repository etc.

As the number of services I used grew, I found it more difficult to track

updates and upgrades, let alone test them in reliable way. The biggest problem

in particular was lack of time to properly document all the different things

I've set-up.

Being familiar with some Puppet-based deployments, I've started looking into the

possibility of using a configuration management system. Ansible emerged as

something that I thought would be easy to use, due to its agent-less nature.

Once I passed some basic tutorials and got to know the system a bit, I decided

to start my journey on implementing the different roles, in the way I want them,

that would let me easily set-up my servers (and reinstall them, amongst other

things).

Majic Ansible Roles documentation

=================================

Majic Ansible Roles is a collection of Ansible roles that are used on regular

basis for deployment and maintenance of Majic infrastructure.

The roles are kept as a separate project in hope of making them potentially

useful to wider audience, and for reference purposes.

Roles cover different aspects of infrastructure, such as mail servers, web

servers, web applications etc. The roles are mainly well-suited for smaller

installations.

Roles are written for use with *Debian GNU/Linux*. For more details on

supported releases, see :ref:`rolereference`.

At the moment, the roles have been written for and tested against **Ansible

2.9.x**.

Contents

========

.. toctree::

   :maxdepth: 2

   about

   usage

   rolereference

   development

   releaseprocedures

   releasenotes

Indices and tables

==================

* :ref:`genindex`

* :ref:`modindex`

* :ref:`search`

Release notes

=============

x.y.z

-----

Dropped support for Python 2.7 and Debian 10 Buster. Added support for

Debian 12 Bookworm. Some minor improvements and fixes.

**Breaking changes:**

* All roles

  * Dropped support for Debian 10 (Buster).

  * Added support for Debian 12 (Bookworm).

  * ``netaddr`` Python package is now required for using the roles.

* ``backup_client`` role

  * Previously the backup would run even if pre-backup scripts would

    fail. This is no longer the case, and all pre-backup scripts must

    exit with non-zero exit code in order for backup process to

    kick-in.

  * Old backups are now automatically purged after successful

    backup. This could lead to longer runtimes for entire backup

    process, as well as higher CPU usage.

* ``common`` role

  * Dropped support for Python 2.7 pip requirements upgrade

    checks. Only Python 3 is supported now.

    Requirements (input) files for Python 3 are now put under the

    ``/etc/pip_check_requirements_upgrades`` directory instead of

    ``/etc/pip_check_requirements_upgrades-py3``.

    The ``pip_check_requirements_py3`` /

    ``pip_check_requirements_py3_in`` role parameters have been

    renamed to ``pip_check_requirements`` /

    ``pip_check_requirements_in``.

  * Parameter ``maintenance_allowed_hosts`` has been dropped and

    replaced with parameter ``maintenance_allowed_sources``. The new

    parameter expects a list of IPv4 and IPv6 addresses (or

  * NTP server configuration is now based on use of pools instead of

    servers. Parameter ``ntp_servers`` has been deprecated and

    replaced with parameter ``ntp_pools``.

* ``ldap_server`` role

  * Starting with Debian 12 Bookworm, the role no longer deploys

    *rsyslog* and *logrotate* configuration for writing and rotating

    the LDAP servers logs under ``/var/log/slapd.log``. Primary

    reason is that Debian 12 Bookworm no longer installs *rsyslog* by

    default, and it is considered to be deprecated at this point. The

    LDAP server logs can be read via ``journalctl -u slapd`` when

    necessary.

* ``mail_server`` role

  * Parameter ``mail_server_tls_protocols`` has been dropped and

    replaced with parameter ``mail_server_minimum_tls_protocol``. Full

    list of TLS protocols can no longer be specified, only the minimum

    one.

* ``wsgi_website`` role

  * Dropped support for Python 2.7. Only Python 3 is supported now.

    The ``python_version`` role parameter has been dropped. The

    ``python_interpreter`` parameter is still available, but it

    defaults to Python 3 binary.

    Python (input) requirements files are now placed under the

    ``/etc/pip_check_requirements_upgrades`` path, in accordance to

    changes made in this release to the ``common`` role.

  * Dropped the ``proxy_headers`` parameter, and replaced it with the

    ``http_header_overrides`` parameter. The new parameter has similar

    function, but the values should no longer include double

    quotes. Main goal is ease of use and consistency between the PHP

    and WSGI website roles.

  smtp_allow_relay_from:

    - ldap.example.com

    - xmpp.example.com

  imap_max_user_connections_per_ip: 50

Mail Forwarder

--------------

The ``mail_forwarder`` role can be used for setting-up a local SMTP server for

sending out mails and receiving mails for local users. The SMTP server is

provided by Postfix.

SMTP service on server set-up this way is not meant to be exposed to the

Internet directly, and should receive delivery failures from the relay server

instead.

The role implements the following:

* Installs and configures Postfix.

* Purges Exim4 configuration (just in case).

* Sets-up aliases for the local recipients.

* Installs SWAKS (utility for testing SMTP servers).

Postfix is configured as follows:

* Local destinations are set-up.

* A relay host is set.

* TLS is enforced for relaying mails, with configurable truststore for server

  certificate verification if SMTP relay is used. If SMTP relay is not used

  (configured), no certificate verification is done.

* Uses 2048-bit Diffie-Hellman parameters for relevant TLS ciphers for

  incoming connections.

Role dependencies

~~~~~~~~~~~~~~~~~

Depends on the following roles:

* **common**

Parameters

~~~~~~~~~~

**local_mail_aliases** (dictionary, optional, ``[]``)

With the operating system installed, it is necessary to install a couple of

packages, and to prepare the environment a bit on the Ansible server:

1. Install the necessary system packages (using the ``root`` account)::

     apt-get install -y virtualenv virtualenvwrapper git python3-pip python3-dev libffi-dev libssl-dev

2. Set-up loading of ``virtualenvwrapper`` via Bash completions (using the ``root`` account)::

     ln -s /usr/share/bash-completion/completions/virtualenvwrapper /etc/bash_completion.d/virtualenvwrapper

3. Set-up the virtual environment (using the ``ansible`` account):

   .. warning::

      If you are already logged-in as user ``ansible`` in the server, you will

      need to log-out and log-in again in order to be able to use

      ``virtualenvwrapper`` commands!

   ::

     mkdir ~/mysite/

     mkvirtualenv -p /usr/bin/python3 -a ~/mysite/ mysite

     pip install -U pip setuptools

.. warning::

Cloning the *Majic Ansible Roles*

---------------------------------

With most of the software pieces in place, the only missing thing is the Majic

Ansible Roles:

1. Clone the git repository::

     git clone https://code.majic.rs/majic-ansible-roles ~/majic-ansible-roles

2. Checkout the correct version of the roles::

     cd ~/majic-ansible-roles/

     git checkout -b 7.1-dev 7.1-dev

Preparing the basic site configuration

--------------------------------------

Phew... Now that was a bit tedious and boring... But at least you are now ready

to set-up your own site :)

ansible~=2.9.0

defusedxml

gimmecert~=0.5.0

molecule~=2.22.0

netaddr

paramiko

pip

pip-tools

python-ldap

python-vagrant

# @TODO: Required for ansible-lint due to breaking changes in newer version.

rich<11.0.0

setuptools

sh~=1.14.0

sphinx

sphinx-rtd-theme

    #   cookiecutter

    #   molecule

    #   pip-tools

    #   python-gilt

click-completion==0.5.2

    # via molecule

colorama==0.4.6

    # via

    #   molecule

    #   python-gilt

    #   rich

commonmark==0.9.1

    # via rich

cookiecutter==2.6.0

    # via molecule

cryptography==3.2.1

    # via

    #   ansible

    #   gimmecert

    #   paramiko

defusedxml==0.7.1

    # via -r requirements.in

distlib==0.3.8

    # via virtualenv

docutils==0.20.1

    # via

    #   sphinx

    #   sphinx-rtd-theme

exceptiongroup==1.2.0

    # via pytest

fasteners==0.19

    # via python-gilt

filelock==3.13.1

    # via virtualenv

flake8==7.0.0

    # via molecule

gimmecert==0.5.0

    # via -r requirements.in

identify==2.5.35

    # via pre-commit

idna==3.6

    # via requests

imagesize==1.4.1

    # via sphinx

importlib-metadata==7.0.1

    # via

    #   build

    #   sphinx

    #   sphinx

markupsafe==2.1.5

    # via jinja2

mccabe==0.7.0

    # via flake8

molecule==2.22

    # via -r requirements.in

netaddr==1.2.1

    # via -r requirements.in

nodeenv==1.8.0

    # via pre-commit

packaging==23.2

    # via

    #   build

    #   pytest

    #   sphinx

paramiko==2.12.0

    # via

    #   -r requirements.in

    #   molecule

pathspec==0.12.1

    # via yamllint

pexpect==4.9.0

    # via molecule

    # via -r requirements.in

platformdirs==4.2.0

    # via virtualenv

pluggy==1.4.0

    # via pytest

pre-commit==1.21.0

    # via molecule

psutil==5.9.8

    # via molecule

ptyprocess==0.7.0

    # via pexpect

pyasn1==0.5.1

    # via

    #   pyasn1-modules

    #   python-ldap

pyasn1-modules==0.3.0

    # via python-ldap

pycodestyle==2.11.1

    # via flake8

pycparser==2.21

    # via cffi

pyflakes==3.2.0

    # via flake8

pygments==2.17.2

    src: "main.cf.j2"

    dest: "/etc/postfix/main.cf"

    owner: root

    group: root

    mode: 0644

  notify:

    - Restart Postfix

- name: Set-up local mail aliases

  lineinfile:

    dest: "/etc/aliases"

    line: "{{ item.key }}: {{ item.value }}"

    regexp: "^{{ item.key }}"

    state: present

  with_dict: "{{ local_mail_aliases }}"

  notify:

    - Rebuild mail aliases

- name: Enable and start postfix service

  service:

    name: postfix

    state: started

    enabled: true

- name: Deploy firewall configuration for mail forwader

  template:

    src: "ferm_mail.conf.j2"

    dest: "/etc/ferm/conf.d/20-mail.conf"

    owner: root

    group: root

    mode: 0640

  notify:

    - Restart ferm

- name: Install SWAKS

  apt:

    name: swaks

    state: present

- name: Explicitly run all handlers

  include: ../handlers/main.yml

  when: "run_handlers | default(False) | bool()"

  tags:

    - handlers

{% if smtp_relay_host and smtp_from_relay_allowed %}

domain ip {

    # Accept incoming connections on port 25 from SMTP relay host.

    table filter {

        chain INPUT {

            # SMTP for server communication.

            proto tcp dport 25 {

            }

        }

    }

}

domain ip6 {

    # Accept incoming connections on port 25 from SMTP relay host.

    table filter {

        chain INPUT {

            # SMTP for server communication.

            proto tcp dport 25 {

            }

        }

    }

}

{% endif %}

{% endif %}