.. _usage:

Usage

=====

Majic Ansible Roles are targeted at sysadmins who wish to deploy services for

their own, small-scale use. This chapter gives a simple tutorial-like set of

instructions for using all of the roles available.

.. contents:: :local:

Overview

--------

There is a number of different roles that can prove useful for setting-up a

small infrastructure of your own.

Some roles are suited for one-off operations during installation, like the

``preseed`` and ``bootstrap``, while some are better suited for periodic runs

for maintaining the users and integrity of the system.

By the end of following the instructions, you will have the following:

* Ansible server, used as controller for configuring and managing the

  remaining servers.

* Communications server, providing the LDAP, mail, and XMPP services.

* Web server, providing the web services.

* Backup server, used for storing all of the backups.

Pre-requisites

--------------

For the set-up outlined in this usage guide you'll need the following:

* One server where Ansible will be installed at. Debian Bookworm will

  be installed on top of this server. The server will be set-up

  manually (this is currently out of scope for the *Majic Ansible

  Roles* automated set-up).

* Three servers where the services will be set-up. All servers must be

  able to communicate over network with each-other, the Ansible

  servers, and with Internet. Debian Bookworm will be installed on top

  of these servers as part of the usage instructions.

* Debian Bookworm network installation CD.

* All servers should be on the same network.

* IP addresses for all servers should be known.

* Netmask for all servers should be known.

* Gateway for all servers should be known.

In case of the servers listed above, it might be safest to have them

as virtual machines - this is cheapest thing to do, and simplest (who

wants to deal with pesky hardware anyway?).

Usage instructions assume the following:

* Domain used for all servers is ``example.com``. If you wish to use a different

  domain, adjust the instructions accordingly.

* Server hostnames are ``ansible``, ``comms``, ``www``, and ``bak`` (for Ansible

  server, communications server, web server, and backup server, respectively).

Installing the OS on Ansible server

-----------------------------------

Start-off by installing the operating system on the Ansible server:

1. Fire-up the ``ansible`` server, and boot from the network installation CD.

2. Select the **Install** option.

3. Pick **English** as language.

4. Pick the country you are living in (or whatever else you want).

5. Pick the **en_US.UTF-8** locale.

6. Pick the **American English** keymap.

7. Configure the network if necessary.

8. Set the hostname to ``ansible``.

9. Set the domain to ``example.com``.

10. Set the root password.

11. Create a new user. For simplicity, call the user **Ansible user**, with

    username **ansible**.

12. Set-up partitioning in any way you want. You can go for **Guided - use

    entire disk** if you want to keep it simple and are just testing things.

13. Wait until the base system has been installed.

14. Pick whatever Debian archive mirror is closest to you.

15. If you have an HTTP proxy, provide its URL.

16. Pick if you want to participate in package survey or not.

17. Make sure that at least the **standard system utilities** and **SSH server**

    options are selected on task selection screen.

18. Wait for packages to be installed.

19. Install the GRUB boot loader on MBR.

20. Finalise the server install, and remove the installation media from server.

Installing required packages

----------------------------

With the operating system installed, it is necessary to install a couple of

packages, and to prepare the environment a bit on the Ansible server:

1. Install the necessary system packages (using the ``root`` account)::

     apt-get install -y virtualenv virtualenvwrapper git python3-pip python3-dev libffi-dev libssl-dev

2. Set-up loading of ``virtualenvwrapper`` via Bash completions (using the ``root`` account)::

     ln -s /usr/share/bash-completion/completions/virtualenvwrapper /etc/bash_completion.d/virtualenvwrapper

3. Set-up the virtual environment (using the ``ansible`` account):

   .. warning::

      If you are already logged-in as user ``ansible`` in the server, you will

      need to log-out and log-in again in order to be able to use

      ``virtualenvwrapper`` commands!

   ::

     mkdir ~/mysite/

     mkvirtualenv -a ~/mysite/ mysite

     pip install -U pip setuptools

     pip install 'ansible~=10.3.0' netaddr

.. warning::

   The ``netaddr`` package is needed for ``ipv4/ipv6`` lookup plugins

   which is used internally by some of the roles.

Cloning the *Majic Ansible Roles*

---------------------------------

With most of the software pieces in place, the only missing thing is the Majic

Ansible Roles:

1. Clone the git repository::

     git clone https://code.majic.rs/majic-ansible-roles ~/majic-ansible-roles

2. Checkout the correct version of the roles::

     cd ~/majic-ansible-roles/

     git checkout -b 8.0-dev 8.0-dev

Preparing the basic site configuration

--------------------------------------

Phew... Now that was a bit tedious and boring... But at least you are now ready

to set-up your own site :)

First of all, let's set-up some basic directory structure and configuration:

1. Create Ansible configuration file.

   .. warning::

      Since Ansible 2.x has introduced much stricter controls over security of

      deployed Python scripts, it is recommended (as in this example) to use the

      ``pipelining`` option (which should also improve performance). This is in

      particular necessary in cases where the SSH user connecting to remote

      machine is *not* ``root``, but there are tasks that use ``become`` with

      non-root ``become_user`` (which is the case in Majic Ansible Roles). See

      `official documentation

      <https://docs.ansible.com/ansible/latest/become.html#becoming-an-unprivileged-user>`_

      and other alternatives to this.

   :file:`~/mysite/ansible.cfg`

   ::

     [defaults]

     roles_path=/home/ansible/majic-ansible-roles/roles:/home/ansible/mysite/roles

     force_handlers = True

     inventory = /home/ansible/mysite/hosts

     [ssh_connection]

     pipelining = True

2. Create directory where retry files will be stored at (so they woudln't

   pollute your home directory)::

     mkdir ~/mysite/retry

3. Create the inventory file.

   :file:`~/mysite/hosts`

   ::

     [preseed]

     localhost ansible_connection=local

     [communications]

     comms.example.com

     [web]

     www.example.com

     [backup]

     bak.example.com

4. Create a number of directories for storing playbooks, group

   variables, SSH keys, X.509 artefacts (for TLS), and GnuPG keyring

   (we'll get to this later)::

     mkdir ~/mysite/playbooks/

     mkdir ~/mysite/group_vars/

     mkdir ~/mysite/ssh/

     mkdir ~/mysite/tls/

     mkdir ~/mysite/gnupg/

5. Create SSH private/public key pair that will be used by Ansible for

   connecting to destination servers, as well as for some roles::

     ssh-keygen -f ~/.ssh/id_rsa -N ''

Protecting communications using TLS

-----------------------------------

In order to protect the communications between users and servers, as

well as between servers themselves, it is important to set-up and

properly configure TLS for each role.

*Majic Ansible Roles* mandates use of TLS wherever possible. In other

words, *you must* have TLS private keys and certificates issued by

some CA for all servers in order to be able to use most of the

roles. The private keys and certificates are primarily meant to be

generated *per service*, and that is the approach we will pursue here

as well.

TLS private keys should be ideally generated locally and kept in a

safe environment (possibly encrypted until needed), while the X.509

certificates should be issued by a relevant certification

authority. You can choose to roll-out your own CA, use one of the

public CAs, or perhaps go for a mix of both.

For the purpose of this guide, we'll set-up a small simple local CA to

issue all the necessary certificates, and we'll generate the private

keys and issue server certificates on the go as needed, storing them

all under the ``~/mysite/tls/`` directory.

So, let us make a slight detour to create a CA of our own:

1. First off, install a couple more tools on the Ansible server. We

   will be using ``certtool`` for our improvised CA needs (run this as

   ``root``)::

     apt-get install -y gnutls-bin

2. Create a template for the ``certtool`` so it would know what

   extensions and content to have in the CA certificate:

   :file:`~/mysite/tls/ca.cfg`

   ::

      organization = "Example Inc."

      country = "SE"

      cn = "Example Inc. Test Site CA"

      expiration_days = 1825

      ca

      cert_signing_key

      crl_signing_key

3. Almost there... Now let us generate the CA private key and

   self-signed certificate::

     certtool --sec-param high --generate-privkey --outfile ~/mysite/tls/ca.key

     certtool --template ~/mysite/tls/ca.cfg --generate-self-signed --load-privkey ~/mysite/tls/ca.key --outfile ~/mysite/tls/ca.pem

4. And just one more small tweak - we need to provide a truststore PEM

   file containing all CA certificates in the chain for services to be

   able to connect to each-other (where necessary). In this particular

   case we have a super-simple hierarchy (root CA is also issuing the

   end entity certificates), so simply make a copy of the ``ca.pem``::

     cp ~/mysite/tls/ca.pem ~/mysite/tls/truststore.pem

.. note::

   A useful feature that all roles implement is a check to see if

   certificates will expire within the next 30 days. This check is

   performed via cronjob at midnight, and failing results will end-up

   being delivered to the ``root`` user on local server. Later on,

   once you have configured the mail server, you should be able to

   set-up the necessary aliases to have the mails delivered to

   non-local accounts too.

Preseed files

-------------

The ``preseed`` role is useful for generating Debian preseed files. Preseed

files can be used for automating the Debian installation process.

Preseed files are created on the Ansible controller, and then supplied

to Debian installer.

So, let's set this up for start:

1. First of all, create the playbook for generating the preseed files locally.

   :file:`~/mysite/playbooks/preseed.yml`

   ::

      ---

      - hosts: preseed

        roles:

          - preseed

2. Now we need to configure the role. Two parameters are mandatory -

   one that specifies where the preseed files are to be stored, and

   one that specifies the public key that should be used to

   pre-populate the SSH authorized keys for the ``root`` account. This

   is required for the initial bootstrap of servers because Debian

   GNU/Linux does not by default allow the ``root`` user to

   authenticate via SSH using a password. We will use the SSH public

   key generated earlier via the ``ssh-keygen`` command. Create the

   configuration file:

   :file:`~/mysite/group_vars/preseed.yml`

   ::

      ---

      # Public key used to authenticate remote logins via SSH for the

      # root account.

      ansible_key: "{{ lookup('file', '~/.ssh/id_rsa.pub') }}"

      # Directory where the preseed files will be output to.

      preseed_directory: "~/mysite/preseed_files/"

3. Now we can generate the pressed files::

     workon mysite && ansible-playbook playbooks/preseed.yml

4. If all went well, you should have the following files created:

   * :file:`~/mysite/preseed_files/comms.example.com.cfg`

   * :file:`~/mysite/preseed_files/www.example.com.cfg`

   * :file:`~/mysite/preseed_files/bak.example.com.cfg`

5. You can have a look at them, but you might notice the settings in the file

   might not be to your liking. In particular, it could be using wrong timezone,

   defaulting to DHCP for network configuration etc. Let's concentrate on making

   the network configuration changes - this is the main thing that will probably

   differ in your environment. Update the preseed configuration file:

   :file:`~/mysite/group_vars/preseed.yml`

   ::

      ---

      # Public key used to authenticate remote logins via SSH for the

      # root account.

      ansible_key: "{{ lookup('file', '~/.ssh/id_rsa.pub') }}"

      # Directory where the preseed files will be output to.

      preseed_directory: "~/mysite/preseed_files/"

      # Set your default (initial) root password.

      preseed_root_password: changeit

      # Use manual network configuration (no DHCP).

      preseed_network_auto: no

      # Set the gateway for all servers.