Some roles are suited for one-off operations during installation, like the

``preseed`` and ``bootstrap``, while some are better suited for periodic runs

for maintaining the users and integrity of the system.

By the end of following the instructions, you will have the following:

* Ansible server, used as controller for configuring and managing the

  remaining servers.

* Communications server, providing the LDAP, mail, and XMPP services.

* Web server, providing the web services.

* Backup server, used for storing all of the backups.

Pre-requisites

--------------

For the set-up outlined in this usage guide you'll need the following:

* One server where Ansible will be installed at. Debian Stretch will

  be installed on top of this server. The server will be set-up

  manually (this is currently out of scope for the *Majic Ansible

  Roles* automated set-up).

* Three servers where the services will be set-up. All servers must be able to

  communicate over network with each-other, the Ansible servers, and with

20. Finalise the server install, and remove the installation media from server.

Installing required packages

----------------------------

With the operating system installed, it is necessary to install a couple of

packages, and to prepare the environment a bit on the Ansible server:

1. Install the necessary system packages (using the ``root`` account)::

2. Set-up the virtual environment (using the ``ansible`` account):

   .. warning::

      If you are already logged-in as user ``ansible`` in the server, you will

      need to log-out and log-in again in order to be able to use

      ``virtualenvwrapper`` commands!

   ::

     mkdir ~/mysite/

     pip install -U pip setuptools

     pip install 'ansible~=2.9.0' dnspython

.. warning::

   The ``dnspython`` package is important since it is used internally via

   ``dig`` lookup plugin.

Cloning the *Majic Ansible Roles*

---------------------------------

With most of the software pieces in place, the only missing thing is the Majic

      <http://docs.ansible.com/ansible/latest/become.html#becoming-an-unprivileged-user>`_

      and other alternatives to this.

   :file:`~/mysite/ansible.cfg`

   ::

     [defaults]

     roles_path=/home/ansible/majic-ansible-roles/roles:/home/ansible/mysite/roles

     force_handlers = True

     inventory = /home/ansible/mysite/hosts

     [ssh_connection]

     pipelining = True

2. Create directory where retry files will be stored at (so they woudln't

   pollute your home directory)::

     mkdir ~/mysite/retry

3. Create the inventory file.

   :file:`~/mysite/hosts`

You have your preseed files now, so you can go ahead and install the

servers ``comms.example.com``, ``www.example.com``, and

``bak.example.com`` using them with network install CD. Have a look at

`Debian instructions

<https://www.debian.org/releases/stretch/amd64/apbs02.html.en>`_ for

more details.

If you need to, you can easily serve the preseed files from the Ansible server

with Python's built-in HTTP server::

  cd ~/mysite/preseed_files/

Then you can point installer to the preseed file selecting the

``Advanced options -> Automated install`` (don't press ``ENTER`` yet),

then pressing ``TAB``, and appending the following at the end (just

fill-in the correct hostname - ``comms``, ``www``, or ``bak``)::

  url=http://ansible.example.com:8000/HOSTNAME.example.com.cfg

Bootstrapping servers for Ansible set-up

----------------------------------------

      # here.Beware the IP spoofing, though! Don't forget to change the bellow

      # IP for your server ;)

      smtp_allow_relay_from:

        - 10.32.64.20

        - 10.32.64.23

4. Let's apply the changes::

     workon mysite && ansible-playbook playbooks/site.yml

5. After this you may want to test out sending mail via web or backup server's

   local SMTP to the root user (to see if the aliasing works), and to some

   something similar to the following on your web server::

     swaks --to root@localhost --server localhost

     swaks --to YOUR_MAIL --server localhost

   If all went well, you should be able to see a new mail in John Doe's mailbox,

   as well as your own mailbox.

Adding XMPP server

------------------

        with_items:

          - /var/www/tbg.example.com/thebuggenie-{{ tbg_version }}/

          - /var/www/tbg.example.com/thebuggenie-{{ tbg_version }}/public/

          - /var/www/tbg.example.com/thebuggenie-{{ tbg_version }}/core/config/

      - name: Create symbolic link to TBG application

        file:

          src: "/var/www/tbg.example.com/thebuggenie-{{ tbg_version }}/public"

          path: "/var/www/tbg.example.com/htdocs"

          state: link

          owner: "admin-tbg_example_com"

          group: "web-tbg_example_com"

        become: yes

        become_user: admin-tbg_example_com

      - name: Install TBG dependencies

        composer:

          command: install

          working_dir: "/var/www/tbg.example.com/thebuggenie-{{ tbg_version }}"

        become: yes

        become_user: admin-tbg_example_com

      - name: Deploy database configuration file for TBG

        copy:

   :file:`~/mysite/roles/wiki/meta/main.yml`

   ::

      ---

      dependencies:

        - role: wsgi_website

          fqdn: wiki.example.com

          # In many cases you need to have some development packages available

          # in order to build Python packages installed via pip

          packages:

            - build-essential

            - libjpeg-dev

          # Here we specify that anything accessing our website with "/static/"

          # URL should be treated as request to a static file, to be served

          # directly by Nginx instead of the WSGI server.

          static_locations:

            - /static/

          # Again, not mandatory, but it is good to have some sort of policy

          # for assigning UIDs.

          uid: 2001

          admin_uid: 3001

          # These are additional packages that should be installed in the

          # virtual environment.

          virtualenv_packages:

          # This is the name of the WSGI application to

          # serve. wiki_example_com.wsgi will be the Python "module" that is

          # accesed, while application is the object instantiated within it (the

          # application itself). The module is referenced relative to the code

          # directory (in our case /var/www/wiki.example.com/code/).

          wsgi_application: wiki_example_com.wsgi:application

        - role: database

          db_name: wiki

          db_password: wiki

3. Let's create a dedicated private key/certificate pair for the wiki website:

   1. Create new template for ``certtool``:

      :file:`~/mysite/tls/wiki.example.com_https.cfg`

      ::

         organization = "Example Inc."

          - urls.py

        notify:

          - Restart wiki

      - name: Deploy project database and deploy static files

        django_manage:

          command: "{{ item }}"

          app_path: "/var/www/wiki.example.com/code/"

          virtualenv: "/var/www/wiki.example.com/virtualenv/"

        become: yes

        become_user: admin-wiki_example_com

        with_items:

          - migrate

          - collectstatic

      - name: Deploy the superadmin creation script

        copy:

          src: "create_superadmin.py"

          dest: "/var/www/wiki.example.com/code/create_superadmin.py"

          owner: admin-wiki_example_com

          group: web-wiki_example_com

          mode: 0750

      - name: Create initial superuser

          state: restarted

5. There is a couple of files that we are deploying through the above

   tasks. Let's create them as well.

   :file:`~/mysite/roles/wiki/files/settings.py`

   ::

      """

      Django settings for wiki_example_com project.

      For more information on this file, see

      For the full list of settings and their values, see

      """

      import os

      # Quick-start development settings - unsuitable for production

      # SECURITY WARNING: keep the secret key used in production secret!

      # SECURITY WARNING: don't run with debug turned on in production!

      DEBUG = False

      ALLOWED_HOSTS = ["wiki.example.com", "localhost"]

      # Application definition

          'django.contrib.admin',

          'django.contrib.auth',

          'django.contrib.contenttypes',

          'django.contrib.sessions',

          'django.contrib.messages',

          'django.contrib.staticfiles',

          'mptt',

          'sekizai',

          'sorl.thumbnail',

          'django.contrib.sessions.middleware.SessionMiddleware',

          'django.middleware.common.CommonMiddleware',

          'django.middleware.csrf.CsrfViewMiddleware',

          'django.contrib.auth.middleware.AuthenticationMiddleware',

          'django.contrib.messages.middleware.MessageMiddleware',

          'django.middleware.clickjacking.XFrameOptionsMiddleware',

      ROOT_URLCONF = 'wiki_example_com.urls'

      WSGI_APPLICATION = 'wiki_example_com.wsgi.application'

      # Database

      DATABASES = {

          'default': {

              'ENGINE': 'django.db.backends.mysql',

              'NAME': 'wiki',

              'USER': 'wiki',

              'PASSWORD': 'wiki',

              'HOST': '127.0.0.1',

              'PORT': '3306',

          }

      }

      # Internationalization

      LANGUAGE_CODE = 'en-us'

      TIME_ZONE = 'Europe/Stockholm'

      USE_I18N = True

      USE_L10N = True

      USE_TZ = True

      # Static files (CSS, JavaScript, Images)

      STATIC_URL = '/static/'

      STATIC_ROOT = '/var/www/wiki.example.com/htdocs/static'

   :file:`~/mysite/roles/wiki/files/urls.py`

   ::

      from django.contrib import admin

   :file:`~/mysite/roles/wiki/files/create_superadmin.py`

   ::

      #!/usr/bin/env python

      import os

      from django import setup

      os.environ['DJANGO_SETTINGS_MODULE']='wiki_example_com.settings'

      setup()

      from django.conf import settings

      from django.contrib.auth.models import User

        become: yes

        roles:

          - common

          - ldap_client

          - mail_forwarder

          - web_server

          - database_server

          - tbg

          - wiki

7. Apply the changes:

   ::

     workon mysite && ansible-playbook playbooks/site.yml

8. At this point Django Wiki has been installed, and you should be able to open

   the URL https://wiki.example.com/ (if you open http://wiki.example.com/ , you

   will be redirected to the HTTPS URL) and log-in into *Django Wiki* with

   username ``admin`` and password ``admin``.

Backups, backups, backups!

--------------------------

   ``backup_client`` role deals with basic set-up of backup client and its

   configuration, the ``backup`` role is used to define what should be

   backed-up. It is important to define unique filename for the backup patterns

   file. Take into account that you can use pretty much any globbing pattern

   supported by Duplicity.

   .. warning::

      Make sure the addition is properly aligned in the yaml file to previous

      role dependency definitions.

   :file:`~/mysite/roles/tbg/meta/main.yml`

        - role: backup

          when: enable_backup

          backup_patterns_filename: "tbg"

          backup_patterns:

            - "/var/www/tbg.example.com/files"

2. Apply the changes::

     workon mysite && ansible-playbook playbooks/site.yml

3. Now rerun the backup on server ``www.example.com`` (as root). If you haven't