- ldap_client

          - ldap_server

          - mail_server

2. Let's configure the role next.

   :file:`~/mysite/group_vars/communications.yml`

   ::

      # Set the LDAP URL to connect through. Keep in mind TLS is required.

      mail_ldap_url: ldap://comms.example.com/

      # Here we need to point to the base DN of LDAP server. A bunch of entries

      # will need to exist under it for service to function correctly, though.

      mail_ldap_base_dn: dc=example,dc=com

      # Separate LDAP entries are used for Postfix/Dovecot

      # authentication. Therefore we have two passwords here.

      mail_ldap_postfix_password: postfix

      mail_ldap_dovecot_password: dovecot

      # Setting uid/gid is optional, but you might have a policy on how to

      # assign UIDs and GIDs, so it is convenient to be able to change this.

      mail_user_uid: 5000

      mail_user_gid: 5000

      # Set private keys and certificates to use for the IMAP service.

      imap_tls_certificate: "{{ lookup('file', '~/mysite/tls/comms.example.com_imap.pem') }}"

      imap_tls_key: "{{ lookup('file', '~/mysite/tls/comms.example.com_imap.key') }}"

      # Set private keys and certificates to use for the SMTP service.

      smtp_tls_certificate: "{{ lookup('file', '~/mysite/tls/comms.example.com_smtp.pem') }}"

      smtp_tls_key: "{{ lookup('file', '~/mysite/tls/comms.example.com_smtp.key') }}"

      # Set the X.509 certificate truststore to use for validating the

      # LDAP server certificate.

      mail_ldap_tls_truststore: "{{ lookup('file', '~/mysite/tls/truststore.pem') }}"

3. There are two distinct mail services that need to access the LDAP directory -

   *Postfix* (serving as an SMTP server), and *Dovecot* (serving as an IMAP

   server). These two need their own dedicated LDAP entries on the LDAP server in

   order to log-in. Luckily, it is easy to create such entries through the options

   provided by the LDAP server role. In addition to this, the Postfix and Dovecot

   services will check if users are members of ``mail`` group in LDAP directory

   before accepting them as valid mail users. Once again, the LDAP server role

   comes with a simple option for creating groups.

   :file:`~/mysite/group_vars/communications.yml`

   ::

      # Don't forget, the passwords here must match with passwords specified

      # under options mail_ldap_postfix_password/mail_ldap_dovecot_password.

      ldap_server_consumers:

        - name: postfix

          password: postfix

        - name: dovecot

          password: dovecot

      ldap_server_groups:

        - name: mail

4. Ok, so now our SMTP and IMAP service can log-in into the LDAP server to

   look-up the mail server information. We have also defined the mail group for

   limitting which users get mail service. However, we don't have any

   user/domain information yet. So let's change that, using the ``ldap_entries``

   option from LDAP server role.

   .. warning::

      Long-term, you probably want to manage these entries manually or through

      other means than the ``ldap_entries`` option. The reason for this is

      because this type of data in LDAP directory can be considered more of an

      operational/application data than configuration data that frequently

      changes (especially the user passwords/info). Backups of LDAP directory on

      regular basis are important. We will get to that at a later point.

   :file:`~/mysite/group_vars/communications.yml`

   ::

      ldap_entries:

        # Create first a couple of user entries. Don't forget to set the

        # "mail" attribute for them.

        - dn: uid=johndoe,ou=people,dc=example,dc=com

          attributes:

            objectClass:

              - inetOrgPerson

            uid: johndoe

            cn: John Doe

            sn: Doe

            userPassword: johndoe

            mail: john.doe@example.com

        - dn: uid=janedoe,ou=people,dc=example,dc=com

          attributes:

            objectClass:

              - inetOrgPerson

            uid: janedoe

            cn: Jane Doe

            sn: Doe

            userPassword: janedoe

            mail: jane.doe@example.com

        # Let's register our domain in LDAP directory.

        - dn: dc=example.com,ou=domains,ou=mail,ou=services,dc=example,dc=com

          attributes:

            objectClass: dNSDomain

            dc: "example.com"

          # Finally, for the lolz, let's also add the standard postmaster alias

          # for our domain. This one will also receive any undeliverable bounced

          # mails.

        - dn: cn=postmaster@example.com,ou=aliases,ou=mail,ou=services,dc=example,dc=com

          attributes:

            objectClass: nisMailAlias

            cn: postmaster@example.com

            rfc822MailMember: john.doe@example.com

5. Once again, before we apply the configuration, we must make sure the

   necessary TLS private keys and certificates are available. In this particular

   case, we need to set-up separate key/certificate pair for both the SMTP and

   IMAP service:

   1. Create new templates for ``certtool``:

      :file:`~/mysite/tls/comms.example.com_smtp.cfg`

      ::

         organization = "Example Inc."

         country = SE

         cn = "Exampe Inc. SMTP Server"

         expiration_days = 365

         dns_name = "comms.example.com"

         tls_www_server

         signing_key

         encryption_key

      :file:`~/mysite/tls/comms.example.com_imap.cfg`

      ::

         organization = "Example Inc."

         country = SE

         cn = "Exampe Inc. IMAP Server"

         expiration_days = 365

         dns_name = "comms.example.com"

         tls_www_server

         signing_key

         encryption_key

   2. Create the keys and certificates for SMTP/IMAP services based on the templates::

        certtool --sec-param normal --generate-privkey --outfile ~/mysite/tls/comms.example.com_smtp.key

        certtool --generate-certificate --load-ca-privkey ~/mysite/tls/ca.key --load-ca-certificate ~/mysite/tls/ca.pem --template ~/mysite/tls/comms.example.com_smtp.cfg --load-privkey ~/mysite/tls/comms.example.com_smtp.key --outfile ~/mysite/tls/comms.example.com_smtp.pem

        certtool --sec-param normal --generate-privkey --outfile ~/mysite/tls/comms.example.com_imap.key

        certtool --generate-certificate --load-ca-privkey ~/mysite/tls/ca.key --load-ca-certificate ~/mysite/tls/ca.pem --template ~/mysite/tls/comms.example.com_imap.cfg --load-privkey ~/mysite/tls/comms.example.com_imap.key --outfile ~/mysite/tls/comms.example.com_imap.pem

6. Configuration and TLS keys have ben set-up, so it is time to apply the changes::

     workon mysite && ansible-playbook playbooks/site.yml

7. Let's add the two users to the mail group (otherwise, the mail

   server will ignore them). We'll use the ``ldap_attr`` module

   directly to make our life a bit easier::

     workon mysite && ansible --become -m ldap_attr -a "dn=cn=mail,ou=groups,dc=example,dc=com state=present name=uniqueMember values=uid=johndoe,ou=people,dc=example,dc=com" communications

     workon mysite && ansible --become -m ldap_attr -a "dn=cn=mail,ou=groups,dc=example,dc=com state=present name=uniqueMember values=uid=janedoe,ou=people,dc=example,dc=com" communications

8. If no errors have been reported, at this point you should have two mail

   accounts - ``john.doe@example.com``, with password ``johndoe``, and

   ``jane.doe@example.com``, with password ``janedoe``. In this particular

   set-up, the mail addresses are used as usernames. If you want to test it out,

   simply install ``swaks`` on your Ansible machine, and run something along the

   lines of

   ::

     swaks --to john.doe@example.com --server comms.example.com

     swaks --to jane.doe@example.com --server comms.example.com

  Of course, free feel to also test out the mail server using any mail client of

  your choice. When doing so, use port 587 for SMTP. Port 25 is reserved for

  unauthenticated server-to-server mail deliveries.

  If you face issues with ISPs or hotels blocking the two ports listed above,

  you can also use alternative ports 26 (redirected to port 587) and 27

  (redirected to port 25).

  TLS has also been hardened on port 587 to allow only TLSv1.2 and PFS ciphers

  (you can override TLS versions/ciphers via role configuration). TLS

  configuration on port 25 has been left unchanged for maximum

  interoperability with other servers.

Setting-up mail relaying from web and backup servers

----------------------------------------------------

With the mail server set-up, the next thing to do would be to set-up the SMTP

server on web and backup servers to relay mails via the communications

server. This way we can make sure that mail that gets sent via local SMTP to

external addresses on those two servers goes through our anti-virus scanner.

1. Update the list of roles for web and backup server to include the mail

   forwarder role.

   :file:`~/mysite/playbooks/web.yml`

   ::

      ---

      - hosts: web

        remote_user: ansible

        become: yes

        roles:

          - common

          - ldap_client

          - mail_forwarder

   :file:`~/mysite/playbooks/backup.yml`

   ::

      ---

      - hosts: backup

        remote_user: ansible

        become: yes

        roles:

          - common

          - mail_forwarder

2. The next thing is to set-up the configuration for the new role. We can define

   this globally for all servers

   :file:`~/mysite/group_vars/all.yml`

   ::

      # Define what X.509 certificates should be used for validating

      # the certificate of server we are relaying the mails through.

      smtp_relay_truststore: "{{ lookup('file', '~/mysite/tls/truststore.pem') }}"

      # Make sure any mails directed to localhost root account get

      # forwarded to one of our mail users as well.

      local_mail_aliases:

        root: root john.doe@example.com

      # Now signal the local SMTP to relay any non-local mails via our

      # communications server. Don't forget to specify your own IP address (or

      # FQDN) here. Without this option, the SMTP would send out the mails

      # directly.

      smtp_relay_host: comms.example.com

3. Although we have told our web and backup servers to use the communications

   server as relay for non-local mail, the communications server is not aware of

   this. This would result in the communications server refusing all relay

   attempts (if not, it would be an open relay, which is bad).

   So, let's fix this a bit - we have a configuration option for the mail server

   for exactly this purpose.

   :file:`~/mysite/group_vars/communications.yml`

   ::

      # We want to allow relaying of mails from our web and backup servers

      # here.Beware the IP spoofing, though! Don't forget to change the bellow

      # IP for your server ;)

      smtp_allow_relay_from:

        - 10.32.64.20

        - 10.32.64.23

4. Let's apply the changes::

     workon mysite && ansible-playbook playbooks/site.yml

5. After this you may want to test out sending mail via web or backup server's

   local SMTP to the root user (to see if the aliasing works), and to some

   external mail address to check if forwarding works correctly too. Run

   something similar to the following on your web server::

     swaks --to root@localhost --server localhost

     swaks --to YOUR_MAIL --server localhost

   If all went well, you should be able to see a new mail in John Doe's mailbox,

   as well as your own mailbox.

Adding XMPP server

------------------

Now that the users can communicate via mail server, we might as well add support

for some instant messaging. For this purpose, we will use the ``xmpp_server``

role.

1. Update the playbook for communications server to include the XMPP server

   role.

   :file:`~/mysite/playbooks/communications.yml`

   ::

      ---

      - hosts: communications

        remote_user: ansible

        become: yes

        roles:

          - common

          - ldap_client

          - ldap_server

          - mail_server

          - xmpp_server

2. Configure the role.

   :file:`~/mysite/group_vars/communications.yml`

   ::

      # Set the TLS private key and certificate.

      xmpp_tls_certificate: "{{ lookup('file', '~/mysite/tls/comms.example.com_xmpp.pem') }}"

      xmpp_tls_key: "{{ lookup('file', '~/mysite/tls/comms.example.com_xmpp.key') }}"

      # Set one of the users to also be an XMPP administrator.

      xmpp_administrators:

        - john.doe@example.com

      # Unfortunately, XMPP can't look-up domains via LDAP, so we need to be

      # explicit here.

      xmpp_domains:

        - example.com

      # Simply point the XMPP server to base DN of LDAP server, and let it use

      # specific directory structure it expects.

      xmpp_ldap_base_dn: dc=example,dc=com

      # Password for logging-in into the LDAP directory.

      xmpp_ldap_password: prosody

      # Where the LDAP server is located at. Full-blown LDAP URIs are _not_

      # supported!

      xmpp_ldap_server: comms.example.com

3. Now, like in case of the mail server role, we need to set-up authentication

   for the XMPP service. In this particular case a single consumer is present -

   Prosody itself. We should also create the group for granting the users right

   to use the service.

   :file:`~/mysite/group_vars/communications.yml`

   ::

      # Just make sure the new entry is added for the prosody user - you can

      # leave the postfix/dovecot intact in your file if you use different

      # passwords. Keep in mind password for prosody user must match with

      # password specified under xmpp_ldap_password.

      ldap_server_consumers:

        - name: postfix

          password: postfix

        - name: dovecot

          password: dovecot

        - name: prosody

          password: prosody

      # And simply append a new group here...

      ldap_server_groups:

        - name: mail

        - name: xmpp

4. Do you know what comes next? Yes! Create some more TLS private keys

   and certificates, this time for our XMPP server ;)

   1. Create new template for ``certtool``:

      :file:`~/mysite/tls/comms.example.com_xmpp.cfg`

      ::

         organization = "Example Inc."

         country = SE

         cn = "Exampe Inc. XMPP Server"

         expiration_days = 365

         dns_name = "example.com"

         tls_www_server

         signing_key

         encryption_key

   2. Create the keys and certificates for XMPP service based on the template::

        certtool --sec-param normal --generate-privkey --outfile ~/mysite/tls/comms.example.com_xmpp.key

        certtool --generate-certificate --load-ca-privkey ~/mysite/tls/ca.key --load-ca-certificate ~/mysite/tls/ca.pem --template ~/mysite/tls/comms.example.com_xmpp.cfg --load-privkey ~/mysite/tls/comms.example.com_xmpp.key --outfile ~/mysite/tls/comms.example.com_xmpp.pem

5. Apply the changes::

     workon mysite && ansible-playbook playbooks/site.yml

6. Ok, configuration of the role is complete. You may have noticed

   that we still haven't added any users to the new LDAP group called

   "xmpp". So let us correct this in similar way as we did for the

   mail server. Since we have the user entries already, no need to

   recreate them here. We will just update the group membership

   instead.

   .. warning::

      Same warning applies here as for mail server role for managing the

      user/group entries! Scroll up and re-read it if you missed it!

   ::

      workon mysite && ansible --become -m ldap_attr -a "dn=cn=xmpp,ou=groups,dc=example,dc=com state=present name=uniqueMember values=uid=johndoe,ou=people,dc=example,dc=com" communications

      workon mysite && ansible --become -m ldap_attr -a "dn=cn=xmpp,ou=groups,dc=example,dc=com state=present name=uniqueMember values=uid=janedoe,ou=people,dc=example,dc=com" communications

7. If no errors have been reported, at this point you should have two users

   capable of using the XMPP service - one with username

   ``john.doe@example.com`` and one with username ``jane.doe@example.com``. Same

   passwords are used as for when you were creating the two users for mail

   server. For testing you can turn to your favourite XMPP client (I don't know

   of any quick CLI-based tools to test the XMPP server functionality,

   unfortunately, but you could try using `mcabber <https://mcabber.com/>`_).

Taking a step back - preparing for web server

---------------------------------------------

Up until now the usage instructions have dealt almost exclusively with the

communications server. That is, we haven't done anything beyond the basic set-up

of the other servers.

Let us first define what we want to deploy on the web server. Here is the plan:

1. First off, we will set-up the web server. This will be necessary no matter

   what web application we decide to deploy later on.

2. Next, we will set-up a database server. Why? Well, most web applications

   need to use some sort of database to store all the data, so we might as well

   try to take that one out of the way.

3. With this basic deployment for a web server in place, we can move on to

   setting-up a couple of web applications. For the purpose of the usage

   instructions, we will deploy the following two:

   1. `The Bug Genie <http://thebuggenie.org/>`_ - an issue tracker. To keep

      things simple, we will not integrate it with our LDAP server (although

      this is supported and possible). Being written in PHP, this will

      demonstrate the role for PHP web application deployment.

   2. `Django Wiki <https://github.com/django-wiki/django-wiki>`_ - a wiki

      application written in Django. This will serve as a demo of how the WSGI

      role works.

It should be noted that the web application deployment roles are a bit more

complex - namely they are not meant to be used directly, but instead as a

dependency for a custom role. They do come with decent amount of batteries

included, and also play nice with the web server role.

As mentioned before, all roles will enforce TLS by default. The web server roles

will additionaly implement HSTS policy by sending connecting clients

``Strict-Transport-Security`` header with value set to ``max-age=31536000;

includeSubDomains``.

With all the above noted, let us finally move on to the next step.

Setting-up the web server

-------------------------

Finally we are moving on to the web server deployment, and we shell start

with... Well, erm, web server deployment! To be more precise, we will set-up

Nginx.

1. Update the playbook for web server to include the web server role.

    :file:`~/mysite/playbooks/web.yml`

    ::

      ---

      - hosts: web

        remote_user: ansible

        become: yes

        roles:

          - common

          - ldap_client

          - mail_forwarder

          - web_server

2. You know the drill, role configuration comes up next. No

   configuration has been deployed before for the web server, so we

   will be creating a new file. Only the TLS parameters are really

   necessary, but we'll spice things up a bit by setting custom title

   and message for default virtual host.

   :file:`~/mysite/group_vars/web.yml`

   ::

      ---

      default_https_tls_certificate: "{{ lookup('file', '~/mysite/tls/www.example.com_https.pem') }}"

      default_https_tls_key: "{{ lookup('file', '~/mysite/tls/www.example.com_https.key') }}"

      web_default_title: "Welcome to default page!"

      web_default_message: "Nothing to see here, move along..."

3. The only thing left now is to create the TLS private key/certificate pair

   that should be used for default virtual host.

   1. Create new template for ``certtool``:

      :file:`~/mysite/tls/www.example.com_https.cfg`

      ::

         organization = "Example Inc."

         country = SE

         cn = "Exampe Inc. Web Server"

         expiration_days = 365

         dns_name = "www.example.com"

         tls_www_server

         signing_key

         encryption_key

   2. Create the keys and certificates for default web server virtual host based

      on the template::

        certtool --sec-param normal --generate-privkey --outfile ~/mysite/tls/www.example.com_https.key

        certtool --generate-certificate --load-ca-privkey ~/mysite/tls/ca.key --load-ca-certificate ~/mysite/tls/ca.pem --template ~/mysite/tls/www.example.com_https.cfg --load-privkey ~/mysite/tls/www.example.com_https.key --outfile ~/mysite/tls/www.example.com_https.pem

4. Apply the changes::

     workon mysite && ansible-playbook playbooks/site.yml

5. If no errors have been reported, at this point you should have a default web

   page available and visible at https://www.example.com/ . By default plaintext

   connections are disabled, and trying to visit http://www.example.com/ should

   simply redirect you to the HTTPS address. Feel free to try it out with some

   browser. Keep in mind you will get a warning about the untrusted certificate!

Adding the database server

--------------------------

Since both of the web applications we want to deploy need a database, we will

proceed to set-up the database server role on the web server itself. *Majic

Ansible Roles* in particular come with a role that will deploy MariaDB database

server.

.. note::

   The ``database_server`` role will set-up unix socket authentication

   for the database ``root`` user. I.e. the ``root`` database user

   will have no password set, but authentication will pass only when

   logging-in as the operating system ``root`` user while connecting

   over database server unix socket.

1. Update the playbook for web server to include the database server role.

    :file:`~/mysite/playbooks/web.yml`

    ::

      ---

      - hosts: web

        remote_user: ansible

        become: yes

        roles:

          - common

          - ldap_client

          - mail_forwarder

          - web_server

          - database_server

2. This particular role has no parameters, and no additional steps are

   necessary to configure it. So move along...

3. No TLS support has been implemented for this role (yet), so simply apply the

   changes::

     workon mysite && ansible-playbook playbooks/site.yml

4. If no errors have been reported, you should have a database server up and

   running on the web server. You should be able to log-in as ``root``

   operating system user by running the following command on the web

   server itself::

     mysql

   Of course, no database has been created for either of the web applications,

   but we will get to that one later (there is a dedicated ``database`` role

   which can be combined with web app roles for this purpose).

Deploying a PHP web application (The Bug Genie)

-----------------------------------------------

We have some basic infrastructure up and running on our web server, so

now we can move on to setting-up a PHP web application on it. As

mentioned before, we will take *The Bug Genie* as an example.

For this we will create a local role in our site to take care of it. This role

will in turn utilise two roles coming from *Majic Ansible Roles* that will make

our life (a little) easier.

To make the example a bit simpler, no parameters will be introduced

for this role (not even the password for database, we'll hard-code

everything).

Before we start, here is a couple of useful pointers regarding the

``php_website`` role we'll be using for the PHP part:

* The role is designed to execute every application via dedicated user and

  group. The user/group name is automatically derived from the FQDN of website,

  for example ``web-tbg_example_com``.

* While running the application, application user's umask is set to ``0007``

  (letting the administrator user be able to manage any files created while the

  application is running).

* An administrative user is created as well, and this user should be used when

  running maintenance and installation commands. Similar to application user,

  the name is also derived from the FQDN of website, for example

  ``admin-tbg_example_com``. Administrative user does not have a dedicated

  group, and instead belongs to same group as the application user.

* PHP applications are executed via FastCGI, using *PHP-FPM*.

* If you ever need to set some additional PHP FPM settings, this can easily be

  done via the ``additional_fpm_config`` role parameter. This particular example

  does not set any, though.

* Mails delivered to local admin/application users are forwarded to ``root``

  account instead (this can be configured via ``website_mail_recipients`` role

  parameter.

* If you ever find yourself mixing-up test and production websites,

  have a look at ``environment_indicator`` role parameter. It lets you

  insert small strip with environment information at bottom of each

  HTML page served by the web server.

* Static content (non-PHP) is served directly by *Nginx*.

* Each web application gets distinct sub-directory under ``/var/www``, named

  after the FQDN. All sub-directories created under there are created with

  ``02750`` permissions, with ownership set to admin user, and group set to the

  application's group. In other words, all directories will have ``SGID`` bit

  set, allowing you to create files/directories that will have their group

  automatically set to the group of the parent directory.

* Files are served (both by *Nginx* and *PHP-FPM*) from sub-directory called

  ``htdocs`` (located in website directory). For example

  ``/var/www/tbg.example.com/htdocs/``. Normally, this can be a symlink to some

  other sub-directory within the website directory (useful for having multiple

  versions for easier downgrades etc).

* Combination of admin user membership in application group, ``SGID``

  permission, and the way ownership of sub-directories is set-up usually means

  that the administrator will be capable of managing application files, and

  application can be granted write permissions to a *minimum* of necessary

  files.

  .. warning::

     Just keep in mind that some file-management commands, like ``mv``, do *not*

     respect the ``SGID`` bit. In fact, I would recommend using ``cp`` when you

     deploy new files to the directory instead (don't simply move them from your

     home directory).

1. Start-off with creating the necessary directories for the new role::

     mkdir -p ~/mysite/roles/tbg/{tasks,meta,files}/

2. Let's set-up role dependencies, reusing some common roles to make our life

   easier.

   :file:`~/mysite/roles/tbg/meta/main.yml`

   ::

      ---

      dependencies:

        # Ok, so this role helps us set-up Nginx virtual host for serving our

        # app.

        - role: php_website

          # Our virtual host will for PHP website will respond to this name.

          fqdn: tbg.example.com

          # TLS key and certificate to use for the virtual host.

          https_tls_certificate: "{{ lookup('file', '~/mysite/tls/tbg.example.com_https.pem') }}"

          https_tls_key: "{{ lookup('file', '~/mysite/tls/tbg.example.com_https.key') }}"

          # Some additional packages are required in order to deploy and use TBG.

          packages:

            - php-gd

            - php-curl

            - php-mbstring

            - php-xml

            - git

            - php-mysql

            - php-apcu

            - php-zip

          # Set-up URL rewriting. This is based on public/.htaccess file from

          # TBG.

          php_rewrite_urls:

            - ^(.*)$ /index.php?url=$1

          # We don't necessarily need this, but in case you have a policy on

          # uid/gid usage, this is useful. Take note that below value is used

          # for both the dedicated uid and gid for application user.

          uid: 2000

          admin_uid: 3000

        # And this role sets up a new dedicated database for our web

        # application.

        - role: database

          # This is both the database name, _and_ name of the database user

          # that will be granted full privileges on the database.

          db_name: tbg

          # This will be the password of our user 'tbg' for accessing the

          # database. Take note the user can only login from localhost.

          db_password: tbg

3. Now for my favourite part again - creating private keys and certificates!

   Why?  Because the ``php_website`` role requires a private key/certificate

   pair to be deployed. So... Moving on:

   1. Create new template for ``certtool``:

      :file:`~/mysite/tls/tbg.example.com_https.cfg`

      ::

         organization = "Example Inc."

         country = SE

         cn = "Exampe Inc. Issue Tracker"

         expiration_days = 365

         dns_name = "tbg.example.com"

         tls_www_server

         signing_key

         encryption_key

   2. Create the keys and certificates for the application::

        certtool --sec-param normal --generate-privkey --outfile ~/mysite/tls/tbg.example.com_https.key

        certtool --generate-certificate --load-ca-privkey ~/mysite/tls/ca.key --load-ca-certificate ~/mysite/tls/ca.pem --template ~/mysite/tls/tbg.example.com_https.cfg --load-privkey ~/mysite/tls/tbg.example.com_https.key --outfile ~/mysite/tls/tbg.example.com_https.pem

4. Time to get our hands a bit more dirty... Up until now we didn't have to write

   custom tasks, but at this point we need to.

   :file:`~/mysite/roles/tbg/tasks/main.yml`

   ::

      ---

      - name: Define TBG version

        set_fact:

          tbg_version: "4.3.1"

          tbg_archive_checksum: "45de72b1ef82142ad46686577d593375ba370156df4367d17386b4e26a37f342"

      - name: Download the TBG archive

        get_url:

          url: "https://github.com/thebuggenie/thebuggenie/archive/v{{ tbg_version }}.tar.gz"

          dest: "/var/www/tbg.example.com/thebuggenie-{{ tbg_version }}.tar.gz"

          sha256sum: "{{ tbg_archive_checksum }}"

        become: yes

        become_user: admin-tbg_example_com

      - name: Download Composer

        get_url:

          url: "https://getcomposer.org/download/1.10.19/composer.phar"

          dest: "/usr/local/bin/composer"

          sha256sum: "688bf8f868643b420dded326614fcdf969572ac8ad7fbbb92c28a631157d39e8"

          owner: root

          group: root

          mode: 0755

      - name: Unpack TBG

        unarchive:

          src: "/var/www/tbg.example.com/thebuggenie-{{ tbg_version }}.tar.gz"

          dest: "/var/www/tbg.example.com/"

          copy: no

          creates: "/var/www/tbg.example.com/thebuggenie-{{ tbg_version }}"

        become: yes

        become_user: admin-tbg_example_com

      - name: Allow use of lib-pcre version 10 (since PHP is built against it in Debian Buster)

        lineinfile:

          dest: "/var/www/tbg.example.com/thebuggenie-{{ tbg_version }}/{{ item }}"

          state: present

          regexp: '.*"lib-pcre".*'

          line: '        "lib-pcre": ">=8.0",'

        with_items:

          - "composer.json"

          - "composer.lock"

      - name: Create TBG cache directory

        file:

          path: "/var/www/tbg.example.com/thebuggenie-{{ tbg_version }}/cache"

          state: directory

          mode: 02770

        become: yes

        become_user: admin-tbg_example_com

      - name: Set-up the necessary write permissions for TBG directories

        file:

          path: "{{ item }}"

          mode: g+w

        with_items:

          - /var/www/tbg.example.com/thebuggenie-{{ tbg_version }}/

          - /var/www/tbg.example.com/thebuggenie-{{ tbg_version }}/public/

          - /var/www/tbg.example.com/thebuggenie-{{ tbg_version }}/core/config/

      - name: Create symbolic link to TBG application

        file:

          src: "/var/www/tbg.example.com/thebuggenie-{{ tbg_version }}/public"

          path: "/var/www/tbg.example.com/htdocs"

          state: link

          owner: "admin-tbg_example_com"

          group: "web-tbg_example_com"

        become: yes

        become_user: admin-tbg_example_com

      - name: Install TBG dependencies

        composer:

          command: install

          working_dir: "/var/www/tbg.example.com/thebuggenie-{{ tbg_version }}"

        become: yes

        become_user: admin-tbg_example_com

      - name: Deploy database configuration file for TBG

        copy:

          src: "b2db.yml"

          dest: "/var/www/tbg.example.com/thebuggenie-{{ tbg_version }}/core/config/b2db.yml"

          mode: 0640

          owner: admin-tbg_example_com

          group: web-tbg_example_com

      - name: Install pexpect package

        apt:

          name: python3-pexpect

          state: present

      - name: Deploy expect script for installing TBG

        copy:

          src: "tbg_expect_install"

          dest: "/var/www/tbg.example.com/tbg_expect_install"

          mode: 0750

          owner: admin-tbg_example_com

          group: web-tbg_example_com

      - name: Run TBG installer via expect script

        command: /var/www/tbg.example.com/tbg_expect_install

        args:

          chdir: "/var/www/tbg.example.com/thebuggenie-{{ tbg_version }}"

          creates: "/var/www/tbg.example.com/thebuggenie-{{ tbg_version }}/installed"

        become: yes

        become_user: admin-tbg_example_com

5. Set-up the files that are deployed by our role.

   :file:`~/mysite/roles/tbg/files/b2db.yml`

   ::

      b2db:

          username: "tbg"

          password: "tbg"

          dsn: "mysql:host=localhost;dbname=tbg"

          tableprefix: ''

          cacheclass: '\thebuggenie\core\framework\Cache'

   :file:`~/mysite/roles/tbg/files/tbg_expect_install`

   ::

      #!/usr/bin/env python3

      import pexpect

      # Spawn the process.

      install_process = pexpect.spawnu('./tbg_cli', args = ["install",

                                                                        "--accept_license=yes",

                                                                        "--url_subdir=/",

                                                                        "--use_existing_db_info=yes",

                                                                        "--enable_all_modules=no",

                                                                        "--setup_htaccess=yes"])

      # If we get EOF, we probably already installed application, and ran

      # into error at the end since no patterns matched.