Changeset - 6b87dd13b24c
[Not reviewed]
0 7 0
Branko Majic (branko) - 9 years ago 2017-04-09 17:54:20
branko@majic.rs
MAR-96: Synchronised introduction in index.rst and about.rst. Added information about distribution compatibility, along with mention in introductory text.Switched to classic theme for documentation to get rid of documentation build warnings. Updated backup_client role so it can work with both Debian 8 and Debian 9. Changes between distro versions include: calling correct GnuPG binary, using correct GnuPG key ID format, installation additional dependencies, using correct Duply settings. Fixed _major_ bug related to additional backup keys (essentially, this never worked correctly due to wrong registered value being used when populating the Duply keyring).
7 files changed with 185 insertions and 7 deletions:
0 comments (0 inline, 0 general)
docs/about.rst
Show inline comments
 
@@ -8,12 +8,16 @@ The roles are kept as a separate project in hope of making them potentially
 
useful to wider audience, and for reference purposes.
 

	
 
Roles cover different aspects of infrastructure, such as mail servers, web
 
servers, web applications etc. The roles are mainly well-suited for smaller
 
installations.
 

	
 
Roles are mainly written for use with *Debian 8 Jessie*, although some support
 
*Debian 9 Stretch* as well. You can find out more about distribution
 
compatibility in :ref:`rolereference`.
 

	
 
At the moment, the roles have been written for and tested against **Ansible
 
1.9.x**.
 

	
 
Roles should work for the most part with **Ansible 2.0.x** and **Ansible
 
2.1.x**, but due to bugs in those releases they should not be used for
 
production purposes. Roles are kept forward-compatible as much as possible until
 
@@ -22,12 +26,16 @@ are:
 

	
 
* Inability to use dynamic name handlers (handlers that include variables in
 
  their name).
 
* Referencing non-existing handlers does not produce error.
 
* Referencing non-existing tags does not produce error.
 

	
 
The role also utilises the ``dig`` lookup plugin which requires ``dnspython``
 
package to be installed. Make sure you have the package available on controller
 
machine.
 

	
 

	
 
Why were these roles created?
 
-----------------------------
 

	
 
For a long time I have had a couple of Internet-facing servers where I hosted
 
all the IT infrastructure I needed for my day-to-day life.
docs/conf.py
Show inline comments
 
@@ -98,13 +98,13 @@ pygments_style = 'sphinx'
 

	
 

	
 
# -- Options for HTML output ----------------------------------------------
 

	
 
# The theme to use for HTML and HTML Help pages.  See the documentation for
 
# a list of builtin themes.
 
html_theme = 'default'
 
html_theme = 'classic'
 

	
 
# Theme options are theme-specific and customize the look and feel of a theme
 
# further.  For a list of options available for each theme, see the
 
# documentation.
 
#html_theme_options = {}
 

	
docs/index.rst
Show inline comments
 
Majic Ansible Roles documentation
 
=================================
 

	
 
Majic Ansible Roles is a collection of Ansible roles that are used on regular
 
basis for deployment and maintenance of Majic infrastructure.
 

	
 
The roles are kept as a separate project in hope of making them potentially
 
useful to wider audience, and for reference purposes.
 

	
 
Roles cover different aspects of infrastructure, such as mail servers, web
 
servers, web applications etc. The roles are mainly well-suited for smaller
 
installations.
 

	
 
Roles are mainly written for use with *Debian 8 Jessie*, although some support
 
*Debian 9 Stretch* as well. You can find out more about distribution
 
compatibility in :ref:`rolereference`.
 

	
 
At the moment, the roles have been written for and tested against **Ansible
 
1.9.x**.
 

	
 
Roles should work for the most part with **Ansible 2.0.x** and **Ansible
 
2.1.x**, but due to bugs in those releases they should not be used for
 
production purposes. Roles are kept forward-compatible as much as possible until
docs/rolereference.rst
Show inline comments
 
@@ -137,12 +137,21 @@ Parameters
 

	
 
**preseed_timezone** (string, optional, ``Europe/Stockholm``)
 
  Timezone that should be used when calculating server time. It is assumed that
 
  the local hardware clock is set to UTC.
 

	
 

	
 
Distribution compatibility
 
~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
Role is compatible with the following distributions:
 

	
 
- Debian 8 (Jessie)
 
- Debian 9 (Stretch)
 

	
 

	
 
Examples
 
~~~~~~~~
 

	
 
Here is an example configuration for a preseed file that sets some global
 
defaults to be used for all servers, and then overrides it for one server:
 

	
 
@@ -200,12 +209,21 @@ Parameters
 

	
 
**ansible_key** (string, optional, ``{{ lookup('file', '~/.ssh/id_rsa.pub') }}``)
 
  SSH public key that should be deployed to authorized_keys truststore for
 
  operating system user ``ansible``.
 

	
 

	
 
Distribution compatibility
 
~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
Role is compatible with the following distributions:
 

	
 
- Debian 8 (Jessie)
 
- Debian 9 (Stretch)
 

	
 

	
 
Examples
 
~~~~~~~~
 

	
 
Since the role is meant to be used just after the server has been installed, and
 
using the ``root`` account, it is probably going to be invoked from a separate
 
playbook.
 
@@ -375,12 +393,21 @@ Parameters
 
  Optional identifier appended to regular Bash prompt, useful for visually
 
  identifying distinct environments. For example, if set to ``test``, resulting
 
  prompt will be similar to ``admin@web[test]:~$``. Setting affects Bash shells
 
  *only*.
 

	
 

	
 
Distribution compatibility
 
~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
Role is compatible with the following distributions:
 

	
 
- Debian 8 (Jessie)
 
- Debian 9 (Stretch)
 

	
 

	
 
Examples
 
~~~~~~~~
 

	
 
Here is an example configuration for setting-up some common users, groups, and
 
packages on all servers:
 

	
 
@@ -449,12 +476,20 @@ Parameters
 
    Name of configuration option.
 

	
 
  **value** (string, mandatory)
 
    Value for configuration option.
 

	
 

	
 
Distribution compatibility
 
~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
Role is compatible with the following distributions:
 

	
 
- Debian 8 (Jessie)
 

	
 

	
 
Examples
 
~~~~~~~~
 

	
 
Here is an example configuration for setting some common LDAP client options:
 

	
 
.. code-block:: yaml
 
@@ -652,12 +687,20 @@ Parameters
 
  cipher specification that should also include what TLS protocol versions
 
  should be used. Value should be compatible with OpenLDAP server option
 
  ``olcTLSCipherSuite``. Default value allows only TLSv1.2 and strong PFS
 
  ciphers.
 

	
 

	
 
Distribution compatibility
 
~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
Role is compatible with the following distributions:
 

	
 
- Debian 8 (Jessie)
 

	
 

	
 
Examples
 
~~~~~~~~
 

	
 
Here is an example configuration for setting-up LDAP server:
 

	
 
.. code-block:: yaml
 
@@ -815,12 +858,20 @@ Parameters
 

	
 
**xmpp_tls_key** (string, optional, ``{{ lookup('file', tls_private_key_dir + '/' + fqdn + '_xmpp.key') }}``)
 
  Private key used for TLS for XMPP service. The file will be stored in
 
  directory ``/etc/ssl/private/`` under name ``{{ ansible_fqdn }}_xmpp.key``.
 

	
 

	
 
Distribution compatibility
 
~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
Role is compatible with the following distributions:
 

	
 
- Debian 8 (Jessie)
 

	
 

	
 
Examples
 
~~~~~~~~
 

	
 
Here is an example configuration for setting-up XMPP server using Prosody:
 

	
 
.. code-block:: yaml
 
@@ -1038,12 +1089,20 @@ Parameters
 
  List of networks from which mail relaying is allowed even without
 
  authentication. Each item in the list is a string defining a network. The
 
  format must be compatible with Postfix ``mynetworks`` setting (for example:
 
  ``192.168.1.0/24``, ``myhost.example.com`` etc).
 

	
 

	
 
Distribution compatibility
 
~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
Role is compatible with the following distributions:
 

	
 
- Debian 8 (Jessie)
 

	
 

	
 
Examples
 
~~~~~~~~
 

	
 
Here is an example configuration for setting-up XMPP server using Prosody:
 

	
 
.. code-block:: yaml
 
@@ -1135,12 +1194,21 @@ Parameters
 
**smtp_relay_truststore** (string, optional, ``{{ lookup('file', tls_certificate_dir + '/truststore.pem') }}``)
 
  X.509 certificate chain used for issuing certificate for the SMTP relay
 
  service. The file will be stored in location
 
  ``/etc/ssl/certs/smtp_relay_truststore.pem``
 

	
 

	
 
Distribution compatibility
 
~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
Role is compatible with the following distributions:
 

	
 
- Debian 8 (Jessie)
 
- Debian 9 (Stretch)
 

	
 

	
 
Examples
 
~~~~~~~~
 

	
 
Here is an example configuration for setting-up the mail forwarder:
 

	
 
.. code-block:: yaml
 
@@ -1218,12 +1286,20 @@ Parameters
 
  TLS ciphers to enable on the web server. This should be an OpenSSL-compatible
 
  cipher specification. Value should be compatible with Nginx configuration
 
  option ``ssl_ciphers``. Default value allows only TLSv1.2 and strong PFS
 
  ciphers.
 

	
 

	
 
Distribution compatibility
 
~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
Role is compatible with the following distributions:
 

	
 
- Debian 8 (Jessie)
 

	
 

	
 
Examples
 
~~~~~~~~
 

	
 
Here is an example configuration for setting-up web server:
 

	
 
.. code-block:: yaml
 
@@ -1406,12 +1482,20 @@ Parameters
 
**website_mail_recipients** (string, optional, ``root``)
 
  Space-separated list of e-mails or local users to which the mails, sent to
 
  either the website admin or website user, should be forwarded to. Forwarding
 
  is configured via ``~/.forward`` configuration file.
 

	
 

	
 
Distribution compatibility
 
~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
Role is compatible with the following distributions:
 

	
 
- Debian 8 (Jessie)
 

	
 

	
 
Examples
 
~~~~~~~~
 

	
 
Here is an example configuration for setting-up two (base) PHP websites (for
 
running *ownCloud* and *The Bug Genie* applications):
 

	
 
@@ -1661,12 +1745,20 @@ Parameters
 
  ``use_paste`` option is enabled, the value should be equal to filename of the
 
  Python Paste ini file, located in the ``code`` sub-directory. It should be
 
  noted that in either case the value should be specsified relative to the
 
  ``code`` sub-directory. I.e. don't use full paths.
 

	
 

	
 
Distribution compatibility
 
~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
Role is compatible with the following distributions:
 

	
 
- Debian 8 (Jessie)
 

	
 

	
 
Examples
 
~~~~~~~~
 

	
 
Here is an example configuration for setting-up a (base) WSGI website (for
 
running a bare Django project):
 

	
 
@@ -1728,12 +1820,20 @@ Parameters
 
~~~~~~~~~~
 

	
 
**db_root_password** (string, mandatory)
 
  Password for the *root* database user.
 

	
 

	
 
Distribution compatibility
 
~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
Role is compatible with the following distributions:
 

	
 
- Debian 8 (Jessie)
 

	
 

	
 
Examples
 
~~~~~~~~
 

	
 
Here is an example configuration for setting-up the database server:
 

	
 
.. code-block:: yaml
 
@@ -1784,12 +1884,20 @@ Parameters
 
  Name of the database that should be created.
 

	
 
**db_password** (string, mandatory)
 
  Password for the database user.
 

	
 

	
 
Distribution compatibility
 
~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
Role is compatible with the following distributions:
 

	
 
- Debian 8 (Jessie)
 

	
 

	
 
Examples
 
~~~~~~~~
 

	
 
Here is an example configuration for creating a single database (for some
 
website):
 

	
 
@@ -1887,12 +1995,20 @@ Parameters
 
    ssh-keygen -f backup_server_dsa_key -N '' -t dsa
 
    ssh-keygen -f backup_server_rsa_key -N '' -t rsa
 
    ssh-keygen -f backup_server_ed25519_key -N '' -t ed25519
 
    ssh-keygen -f backup_server_ecdsa_key -N '' -t ecdsa
 

	
 

	
 
Distribution compatibility
 
~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
Role is compatible with the following distributions:
 

	
 
- Debian 8 (Jessie)
 

	
 

	
 
Examples
 
~~~~~~~~
 

	
 
Here is an example configuration for setting-up the backup server role:
 

	
 
.. code-block:: yaml
 
@@ -2008,12 +2124,21 @@ Parameters
 
  Port on the backup server to connect to for accessing the SFTP service.
 

	
 
**backup_ssh_key** (string, mandatory)
 
  SSH private key for logging-in into the backup server.
 

	
 

	
 
Distribution compatibility
 
~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
Role is compatible with the following distributions:
 

	
 
- Debian 8 (Jessie)
 
- Debian 9 (Stretch)
 

	
 

	
 
Examples
 
~~~~~~~~
 

	
 
Here is an example configuration for setting-up the role (take note that lookup
 
plugin is quite useful here for fetching key values from some local directory):
 

	
 
@@ -2071,12 +2196,21 @@ Parameters
 

	
 
**backup_patterns** (list, optional, ``[]``)
 
  List of globbing patterns defining which files or directories should be
 
  backed-up.
 

	
 

	
 
Distribution compatibility
 
~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
Role is compatible with the following distributions:
 

	
 
- Debian 8 (Jessie)
 
- Debian 9 (Stretch)
 

	
 

	
 
Examples
 
~~~~~~~~
 

	
 
Here is an example configuration for setting-up the role:
 

	
 
.. code-block:: yaml
roles/backup_client/handlers/main.yml
Show inline comments
 
---
 

	
 
- name: Clean-up GnuPG keyring for import of new keys
 
  shell: rm -f /etc/duply/main/gnupg/*
 

	
 
- name: Import private keys
 
  command: gpg2 --homedir /etc/duply/main/gnupg --import /etc/duply/main/private_keys.asc
 
  command: "{{ gnupg_binary }} --homedir /etc/duply/main/gnupg --import /etc/duply/main/private_keys.asc"
 

	
 
- name: Import public keys
 
  command: gpg2 --homedir /etc/duply/main/gnupg --import /etc/duply/main/public_keys.asc
 
  command: "{{ gnupg_binary }} --homedir /etc/duply/main/gnupg --import /etc/duply/main/public_keys.asc"
 
  when: backup_additional_encryption_keys
roles/backup_client/tasks/main.yml
Show inline comments
 
---
 

	
 
# Determine how to invoke the GnuPG binary based on Debian version.
 
- set_fact: gnupg_binary="gpg2"
 
  when: "ansible_distribution == 'Debian' and ansible_distribution_release == 'jessie'"
 
- set_fact: gnupg_binary="gpg"
 
  when: "ansible_distribution == 'Debian' and ansible_distribution_release == 'stretch'"
 

	
 
# Determine cut-off for the GnuPG key ID (long vs short) based on Debian
 
# version.
 
- set_fact: gnupg_key_cutoff="{8}"
 
  when: "ansible_distribution == 'Debian' and ansible_distribution_release == 'jessie'"
 
- set_fact: gnupg_key_cutoff="{0}"
 
  when: "ansible_distribution == 'Debian' and ansible_distribution_release == 'stretch'"
 

	
 
- name: Install pexpect for pexpect+sftp Duplicity backend (only on Stretch)
 
  apt: name="python-pexpect" state=installed
 
  when: "ansible_distribution == 'Debian' and ansible_distribution_release == 'stretch'"
 

	
 
- name: Install backup software
 
  apt: name="{{ item }}" state=installed
 
  with_items:
 
    - duplicity
 
    - duply
 

	
 
@@ -31,19 +48,19 @@
 
  notify:
 
    - Clean-up GnuPG keyring for import of new keys
 
    - Import private keys
 
    - Import public keys
 

	
 
- name: Extract encryption key identifier (Duplicty requires key ID in hexadecimal format)
 
  shell: "gpg2 --list-packets /etc/duply/main/private_keys.asc | grep keyid: | head -n1 | sed -e 's/.*: //' | sed -re 's/^.{8}//'"
 
  shell: "{{ gnupg_binary }} --list-packets /etc/duply/main/private_keys.asc | grep keyid: | head -n1 | sed -e 's/.*: //' | sed -re 's/^.{{gnupg_key_cutoff}}//'"
 
  register: backup_encryption_key_id
 
  changed_when: False
 
  failed_when: backup_encryption_key_id.stdout == ""
 

	
 
- name: Extract additional encryption keys identifiers (Duplicty requires key ID in hexadecimal format)
 
  shell: "gpg2 --list-packets /etc/duply/main/private_keys.asc | grep keyid: | head -n1 | sed -e 's/.*: //' | sort -u | sed -re 's/^.{8}//' | tr '\n' ',' | sed -e 's/,$//'"
 
  shell: "{{ gnupg_binary }} --list-packets /etc/duply/main/public_keys.asc | grep keyid: | head -n1 | sed -e 's/.*: //' | sort -u | sed -re 's/^.{{gnupg_key_cutoff}}//' | tr '\n' ',' | sed -e 's/,$//'"
 
  register: backup_additional_encryption_keys_ids
 
  when: backup_additional_encryption_keys
 
  changed_when: False
 
  failed_when: backup_additional_encryption_keys_ids.stdout == ""
 

	
 
- name: Deploy private SSH key for logging-in into backup server
roles/backup_client/templates/duply_main_conf.j2
Show inline comments
 
# GnuPG keys that should be used for encryption. Normally the encryption key is
 
# not available locally.
 
GPG_KEYS_ENC='{{ backup_encryption_key_id.stdout }}{% if backup_additional_encryption_keys %},{{ backup_additional_encryption_keys_ids.stdout }}{% endif %}'
 

	
 
# GnuPG key used for signing.
 
GPG_KEY_SIGN='{{backup_encryption_key_id.stdout }}'
 
GPG_KEY_SIGN='{{ backup_encryption_key_id.stdout }}'
 

	
 
# Trust all keys available in the GnuPG keyring.
 
GPG_OPTS="--homedir /etc/duply/main/gnupg/ --trust-model always"
 

	
 
# Destination where the backups are stored at.
 
{% if ansible_distribution == 'Debian' and ansible_distribution_release == 'stretch' %}
 
TARGET='pexpect+sftp://{{ backup_client_username }}@{{ backup_server }}:{{ backup_server_port }}//{{ backup_server_destination }}'
 
{% else %}
 
TARGET='sftp://{{ backup_client_username }}@{{ backup_server }}:{{ backup_server_port }}//{{ backup_server_destination }}'
 
{% endif %}
 

	
 
# Base directory to backup (root). File selection is done via include/exclude
 
# patterns.
 
SOURCE='/'
 

	
 
# Maximum age for preserving old backups. Used when running the "purge"
 
@@ -44,10 +48,14 @@ ARCH_DIR="/var/cache/duply/main/"
 
DUPL_PARAMS="$DUPL_PARAMS --use-agent"
 

	
 
# Use the pexepct backend for Duplicity so we can pass in all the
 
# ssh-options. Use dedicated known hosts and identity file when connecting over
 
# SFTP. Using -oLogLevel=ERROR makes output a bit less verbose. This is mainly
 
# to avoid output from sftp telling us it added IP address to known_hosts.
 
{% if ansible_distribution == 'Debian' and ansible_distribution_release == 'stretch' %}
 
DUPL_PARAMS="$DUPL_PARAMS --ssh-options='-oLogLevel=ERROR -oUserKnownHostsFile=/dev/null -oGlobalKnownHostsFile=/etc/duply/main/ssh/known_hosts -oIdentityFile=/etc/duply/main/ssh/identity'"
 
{% else %}
 
DUPL_PARAMS="$DUPL_PARAMS --ssh-backend pexpect --ssh-options='-oLogLevel=ERROR -oUserKnownHostsFile=/dev/null -oGlobalKnownHostsFile=/etc/duply/main/ssh/known_hosts -oIdentityFile=/etc/duply/main/ssh/identity'"
 
{% endif %}
 

	
 
# By default we exclude everything, and then include only specific patterns.
 
DUPL_PARAMS="$DUPL_PARAMS --include-globbing-filelist /etc/duply/main/include"
 
\ No newline at end of file
 
DUPL_PARAMS="$DUPL_PARAMS --include-globbing-filelist /etc/duply/main/include"
0 comments (0 inline, 0 general)