Changeset - 30d5b3fa5b93
[Not reviewed]
0 4 2
Branko Majic (branko) - 7 years ago 2018-08-18 13:51:47
branko@majic.rs
MAR-132: Added support for Debian 9 (Stretch) to database_server role:

- Updated Molecule test configuration to include Debian 9 Stretch in
test matrix.
- Updated tests related to UTF-8 configuration (differences between
Debian 8 and 9).
- Deploy UTF-8 configuration in alternate locations depending on what
distro is being used.
- Force set-up of root password on Debian Stretch (default is to use
the unix_socket authentication).
6 files changed with 101 insertions and 17 deletions:
0 comments (0 inline, 0 general)
docs/rolereference.rst
Show inline comments
 
@@ -1862,192 +1862,193 @@ Parameters
 
  To create complete requirements list, it is recommended to use `pip-tools
 
  <https://github.com/jazzband/pip-tools>`_ (the ``pip-compile`` utility) with
 
  ``gunicorn`` and ``futures`` in the ``.in.`` file.
 

	
 

	
 
Distribution compatibility
 
~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
Role is compatible with the following distributions:
 

	
 
- Debian 8 (Jessie)
 
- Debian 9 (Stretch)
 

	
 

	
 
Examples
 
~~~~~~~~
 

	
 
Here is an example configuration for setting-up a (base) WSGI website (for
 
running a bare Django project):
 

	
 
.. code-block:: yaml
 

	
 
    # Sample for a Django installation.
 
    - role: wsgi_website
 
      fqdn: django.example.com
 
      static_locations:
 
        - /static
 
        - /media
 
      uid: 2004
 
      virtualenv_packages:
 
        - django
 
      wsgi_application: django_example_com.wsgi:application
 
      environment_variables:
 
        DJANGO_SETTINGS_MODULE: "django_example_com.settings.production"
 
      https_tls_key: "{{ lookup('file', inventory_dir + '/tls/wsgi.example.com_https.key') }}"
 
      https_tls_certificate: "{{ lookup('file', inventory_dir + '/tls/wsgi.example.com_https.pem') }}"
 
      futures_version: 3.0.5
 
      gunicorn_version: 19.6.0
 
      additional_nginx_config:
 
        - comment: Use custom page for forbidden files.
 
          value: error_page 403 /static/403.html;
 
        - comment: Use custom page for non-existing locations/files.
 
          value: error_page 404 /static/404.html;
 
      website_mail_recipients: "root john.doe@example.com"
 
      environment_indicator:
 
        background_colour: "green"
 
        text_colour: "black"
 
        text: "TEST ENVIRONMENT"
 
      proxy_headers:
 
        Accept-Encoding: '""'
 

	
 
    # Use wsgi_requirements to deploy Gunicorn.
 
    - role: wsgi_website
 
      fqdn: wsgi.example.com
 
      wsgi_application: wsgi:main
 
      wsgi_requirements:
 
        - gunicorn==19.7.1
 
	- futures==3.1.1
 

	
 

	
 
Database Server
 
---------------
 

	
 
The ``database_server`` role can be used for setting-up a MariaDB database
 
server on destination machine.
 

	
 
The role implements the following:
 

	
 
* Installs MariaDB server and client.
 
* Configures MariaDB server and client to use *UTF-8* encoding by default.
 
* Sets password for the database root user.
 
* Deploys MariaDB client configuration in location ``/root/.my.cnf`` that
 
  contains username and password for the root database user.
 

	
 

	
 
Role dependencies
 
~~~~~~~~~~~~~~~~~
 

	
 
Depends on the following roles:
 

	
 
* **common**
 

	
 

	
 
Parameters
 
~~~~~~~~~~
 

	
 
**db_root_password** (string, mandatory)
 
  Password for the *root* database user.
 

	
 

	
 
Distribution compatibility
 
~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
Role is compatible with the following distributions:
 

	
 
- Debian 8 (Jessie)
 
- Debian 9 (Stretch)
 

	
 

	
 
Examples
 
~~~~~~~~
 

	
 
Here is an example configuration for setting-up the database server:
 

	
 
.. code-block:: yaml
 

	
 
   ---
 

	
 
   db_root_password: root
 

	
 

	
 
Database
 
--------
 

	
 
The ``database`` role can be used for creating a MariaDB database and
 
accompanying user on destination machine.
 

	
 
The role implements the following:
 

	
 
* Creates MariaDB database.
 
* Creates a dedicated user capable of performing any operation on the created
 
  database. Username is set to be same as the name of database.
 
* Sets-up pre-backup task that creates database dump in location
 
  ``/srv/backup/mariadb/{{ db_name }}.sql``.
 

	
 

	
 
Role dependencies
 
~~~~~~~~~~~~~~~~~
 

	
 
Depends on the following roles:
 

	
 
* **database_server**
 
* **backup_client**
 

	
 

	
 
Backups
 
~~~~~~~
 

	
 
If the backup for this role has been enabled, the following paths are backed-up:
 

	
 
**/srv/backup/maraidb/{{ db_name }}.sql**
 
  Dump of the database. Database dump is created every day at 01:45 in the
 
  morning.
 

	
 

	
 
Parameters
 
~~~~~~~~~~
 

	
 
**db_name** (string, mandatory)
 
  Name of the database that should be created.
 

	
 
**db_password** (string, mandatory)
 
  Password for the database user.
 

	
 

	
 
Distribution compatibility
 
~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
Role is compatible with the following distributions:
 

	
 
- Debian 8 (Jessie)
 

	
 

	
 
Examples
 
~~~~~~~~
 

	
 
Here is an example configuration for creating a single database (for some
 
website):
 

	
 
.. code-block:: yaml
 

	
 
  - role: database
 
    db_name: phpinfo_example_com
 
    db_password: phpinfo_example_com
 

	
 

	
 
Backup Server
 
-------------
 

	
 
The ``backup_server`` role can be used for setting-up a server to act as backup
 
storage for the backup clients. Storage is made available to the clients
 
exclusively via SFTP on a dedicated port and dedicated OpenSSH server
 
instance. This instance is specifically configured and tailored for this
 
purpose.
 

	
 
The role is primarily aimed for use with `Duplicity
 
<http://duplicity.nongnu.org/>`_, but should be also usable for generic SFTP
 
uploads.
 

	
 
The role implements the following:
 

	
 
* Installs backup software (Duplicity, Duply).
 
* Creates a dedicated directory structure for backups with the following structure:
roles/database_server/molecule/default/molecule.yml
Show inline comments
 
---
 

	
 
dependency: {}
 

	
 
driver:
 
  name: vagrant
 
  provider:
 
    name: virtualbox
 

	
 
lint:
 
  name: yamllint
 
  options:
 
    config-file: ../../.yamllint.yml
 

	
 
platforms:
 

	
 
  - name: parameters-mandatory-jessie64
 
    groups:
 
      - parameters-mandatory
 
    box: debian/contrib-jessie64
 
    memory: 512
 
    cpus: 1
 

	
 
  - name: parameters-mandatory-stretch64
 
    groups:
 
      - parameters-mandatory
 
    box: debian/contrib-stretch64
 
    memory: 512
 
    cpus: 1
 

	
 
provisioner:
 
  name: ansible
 
  config_options:
 
    ssh_connection:
 
      pipelining: "True"
 
  lint:
 
    name: ansible-lint
 

	
 
scenario:
 
  name: default
 

	
 
verifier:
 
  name: testinfra
 
  lint:
 
    name: flake8
roles/database_server/molecule/default/tests/test_default.py
Show inline comments
 
import os
 

	
 
import testinfra.utils.ansible_runner
 

	
 

	
 
testinfra_hosts = testinfra.utils.ansible_runner.AnsibleRunner(
 
    os.environ['MOLECULE_INVENTORY_FILE']).get_hosts(['parameters-mandatory'])
 

	
 

	
 
def test_installed_packages(host):
 
    """
 
    Tests if the correct packages have been installed.
 
    """
 

	
 
    assert host.package('mariadb-client').is_installed
 
    assert host.package('mariadb-server').is_installed
 
    assert host.package('python-mysqldb').is_installed
 

	
 

	
 
def test_service(host):
 
    """
 
    Tests if the database server service is enabled on boot and running.
 
    """
 

	
 
    service = host.service('mysql')
 

	
 
    assert service.is_enabled
 
    assert service.is_running
 

	
 

	
 
def test_root_password(host):
 
    """
 
    Tests if the root password has been set correctly.
 
    """
 

	
 
    login = host.run("mysql -uroot -proot_password -BNe 'show databases'")
 

	
 
    assert login.rc == 0
 
    assert "information_schema" in login.stdout
 
    assert "mysql" in login.stdout
 
    assert "performance_schema" in login.stdout
 

	
 

	
 
def test_root_my_cnf(host):
 
    """
 
    Tests if the root my.cnf configuration has been set-up with correct
 
    user/password and permissions.
 
    """
 

	
 
    with host.sudo():
 

	
 
        my_cnf = host.file('/root/.my.cnf')
 

	
 
        assert my_cnf.is_file
 
        assert my_cnf.user == 'root'
 
        assert my_cnf.group == 'root'
 
        assert my_cnf.mode == 0o400
 
        assert "user=root" in my_cnf.content
 
        assert "password=root_password" in my_cnf.content
 

	
 

	
 
def test_root_my_cnf_login(host):
 
    """
 
    Tets if the database server root login works using just the my.cnf
 
    configuration file.
 
    """
 

	
 
    with host.sudo():
 

	
 
        login = host.run("mysql -BNe 'show databases'")
 

	
 
        assert "information_schema" in login.stdout
 
        assert "mysql" in login.stdout
 
        assert "performance_schema" in login.stdout
 

	
 

	
 
def test_utf8_configuration_file(host):
 
    """
 
    Tests if UTF-8 database server configuration file has been deployed
 
    correctly.
 
    """
 

	
 
    config = host.file('/etc/mysql/conf.d/utf8.cnf')
 

	
 
    assert config.is_file
 
    assert config.user == 'root'
 
    assert config.group == 'root'
 
    assert config.mode == 0o644
 

	
 

	
 
def test_utf8_configuration(host):
 
    """
 
    Tests if UTF-8 configuration has been applied correctly to server.
 
    """
 

	
 
    assert host.run("mysql -uroot -proot_password -BNe 'drop database if exists test'").rc == 0
 
    assert host.run("mysql -uroot -proot_password -BNe 'create database test'").rc == 0
 

	
 
    check_server = host.run("mysql -uroot -proot_password test -BNe 'select @@character_set_server; select @@collation_server'")
 

	
 
    assert check_server.rc == 0
 
    assert check_server.stdout == "utf8\nutf8_general_ci"
 

	
 
    check_database = host.run("mysql -uroot -proot_password test -BNe 'select @@character_set_database; select @@collation_database'")
 

	
 
    assert check_database.rc == 0
 
    assert check_database.stdout == "utf8\nutf8_general_ci"
 

	
 
    check_database = host.run("mysql -uroot -proot_password -BNe 'select @@character_set_connection; select @@collation_connection'")
 

	
 
    assert check_database.rc == 0
 
    assert check_database.stdout == "utf8\nutf8_general_ci"
roles/database_server/molecule/default/tests/test_default_jessie64.py
Show inline comments
 
new file 100644
 
import os
 

	
 
import testinfra.utils.ansible_runner
 

	
 

	
 
testinfra_hosts = testinfra.utils.ansible_runner.AnsibleRunner(
 
    os.environ['MOLECULE_INVENTORY_FILE']).get_hosts(['parameters-mandatory-jessie64'])
 

	
 

	
 
def test_utf8_configuration_file(host):
 
    """
 
    Tests if UTF-8 database server configuration file has been deployed
 
    correctly.
 
    """
 

	
 
    config = host.file('/etc/mysql/conf.d/utf8.cnf')
 

	
 
    assert config.is_file
 
    assert config.user == 'root'
 
    assert config.group == 'root'
 
    assert config.mode == 0o644
 

	
 

	
 
def test_stretch_utf8_configuration_file_absent(host):
 
    """
 
    Tests if the Stretch configuration file is absent.
 
    """
 

	
 
    config = host.file('/etc/mysql/mariadb.conf.d/90-utf8.cnf')
 

	
 
    assert not config.exists
roles/database_server/molecule/default/tests/test_default_stretch64.py
Show inline comments
 
new file 100644
 
import os
 

	
 
import testinfra.utils.ansible_runner
 

	
 

	
 
testinfra_hosts = testinfra.utils.ansible_runner.AnsibleRunner(
 
    os.environ['MOLECULE_INVENTORY_FILE']).get_hosts(['parameters-mandatory-stretch64'])
 

	
 

	
 
def test_utf8_configuration_file(host):
 
    """
 
    Tests if UTF-8 database server configuration file has been deployed
 
    correctly.
 
    """
 

	
 
    config = host.file('/etc/mysql/mariadb.conf.d/90-utf8.cnf')
 

	
 
    assert config.is_file
 
    assert config.user == 'root'
 
    assert config.group == 'root'
 
    assert config.mode == 0o644
 

	
 

	
 
def test_jessie_utf8_configuration_file_absent(host):
 
    """
 
    Tests if the Jessie configuration file is absent.
 
    """
 

	
 
    config = host.file('/etc/mysql/conf.d/utf8.cnf')
 

	
 
    assert not config.exists
roles/database_server/tasks/main.yml
Show inline comments
 
---
 

	
 
- name: Install MariaDB
 
  apt:
 
    name: "{{ item }}"
 
    state: present
 
  with_items:
 
    - mariadb-client
 
    - mariadb-server
 
    - python-mysqldb
 

	
 
- name: Enable MariaDB service on boot (workaround for systemctl broken handling of SysV)
 
  command: rcconf -on mysql
 
  register: result
 
  changed_when: result.stderr == ""
 

	
 
- name: Enable and start MariaDB
 
  service:
 
    name: mysql
 
    state: started
 

	
 
- name: Set password for the root database user
 
  mysql_user:
 
    check_implicit_admin: true
 
    name: root
 
    password: "{{ db_root_password }}"
 

	
 
- name: Deploy username and password for the root database user
 
  template:
 
    src: "root_my.cnf.j2"
 
    dest: "/root/.my.cnf"
 
    owner: root
 
    group: root
 
    mode: 0400
 

	
 
- name: Set UTF-8 encoding as default for MariaDB
 
- name: Check if root user authentication is based on use of unix_socket module (Stretch default)
 
  command: mysql --skip-column-names -B -e "select 1 from mysql.user where user='root' and plugin='unix_socket';"
 
  when: "ansible_distribution_release == 'stretch'"
 
  register: "root_using_unix_socket_authentication"
 
  changed_when: false
 

	
 
- name: Disable use of unix socket login on Debian Stretch (temporary workaround)
 
  command: "mysql -B -e \"update mysql.user set plugin='' where user='root' and plugin='unix_socket'; flush privileges;\""
 
  when: "ansible_distribution_release == 'stretch' and root_using_unix_socket_authentication.stdout != ''"
 

	
 
- name: Remove UTF-8 encoding configuration file from the old location on Debian Stretch
 
  file:
 
    path: "/etc/mysql/conf.d/utf8.cnf"
 
    state: absent
 
  when: "ansible_distribution_release == 'stretch'"
 
  register: mariadb_utf8_configuration_stretch
 

	
 
- name: Set UTF-8 encoding as default for MariaDB (on Jessie)
 
  copy:
 
    src: "utf8.cnf"
 
    dest: "/etc/mysql/conf.d/utf8.cnf"
 
    owner: root
 
    group: root
 
    mode: 0644
 
  register: mariadb_utf8_configuration
 
  when: "ansible_distribution_release == 'jessie'"
 
  register: mariadb_utf8_configuration_jessie
 

	
 
- name: Set UTF-8 encoding as default for MariaDB (on Stretch)
 
  copy:
 
    src: "utf8.cnf"
 
    dest: "/etc/mysql/mariadb.conf.d/90-utf8.cnf"
 
    owner: root
 
    group: root
 
    mode: 0644
 
  when: "ansible_distribution_release == 'stretch'"
 
  register: mariadb_utf8_configuration_stretch
 

	
 
- name: Restart MariaDB in order to use UTF-8 as default character set
 
  service:
 
    name: mysql
 
    state: restarted
 
  when: mariadb_utf8_configuration.changed
 
  when: mariadb_utf8_configuration_jessie.changed or mariadb_utf8_configuration_stretch.changed
 
  tags:
 
    # [ANSIBLE0016] Tasks that run when changed should likely be handlers
 
    #   UTF-8 configuration must be applied immediatelly in order to ensure that
 
    #   subsequent tasks that create databases will end-up with correct (UTF-8)
 
    #   encoding. Otherwise they will be created using default latin1.
 
    - skip_ansible_lint
 

	
 
- name: Explicitly run all handlers
 
  include: ../handlers/main.yml
 
  when: "handlers | default(False) | bool() == True"
 
  tags:
 
    - handlers
0 comments (0 inline, 0 general)