Release notes

=============

x.y.z

-----

Dropped support for Debian 10 (Stretch), alongside changes required to

run applications using Debian-only repositories.

**Breaking changes:**

* All roles

  * Dropped support for Debian 10 (Stretch).

* ``xmpp_server`` role

  * Parameter ``xmpp_prosody_package`` has been dropped.

**New features/improvements**

* ``xmpp_server`` role

  * Drop dependency on the external (Prosody) package

    repository. Install everything using official Debian

    repositories. This should help avoid future issues with Prosody

    project removing older versions of packages or dropping entire

    repository archives for older Debian releases.

  * Prosody package and some of its dependencies are installed from

    Debian backports to get more featureful release installed.

6.0.0

-----

Added support for Debian 10 (Buster), alongside a couple of minor

changes and features/improvements.

**Breaking changes:**

* ``ldap_server`` role

  * Use 2048-bit Diffie-Hellman parameters for relevant TLS

    ciphers. This could introduce incompatibility with older

    clients/servers trying to connect to the LDAP server. This change

    is applicable only under Debian Buster.

**New features/improvements:**

* All roles

  * Added support for Debian 10 (Buster).

* ``common`` role

  * Added parameters ``maintenance`` and ``maintenance_allowed_hosts``

    for enabling maintenance mode. In maintenance mode only the listed

    hosts are allowed to connect to the server.

**Bug fixes:**

* ``ldap_server`` role

  * Allow use of DHE TLS ciphers by generating the necessary

    Diffie-Hellman parameters. This bug fix is applicable only under

    Debian Buster.

* ``wsgi_website_`` role

  * When the virtual environment is created, the ``setuptools`` and

    ``pip`` packages will not get pinned to any specific version,

    allowing roles that are based on ``wsgi_website`` to easily

    install preferred versions, and avoid idempotence problems in the

    process.

5.0.0

-----

Upgrade to Ansible 2.9.x, dropping support for Debian 8 Jessie,

upgrade to Python 3.x, dropping support for Python 2.7. A number of

parameters have been made mandatory or deprecated. Security has been

slightly improved in a number of roles, and there is plenty of

bug-fixes and minor improvements throughout as well.

**Breaking changes:**

* Switched to Ansible 2.9.x, removing support for older versions. All

  documentation has been updated.

* Switched to using Python 3 on both controller and managed server

  side. Python 2.7 can no longer be used for this purpose. Support for

  WSGI applications running on Python 2.7 remains.

* All roles

  * Support for Debian 8 Jessie has been dropped.

  * Common parameters ``tls_private_key_dir`` and

    ``tls_certificate_dir`` are no longer used.

  * TLS private key and certificate parameters are now mandatory.

* ``bootstrap`` role

  * Parameter ``ansible_key`` is now mandatory.

* ``common`` role``

  * Minimum version of ``pip-tools`` in the ``pip_check_requirements``

    and ``pip_check_requirements_py3`` is now 5.3.0. This change was

    required in order to fix the deprecation warnings being sent out

    when the ``pip_check_requirements_upgrades.sh`` script is run.

* ``database_server`` role

  * Parameter ``db_root_password`` has been deprecated. The root user

    can now login into the database (as the root database user) via

    unix socket authentication.

  * Role will drop the use of Debian system maintenance user

    (``debian-sys-maint``) in favour of using the root account with

    UNIX socket authentication if the database server has not already

    been set-up in that manner. This is the default behaviour starting

    from Debian Stretch, and the ``debian-sys-main`` will be present

    only if the server has been upgraded from older releases.

* ``ldap_server`` role

  * Parameter ``ldap_server_domain`` is now mandatory.

  * Updated default set of TLS ciphers used by server

    (``ldap_tls_ciphers`` parameter). All CBC ciphers have been

    dropped. This could introduce incompatibility with older clients

    trying to connect to the LDAP server.

* ``mail_forwarder`` role

  * Use 2048-bit Diffie-Hellman parameters for relevant TLS

    ciphers. This could introduce incompatibility with older

    clients/servers trying to connect to the SMTP server.

* ``mail_server`` role

  * Use 2048-bit Diffie-Hellman parameters for relevant TLS

    ciphers. This could introduce incompatibility with older

    clients/servers trying to connect to the SMTP/IMAP server.

  * Updated default set of TLS ciphers used by IMAP/SMTP servers

    (``mail_server_tls_ciphers`` parameter). All CBC ciphers have been

    dropped. This could introduce incompatibility with older clients

    trying to connect to the IMAP/SMTP server.

  * Dropped the use of ``procmail`` for local mail deliveries.

* ``php_website`` role

  * Parameter ``enforce_https`` has been deprecated and

    removed. HTTPS is now mandatory in all cases.

* ``preseed`` role

  * Parameter ``ansible_key`` is now mandatory.

  * Parameter ``preseed_directory`` is now mandatory.

* ``web_server`` role

  * Use 2048-bit Diffie-Hellman parameters for relevant TLS

    ciphers. This could introduce incompatibility with older clients

    trying to connect to the web server.

  * Updated default set of TLS ciphers used by the server

    (``web_server_tls_ciphers`` parameter). All CBC ciphers have been

    dropped. This could introduce incompatibility with older clients

    trying to connect to the server.

  * Parameter ``default_enforce_https`` has been deprecated and

    removed. HTTPS is now mandatory in all cases.

* ``wsgi_website`` role

  * Parameters ``gunicorn_version`` and ``futures_version`` have been

    deprecated and removed. Existing roles should be updated to

    utilise the ``wsgi_requirements`` parameter instead.

  * Parameter ``enforce_https`` has been deprecated and

    removed. HTTPS is now mandatory in all cases.

  * Added parameter ``wsgi_requirements_in`` for listing top-level

    packages for performing pip requirements upgrade checks for

    Gunicorn requirements (listed via existing ``wsgi_requirements``

    parameter).

* ``xmpp_server`` role

  * Parameter ``xmpp_domains`` is now mandatory.

  * Use 2048-bit Diffie-Hellman parameters for relevant TLS

    ciphers. This could introduce incompatibility with older

    clients/servers trying to connect to the XMPP server.

  * TLS hardening is now applied to the *c2s* (client) connections on

    both the standard (``5222``) and legacy (``5223``) ports. Protocol

    version and ciphers are configurable via new

    ``xmpp_server_tls_protocol`` and ``xmpp_server_tls_ciphers``

    parameters with defaults enforcing TLSv1.2+ and PFS (perfect

    forward secrecy) ciphers.

  * Support for older Prosody versions (``0.9.x``) has been

    dropped. Only Prosody ``0.10.x`` is supported at the moment (due

    to missing Lua LDAP bindings in Debian 9 Stretch).

  * Support for running Prosody 0.11.x has been added. This is also

    the new default version of Prosody that gets deployed to the

    target system.

**Bug fixes:**

* ``common`` role

  * Run apticron at least once during initial installation to avoid

    accidental locking later on during the same playbook run.

* ``mail_server`` role

import os

import testinfra.utils.ansible_runner

testinfra_hosts = testinfra.utils.ansible_runner.AnsibleRunner(

    os.environ['MOLECULE_INVENTORY_FILE']).get_hosts('parameters-*')

testinfra_hosts += testinfra.utils.ansible_runner.AnsibleRunner(

    os.environ['MOLECULE_INVENTORY_FILE']).get_hosts('deprecated')

def test_supporting_packages_installed(host):

    """

    Tests if all the necessary supporting packages have been

    installed.

    """

    assert host.package('python-apt').is_installed

    assert host.package('lua-ldap').is_installed

def test_prosody_user(host):

    """

    Tests if Prosody user has been set-up correctly to access TLS material.

    """

    assert 'ssl-cert' in host.user('prosody').groups

def test_prosody_modules_directory(host):

    """

    Tests if directory for storing additional Prosody modules is set-up

    correctly.

    """

    directory = host.file('/usr/local/lib/prosody/modules')

    assert directory.is_directory

    assert directory.user == 'root'

    assert directory.group == 'root'

    assert directory.mode == 0o755

def test_prosody_mod_auth_ldap(host):

    """

    Tests if Prosody module mod_auth_ldap has been deployed correctly.

    """

    module = host.file('/usr/local/lib/prosody/modules/mod_auth_ldap.lua')

    assert module.is_file

    assert module.user == 'root'

    assert module.group == 'root'

    assert module.mode == 0o644

    assert 'module:provides("auth", provider);' in module.content_string

    assert 'mod_auth_ldap' in module.content_string

def test_prosody_configuration_file(host):

    """

    Tests if Prosody configuration file has correct permissions.

    """

    with host.sudo():

        config = host.file('/etc/prosody/prosody.cfg.lua')

        assert config.is_file

        assert config.user == 'root'

        assert config.group == 'prosody'

        assert config.mode == 0o640

def test_services(host):

    """

    Tests if services are enabled and running.

    """

    service = host.service('prosody')

    assert service.is_enabled

    assert service.is_running

def test_firewall_configuration_file(host):

    """

    Tests if firewall configuration file has been deployed correctly.

    """

    with host.sudo():

        config = host.file('/etc/ferm/conf.d/30-xmpp.conf')

        assert config.is_file

        assert config.user == 'root'

        assert config.group == 'root'

        assert config.mode == 0o640

def test_xmpp_server_dh_parameters_file(host):

    """

    Tests if the Diffie-Hellman parameter file has been generated

    correctly.

    """

    fqdn = host.run('hostname -f').stdout.strip()

    dhparam_file_path = '/etc/ssl/private/%s_xmpp.dh.pem' % fqdn

    with host.sudo():

        dhparam_file = host.file(dhparam_file_path)

        assert dhparam_file.is_file

        assert dhparam_file.user == 'root'

        assert dhparam_file.group == 'prosody'

        assert dhparam_file.mode == 0o640

        dhparam_info = host.run("openssl dhparam -noout -text -in %s", dhparam_file_path)

        assert "DH Parameters: (2048 bit)" in dhparam_info.stdout

def test_prosody_tls_files(host):

    """

    Tests if Prosody TLS private key and certificage have been deployed

    correctly.

    """

    hostname = host.run('hostname -f').stdout.strip()

    with host.sudo():

        tls_file = host.file('/etc/ssl/private/%s_xmpp.key' % hostname)

        assert tls_file.is_file

        assert tls_file.user == 'root'

        assert tls_file.group == 'prosody'

        assert tls_file.mode == 0o640

        assert tls_file.content_string == open("tests/data/x509/server/%s_xmpp.key.pem" % hostname, "r").read().rstrip()

        tls_file = host.file('/etc/ssl/certs/%s_xmpp.pem' % hostname)

        assert tls_file.is_file

        assert tls_file.user == 'root'

        assert tls_file.group == 'root'

        assert tls_file.mode == 0o644

        assert tls_file.content_string == open("tests/data/x509/server/%s_xmpp.cert.pem" % hostname, "r").read().rstrip()

def test_certificate_validity_check_configuration(host):

    """

    Tests if certificate validity check configuration file has been deployed

    correctly.

    """

    hostname = host.run('hostname').stdout.strip()

    config = host.file('/etc/check_certificate/%s_xmpp.conf' % hostname)

    assert config.is_file

    assert config.user == 'root'

    assert config.group == 'root'

    assert config.mode == 0o644

    assert config.content_string == "/etc/ssl/certs/%s_xmpp.pem" % hostname

def test_xmpp_server_uses_correct_dh_parameters(host):

    """

    Tests if the HTTP server uses the generated Diffie-Hellman parameter.

    """

    fqdn = host.run('hostname -f').stdout.strip()

    # Use first defined domain for testing.

    domain = host.ansible.get_variables()['xmpp_domains'][0]

    with host.sudo():

        expected_dhparam = host.file('/etc/ssl/private/%s_xmpp.dh.pem' % fqdn).content_string.rstrip()

    connection = host.run("gnutls-cli --no-ca-verification --starttls-proto=xmpp --port 5222 "

                          "--priority 'NONE:+VERS-TLS1.2:+CTYPE-X509:+COMP-NULL:+SIGN-RSA-SHA384:+DHE-RSA:+SHA384:+AEAD:+AES-256-GCM' --verbose %s", domain)

    output = connection.stdout

    begin_marker = "-----BEGIN DH PARAMETERS-----"

    end_marker = "-----END DH PARAMETERS-----"

    used_dhparam = output[output.find(begin_marker):output.find(end_marker) + len(end_marker)]

    assert used_dhparam == expected_dhparam

def test_tls_connectivity(host):

    """

    Tests if it is possible to connect to the XMPP server using

    STARTTLS/TLS.

    """

    # Use first defined domain for testing.

    domain = host.ansible.get_variables()['xmpp_domains'][0]

    starttls = host.run('echo "test" | openssl s_client -quiet -starttls xmpp -xmpphost %s -connect localhost:5222', domain)

    assert starttls.rc == 0

    assert 'jabber:client' in starttls.stdout

    assert 'not-well-formed' in starttls.stdout

    tls = host.run('echo "test" | openssl s_client -quiet -connect %s:5223', domain)

    assert tls.rc == 0

    assert 'jabber:client' in starttls.stdout

    assert 'not-well-formed' in starttls.stdout

    s2s = host.run('echo "test" | openssl s_client -quiet -starttls xmpp-server -xmpphost %s -connect localhost:5222', domain)

    assert s2s.rc == 0

    assert 'jabber:client' in s2s.stdout

    assert 'not-well-formed' in s2s.stdout

def test_prosody_configuration_validity(host):

    """

    Tests the Prosody configuration file using the 'prosodyctl check'

    command.

    """

    with host.sudo():

        check_config = host.run("prosodyctl check config")

    assert check_config.rc == 0, check_config.stdout

def test_enabled_modules(host):

    """

    Tests if correct modules have been enabled.

    """

    expected_modules = [

        "admin_adhoc",

        "announce",

        "blocklist",

        "carbons",

        "dialback",

        "disco",

        "legacyauth",

        "mam",

        "pep",

        "ping",

        "posix",

        "private",

        "register",

        "roster",

        "saslauth",

        "time",

        "tls",

        "uptime",

        "vcard",

        "version",

    ]

    with host.sudo():

        module_list_command = host.run("/usr/local/bin/list_prosody_modules.lua")

    enabled_modules = sorted(module_list_command.stdout.strip().splitlines())

    assert enabled_modules == expected_modules

def test_certificate_configuration(host):

    """

    Tests if certificates have been issued and configured correctly

    for use with Prosody. Relies on Prosody's own internal check

    command.

    """

    with host.sudo():

        check_certs = host.run("prosodyctl check certs")

    assert check_certs.rc == 0, check_certs.stdout

def test_prosody_certificate_checker_script(host):

    """

    Tests if Prosody certificate checker script has been correctly

    deployed.

    """

    with host.sudo():

        script = host.file("/usr/local/bin/check_prosody_certificate.sh")

        assert script.is_file

        assert script.user == 'root'

        assert script.group == 'root'

        assert script.mode == 0o755

def test_prosody_certificate_checker_crontab(host):

    """

    Tests if crontab entry has been deployed for running the Prosody

    certificate checker script.

    """

    crontab = host.file('/etc/cron.d/check_prosody_certificate')

    assert crontab.is_file

    assert crontab.user == 'root'

    assert crontab.group == 'root'

    assert crontab.mode == 0o644

    assert "MAILTO=root" in crontab.content_string

    assert "/usr/local/bin/check_prosody_certificate.sh" in crontab.content_string

def test_backports_repository(host):

    """

    Tests if the backports repository has been configured.

    """

    repository = host.file("/etc/apt/sources.list.d/backports.list")

    distribution_release = host.ansible("setup")["ansible_facts"]["ansible_distribution_release"]

    expected_content = "deb http://deb.debian.org/debian %s-backports main" % distribution_release

    assert repository.is_file

    assert repository.user == "root"

    assert repository.group == "root"

    assert repository.mode == 0o644

    assert repository.content_string.rstrip() == expected_content

def test_backports_prosody_pinning(host):

    """

    Tests if the backports pin for Prosody has been deployed correctly.

    """

    pin = host.file("/etc/apt/preferences.d/prosody")

    assert pin.is_file

    assert pin.user == "root"

    assert pin.group == "root"

    assert pin.mode == 0o644

    prosody_package = host.package("prosody")

    lua_ldap_package = host.package("lua-sec")

    assert "bpo" in prosody_package.version

    assert "bpo" in lua_ldap_package.version

# @TODO: Tests which were not implemented due to lack of out-of-box tools:

#

# - Proxy capability.

# - MUC.

# - Server administration through XMPP.

---

- name: Install Python apt bindings

  apt:

    name: python-apt

- name: Set-up the Debian backports repository

  template:

    src: backports.list.j2

    dest: /etc/apt/sources.list.d/backports.list

    owner: root

    group: root

    mode: 0644

  register: backports_repository_configuration

- name: Update apt cache if backports repository configuration changed (for immediate use)

  apt:

    update_cache: true

  when: backports_repository_configuration.changed

- name: Configure package pins to backports for Prosody

  template:

    src: prosody_backports_pin.j2

    dest: /etc/apt/preferences.d/prosody

    owner: root

    group: root

    mode: 0644

- name: Collect information about installed packages

  package_facts:

- name: Uninstall Prosody from project-provided repository

  apt:

    name: prosody

    state: absent

  when:

    - "ansible_facts.packages['prosody'] is defined"

    - "'nightly' in ansible_facts.packages['prosody'][0].version"

- name: Uninstall Prosody dependencies from project-provided repository

  apt:

    name:

      - lua-expat

      - lua-filesystem

      - lua-sec

      - lua-socket

    state: absent

  when: >-

    (ansible_facts.packages['lua-expat'] is defined and 'prosody' in ansible_facts.packages['lua-expat'][0].version)

    or (ansible_facts.packages['lua-filesystem'] is defined and 'prosody' in ansible_facts.packages['lua-filesystem'][0].version)

    or (ansible_facts.packages['lua-sec'] is defined and 'prosody' in ansible_facts.packages['lua-sec'][0].version)

    or (ansible_facts.packages['lua-socket'] is defined and 'prosody' in ansible_facts.packages['lua-socket'][0].version)

- name: Remove Prosody project-provided apt key

  apt_key:

    id: "{{ item }}"

    state: absent

  with_items:

    - "107D65A0A148C237FDF00AB47393D7E674D9DBB5"

    - "44AB6DD06DA46979CFAF997F9B1B82786C8F28BA"

- name: Remove Prosody project-provided repository

  apt_repository:

    repo: "deb http://packages.prosody.im/debian {{ ansible_distribution_release }} main"

    state: absent

  apt:

    state: present

  notify:

    - Restart Prosody

- name: Install Prosody

  apt:

    name: prosody

    state: present

  notify:

    - Restart Prosody

- name: Allow Prosody user to traverse the directory with TLS private keys

  user:

    name: prosody

    append: true

    groups: ssl-cert

- name: Deploy XMPP TLS private key

  copy:

    dest: "/etc/ssl/private/{{ ansible_fqdn }}_xmpp.key"

    content: "{{ xmpp_tls_key }}"

    owner: root

    group: prosody

    mode: 0640

  notify:

    - Restart Prosody

- name: Deploy XMPP TLS certificate

  copy:

    dest: "/etc/ssl/certs/{{ ansible_fqdn }}_xmpp.pem"

    content: "{{ xmpp_tls_certificate }}"

    owner: root

    group: root

    mode: 0644

  notify:

    - Restart Prosody

- name: Generate the XMPP server Diffie-Hellman parameter

  openssl_dhparam:

    owner: root

    group: prosody

    mode: 0640

    path: "/etc/ssl/private/{{ ansible_fqdn }}_xmpp.dh.pem"

    size: 2048

  notify:

    - Restart Prosody

- name: Deploy configuration file for checking certificate validity via cron

  copy:

    content: "/etc/ssl/certs/{{ ansible_fqdn }}_xmpp.pem"

    dest: "/etc/check_certificate/{{ ansible_fqdn }}_xmpp.conf"

    owner: root

    group: root

    mode: 0644

- name: Deploy script for validating Prosody certificate

  copy:

    src: "check_prosody_certificate.sh"

    dest: "/usr/local/bin/check_prosody_certificate.sh"

    owner: root

    group: root

    mode: 0755

- name: Set-up crontab task that runs the Prosody certificate checker script once a day

  copy:

    src: "cron_check_prosody_certificate"

    dest: "/etc/cron.d/check_prosody_certificate"

    owner: root

    group: root

    mode: 0644

- name: Set-up directory for storing additional Prosody modules

  file:

    path: "/usr/local/lib/prosody/modules/"

    state: directory

    owner: root

    group: root

    mode: 0755

- name: Deploy the Prosody mod_auth_ldap module

  get_url:

    url: "https://hg.prosody.im/prosody-modules/raw-file/tip/mod_auth_ldap/mod_auth_ldap.lua"

    dest: "/usr/local/lib/prosody/modules/mod_auth_ldap.lua"

- name: Set-up file permissions for the Prosody mod_auth_ldap module

  file:

    dest: "/usr/local/lib/prosody/modules/mod_auth_ldap.lua"

    owner: root

    group: root

    mode: 0644

- name: Deploy Prosody configuration file

  template:

    src: "prosody.cfg.lua.j2"

    dest: "/etc/prosody/prosody.cfg.lua"

    owner: root

    group: prosody

    mode: 0640

  notify:

    - Restart Prosody

- name: Enable and start Prosody service

  service:

    name: prosody

    state: started

    enabled: true

- name: Deploy firewall configuration for XMPP server

  copy:

    src: "ferm_xmpp.conf"

    dest: "/etc/ferm/conf.d/30-xmpp.conf"

    owner: root

    group: root

    mode: 0640

  notify:

    - Restart ferm

- name: Explicitly run all handlers

  include: ../handlers/main.yml

  when: "run_handlers | default(False) | bool()"

  tags:

    - handlers

#

# Pins Prosody and some related packages to Debian backports in order

# to get more up-to-date features and bug/security updates.

#

Pin: release a={{ ansible_distribution_release }}-backports

Pin-Priority: 600