Changeset - ef35c565bb0a
[Not reviewed]
0 7 0
Branko Majic (branko) - 10 months ago 2025-02-13 22:37:03
branko@majic.rs
MAR-242: Added role parameters for xmpp_server role to configure HTTP file upload limits (XEP-0363):

- Refactor the daily quota tests to be a bit more flexible.
7 files changed with 52 insertions and 24 deletions:
0 comments (0 inline, 0 general)
docs/rolereference.rst
Show inline comments
 
@@ -879,50 +879,59 @@ The role implements the following:
 

	
 
* Installs Prosody.
 
* Configures Prosody.
 
* Configures firewall to allow incoming connections to the XMPP server.
 

	
 
Prosody is configured as follows:
 

	
 
* Modules enabled: roster, saslauth, tls, dialback, posix, private, vcard,
 
  version, uptime, time, ping, pep, register, admin_adhoc, announce,
 
  legacyauth, carbons, mam.
 
* Self-registration is not allowed.
 
* TLS is configured. Legacy TLS is available on port 5223.
 
* Client-to-server communication requires encryption (TLS).
 
* Uses 2048-bit Diffie-Hellman parameters for relevant TLS ciphers for
 
  incoming connections.
 
* Configures TLS versions and ciphers supported by Prosody (for
 
  *c2s*/client connections only).
 
* Authentication is done via LDAP. For setting the LDAP TLS truststore, see
 
  :ref:`LDAP Client <ldap_client>`.
 
* Internal storage is used.
 
* For each domain specified, a dedicated conference/multi-user chat (MUC)
 
  service is set-up, with FQDN set to ``conference.DOMAIN``.
 
* For each domain specified, a dedicated file proxy service will be set-up, with
 
  FQDN set to ``proxy.DOMAIN``.
 
* For each domain specified, a dedicated http file share service will be set-up,
 
  with FQDN set to ``upload.DOMAIN``.
 
* For each domain specified, a dedicated http file share service is
 
  set-up, with FQDN set to ``upload.DOMAIN``. Service is configured
 
  with maximum upload file size limit, as well as per-user daily
 
  quota. This allows clients to use `XEP-0363: HTTP File Upload
 
  <https://xmpp.org/extensions/xep-0363.html>`_ for exchanging files.
 

	
 
  .. warning::
 
     Due to `bug related to global quotas
 
     <https://issues.prosody.im/1891>`_, the role currently does not
 
     configure global quotas in any way. This might change in the
 
     future.
 

	
 
Prosody expects a specific directory structure in LDAP when doing look-ups:
 

	
 
* Prosody will log-in to LDAP as user
 
  ``cn=prosody,ou=services,XMPP_LDAP_BASE_DN``.
 
* User entries are read from sub-tree (first-level only)
 
  ``ou=people,XMPP_LDAP_BASE_DN``. Query filter used for finding users is
 
  ``(&(mail=$user@$host)(memberOf=cn=xmpp,ou=groups,XMPP_LDAP_BASE_DN))``. This
 
  allows group-based granting of XMPP service to users.
 

	
 

	
 
LDIF Templates
 
~~~~~~~~~~~~~~
 

	
 
For adding user to a group, use::
 

	
 
  dn: cn=xmpp,ou=groups,BASE_DN
 
  changetype: modify
 
  add: uniqueMember
 
  uniqueMember: uid=USERNAME,ou=people,BASE_DN
 

	
 

	
 
Role dependencies
 
~~~~~~~~~~~~~~~~~
 
@@ -947,48 +956,57 @@ If the backup for this role has been enabled, the following paths are backed-up:
 
Parameters
 
~~~~~~~~~~
 

	
 
**xmpp_administrators** (list, mandatory)
 
  List of Prosody users that should be granted administrator privileges over
 
  Prosody. Each item is a string with value equal to XMPP user ID
 
  (i.e. ``john.doe@example.com``).
 

	
 
**xmpp_domains** (list, mandatory)
 
  List of domains that are served by this Prosody instance. Each item is a
 
  string specifying a domain.
 

	
 
**xmpp_ldap_base_dn** (string, mandatory)
 
  Base DN on the LDAP server. A specific directory structure is expected under
 
  this entry (as explained above) in order to locate the available domains,
 
  users, aliases etc.
 

	
 
**xmpp_ldap_password** (string, mandatory)
 
  Password used for authenticating to the LDAP server.
 

	
 
**xmpp_ldap_server** (string, mandatory)
 
  Fully qualified domain name, hostname, or IP address of the LDAP server used
 
  for user authentication and listing.
 

	
 
**xmpp_http_file_share_daily_quota** (integer, optional, ``104857600``)
 
  Daily quota for individual users - maximum file size in bytes that a
 
  particular user can upload per day (`XEP-0363: HTTP File Upload
 
  <https://xmpp.org/extensions/xep-0363.html>`_).
 

	
 
**xmpp_http_file_share_size_limit** (integer, optional, ``10485760``)
 
  Maximum file size in bytes to allow for upload (`XEP-0363: HTTP File
 
  Upload <https://xmpp.org/extensions/xep-0363.html>`_).
 

	
 
**xmpp_server_archive_expiration** (string, optional, ``never``)
 
  Expiration period for messages stored server-side using `XEP-0313:
 
  Message Archive Management
 
  <https://xmpp.org/extensions/xep-0313.html>`_. The value should be
 
  compatible with `Prosody mod_mam
 
  <https://prosody.im/doc/modules/mod_mam>`_ configuration option
 
  ``archive_expires_after``.
 

	
 
**xmpp_server_tls_ciphers** (string, optional,  ``DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-CHACHA20-POLY1305:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-CHACHA20-POLY1305:TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:!aNULL:!MD5:!EXPORT``)
 
  TLS ciphers to enable on the XMPP server. This should be an
 
  OpenSSL-compatible cipher specification. Value should be compatible
 
  with Prosody's option ``ciphers`` normally defined within the
 
  ``ssl`` section of configuration file (see `official documentation
 
  <https://prosody.im/doc/advanced_ssl_config#ciphers>`_ for details).
 
  Default value enables TLSv1.2-compatible strong PFS ciphers and RSA
 
  private keys. TLSv1.3-specific ciphers (``TLS_*``)
 
  are not configurable due to OpenSSL TLS context initialisation
 
  specifics/implementation details. They are included here for
 
  documentation/reference purposes only.
 

	
 
  .. warning::
 
     If the mail server minimum TLS version is set to TLSv1.3, it is
 
     still necessary to include at least one TLSv1.2-compatible cipher
 
     in the list in order to ensure TLSv1.3 initialisation. This is
roles/xmpp_server/defaults/main.yml
Show inline comments
 
---
 

	
 
enable_backup: false
 
xmpp_http_file_share_daily_quota: 104857600  # 100MiB
 
xmpp_http_file_share_size_limit: 10485760  # 10MiB
 
xmpp_server_archive_expiration: "never"
 
xmpp_server_tls_protocol: "tlsv1_2+"
 

	
 
# TLS_* ciphers are mandated by the TLSv1.3-related standards and
 
# cannot be disabled when TLSv1.3 is enabled on the server.
 
xmpp_server_tls_ciphers: "\
 
DHE-RSA-AES128-GCM-SHA256:\
 
DHE-RSA-AES256-GCM-SHA384:\
 
DHE-RSA-CHACHA20-POLY1305:\
 
ECDHE-RSA-AES128-GCM-SHA256:\
 
ECDHE-RSA-AES256-GCM-SHA384:\
 
ECDHE-RSA-CHACHA20-POLY1305:\
 
TLS_AES_128_GCM_SHA256:\
 
TLS_AES_256_GCM_SHA384:\
 
TLS_CHACHA20_POLY1305_SHA256:\
 
!aNULL:!MD5:!EXPORT"
roles/xmpp_server/molecule/default/group_vars/parameters-optional.yml
Show inline comments
 
---
 

	
 
xmpp_administrators:
 
  - jane.doe@domain2
 
  - mick.doe@domain3
 
xmpp_domains:
 
  - domain2
 
  - domain3
 
xmpp_http_file_share_daily_quota: 73400320  # 70MiB
 
xmpp_http_file_share_size_limit: 20971520  # 20MiB
 
xmpp_ldap_base_dn: dc=local
 
xmpp_ldap_password: prosodypassword
 
xmpp_ldap_server: ldap-server
 
xmpp_server_archive_expiration: "1w"
 
xmpp_tls_certificate: "{{ lookup('file', 'tests/data/x509/server/{{ ansible_fqdn }}_xmpp.cert.pem') }}"
 
xmpp_tls_key: "{{ lookup('file', 'tests/data/x509/server/{{ ansible_fqdn }}_xmpp.key.pem') }}"
 
xmpp_server_tls_protocol: "tlsv1_3+"
 
# At least one non-TLSv1.3 cipher has to be included in order to
 
# ensure TLSv1.3 gets initialised. TLSv1.3 ciphers (TLS_*) are not
 
# configurable and listed for documentation/reference purposes.
 
xmpp_server_tls_ciphers: "\
 
ECDHE-RSA-CHACHA20-POLY1305:\
 
TLS_AES_128_GCM_SHA256:\
 
TLS_AES_256_GCM_SHA384:\
 
TLS_CHACHA20_POLY1305_SHA256:\
 
!aNULL:!MD5:!EXPORT"
 

	
 
# common
 
ca_certificates:
 
  testca: "{{ lookup('file', 'tests/data/x509/ca/level1.cert.pem') }}"
 

	
 
# backup_client
 
enable_backup: true
 
backup_client_username: "bak-parameters-optional-{{ ansible_distribution_release }}"
roles/xmpp_server/molecule/default/tests/test_client.py
Show inline comments
 
@@ -124,88 +124,86 @@ def test_http_file_upload(host, server_host, username, password, domain):
 
    send = host.run(f"go-sendxmpp --debug --username {username}@{domain} --password {password} --jserver {domain}:5222 "
 
                    f"--http-upload /tmp/http_file_upload_sample.txt "
 
                    f"{username}@{domain}")
 
    assert send.rc == 0
 
    assert "No http upload component found." not in send.stderr
 

	
 
    # Verify content on server.
 
    with server_host.sudo():
 
        upload_directory = server_host.file(upload_directory_path)
 
        assert upload_directory.is_directory
 
        assert upload_directory.user == "prosody"
 
        assert upload_directory.group == "prosody"
 
        assert upload_directory.mode == 0o750
 
        assert len(upload_directory.listdir()) == 1
 

	
 
        uploaded_file_name = upload_directory.listdir()[0]
 
        uploaded_file = server_host.file(os.path.join(upload_directory_path, uploaded_file_name))
 
        assert uploaded_file.is_file
 
        assert uploaded_file.user == "prosody"
 
        assert uploaded_file.group == "prosody"
 
        assert uploaded_file.mode == 0o640
 
        assert uploaded_file.content_string == expected_content
 

	
 

	
 
@pytest.mark.parametrize("username, password, domain, server", [
 
    ["john.doe", "johnpassword", "domain1", "parameters-mandatory"],
 
    ["jane.doe", "janepassword", "domain2", "parameters-optional"],
 
    ["mick.doe", "mickpassword", "domain3", "parameters-optional"],
 
@pytest.mark.parametrize("username, password, domain, server, file_size_limit", [
 
    ["john.doe", "johnpassword", "domain1", "parameters-mandatory", 10 * 1024 * 1024],
 
    ["jane.doe", "janepassword", "domain2", "parameters-optional", 20 * 1024 * 1024],
 
    ["mick.doe", "mickpassword", "domain3", "parameters-optional", 20 * 1024 * 1024],
 
])
 
@pytest.mark.usefixtures("server_clean_domain_uploads")
 
def test_http_file_share_size_limit(host, username, password, domain):
 
def test_http_file_share_size_limit(host, username, password, domain, file_size_limit):
 
    """
 
    Tests the maximum file size for files uploaded via XEP-0363.
 
    """
 

	
 
    file_size_limit = 10 * 1024 * 1024
 

	
 
    # Test exact size limit.
 
    create_sample_file = host.run("dd if=/dev/zero of=/tmp/http_file_upload_sample.txt bs=%sB count=1", str(file_size_limit))
 
    assert create_sample_file.rc == 0
 

	
 
    send = host.run(f"go-sendxmpp --debug --username {username}@{domain} --password {password} --jserver {domain}:5222 "
 
                    f"--http-upload /tmp/http_file_upload_sample.txt "
 
                    f"{username}@{domain}")
 
    assert "file-too-large" not in send.stderr
 

	
 
    # Test exceeded size limit.
 
    create_sample_file = host.run("dd if=/dev/zero of=/tmp/http_file_upload_sample.txt bs=%sB count=1", str(file_size_limit + 1))
 
    assert create_sample_file.rc == 0
 

	
 
    send = host.run(f"go-sendxmpp --debug --username {username}@{domain} --password {password} --jserver {domain}:5222 "
 
                    f"--http-upload /tmp/http_file_upload_sample.txt "
 
                    f"{username}@{domain}")
 
    assert "file-too-large" in send.stderr
 

	
 

	
 
@pytest.mark.parametrize("username, password, domain, server", [
 
    ["john.doe", "johnpassword", "domain1", "parameters-mandatory"],
 
    ["jane.doe", "janepassword", "domain2", "parameters-optional"],
 
    ["mick.doe", "mickpassword", "domain3", "parameters-optional"],
 
@pytest.mark.parametrize("username, password, domain, server, file_size_limit, user_daily_quota", [
 
    ["john.doe", "johnpassword", "domain1", "parameters-mandatory", 10 * 1024 * 1024, 100 * 1024 * 1024],
 
    ["jane.doe", "janepassword", "domain2", "parameters-optional", 20 * 1024 * 1024, 70 * 1024 * 1024],
 
    ["mick.doe", "mickpassword", "domain3", "parameters-optional", 20 * 1024 * 1024, 70 * 1024 * 1024],
 
])
 
@pytest.mark.usefixtures("server_clean_domain_uploads")
 
def test_http_file_share_daily_quota(host, username, password, domain):
 
def test_http_file_share_daily_quota(host, username, password, domain, file_size_limit, user_daily_quota):
 
    """
 
    Tests the user's daily quota for files uploaded via XEP-0363.
 
    """
 

	
 
    # Equivalent of 100MiB.
 
    file_size_limit = 10 * 1024 * 1024
 
    file_count = 10
 
    remaining_quota = user_daily_quota
 
    while remaining_quota > 0:
 
        file_size = file_size_limit if file_size_limit < remaining_quota else remaining_quota
 
        create_sample_file = host.run("dd if=/dev/zero of=/tmp/http_file_upload_sample.txt bs=%sB count=1", str(file_size))
 
        assert create_sample_file.rc == 0
 

	
 
    # Fill-up the daily quota.
 
    create_sample_file = host.run("dd if=/dev/zero of=/tmp/http_file_upload_sample.txt bs=%sB count=1", str(file_size_limit))
 
    assert create_sample_file.rc == 0
 
    for _ in range(file_count):
 
        send = host.run(f"go-sendxmpp --debug --username {username}@{domain} --password {password} --jserver {domain}:5222 "
 
                        f"--http-upload /tmp/http_file_upload_sample.txt "
 
                        f"{username}@{domain}")
 
        assert send.rc == 0
 

	
 
        remaining_quota -= file_size
 

	
 
    # Test exceeded daily quota.
 
    create_sample_file = host.run("dd if=/dev/zero of=/tmp/http_file_upload_sample.txt bs=1B count=1")
 
    assert create_sample_file.rc == 0
 

	
 
    send = host.run(f"go-sendxmpp --debug --username {username}@{domain} --password {password} --jserver {domain}:5222 "
 
                    f"--http-upload /tmp/http_file_upload_sample.txt "
 
                    f"{username}@{domain}")
 
    assert "Daily quota reached" in send.stderr
roles/xmpp_server/molecule/default/tests/test_mandatory.py
Show inline comments
 
@@ -17,49 +17,51 @@ def test_prosody_configuration_file_content(host):
 
    """
 

	
 
    hostname = host.run('hostname').stdout.strip()
 

	
 
    with host.sudo():
 

	
 
        config = host.file('/etc/prosody/prosody.cfg.lua')
 

	
 
        assert "admins = { \"john.doe@domain1\",  }" in config.content_string
 
        assert "key = \"/etc/ssl/private/%s_xmpp.key\";" % hostname in config.content_string
 
        assert "certificate = \"/etc/ssl/certs/%s_xmpp.pem\";" % hostname in config.content_string
 
        assert "ldap_server = \"ldap-server\"" in config.content_string
 
        assert "ldap_rootdn = \"cn=prosody,ou=services,dc=local\"" in config.content_string
 
        assert "ldap_password = \"prosodypassword\"" in config.content_string
 
        assert "ldap_filter = \"(&(mail=$user@$host)(memberOf=cn=xmpp,ou=groups,dc=local))\"" in config.content_string
 
        assert "ldap_base = \"ou=people,dc=local\"" in config.content_string
 
        assert "archive_expires_after = \"never\"" in config.content_string
 

	
 
        assert """VirtualHost "domain1"
 
Component "conference.domain1" "muc"
 
  restrict_room_creation = "local"
 
Component "proxy.domain1" "proxy65"
 
  proxy65_acl = { "domain1" }
 
Component "upload.domain1" "http_file_share"
 
  http_file_share_access = { "domain1" }""" in config.content_string
 
  http_file_share_access = { "domain1" }
 
  http_file_share_size_limit = 10485760
 
  http_file_share_daily_quota = 104857600""" in config.content_string
 

	
 

	
 
def test_xmpp_server_uses_correct_dh_parameters(host):
 
    """
 
    Tests if the HTTP server uses the generated Diffie-Hellman parameter.
 
    """
 

	
 
    fqdn = host.run('hostname -f').stdout.strip()
 

	
 
    # Use first defined domain for testing.
 
    domain = host.ansible.get_variables()['xmpp_domains'][0]
 

	
 
    with host.sudo():
 
        expected_dhparam = host.file('/etc/ssl/private/%s_xmpp.dh.pem' % fqdn).content_string.rstrip()
 

	
 
    connection = host.run("gnutls-cli --no-ca-verification --starttls-proto=xmpp --port 5222 "
 
                          "--priority 'NONE:+VERS-TLS1.2:+CTYPE-X509:+COMP-NULL:+SIGN-RSA-SHA384:+DHE-RSA:+SHA384:+AEAD:+AES-256-GCM' --verbose %s", domain)
 

	
 
    output = connection.stdout
 
    begin_marker = "-----BEGIN DH PARAMETERS-----"
 
    end_marker = "-----END DH PARAMETERS-----"
 
    used_dhparam = output[output.find(begin_marker):output.find(end_marker) + len(end_marker)]
 

	
 
    assert used_dhparam == expected_dhparam
roles/xmpp_server/molecule/default/tests/test_optional.py
Show inline comments
 
@@ -17,57 +17,61 @@ def test_prosody_configuration_file_content(host):
 
    """
 

	
 
    hostname = host.run('hostname').stdout.strip()
 

	
 
    with host.sudo():
 

	
 
        config = host.file('/etc/prosody/prosody.cfg.lua')
 

	
 
        assert "admins = { \"jane.doe@domain2\", \"mick.doe@domain3\",  }" in config.content_string
 
        assert "key = \"/etc/ssl/private/%s_xmpp.key\";" % hostname in config.content_string
 
        assert "certificate = \"/etc/ssl/certs/%s_xmpp.pem\";" % hostname in config.content_string
 
        assert "ldap_server = \"ldap-server\"" in config.content_string
 
        assert "ldap_rootdn = \"cn=prosody,ou=services,dc=local\"" in config.content_string
 
        assert "ldap_password = \"prosodypassword\"" in config.content_string
 
        assert "ldap_filter = \"(&(mail=$user@$host)(memberOf=cn=xmpp,ou=groups,dc=local))\"" in config.content_string
 
        assert "ldap_base = \"ou=people,dc=local\"" in config.content_string
 
        assert "archive_expires_after = \"1w\"" in config.content_string
 

	
 
        assert """VirtualHost "domain2"
 
Component "conference.domain2" "muc"
 
  restrict_room_creation = "local"
 
Component "proxy.domain2" "proxy65"
 
  proxy65_acl = { "domain2" }
 
Component "upload.domain2" "http_file_share"
 
  http_file_share_access = { "domain2" }""" in config.content_string
 
  http_file_share_access = { "domain2" }
 
  http_file_share_size_limit = 20971520
 
  http_file_share_daily_quota = 73400320""" in config.content_string
 

	
 
        assert """VirtualHost "domain3"
 
Component "conference.domain3" "muc"
 
  restrict_room_creation = "local"
 
Component "proxy.domain3" "proxy65"
 
  proxy65_acl = { "domain3" }
 
Component "upload.domain3" "http_file_share"
 
  http_file_share_access = { "domain3" }""" in config.content_string
 
  http_file_share_access = { "domain3" }
 
  http_file_share_size_limit = 20971520
 
  http_file_share_daily_quota = 73400320""" in config.content_string
 

	
 

	
 
@pytest.mark.parametrize("port", [
 
    5222,
 
    5223
 
])
 
def test_xmpp_c2s_tls_version_and_ciphers(host, port):
 
    """
 
    Tests if the correct TLS version and ciphers have been enabled for
 
    XMPP C2S ports.
 
    """
 

	
 
    expected_tls_versions = ["TLSv1.3"]
 
    expected_tls_ciphers = [
 
        "TLS_AKE_WITH_AES_128_GCM_SHA256",
 
        "TLS_AKE_WITH_AES_256_GCM_SHA384",
 
        "TLS_AKE_WITH_CHACHA20_POLY1305_SHA256",
 
    ]
 

	
 
    # Run the nmap scanner against the server, and fetch the results.
 
    nmap = host.run("nmap -sV --script ssl-enum-ciphers -p %s domain2 -oX /tmp/report.xml", str(port))
 
    assert nmap.rc == 0
 
    report_content = host.file('/tmp/report.xml').content_string
 

	
roles/xmpp_server/templates/prosody.cfg.lua.j2
Show inline comments
 
@@ -91,25 +91,27 @@ ldap_base = "ou=people,{{ xmpp_ldap_base_dn }}"
 

	
 
-- Message Archives (mod_mam) configuration.
 
archive_expires_after = "{{ xmpp_server_archive_expiration }}"
 

	
 
-- Storage backend.
 
storage = "internal"
 

	
 
-- Logging configuration.
 
log = {
 
  info = "/var/log/prosody/prosody.log"; -- Change 'info' to 'debug' for verbose logging
 
  error = "/var/log/prosody/prosody.err";
 
  "*syslog";
 
}
 

	
 
-- Domains which should be handled by Prosody, with dedicated MUC and file
 
-- proxying components.
 
{% for domain in xmpp_domains -%}
 
VirtualHost "{{ domain }}"
 
Component "conference.{{ domain }}" "muc"
 
  restrict_room_creation = "local"
 
Component "proxy.{{ domain }}" "proxy65"
 
  proxy65_acl = { "{{ domain }}" }
 
Component "upload.{{ domain }}" "http_file_share"
 
  http_file_share_access = { "{{ domain }}" }
 
  http_file_share_size_limit = {{ xmpp_http_file_share_size_limit }}
 
  http_file_share_daily_quota = {{ xmpp_http_file_share_daily_quota }}
 
{% endfor -%}
0 comments (0 inline, 0 general)