Changeset - ef35c565bb0a
[Not reviewed]
0 7 0
Branko Majic (branko) - 10 months ago 2025-02-13 22:37:03
branko@majic.rs
MAR-242: Added role parameters for xmpp_server role to configure HTTP file upload limits (XEP-0363):

- Refactor the daily quota tests to be a bit more flexible.
7 files changed with 52 insertions and 24 deletions:
0 comments (0 inline, 0 general)
docs/rolereference.rst
Show inline comments
 
@@ -891,26 +891,35 @@ Prosody is configured as follows:
 
* Client-to-server communication requires encryption (TLS).
 
* Uses 2048-bit Diffie-Hellman parameters for relevant TLS ciphers for
 
  incoming connections.
 
* Configures TLS versions and ciphers supported by Prosody (for
 
  *c2s*/client connections only).
 
* Authentication is done via LDAP. For setting the LDAP TLS truststore, see
 
  :ref:`LDAP Client <ldap_client>`.
 
* Internal storage is used.
 
* For each domain specified, a dedicated conference/multi-user chat (MUC)
 
  service is set-up, with FQDN set to ``conference.DOMAIN``.
 
* For each domain specified, a dedicated file proxy service will be set-up, with
 
  FQDN set to ``proxy.DOMAIN``.
 
* For each domain specified, a dedicated http file share service will be set-up,
 
  with FQDN set to ``upload.DOMAIN``.
 
* For each domain specified, a dedicated http file share service is
 
  set-up, with FQDN set to ``upload.DOMAIN``. Service is configured
 
  with maximum upload file size limit, as well as per-user daily
 
  quota. This allows clients to use `XEP-0363: HTTP File Upload
 
  <https://xmpp.org/extensions/xep-0363.html>`_ for exchanging files.
 

	
 
  .. warning::
 
     Due to `bug related to global quotas
 
     <https://issues.prosody.im/1891>`_, the role currently does not
 
     configure global quotas in any way. This might change in the
 
     future.
 

	
 
Prosody expects a specific directory structure in LDAP when doing look-ups:
 

	
 
* Prosody will log-in to LDAP as user
 
  ``cn=prosody,ou=services,XMPP_LDAP_BASE_DN``.
 
* User entries are read from sub-tree (first-level only)
 
  ``ou=people,XMPP_LDAP_BASE_DN``. Query filter used for finding users is
 
  ``(&(mail=$user@$host)(memberOf=cn=xmpp,ou=groups,XMPP_LDAP_BASE_DN))``. This
 
  allows group-based granting of XMPP service to users.
 

	
 

	
 
LDIF Templates
 
@@ -959,24 +968,33 @@ Parameters
 
**xmpp_ldap_base_dn** (string, mandatory)
 
  Base DN on the LDAP server. A specific directory structure is expected under
 
  this entry (as explained above) in order to locate the available domains,
 
  users, aliases etc.
 

	
 
**xmpp_ldap_password** (string, mandatory)
 
  Password used for authenticating to the LDAP server.
 

	
 
**xmpp_ldap_server** (string, mandatory)
 
  Fully qualified domain name, hostname, or IP address of the LDAP server used
 
  for user authentication and listing.
 

	
 
**xmpp_http_file_share_daily_quota** (integer, optional, ``104857600``)
 
  Daily quota for individual users - maximum file size in bytes that a
 
  particular user can upload per day (`XEP-0363: HTTP File Upload
 
  <https://xmpp.org/extensions/xep-0363.html>`_).
 

	
 
**xmpp_http_file_share_size_limit** (integer, optional, ``10485760``)
 
  Maximum file size in bytes to allow for upload (`XEP-0363: HTTP File
 
  Upload <https://xmpp.org/extensions/xep-0363.html>`_).
 

	
 
**xmpp_server_archive_expiration** (string, optional, ``never``)
 
  Expiration period for messages stored server-side using `XEP-0313:
 
  Message Archive Management
 
  <https://xmpp.org/extensions/xep-0313.html>`_. The value should be
 
  compatible with `Prosody mod_mam
 
  <https://prosody.im/doc/modules/mod_mam>`_ configuration option
 
  ``archive_expires_after``.
 

	
 
**xmpp_server_tls_ciphers** (string, optional,  ``DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-CHACHA20-POLY1305:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-CHACHA20-POLY1305:TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:!aNULL:!MD5:!EXPORT``)
 
  TLS ciphers to enable on the XMPP server. This should be an
 
  OpenSSL-compatible cipher specification. Value should be compatible
 
  with Prosody's option ``ciphers`` normally defined within the
roles/xmpp_server/defaults/main.yml
Show inline comments
 
---
 

	
 
enable_backup: false
 
xmpp_http_file_share_daily_quota: 104857600  # 100MiB
 
xmpp_http_file_share_size_limit: 10485760  # 10MiB
 
xmpp_server_archive_expiration: "never"
 
xmpp_server_tls_protocol: "tlsv1_2+"
 

	
 
# TLS_* ciphers are mandated by the TLSv1.3-related standards and
 
# cannot be disabled when TLSv1.3 is enabled on the server.
 
xmpp_server_tls_ciphers: "\
 
DHE-RSA-AES128-GCM-SHA256:\
 
DHE-RSA-AES256-GCM-SHA384:\
 
DHE-RSA-CHACHA20-POLY1305:\
 
ECDHE-RSA-AES128-GCM-SHA256:\
 
ECDHE-RSA-AES256-GCM-SHA384:\
 
ECDHE-RSA-CHACHA20-POLY1305:\
roles/xmpp_server/molecule/default/group_vars/parameters-optional.yml
Show inline comments
 
---
 

	
 
xmpp_administrators:
 
  - jane.doe@domain2
 
  - mick.doe@domain3
 
xmpp_domains:
 
  - domain2
 
  - domain3
 
xmpp_http_file_share_daily_quota: 73400320  # 70MiB
 
xmpp_http_file_share_size_limit: 20971520  # 20MiB
 
xmpp_ldap_base_dn: dc=local
 
xmpp_ldap_password: prosodypassword
 
xmpp_ldap_server: ldap-server
 
xmpp_server_archive_expiration: "1w"
 
xmpp_tls_certificate: "{{ lookup('file', 'tests/data/x509/server/{{ ansible_fqdn }}_xmpp.cert.pem') }}"
 
xmpp_tls_key: "{{ lookup('file', 'tests/data/x509/server/{{ ansible_fqdn }}_xmpp.key.pem') }}"
 
xmpp_server_tls_protocol: "tlsv1_3+"
 
# At least one non-TLSv1.3 cipher has to be included in order to
 
# ensure TLSv1.3 gets initialised. TLSv1.3 ciphers (TLS_*) are not
 
# configurable and listed for documentation/reference purposes.
 
xmpp_server_tls_ciphers: "\
 
ECDHE-RSA-CHACHA20-POLY1305:\
roles/xmpp_server/molecule/default/tests/test_client.py
Show inline comments
 
@@ -136,76 +136,74 @@ def test_http_file_upload(host, server_host, username, password, domain):
 
        assert upload_directory.mode == 0o750
 
        assert len(upload_directory.listdir()) == 1
 

	
 
        uploaded_file_name = upload_directory.listdir()[0]
 
        uploaded_file = server_host.file(os.path.join(upload_directory_path, uploaded_file_name))
 
        assert uploaded_file.is_file
 
        assert uploaded_file.user == "prosody"
 
        assert uploaded_file.group == "prosody"
 
        assert uploaded_file.mode == 0o640
 
        assert uploaded_file.content_string == expected_content
 

	
 

	
 
@pytest.mark.parametrize("username, password, domain, server", [
 
    ["john.doe", "johnpassword", "domain1", "parameters-mandatory"],
 
    ["jane.doe", "janepassword", "domain2", "parameters-optional"],
 
    ["mick.doe", "mickpassword", "domain3", "parameters-optional"],
 
@pytest.mark.parametrize("username, password, domain, server, file_size_limit", [
 
    ["john.doe", "johnpassword", "domain1", "parameters-mandatory", 10 * 1024 * 1024],
 
    ["jane.doe", "janepassword", "domain2", "parameters-optional", 20 * 1024 * 1024],
 
    ["mick.doe", "mickpassword", "domain3", "parameters-optional", 20 * 1024 * 1024],
 
])
 
@pytest.mark.usefixtures("server_clean_domain_uploads")
 
def test_http_file_share_size_limit(host, username, password, domain):
 
def test_http_file_share_size_limit(host, username, password, domain, file_size_limit):
 
    """
 
    Tests the maximum file size for files uploaded via XEP-0363.
 
    """
 

	
 
    file_size_limit = 10 * 1024 * 1024
 

	
 
    # Test exact size limit.
 
    create_sample_file = host.run("dd if=/dev/zero of=/tmp/http_file_upload_sample.txt bs=%sB count=1", str(file_size_limit))
 
    assert create_sample_file.rc == 0
 

	
 
    send = host.run(f"go-sendxmpp --debug --username {username}@{domain} --password {password} --jserver {domain}:5222 "
 
                    f"--http-upload /tmp/http_file_upload_sample.txt "
 
                    f"{username}@{domain}")
 
    assert "file-too-large" not in send.stderr
 

	
 
    # Test exceeded size limit.
 
    create_sample_file = host.run("dd if=/dev/zero of=/tmp/http_file_upload_sample.txt bs=%sB count=1", str(file_size_limit + 1))
 
    assert create_sample_file.rc == 0
 

	
 
    send = host.run(f"go-sendxmpp --debug --username {username}@{domain} --password {password} --jserver {domain}:5222 "
 
                    f"--http-upload /tmp/http_file_upload_sample.txt "
 
                    f"{username}@{domain}")
 
    assert "file-too-large" in send.stderr
 

	
 

	
 
@pytest.mark.parametrize("username, password, domain, server", [
 
    ["john.doe", "johnpassword", "domain1", "parameters-mandatory"],
 
    ["jane.doe", "janepassword", "domain2", "parameters-optional"],
 
    ["mick.doe", "mickpassword", "domain3", "parameters-optional"],
 
@pytest.mark.parametrize("username, password, domain, server, file_size_limit, user_daily_quota", [
 
    ["john.doe", "johnpassword", "domain1", "parameters-mandatory", 10 * 1024 * 1024, 100 * 1024 * 1024],
 
    ["jane.doe", "janepassword", "domain2", "parameters-optional", 20 * 1024 * 1024, 70 * 1024 * 1024],
 
    ["mick.doe", "mickpassword", "domain3", "parameters-optional", 20 * 1024 * 1024, 70 * 1024 * 1024],
 
])
 
@pytest.mark.usefixtures("server_clean_domain_uploads")
 
def test_http_file_share_daily_quota(host, username, password, domain):
 
def test_http_file_share_daily_quota(host, username, password, domain, file_size_limit, user_daily_quota):
 
    """
 
    Tests the user's daily quota for files uploaded via XEP-0363.
 
    """
 

	
 
    # Equivalent of 100MiB.
 
    file_size_limit = 10 * 1024 * 1024
 
    file_count = 10
 

	
 
    # Fill-up the daily quota.
 
    create_sample_file = host.run("dd if=/dev/zero of=/tmp/http_file_upload_sample.txt bs=%sB count=1", str(file_size_limit))
 
    remaining_quota = user_daily_quota
 
    while remaining_quota > 0:
 
        file_size = file_size_limit if file_size_limit < remaining_quota else remaining_quota
 
        create_sample_file = host.run("dd if=/dev/zero of=/tmp/http_file_upload_sample.txt bs=%sB count=1", str(file_size))
 
        assert create_sample_file.rc == 0
 
    for _ in range(file_count):
 

	
 
        send = host.run(f"go-sendxmpp --debug --username {username}@{domain} --password {password} --jserver {domain}:5222 "
 
                        f"--http-upload /tmp/http_file_upload_sample.txt "
 
                        f"{username}@{domain}")
 
        assert send.rc == 0
 

	
 
        remaining_quota -= file_size
 

	
 
    # Test exceeded daily quota.
 
    create_sample_file = host.run("dd if=/dev/zero of=/tmp/http_file_upload_sample.txt bs=1B count=1")
 
    assert create_sample_file.rc == 0
 

	
 
    send = host.run(f"go-sendxmpp --debug --username {username}@{domain} --password {password} --jserver {domain}:5222 "
 
                    f"--http-upload /tmp/http_file_upload_sample.txt "
 
                    f"{username}@{domain}")
 
    assert "Daily quota reached" in send.stderr
roles/xmpp_server/molecule/default/tests/test_mandatory.py
Show inline comments
 
@@ -29,25 +29,27 @@ def test_prosody_configuration_file_content(host):
 
        assert "ldap_rootdn = \"cn=prosody,ou=services,dc=local\"" in config.content_string
 
        assert "ldap_password = \"prosodypassword\"" in config.content_string
 
        assert "ldap_filter = \"(&(mail=$user@$host)(memberOf=cn=xmpp,ou=groups,dc=local))\"" in config.content_string
 
        assert "ldap_base = \"ou=people,dc=local\"" in config.content_string
 
        assert "archive_expires_after = \"never\"" in config.content_string
 

	
 
        assert """VirtualHost "domain1"
 
Component "conference.domain1" "muc"
 
  restrict_room_creation = "local"
 
Component "proxy.domain1" "proxy65"
 
  proxy65_acl = { "domain1" }
 
Component "upload.domain1" "http_file_share"
 
  http_file_share_access = { "domain1" }""" in config.content_string
 
  http_file_share_access = { "domain1" }
 
  http_file_share_size_limit = 10485760
 
  http_file_share_daily_quota = 104857600""" in config.content_string
 

	
 

	
 
def test_xmpp_server_uses_correct_dh_parameters(host):
 
    """
 
    Tests if the HTTP server uses the generated Diffie-Hellman parameter.
 
    """
 

	
 
    fqdn = host.run('hostname -f').stdout.strip()
 

	
 
    # Use first defined domain for testing.
 
    domain = host.ansible.get_variables()['xmpp_domains'][0]
 

	
roles/xmpp_server/molecule/default/tests/test_optional.py
Show inline comments
 
@@ -29,33 +29,37 @@ def test_prosody_configuration_file_content(host):
 
        assert "ldap_rootdn = \"cn=prosody,ou=services,dc=local\"" in config.content_string
 
        assert "ldap_password = \"prosodypassword\"" in config.content_string
 
        assert "ldap_filter = \"(&(mail=$user@$host)(memberOf=cn=xmpp,ou=groups,dc=local))\"" in config.content_string
 
        assert "ldap_base = \"ou=people,dc=local\"" in config.content_string
 
        assert "archive_expires_after = \"1w\"" in config.content_string
 

	
 
        assert """VirtualHost "domain2"
 
Component "conference.domain2" "muc"
 
  restrict_room_creation = "local"
 
Component "proxy.domain2" "proxy65"
 
  proxy65_acl = { "domain2" }
 
Component "upload.domain2" "http_file_share"
 
  http_file_share_access = { "domain2" }""" in config.content_string
 
  http_file_share_access = { "domain2" }
 
  http_file_share_size_limit = 20971520
 
  http_file_share_daily_quota = 73400320""" in config.content_string
 

	
 
        assert """VirtualHost "domain3"
 
Component "conference.domain3" "muc"
 
  restrict_room_creation = "local"
 
Component "proxy.domain3" "proxy65"
 
  proxy65_acl = { "domain3" }
 
Component "upload.domain3" "http_file_share"
 
  http_file_share_access = { "domain3" }""" in config.content_string
 
  http_file_share_access = { "domain3" }
 
  http_file_share_size_limit = 20971520
 
  http_file_share_daily_quota = 73400320""" in config.content_string
 

	
 

	
 
@pytest.mark.parametrize("port", [
 
    5222,
 
    5223
 
])
 
def test_xmpp_c2s_tls_version_and_ciphers(host, port):
 
    """
 
    Tests if the correct TLS version and ciphers have been enabled for
 
    XMPP C2S ports.
 
    """
 

	
roles/xmpp_server/templates/prosody.cfg.lua.j2
Show inline comments
 
@@ -103,13 +103,15 @@ log = {
 
}
 

	
 
-- Domains which should be handled by Prosody, with dedicated MUC and file
 
-- proxying components.
 
{% for domain in xmpp_domains -%}
 
VirtualHost "{{ domain }}"
 
Component "conference.{{ domain }}" "muc"
 
  restrict_room_creation = "local"
 
Component "proxy.{{ domain }}" "proxy65"
 
  proxy65_acl = { "{{ domain }}" }
 
Component "upload.{{ domain }}" "http_file_share"
 
  http_file_share_access = { "{{ domain }}" }
 
  http_file_share_size_limit = {{ xmpp_http_file_share_size_limit }}
 
  http_file_share_daily_quota = {{ xmpp_http_file_share_daily_quota }}
 
{% endfor -%}
0 comments (0 inline, 0 general)