majic-ansible-roles Changeset

Changeset - d0c6f26ece9f

Parent rev.

Child rev.

[Not reviewed]

1 7 1

Branko Majic (branko) - 2 years ago 2023-11-19 22:24:55
branko@majic.rs

MAR-182: Added ability to specify input requirements for running pip package upgrade checks against helper virtual environments:

- These environments are used for running the package upgrade checks
themselves.

9 files changed with 117 insertions and 11 deletions:

docs/releasenotes.rst

docs/rolereference.rst

roles/common/defaults/main.yml

roles/common/files/pipreqcheck_requirements.in

roles/common/molecule/default/group_vars/parameters-optional.yml

roles/common/molecule/default/tests/test_parameters_mandatory.py

roles/common/molecule/default/tests/test_parameters_optional.py

roles/common/tasks/main.yml

roles/common/templates/pipreqcheck_requirements.in.j2

0 comments (0 inline, 0 general)

docs/releasenotes.rst

➞

Show inline comments

@@ @@ -17,12 +17,23 @@ run applications using Debian-only repositories. @@
 * ``xmpp_server`` role
   * Parameter ``xmpp_prosody_package`` has been dropped.
 **New features/improvements**
 * ``common`` role
   * Added parameters ``pip_check_requirements_in`` and
     ``pip_check_requirements_py3_in`` that can be used for specifying
     input requirements when checking for available package upgrades
     for Python virtual environments that are used for the checks
     themselves. This is particularly helpful in cases where Python
     version gets deprecated and some packages do not correctly declare
     the minimum version required, allowing to be more specific to
     avoid unnecessary warning mails being sent out.
 * ``xmpp_server`` role
   * Drop dependency on the external (Prosody) package
     repository. Install everything using official Debian
     repositories. This should help avoid future issues with Prosody
     project removing older versions of packages or dropping entire

docs/rolereference.rst

➞

Show inline comments

@@ @@ -413,27 +413,39 @@ Parameters @@
     - "0.debian.pool.ntp.org"
     - "1.debian.pool.ntp.org"
     - "2.debian.pool.ntp.org"
     - "3.debian.pool.ntp.org"
 **pip_check_requirements_in** (list, optional, ``[pip, pip-tools, setuptools, wheel]``)
   List of Python package requirements inputs to use for checking for
   package upgrades for the Python 2 virtual environment used to run
   the check itself. For Python 3, see the dedicated parameter
   ``pip_check_requirements_py3`` below.
 **pip_check_requirements** (list, optional, ``[click==7.0, pip-tools==4.0.0, pip==19.2.3, setuptools==41.2.0, six==1.12.0, wheel==0.33.6]``)
   List of Python package requirements to install in Python 2 virtual
   environment in order to be able to run the ``pip-tools``
   applications as part of pip requirements upgrade checks. This list
   needs to be updated from time to time as the new releases of
   ``pip-tools`` and related packages are coming out. For Python 3, see
   the dedicatd parameter ``pip_check_requirements_py3`` below.
   the dedicated parameter ``pip_check_requirements_py3`` below.
 **pip_check_requirements_py3_in** (list, optional, ``[pip, pip-tools, setuptools, wheel]``)
   List of Python package requirements inputs to use for checking for
   package upgrades for the Python 3 virtual environment used to run
   the check itself. For Python 2, see the dedicated parameter
   ``pip_check_requirements`` above.
 **pip_check_requirements_py3** (list, optional, ``[click==7.0, pip-tools==3.9.0, pip==19.1.1, setuptools==41.2.0, six==1.12.0, wheel==0.33.6]``)
   List of Python package requirements to install in Python 3 virtual
   environment in order to be able to run the ``pip-tools``
   applications as part of pip requirements upgrade checks. This list
   needs to be updated from time to time as the new releases of
   ``pip-tools`` and related packages are coming out. For Python 2, see
-  the dedicatd parameter ``pip_check_requirements`` above.
+  the dedicated parameter ``pip_check_requirements`` above.
 **pipreqcheck_uid** (integer, optional, ``whatever OS picks``)
   UID for user running the pip requirements upgrade checks. User is created with
   name ``pipreqcheck``.
 **pipreqcheck_gid** (integer, optional, ``whatever OS picks``)

roles/common/defaults/main.yml

➞

Show inline comments

@@ @@ -9,19 +9,29 @@ incoming_connection_limit: 3/second @@
 incoming_connection_limit_burst: 9
 prompt_colour: none
 prompt_id: null
 extra_backup_patterns:
   - "/root"
   - "/home"
 pip_check_requirements_in:
   - pip
   - pip-tools
   - setuptools
   - wheel
 pip_check_requirements:
   - click==7.1.2
   - pip-tools==5.3.1
   - pip==20.2.4
   - setuptools==44.1.1
   - six==1.15.0
   - wheel==0.35.1
 pip_check_requirements_py3_in:
   - pip
   - pip-tools
   - setuptools
   - wheel
 pip_check_requirements_py3:
   - click==7.1.2
   - pip-tools==5.3.1
   - pip==20.2.4
   - setuptools==50.3.2
   - six==1.15.0

roles/common/files/pipreqcheck_requirements.in

➞

Show inline comments

deleted file

roles/common/molecule/default/group_vars/parameters-optional.yml

➞

Show inline comments

@@ @@ -50,12 +50,22 @@ ntp_servers: @@
   - "0.debian.pool.ntp.org"
   - "1.debian.pool.ntp.org"
   - "2.debian.pool.ntp.org"
 maintenance: true
 maintenance_allowed_hosts:
   - client1
 pip_check_requirements_in:
   - pip >= 0.2.1
   - pip-tools >= 0.2.2
   - setuptools >= 0.2.3
   - wheel >= 0.2.4
 pip_check_requirements_py3_in:
   - pip >= 0.3.1
   - pip-tools >= 0.3.2
   - setuptools >= 0.3.3
   - wheel >= 0.3.4
 # From backup_client role meta dependency.
 backup_encryption_key: "{{ lookup('file', 'tests/data/gnupg/backup_encryption_key') }}"
 backup_server: backup-server
 backup_server_host_ssh_public_keys:
   - bougs-backup-server-key-1

roles/common/molecule/default/tests/test_parameters_mandatory.py

➞

Show inline comments

@@ @@ -2,12 +2,14 @@ import os @@
 import socket
 import paramiko
 import testinfra.utils.ansible_runner
 import pytest
 testinfra_hosts = testinfra.utils.ansible_runner.AnsibleRunner(
     os.environ['MOLECULE_INVENTORY_FILE']).get_hosts('parameters-mandatory')
 def test_apt_proxy(host):
@@ @@ -128,6 +130,36 @@ def test_ntp_software_not_installed(host): @@
 def test_ntp_listening_interfaces(host):
     """
     Tests if NTP server is not listening.
     """
     assert not host.socket('udp://:::123').is_listening
 @pytest.mark.parametrize("requirements_path, expected_requirements", [
     ('/etc/pip_check_requirements_upgrades/pipreqcheck/requirements.in', [
         "pip",
         "pip-tools",
         "setuptools",
         "wheel"
     ]),
     ('/etc/pip_check_requirements_upgrades-py3/pipreqcheck/requirements.in', [
         "pip",
         "pip-tools",
         "setuptools",
         "wheel"
     ])
 ])
 def test_pipreqcheck_input_content(host, requirements_path, expected_requirements):
     """
     Tests content of requirements input file used for virtual
     environment utilised by script that performs pip requirements
     upgrade checks.
     """
     with host.sudo():
         deployed_requirements = host.file(requirements_path).content_string
         expected_requirements = sorted([line.lower() for line in expected_requirements])
         actual_requirements = sorted(deployed_requirements.lower().strip().split("\n"))
         assert actual_requirements == expected_requirements

roles/common/molecule/default/tests/test_parameters_optional.py

➞

Show inline comments

@@ @@ -320,6 +320,36 @@ def test_ntp_query_server_count(host): @@
 def test_ntp_listening_interfaces(host):
     """
     Tests if NTP server is listening on correct ports.
     """
     assert host.socket('udp://:::123').is_listening
 @pytest.mark.parametrize("requirements_path, expected_requirements", [
     ('/etc/pip_check_requirements_upgrades/pipreqcheck/requirements.in', [
         "pip >= 0.2.1",
         "pip-tools >= 0.2.2",
         "setuptools >= 0.2.3",
         "wheel >= 0.2.4"
     ]),
     ('/etc/pip_check_requirements_upgrades-py3/pipreqcheck/requirements.in', [
         "pip >= 0.3.1",
         "pip-tools >= 0.3.2",
         "setuptools >= 0.3.3",
         "wheel >= 0.3.4"
     ])
 ])
 def test_pipreqcheck_input_content(host, requirements_path, expected_requirements):
     """
     Tests content of requirements input file used for virtual
     environment utilised by script that performs pip requirements
     upgrade checks.
     """
     with host.sudo():
         deployed_requirements = host.file(requirements_path).content_string
         expected_requirements = sorted([line.lower() for line in expected_requirements])
         actual_requirements = sorted(deployed_requirements.lower().strip().split("\n"))
         assert actual_requirements == expected_requirements

roles/common/tasks/main.yml

➞

Show inline comments

@@ @@ -350,21 +350,23 @@ @@
     mode: 0750
   with_items:
     - "/etc/pip_check_requirements_upgrades/pipreqcheck"
     - "/etc/pip_check_requirements_upgrades-py3/pipreqcheck"
 - name: Deploy .in file for pip requirements in pip-tools virtual environment
   copy:
     src: "pipreqcheck_requirements.in"
     dest: "{{ item }}"
   template:
     src: "pipreqcheck_requirements.in.j2"
     dest: "{{ item.path }}"
     owner: root
     group: pipreqcheck
     mode: 0640
   with_items:
     - "/etc/pip_check_requirements_upgrades/pipreqcheck/requirements.in"
     - "/etc/pip_check_requirements_upgrades-py3/pipreqcheck/requirements.in"
     - path: "/etc/pip_check_requirements_upgrades/pipreqcheck/requirements.in"
       requirements: "{{ pip_check_requirements_in }}"
     - path: "/etc/pip_check_requirements_upgrades-py3/pipreqcheck/requirements.in"
       requirements: "{{ pip_check_requirements_py3_in }}"
 - name: Deploy requirements file for pipreqcheck virtual environment
   template:
     src: "pipreqcheck_requirements.txt.j2"
     dest: "{{ item.file }}"
     owner: root

roles/common/templates/pipreqcheck_requirements.in.j2

➞

Show inline comments

@@ new file 100644 @@
 {% for requirement in item.requirements %}
 {{ requirement }}
 {% endfor %}

0 comments (0 inline, 0 general)