**Breaking changes:**

* All roles

  * Dropped support for Debian 10 (Buster).

**Bug fixes:**

* ``common`` role

  * Fix deprecation warnings for Python requirements upgrade checks

    when using pip-tools 7.3.0. This would result in unnecessary

  daily basis, using crontab (resulting in failures being sent out to

  the ``root`` user).

* Deploys ``apticron`` package that performs checks for available package

  upgrades on daily basis. Mails are delivered to local ``root`` account, and

  can be redirected elsewhere via aliases. If using ``mail_forwarder`` or

  ``mail_server`` roles on the same server, aliases can be set-up through them.

* Optionally configures time synchronisation using NTP (if

  ``ntp_servers`` parameter is set).

Role dependencies

~~~~~~~~~~~~~~~~~

    - "0.debian.pool.ntp.org"

    - "1.debian.pool.ntp.org"

    - "2.debian.pool.ntp.org"

    - "3.debian.pool.ntp.org"

**pip_check_requirements_in** (list, optional, ``[pip, pip-tools, setuptools, wheel]``)

  List of Python package requirements inputs to use for checking for

  package upgrades for the Python 3 virtual environment used to run

  List of Python package requirements to install in Python 3 virtual

  environment in order to be able to run the ``pip-tools``

  applications as part of pip requirements upgrade checks. This list

  needs to be updated from time to time as the new releases of

  Default value is:

  .. code-block:: yaml

    - build==1.0.3

package is *not* installed by the ``common`` role out-of-the-box, but you can

easily do so by updating the ``common_packages`` setting.

In addition to system packages, the ``common`` role makes it easy to check if

any of the pip requirements files are outdated as well. It should be noted,

though, that this check does *not* verify the Python virtual environments

This is primarily useful when you use `pip-tools

<https://github.com/jazzband/pip-tools>`_ for maintaining the

requirements files. In fact, I would encourage you to utilise

``pip-tools`` for both this purpose and for keeping the virtual

environment in sync and up-to-date.

Roles that want to take advantage of this would:

- Create a sub-directory under

- Deploy ``.in`` and ``.txt`` files within the sub-directory (see ``pip-tools``

  docs for explanation of how the ``.in`` files work).

- Ensure the created sub-directory and files have ownership set to

  ``root:pipreqcheck``.

.. note::

pip_check_requirements_in:

  - pip

  - pip-tools

  - setuptools

  - wheel

pip_check_requirements:

  - build==1.0.3

  - click==8.1.7

  - importlib-metadata==6.7.0

  - packaging==23.2

  - pip-tools==6.14.0

  - pip==23.1.2

  - "1.debian.pool.ntp.org"

  - "2.debian.pool.ntp.org"

maintenance: true

maintenance_allowed_hosts:

  - client1

pip_check_requirements_in:

  - pip >= 0.3.1

  - pip-tools >= 0.3.2

  - setuptools >= 0.3.3

  - wheel >= 0.3.4

# From backup_client role meta dependency.

        group: pipreqcheck

        mode: 0750

      with_items:

        - "/tmp/pip_check_requirements_upgrades"

        - "/tmp/pip_check_requirements_upgrades/with_updates"

        - "/tmp/pip_check_requirements_upgrades/without_updates"

    - name: Deploy files for testing pip requirements upgrade checks script

      copy:

        src: "{{ item }}"

        dest: "/tmp/{{ item }}"

        owner: root

        directory_mode: 0750

      with_items:

        - "pip_check_requirements_upgrades/with_updates/requirements.in"

        - "pip_check_requirements_upgrades/with_updates/requirements.txt"

        - "pip_check_requirements_upgrades/without_updates/requirements.in"

        - "pip_check_requirements_upgrades/without_updates/requirements.txt"

    - name: Install web server for testing connectivity

      apt:

        name: nginx

        state: present

        - filter

        - nat

        - mangle

        - security

        - raw

- hosts: parameters-mandatory,parameters-optional

  become: true

  tasks:

    - name: Remove the ss utility (see https://github.com/philpep/testinfra/pull/320)

      file:

    assert check_certificate_crontab.user == 'root'

    assert check_certificate_crontab.group == 'root'

    assert check_certificate_crontab.mode == 0o644

    assert "0 0 * * * nobody /usr/local/bin/check_certificate.sh -q expiration" in check_certificate_crontab.content_string

    """

    Tests creation of Python virtual environment used for performing pip

    requirements upgrade checks.

    """

    with host.sudo():

        virtualenv_activate = host.file(virtualenv_activate_path)

        assert virtualenv_activate.is_file

        assert virtualenv_activate.user == 'pipreqcheck'

        assert virtualenv_activate.group == 'pipreqcheck'

        assert virtualenv_activate.mode == 0o644

    """

    Tests creation of directories used for storing configuration used by script

    that performs pip requirements upgrade checks.

    """

    with host.sudo():

        pipreqcheck_config_directory = host.file(config_dir)

        assert pipreqcheck_config_directory.is_directory

        assert pipreqcheck_config_directory.user == 'root'

        assert pipreqcheck_config_directory.group == 'pipreqcheck'

        assert pipreqcheck_config_directory.mode == 0o750

        assert pipreqcheck_config_directory_pipreqcheck.is_directory

        assert pipreqcheck_config_directory_pipreqcheck.user == 'root'

        assert pipreqcheck_config_directory_pipreqcheck.group == 'pipreqcheck'

        assert pipreqcheck_config_directory_pipreqcheck.mode == 0o750

    """

    Tests deployment of requirements input and text file used for virtual

    environment utilised by script that perform pip requirements upgrade checks.

    """

    with host.sudo():

        requirements_in = host.file(requirements_in_path)

        assert requirements_in.is_file

        assert requirements_in.user == 'root'

        assert requirements_in.group == 'pipreqcheck'

        assert requirements_in.mode == 0o640

        requirements_txt.is_file

        assert requirements_txt.user == 'root'

        assert requirements_txt.group == 'pipreqcheck'

        assert requirements_txt.mode == 0o640

        "build==1.0.3",

        "click==8.1.7",

        "importlib-metadata==6.7.0",

        "packaging==23.2",

        "pip-tools==6.14.0",

        "pip==23.1.2",

        "pyproject_hooks==1.0.0",

        "setuptools==68.0.0",

        "tomli==2.0.1",

        "typing_extensions==4.7.1",

        "wheel==0.41.3",

        "zipp==3.15.0",

    packages = host.run("sudo -u pipreqcheck %s freeze --all", pip_path)

    # Normalise package names and order.

    expected_packages = sorted([p.lower() for p in expected_packages])

    actual_packages = sorted(packages.stdout.lower().strip().split("\n"))

    assert pipreqcheck_script.is_file

    assert pipreqcheck_script.user == 'root'

    assert pipreqcheck_script.group == 'root'

    assert pipreqcheck_script.mode == 0o755

    """

    Tests if crontab entry is set-up correctly for running the pip requirements

    upgrade checks.

    """

    crontab = host.file(crontab_path)

    assert crontab.is_file

    assert crontab.user == 'root'

    assert crontab.group == 'root'

    assert crontab.mode == 0o644

    assert "MAILTO=root" in crontab.content_string

    assert virtualenv_path in crontab.content_string.split(" ")

    """

    Tests if Python virtual environment for pipreqcheck has been

    set-up correctly.

    """

    with host.sudo('pipreqcheck'):

        major_version = host.run("%s -c %s", python_path, "import sys; print(sys.version_info.major)")

    assert major_version.rc == 0

    assert major_version.stdout.strip() == expected_major_version

    """

    Tests if the pip_check_requirements_upgrades.sh script properly

    reports available updates or not.

    """

    expected_line_count = 9

    expected_warning_message = "[WARN]  Upgrades available for: %s/with_updates/requirements.txt" % config_directory

    expected_package_diff = "@@ -1 +1 @@\n-urllib3==1.24.2\n+urllib3==1.24.3"

    with host.sudo("pipreqcheck"):

        # Clean-up the SSH warning from the beginning of stderr if

        # present.

        stderr = re.sub("^Warning: Permanently added.*?\r\n", "", report.stderr)

    assert stderr == ""

import socket

import paramiko

import testinfra.utils.ansible_runner

testinfra_hosts = testinfra.utils.ansible_runner.AnsibleRunner(

    os.environ['MOLECULE_INVENTORY_FILE']).get_hosts('parameters-mandatory')

def test_apt_proxy(host):

    Tests if NTP server is not listening.

    """

    assert not host.socket('udp://:::123').is_listening

    """

    Tests content of requirements input file used for virtual

    environment utilised by script that performs pip requirements

    upgrade checks.

    """

    with host.sudo():

        deployed_requirements = host.file(requirements_path).content_string

        expected_requirements = sorted([line.lower() for line in expected_requirements])

        actual_requirements = sorted(deployed_requirements.lower().strip().split("\n"))

    Tests if NTP server is listening on correct ports.

    """

    assert host.socket('udp://:::123').is_listening

    """

    Tests content of requirements input file used for virtual

    environment utilised by script that performs pip requirements

    upgrade checks.

    """

    with host.sudo():

        deployed_requirements = host.file(requirements_path).content_string

        expected_requirements = sorted([line.lower() for line in expected_requirements])

        actual_requirements = sorted(deployed_requirements.lower().strip().split("\n"))

---

- name: Enable use of proxy for retrieving system packages via apt

  template:

    src: "apt_proxy.j2"

    dest: "/etc/apt/apt.conf.d/00proxy"

    owner: root

    group: root

    state: present

- name: Check for presence of wrong Python versions

  stat:

    path: "{{ item.offending_binary }}"

  with_items:

      virtualenv_directory: "/var/lib/pipreqcheck/virtualenv"

  register: "pipreqcheck_wrong_python_version_check"

- name: Drop Python virtual environment if wrong Python version has been detected within

  file:

    path: "{{ item.item.virtualenv_directory }}"

    state: absent

    path: "{{ item }}"

    state: directory

    owner: pipreqcheck

    group: pipreqcheck

    mode: 0750

  with_items:

    - "/var/lib/pipreqcheck/virtualenv"

- name: Create Python virtual environment used for installing/running pip-tools

  command: "/usr/bin/virtualenv --python '{{ item.python_path }}' --prompt '({{ item.name }})' '{{ item.virtualenv_path }}'"

  args:

    creates: "{{ item.creates }}"

  become: true

  become_user: "pipreqcheck"

  with_items:

    - name: pipreqcheck

      virtualenv_path: "/var/lib/pipreqcheck/virtualenv"

      python_path: "/usr/bin/python3"

- name: Create directory for storing pip requirements files

  file:

    path: "{{ item }}"

    state: "directory"

    owner: root

    group: pipreqcheck

    mode: 0750

  with_items:

    - "/etc/pip_check_requirements_upgrades"

- name: Set-up directory for storing pip requirements file for pip-tools virtual environment itself

  file:

    path: "{{ item }}"

    state: "directory"

    owner: root

    group: pipreqcheck

    mode: 0750

  with_items:

    - "/etc/pip_check_requirements_upgrades/pipreqcheck"

- name: Deploy .in file for pip requirements in pip-tools virtual environment

  template:

    src: "pipreqcheck_requirements.in.j2"

    dest: "{{ item.path }}"

    owner: root

    group: pipreqcheck

    mode: 0640

  with_items:

    - path: "/etc/pip_check_requirements_upgrades/pipreqcheck/requirements.in"

      requirements: "{{ pip_check_requirements_in }}"

- name: Deploy requirements file for pipreqcheck virtual environment

  template:

    src: "pipreqcheck_requirements.txt.j2"

    dest: "{{ item.file }}"

    owner: root

    group: pipreqcheck

    mode: 0640

  with_items:

    - file: "/etc/pip_check_requirements_upgrades/pipreqcheck/requirements.txt"

      requirements: "{{ pip_check_requirements }}"

- name: Install requirements in the pipreqcheck virtual environment

  pip:

    requirements: "{{ item.requirements }}"

    virtualenv: "{{ item.virtualenv }}"

  become: true

  become_user: pipreqcheck

  with_items:

    - virtualenv: "~pipreqcheck/virtualenv"

      requirements: "/etc/pip_check_requirements_upgrades/pipreqcheck/requirements.txt"

  shell: "source ~pipreqcheck/virtualenv/bin/activate && pip-sync /etc/pip_check_requirements_upgrades/pipreqcheck/requirements.txt"

  args:

    executable: /bin/bash

  become: true

  become_user: "pipreqcheck"

  register: pipreqcheck_pip_sync

  changed_when: "pipreqcheck_pip_sync.stdout != 'Everything up-to-date'"

- name: Deploy script for checking available upgrades

  copy:

    src: "pip_check_requirements_upgrades.sh"

    dest: "/usr/local/bin/pip_check_requirements_upgrades.sh"

    owner: root

    group: root

    src: "cron_check_pip_requirements"

    dest: "/etc/cron.d/check_pip_requirements"

    owner: root

    group: root

    mode: 0644

- name: Install NTP packages

  apt:

    name:

      - ntp

      - ntpdate

    state: present