majic-ansible-roles Changeset

Changeset - 380556b776cd

Parent rev.

Child rev.

[Not reviewed]

0 5 0

Branko Majic (branko) - 5 years ago 2018-12-10 20:58:52
branko@majic.rs

MAR-141: Fix handling of pipreqcheck virtualenv package installation:

- Deploy all requirements through requirements file, even if using
pip. Improves reproducability of installed packages and solves some
double-versioning issues for packages like pip etc.
- Updated test for validating installed packages.

5 files changed with 47 insertions and 33 deletions:

roles/common/defaults/main.yml

roles/common/files/pip_check_requirements_upgrades.sh

roles/common/files/pipreqcheck_requirements.in

roles/common/molecule/default/tests/test_default.py

roles/common/tasks/main.yml

0 comments (0 inline, 0 general)

roles/common/defaults/main.yml

➞

Show inline comments

@@ @@ -13,10 +13,12 @@ extra_backup_patterns: @@
   - "/root"
   - "/home"
 pip_check_requirements:
   - click==6.7
   - first==2.0.1
   - pip-tools==2.0.2
   - click==7.0
   - pip-tools==3.1.0
   - pip==18.1
   - setuptools==40.6.2
   - six==1.11.0
   - wheel==0.32.3
 ntp_servers: []
 # Internal use only.

roles/common/files/pip_check_requirements_upgrades.sh

➞

Show inline comments

@@ @@ -254,7 +254,7 @@ for environment in "$config_dir"/*; do @@
 	# Deploy the existing requirements file and the new one.
 	sed -e 's/[[:blank:]]*#.*//' "$req_txt" | grep -v "^$" | sort -u > "$tmp_current"
 	if ! pip-compile --no-header --no-annotate --no-index --output-file "$tmp_new" --upgrade "$req_in" > /dev/null; then
+	if ! pip-compile --allow-unsafe --no-header --no-annotate --no-index --output-file "$tmp_new" --upgrade "$req_in" > /dev/null; then
 	    error "Failed while running pip-compile command against (see error stack trace above): $req_in"
 	    continue
 	fi

roles/common/files/pipreqcheck_requirements.in

➞

Show inline comments

 pip-tools
@@ \ No newline at end of file @@
 pip
 pip-tools
 setuptools
@@ \ No newline at end of file @@

roles/common/molecule/default/tests/test_default.py

➞

Show inline comments

         assert requirements_txt.mode == 0o640
 @pytest.mark.parametrize('pip_path', [
     '/var/lib/pipreqcheck/virtualenv/bin/pip',
     '/var/lib/pipreqcheck/virtualenv-py3/bin/pip',
 @pytest.mark.parametrize("pip_path, expected_packages",  [
     ('/var/lib/pipreqcheck/virtualenv/bin/pip', [
         "Click==7.0",
         "pip==18.1",
         "pip-tools==3.1.0",
         "setuptools==40.6.2",
         "six==1.11.0",
         "wheel==0.32.3"
     ]),
     ('/var/lib/pipreqcheck/virtualenv-py3/bin/pip', [
         "Click==7.0",
         "pip==18.1",
         "pip-tools==3.1.0",
         "setuptools==40.6.2",
         "six==1.11.0",
         "wheel==0.32.3",
     ]),
 ])
-def test_pipreqcheck_packages(host, pip_path):
+def test_pipreqcheck_virtualenv_packages(host, pip_path, expected_packages):
     """
     Tests if Python virtual environment used for running the pip requirements
     upgrade checks has correct version of pip installed.
     Tests if correct packages are installed in virtualenv used for pip
     requirements checks..
     """
     with host.sudo():
         packages = host.pip_package.get_packages(pip_path=pip_path)
     packages = host.run("sudo -u %s %s freeze --all" % ('pipreqcheck', pip_path))
     # Normalise package names and order.
     expected_packages = sorted([unicode(p.lower()) for p in expected_packages])
     actual_packages = sorted(packages.stdout.lower().split("\n"))
     # This is a dummy distro-provided package ignored by the pip-tools.
     if "pkg-resources==0.0.0" in actual_packages:
         actual_packages.remove("pkg-resources==0.0.0")
         assert packages['pip']['version'].rsplit('.', 1)[0] == '18'
         assert 'pip-tools' in packages
     assert actual_packages == expected_packages
 def test_pipreqcheck_script(host):

roles/common/tasks/main.yml

➞

Show inline comments

@@ @@ -368,27 +368,17 @@ @@
     - "/etc/pip_check_requirements_upgrades/pipreqcheck/requirements.txt"
     - "/etc/pip_check_requirements_upgrades-py3/pipreqcheck/requirements.txt"
-- name: Install latest pip in pip-tools virtual environment
+- name: Install requirements in the pipreqcheck virtual environment
   pip:
     name:
       - "pip>=18.0.0,<19.0.0"
     virtualenv: "{{ item }}"
   become: true
   become_user: "pipreqcheck"
   with_items:
     - "~pipreqcheck/virtualenv"
     - "~pipreqcheck/virtualenv-py3"
 - name: Install pip-tools if not present
   pip:
     name: pip-tools>=2.0.2
     state: present
     virtualenv: "{{ item }}"
     requirements: "{{ item.requirements }}"
     virtualenv: "{{ item.virtualenv }}"
   become: true
-  become_user: "pipreqcheck"
   become_user: pipreqcheck
   with_items:
     - "~pipreqcheck/virtualenv"
     - "~pipreqcheck/virtualenv-py3"
     - virtualenv: "~pipreqcheck/virtualenv"
       requirements: "/etc/pip_check_requirements_upgrades/pipreqcheck/requirements.txt"
     - virtualenv: "~pipreqcheck/virtualenv-py3"
       requirements: "/etc/pip_check_requirements_upgrades-py3/pipreqcheck/requirements.txt"
 - name: Synchronise pip-tools virtual environment via deployed requirements file (Python 2)
   shell: "source ~pipreqcheck/virtualenv/bin/activate && pip-sync /etc/pip_check_requirements_upgrades/pipreqcheck/requirements.txt"

0 comments (0 inline, 0 general)