Changeset - b757d690af42
[Not reviewed]
0 7 1
Branko Majic (branko) - 9 years ago 2016-11-26 19:49:53
branko@majic.rs
MAR-75: Implemented support for specifying additional environment variables for the wsgi_website role. Environment is set-up for both the systemd service and for application admin user. Updated testsite implementation to use it for sample WSGI website.
8 files changed with 28 insertions and 2 deletions:
0 comments (0 inline, 0 general)
docs/rolereference.rst
Show inline comments
 
@@ -1439,48 +1439,53 @@ Parameters
 
**additional_nginx_config** (list, optional, ``[]``)
 
  List providing additional Nginx configuration options to include. This can be
 
  useful for specifying things like error pages. Options are applied inside of a
 
  **server** context of Nginx configuration file.
 

	
 
  Each item is a dictionary with the following options describing the extra
 
  configuration option:
 

	
 
  **comment** (string, mandatory)
 
    Comment describing the configuration option.
 

	
 
  **value** (string, mandatory)
 
    Configuration option.
 

	
 
**admin_uid** (integer, optional, ``whatever OS picks``)
 
  UID of the dedicated website administrator user. The user will be member of
 
  website group.
 

	
 
**enforce_https** (boolean, optional, ``True``)
 
  Specify if HTTPS should be enforced for the website or not. If enforced,
 
  clients connecting via plaintext will be redirected to HTTPS, and clients will
 
  be served with ``Strict-Transport-Security`` header with value of
 
  ``max-age=31536000; includeSubDomains``.
 

	
 
**environment_variables** (dict, optional, ``{}``)
 
  Specify additional environment variables that should be set for running the
 
  service. Environment variables will be set in both the systemd service and for
 
  the application's administrator user (when logged in as one).
 

	
 
**fqdn** (string, mandatory)
 
  Fully-qualified domain name where the website is reachable. This value is used
 
  for calculating the user/group name for dedicated website user, as well as
 
  home directory of the website user (where data/code should be stored at).
 

	
 
**futures_version** (string, optional, ``3.0.5``)
 
  Version of ``futures`` package to deploy in virtual environment. Required by
 
  Gunicorn when using Python 2.7. Default version is tested with the test site.
 

	
 
**gunicorn_version** (string, optional, ``19.6.0``)
 
  Version of Gunicorn to deploy in virtual environment for running the WSGI
 
  application. Default version is tested with the test site.
 

	
 
**https_tls_certificate** (string, optional, ``{{ lookup('file', tls_certificate_dir + '/' + fqdn + '_https.pem') }}``)
 
  X.509 certificate used for TLS for HTTPS service. The file will be stored in
 
  directory ``/etc/ssl/certs/`` under name ``{{ fqdn }}_https.pem``.
 

	
 
**https_tls_key** (string, optional, ``{{ lookup('file', tls_private_key_dir + '/' + fqdn + '_https.key') }}``)
 
  Private key used for TLS for HTTPS service. The file will be stored in
 
  directory ``/etc/ssl/private/`` under name ``{{ fqdn }}_https.key``.
 

	
 
**packages** (list, optional, ``[]``)
 
  A list of additional packages to install for this particular WSGI
 
  website. This is usually going to be development libraries for building Python
 
@@ -1515,48 +1520,50 @@ Parameters
 
  conformant to what the ``gunicorn`` command-line tool accepts. If the
 
  ``use_paste`` option is enabled, the value should be equal to filename of the
 
  Python Paste ini file, located in the ``code`` sub-directory. It should be
 
  noted that in either case the value should be specsified relative to the
 
  ``code`` sub-directory. I.e. don't use full paths.
 

	
 

	
 
Examples
 
~~~~~~~~
 

	
 
Here is an example configuration for setting-up a (base) WSGI website (for
 
running a bare Django project):
 

	
 
.. code-block:: yaml
 

	
 
    - role: wsgi_website
 
      fqdn: django.example.com
 
      static_locations:
 
        - /static
 
        - /media
 
      uid: 2004
 
      virtualenv_packages:
 
        - django
 
      wsgi_application: django_example_com.wsgi:application
 
      environment_variables:
 
        DJANGO_SETTINGS_MODULE: "django_example_com.settings.production"
 
      https_tls_key: "{{ lookup('file', inventory_dir + '/tls/wsgi.example.com_https.key') }}"
 
      https_tls_certificate: "{{ lookup('file', inventory_dir + '/tls/wsgi.example.com_https.pem') }}"
 
      futures_version: 3.0.5
 
      gunicorn_version: 19.6.0
 
      additional_nginx_config:
 
        - comment: Use custom page for forbidden files.
 
          value: error_page 403 /static/403.html;
 
        - comment: Use custom page for non-existing locations/files.
 
          value: error_page 404 /static/404.html;
 

	
 

	
 
Database Server
 
---------------
 

	
 
The ``database_server`` role can be used for setting-up a MariaDB database
 
server on destination machine.
 

	
 
The role implements the following:
 

	
 
* Installs MariaDB server and client.
 
* Configures MariaDB server and client to use *UTF-8* encoding by default.
 
* Sets password for the database root user.
 
* Deploys MariaDB client configuration in location ``/root/.my.cnf`` that
 
  contains username and password for the root database user.
docs/usage.rst
Show inline comments
 
@@ -1451,48 +1451,51 @@ Next thing up will be to deploy a WSGI Python application.
 

	
 
Similar to the PHP application deployment, we will use a couple of roles to make
 
it easier to deploy it in a standardised manner, and we will not have any kind
 
of parameters for configuring the role to keep things simple.
 

	
 
Most of the notes on how a ``php_website`` role is deployed also stand for the
 
``wsgi_website`` role, but we will reiterate and clarify them a bit just to be
 
on the safe side:
 

	
 
* The role is designed to execute every application via dedicated user and
 
  group. The user/group name is automatically derived from the FQDN of website,
 
  for example ``web-wiki_example_com``.
 
* While running the application, application user's umask is set to ``0007``
 
  (letting the administrator user be able to manage any files created while the
 
  application is running).
 
* An administrative user is created as well, and this user should be used when
 
  running maintenance and installation commands. Similar to application user,
 
  the name is also derived from the FQDN of website, for example
 
  ``admin-wiki_example_com``. Administrative user does not have a dedicated
 
  group, and instead belongs to same group as the application user. As
 
  convenience, whenever you switch to this user the Python virtual environment
 
  will be automatically activated for you.
 
* WSGI applications are executed via *Gunicorn*. The WSGI server listens on a
 
  Unix socket, making the socket accessible by *Nginx*.
 
* If you ever need to set some environment variables, this can easily be done
 
  via the ``environment_variables`` role parameter. This particular example does
 
  not set any, though.
 
* Static content is served directly by *Nginx*.
 
* Each web application gets distinct sub-directory under ``/var/www``, named
 
  after the FQDN. All sub-directories created under there are created with
 
  ``2750`` permissions, with ownership set to admin user, and group set to the
 
  application's group. In other words, all directories will have ``SGID`` bit
 
  set-up, allowing you to create files/directories that will have their group
 
  automatically set to the group of the parent directory.
 
* Each WSGI website gets a dedicated virtual environment, stored in the
 
  sub-directory ``virtualenv`` of the website directory, for example
 
  ``/var/www/wiki.example.com/virtualenv``.
 
* Static files are served from sub-directory ``htdocs`` in the website
 
  directory, for example ``/var/www/wiki.example.com/htdocs/``.
 
* The base directory where your website/application code should be at is
 
  expected to be in sub-directory ``code`` in the website directory, for example
 
  ``/var/www/wiki.example.com/code/``.
 
* Combination of admin user membership in application group, ``SGID``
 
  permission, and the way ownership of sub-directories is set-up usually means
 
  that the administrator will be capable of managing application files, and
 
  application can be granted write permissions to a *minimum* of necessary
 
  files.
 

	
 
  .. warning::
 
     Just keep in mind that some file-management commands, like ``mv``, do *not*
 
     respect the ``SGID`` bit. In fact, I would recommend using ``cp`` when you
roles/wsgi_website/defaults/main.yml
Show inline comments
 
---
 

	
 
additional_nginx_config: {}
 
enforce_https: True
 
packages: []
 
rewrites: []
 
static_locations: []
 
use_paste: False
 
virtualenv_packages: []
 
environment_variables: {}
 
admin: "web-{{ fqdn | replace('.', '_') }}"
 
https_tls_certificate: "{{ lookup('file', tls_certificate_dir + '/' + fqdn + '_https.pem') }}"
 
https_tls_key: "{{ lookup('file', tls_private_key_dir + '/' + fqdn + '_https.key') }}"
 
gunicorn_version: "19.6.0"
 
futures_version: "3.0.5"
 
\ No newline at end of file
roles/wsgi_website/tasks/main.yml
Show inline comments
 
---
 

	
 
- set_fact:
 
    admin: "admin-{{ fqdn | replace('.', '_') }}"
 
    user: "web-{{ fqdn | replace('.', '_') }}"
 
    home: "/var/www/{{ fqdn }}"
 

	
 
- name: Create WSGI website group
 
  group: name="{{ user }}" gid="{{ uid | default(omit) }}" state=present
 

	
 
- name: Create WSGI website admin user
 
  user: name="{{ admin }}" uid="{{ admin_uid | default(omit) }}" group="{{ user }}"
 
        shell=/bin/bash createhome=yes home="{{ home }}" state=present
 

	
 
- name: Set-up directory for storing user profile configuration files
 
  file: path="{{ home }}/.profile.d" state=directory
 
        owner="{{ admin }}" group="{{ user }}" mode=750
 

	
 
- name: Deploy profile configuration file for auto-activating the virtual environment
 
  copy: src="profile_virtualenv.sh" dest="{{ home }}/.profile.d/virtualenv.sh"
 
        owner="root" group="{{ user }}" mode="640"
 

	
 
- name: Deploy profile configuration file for setting environment variables
 
  template: src="environment.sh.j2" dest="{{ home }}/.profile.d/environment.sh"
 
            owner="root" group="{{ user }}" mode=640
 

	
 
- name: Create WSGI website user
 
  user: name="{{ user }}" uid="{{ uid | default(omit) }}" group="{{ user }}" comment="umask=0007"
 
        system=yes createhome=no state=present
 

	
 
- name: Add nginx user to website group
 
  user: name="www-data" groups="{{ user }}" append="yes"
 
  notify:
 
    - Restart nginx
 

	
 
- name: Install extra packages for website
 
  apt: name="{{ item }}" state=present
 
  with_items: "{{ packages }}"
 
  notify:
 
    - "Restart website {{ fqdn }}"
 

	
 
- name: Set-up MariaDB mysql_config symbolic link for compatibility (workaround for Debian bug 766996)
 
  file: src="/usr/bin/mariadb_config" dest="/usr/bin/mysql_config" state=link
 
  when: "'libmariadb-client-lgpl-dev-compat' in packages"
 

	
 
- name: Create directory for storing the Python virtual environment
 
  file: path="{{ home }}/virtualenv" state=directory
 
        owner="{{ admin }}" group="{{ user }}" mode="2750"
 

	
 
- name: Create Python virtual environment
roles/wsgi_website/templates/environment.sh.j2
Show inline comments
 
new file 100644
 
{% for var, val in environment_variables.iteritems() %}
 
export {{ var }}='{{ val }}'
 
{% endfor %}
roles/wsgi_website/templates/systemd_wsgi_website.service.j2
Show inline comments
 
[Unit]
 
Description=Website {{ fqdn }}
 
Requires={{ fqdn }}.socket
 
After=network.target
 

	
 
[Service]
 
User={{ user }}
 
Group={{ user }}
 
WorkingDirectory={{ home }}/code
 
ExecStart={{ home }}/virtualenv/bin/gunicorn --bind unix:/run/wsgi/{{ fqdn }}.sock {% if use_paste %}--paste {{home}}/code/{{ wsgi_application }}{% else %}{{ wsgi_application }}{% endif %}
 

	
 
{% for var, val in environment_variables.iteritems() %}
 
Environment="{{ var }}={{ val }}"
 
{% endfor %}
 

	
 
ExecReload=/bin/kill -s HUP $MAINPID
 
ExecStop=/bin/kill -s TERM $MAINPID
 
PrivateTmp=true
 
UMask=0007
 

	
 
[Install]
 
WantedBy=multi-user.target
 
\ No newline at end of file
 
WantedBy=multi-user.target
testsite/playbooks/roles/wsgihello/files/hello.wsgi
Show inline comments
 
#!/usr/bin/env python
 

	
 
import os
 

	
 
def application(environ, start_response):
 
    status = '200 OK'
 
    output = 'Hello, world one!'
 
    output = 'Hello, world one! I am website %s' % os.environ.get("WEBSITE_NAME", "that nobody set a name for :(")
 

	
 
    response_headers = [('Content-type', 'text/plain'),
 
                        ('Content-Length', str(len(output)))]
 
    start_response(status, response_headers)
 

	
 
    return [output]
testsite/playbooks/roles/wsgihello/meta/main.yml
Show inline comments
 
---
 

	
 
dependencies:
 
  - role: wsgi_website
 
    fqdn: wsgi.{{ testsite_domain }}
 
    admin_uid: 3001
 
    uid: 2001
 
    wsgi_application: wsgi:application
 
    static_locations:
 
      - /static/
 
    https_tls_key: "{{ lookup('file', inventory_dir + '/tls/wsgi.' + testsite_domain + '_https.key') }}"
 
    https_tls_certificate: "{{ lookup('file', inventory_dir + '/tls/wsgi.' + testsite_domain + '_https.pem') }}"
 
    environment_variables:
 
      WEBSITE_NAME: "Majic Ansible Roles Test Site"
 
  - role: database
 
    db_name: wsgi_{{ testsite_domain_underscores }}
 
    db_password: wsgi_{{ testsite_domain_underscores }}
 
\ No newline at end of file
0 comments (0 inline, 0 general)