Changeset - 3352797ee517
[Not reviewed]
0 5 0
Branko Majic (branko) - 9 years ago 2016-11-22 22:25:09
branko@majic.rs
MAR-68: When enforcing TLS in web_server, wsgi_website, or php_website roles, send connecting clients proper HSTS policy via Strict-Transport-Security header.
5 files changed with 33 insertions and 3 deletions:
0 comments (0 inline, 0 general)
docs/rolereference.rst
Show inline comments
 
@@ -1089,97 +1089,99 @@ Here is an example configuration for setting-up the mail forwarder:
 
.. code-block:: yaml
 

	
 
  ---
 

	
 
  # All mails sent to local user root will be forwarded to external account as
 
  # well.
 
  local_mail_aliases:
 
    root: "root john.doe@example.com"
 

	
 
  smtp_relay_host: mail.example.com
 

	
 
  smtp_relay_truststore: /etc/ssl/certs/example_ca_chain.pem
 

	
 

	
 
Web Server
 
----------
 

	
 
The ``web_server`` role can be used for setting-up a web server on destination
 
machine.
 

	
 
The role is supposed to be very lightweight, providing a basis for deployment of
 
web applications.
 

	
 
The role implements the following:
 

	
 
* Installs and configures nginx with a single, default vhost with a small static
 
  index page.
 
* Deploys the HTTPS TLS private key and certificate (for default vhost).
 
* Hardens TLS configuration by allowing only TLSv1.2 and PFS ciphers.
 
* Configures firewall to allow incoming connections to the web server.
 
* Installs and configures virtualenv and virtualenvwrapper as a common base for
 
  Python apps.
 
* Installs and configures PHP FPM as a common base for PHP apps.
 

	
 

	
 
Role dependencies
 
~~~~~~~~~~~~~~~~~
 

	
 
Depends on the following roles:
 

	
 
* **common**
 

	
 

	
 
Parameters
 
~~~~~~~~~~
 

	
 
**default_enforce_https** (boolean, optional, ``True``)
 
  Specify if HTTPS should be enforced for the default virtual host or not. If
 
  enforced, clients connecting via plaintext will be redirected to HTTPS.
 
  enforced, clients connecting via plaintext will be redirected to HTTPS, and
 
  clients will be served with ``Strict-Transport-Security`` header with value of
 
  ``max-age=31536000; includeSubDomains``.
 

	
 
**default_https_tls_certificate** (string, optional, ``{{ lookup('file', tls_certificate_dir + '/' + ansible_fqdn + '_https.pem') }}``)
 
  X.509 certificate used for TLS for HTTPS service. The file will be stored in
 
  directory ``/etc/ssl/certs/`` under name ``{{ ansible_fqdn }}_https.pem``.
 

	
 
**default_https_tls_key** (string, optional, ``{{ lookup('file', tls_private_key_dir + '/' + ansible_fqdn + '_https.key') }}``)
 
  Private key used for TLS for HTTPS service. The file will be stored in
 
  directory ``/etc/ssl/private/`` under name ``{{ ansible_fqdn }}_https.key``.
 

	
 
**web_default_title** (string, optional, ``Welcome``)
 
  Title for the default web page shown to users (if no other vhosts were matched).
 

	
 
**web_default_message** (string, optional, ``You are attempting to access the web server using a wrong name or an IP address. Please check your URL.``)
 
  Message for the default web page shown to users (if no other vhosts were
 
  matched).
 

	
 

	
 
Examples
 
~~~~~~~~
 

	
 
Here is an example configuration for setting-up web server:
 

	
 
.. code-block:: yaml
 

	
 
  ---
 

	
 
  default_https_tls_key: "{{ lookup('file', inventory_dir + '/tls/web.example.com_https.key') }}"
 
  default_https_tls_certificate: "{{ lookup('file', inventory_dir + '/tls/web.example.com_https.pem') }}"
 

	
 
  web_default_title: "Welcome to Example Inc."
 
  web_default_message: "You are attempting to access the web server using a wrong name or an IP address. Please check your URL."
 

	
 

	
 
PHP Website
 
-----------
 

	
 
The ``php_website`` role can be used for setting-up a website powered by PHP on
 
destination machine.
 

	
 
This role is normally not supposed to be used directly, but should instead serve
 
as the basis for writing website-specific roles. Therefore the role is written
 
in quite generic way, allowing the integrator to write his/her own logic for
 
deploying the necessary PHP applications, while still reusing a common base and
 
reducing the workload.
 

	
 
The role implements the following:
 

	
 
* Creates a dedicated user/group for running the PHP scripts.
 
@@ -1214,97 +1216,99 @@ The role is implemented with the following layout/logic in mind:
 
  the website user (running the PHP scripts) without having to become root.
 
* All files placed in the website directory should be either created there
 
  directly, or copied to the directory in order to make sure the ``SGID`` gets
 
  honored. **Do not move the files, the permissions will not be set correctly.**
 
* Within the website directory, nginx/php5-fpm will expect to find the relevant
 
  files within the htdocs sub-directory (this can be symlink too).
 
* nginx communicates with PHP FPM over a dedicated Unix socket for each website.
 

	
 

	
 
Role dependencies
 
~~~~~~~~~~~~~~~~~
 

	
 
Depends on the following roles:
 

	
 
* **common**
 
* **web_server**
 

	
 

	
 
Parameters
 
~~~~~~~~~~
 

	
 
**additional_nginx_config** (list, optional, ``[]``)
 
  List providing additional Nginx configuration options to include. This can be
 
  useful for specifying things like error pages. Options are applied inside of a
 
  **server** context of Nginx configuration file.
 

	
 
  Each item is a dictionary with the following options describing the extra
 
  configuration option:
 

	
 
  **comment** (string, mandatory)
 
    Comment describing the configuration option.
 

	
 
  **value** (string, mandatory)
 
    Configuration option.
 

	
 
**admin_uid** (integer, optional, ``whatever OS picks``)
 
  UID of the dedicated website administrator user. The user will be member of
 
  website group.
 

	
 
**deny_files_regex** (list, optional, ``[]``)
 
  List of regular expressions for matching files/locations to which the web
 
  server should deny access. This is useful to block access to any sensitive
 
  files that should not be served directly by the web server. The format must be
 
  compatible with regular expressions used by ``nginx`` for ``location ~``
 
  syntax.
 

	
 
**enforce_https** (boolean, optional, ``True``)
 
  Specify if HTTPS should be enforced for the website or not. If enforced,
 
  clients connecting via plaintext will be redirected to HTTPS.
 
  clients connecting via plaintext will be redirected to HTTPS, and clients will
 
  be served with ``Strict-Transport-Security`` header with value of
 
  ``max-age=31536000; includeSubDomains``.
 

	
 
**fqdn** (string, mandatory)
 
  Fully-qualified domain name where the website is reachable. This value is used
 
  for calculating the user/group name for dedicated website user, as well as
 
  home directory of the website user (where data/code should be stored at).
 

	
 
**index** (string, optional, ``index.php``)
 
  Space-separated list of files which should be treated as index files by the
 
  web server. The web server will attempt opening these index files, in
 
  succession, until the first match, or until it runs out of matches, when a
 
  client requests an URI pointing to directory.
 

	
 
**https_tls_certificate** (string, optional, ``{{ lookup('file', tls_certificate_dir + '/' + fqdn + '_https.pem') }}``)
 
  X.509 certificate used for TLS for HTTPS service. The file will be stored in
 
  directory ``/etc/ssl/certs/`` under name ``{{ fqdn }}_https.pem``.
 

	
 
**https_tls_key** (string, optional, ``{{ lookup('file', tls_private_key_dir + '/' + fqdn + '_https.key') }}``)
 
  Private key used for TLS for HTTPS service. The file will be stored in
 
  directory ``/etc/ssl/private/`` under name ``{{ fqdn }}_https.key``.
 

	
 
**php_file_regex** (string, optional, ``\.php$``)
 
  Regular expression used for determining which file should be interepted via
 
  PHP.
 

	
 
**php_rewrite_urls** (list, optional, ``[]``)
 
  A list of rewrite rules that are applied to incoming requests. These rewrite
 
  rules are specifically targetted at prettying-up the URLs used by the PHP
 
  scripts. Each element of the list should be a string value compatible with the
 
  format of ``nginx`` option ``rewrite``. The keyword ``rewrite`` itself should
 
  be omitted, as well as trailing semi-colon (``;``).
 

	
 
**rewrites** (list, optional, ``[]``)
 
  A list of rewrite rules that are applied to incoming requests. Each element of
 
  the list should be a string value compatible with the format of ``nginx``
 
  option ``rewrite``. The keyword ``rewrite`` itself should be omitted, as well
 
  as trailing semi-colon (``;``).
 

	
 
**packages** (list, optional, ``[]``)
 
  A list of additional packages to install for this particular PHP
 
  appliction. This is usually going to be different PHP extensions.
 

	
 
**uid** (integer, optional, ``whatever OS picks``)
 
  UID/GID (they are set-up to be the same) of the dedicated website
 
  user/group.
 

	
 

	
 
Examples
 
~~~~~~~~
 
@@ -1407,97 +1411,99 @@ The role is implemented with the following layout/logic in mind:
 
  the website user (running the WSGI application) without having to become root.
 
* All files placed in the website directory should be either created there
 
  directly, or copied to the directory in order to make sure the ``SGID`` gets
 
  honored. **Do not move the files, the permissions will not be set correctly.**
 
* Within the website directory, Python virtual environment can be found within
 
  the ``virtualenv`` sub-directory. Switching to administrator user via login
 
  shell will automatically activate the virtual environment.
 
* Within the website directory, nginx will expect to find the static files
 
  within the ``htdocs`` sub-directory (this can be symlink too). Locations/aliases
 
  can be configured for static file serving.
 
* Within the website directory, systemd service will expect to find the website
 
  code within the ``code`` sub-directory (this can be symlink too).
 
* nginx communicates with WSGI server over a dedicated Unix socket for each
 
  website.
 

	
 

	
 
Role dependencies
 
~~~~~~~~~~~~~~~~~
 

	
 
Depends on the following roles:
 

	
 
* **common**
 
* **web_server**
 

	
 

	
 
Parameters
 
~~~~~~~~~~
 

	
 
**additional_nginx_config** (list, optional, ``[]``)
 
  List providing additional Nginx configuration options to include. This can be
 
  useful for specifying things like error pages. Options are applied inside of a
 
  **server** context of Nginx configuration file.
 

	
 
  Each item is a dictionary with the following options describing the extra
 
  configuration option:
 

	
 
  **comment** (string, mandatory)
 
    Comment describing the configuration option.
 

	
 
  **value** (string, mandatory)
 
    Configuration option.
 

	
 
**admin_uid** (integer, optional, ``whatever OS picks``)
 
  UID of the dedicated website administrator user. The user will be member of
 
  website group.
 

	
 
**enforce_https** (boolean, optional, ``True``)
 
  Specify if HTTPS should be enforced for the website or not. If enforced,
 
  clients connecting via plaintext will be redirected to HTTPS.
 
  clients connecting via plaintext will be redirected to HTTPS, and clients will
 
  be served with ``Strict-Transport-Security`` header with value of
 
  ``max-age=31536000; includeSubDomains``.
 

	
 
**fqdn** (string, mandatory)
 
  Fully-qualified domain name where the website is reachable. This value is used
 
  for calculating the user/group name for dedicated website user, as well as
 
  home directory of the website user (where data/code should be stored at).
 

	
 
**futures_version** (string, optional, ``3.0.5``)
 
  Version of ``futures`` package to deploy in virtual environment. Required by
 
  Gunicorn when using Python 2.7. Default version is tested with the test site.
 

	
 
**gunicorn_version** (string, optional, ``19.6.0``)
 
  Version of Gunicorn to deploy in virtual environment for running the WSGI
 
  application. Default version is tested with the test site.
 

	
 
**https_tls_certificate** (string, optional, ``{{ lookup('file', tls_certificate_dir + '/' + fqdn + '_https.pem') }}``)
 
  X.509 certificate used for TLS for HTTPS service. The file will be stored in
 
  directory ``/etc/ssl/certs/`` under name ``{{ fqdn }}_https.pem``.
 

	
 
**https_tls_key** (string, optional, ``{{ lookup('file', tls_private_key_dir + '/' + fqdn + '_https.key') }}``)
 
  Private key used for TLS for HTTPS service. The file will be stored in
 
  directory ``/etc/ssl/private/`` under name ``{{ fqdn }}_https.key``.
 

	
 
**packages** (list, optional, ``[]``)
 
  A list of additional packages to install for this particular WSGI
 
  website. This is usually going to be development libraries for building Python
 
  packages.
 

	
 
**rewrites** (list, optional, ``[]``)
 
  A list of rewrite rules that are applied to incoming requests. Each element of
 
  the list should be a string value compatible with the format of ``nginx``
 
  option ``rewrite``. The keyword ``rewrite`` itself should be omitted, as well
 
  as trailing semi-colon (``;``).
 

	
 
**static_locations** (list, optional, ``[]``)
 
  List of locations that should be treated as static-only, and not processed by
 
  the WSGI application at all. This is normally used for designating serving of
 
  static/media files by Nginx (for example, in case of Django projects for
 
  ``/static/`` and ``/media/``).
 

	
 
**uid** (integer, optional, ``whatever OS picks``)
 
  UID/GID (they are set-up to be the same) of the dedicated website
 
  user/group.
 

	
 
**use_paste** (boolean, optional, ``False``)
 
  Tell Gunicorn to assume that the passed-in ``wsgi_application`` value is a
 
  filename of a Python Paste ``ini`` file instead of WSGI application.
 

	
 
**virtuaelnv_packages** (list, optional, ``[]``)
docs/usage.rst
Show inline comments
 
@@ -1027,96 +1027,102 @@ role.
 

	
 
6. Apply the changes::
 

	
 
     workon mysite && ansible-playbook playbooks/site.yml
 

	
 
7. If no errors have been reported, at this point you should have two users
 
   capable of using the XMPP service - one with username
 
   ``john.doe@example.com`` and one with username ``jane.doe@example.com``. Same
 
   passwords are used as for when you were creating the two users for mail
 
   server. For testing you can turn to your favourite XMPP client (I don't know
 
   of any quick CLI-based tools to test the XMPP server functionality,
 
   unfortunately, but you could try using `mcabber <https://mcabber.com/>`_).
 

	
 

	
 
Taking a step back - preparing for web server
 
---------------------------------------------
 

	
 
Up until now the usage instructions have dealt almost exclusively with the
 
communications server. That is, we haven't done anything beyond the basic set-up
 
of the other servers.
 

	
 
Let us first define what we want to deploy on the web server. Here is the plan:
 

	
 
1. First off, we will set-up the web server. This will be necessary no matter
 
   what web application we decide to deploy later on.
 

	
 
2. Next, we will set-up a database server. Why? Well, most web applications
 
   need to use some sort of database to store all the data, so we might as well
 
   try to take that one out of the way.
 

	
 
3. With this basic deployment for a web server in place, we can move on to
 
   setting-up a couple of web applications. For the purpose of the usage
 
   instructions, we will deploy the following two:
 

	
 
   1. `The Bug Genie <http://thebuggenie.org/>`_ - an issue tracker. To keep
 
      things simple, we will not integrate it with our LDAP server (although
 
      this is supported and possible). Being written in PHP, this will
 
      demonstrate the role for PHP web application deployment.
 

	
 
   2. `Django Wiki <https://github.com/django-wiki/django-wiki>`_ - a wiki
 
      application written in Django. This will serve as a demo of how the WSGI
 
      role works.
 

	
 
It should be noted that the web application deployment roles are a bit more
 
complex - namely they are not meant to be used directly, but instead as a
 
dependency for a custom role. They do come with decent amount of batteries
 
included, and also play nice with the web server role.
 

	
 
As mentioned before, all roles will enforce TLS by default. The web server roles
 
will additionaly implement HSTS policy by sending connecting clients
 
``Strict-Transport-Security`` header with value set to ``max-age=31536000;
 
includeSubDomains`` (if you disable enforcement of TLS, the header will not be
 
sent).
 

	
 
With all the above noted, let us finally move on to the next step.
 

	
 

	
 
Setting-up the web server
 
-------------------------
 

	
 
Finally we are moving on to the web server deployment, and we shell start
 
with... Well, erm, web server deployment! To be more precise, we will set-up
 
Nginx.
 

	
 
1. Update the playbook for web server to include the web server role.
 

	
 

	
 
    :file:`~/mysite/playbooks/web.yml`
 
    ::
 

	
 
      ---
 
      - hosts: web
 
        remote_user: ansible
 
        become: yes
 
        roles:
 
          - common
 
          - ldap_client
 
          - mail_forwarder
 
          - web_server
 

	
 
2. You know the drill, role configuration comes up next. Actually... The web
 
   server role parameters are all optional, and they default to some ok-ish
 
   values. But let us spicen up things a bit nevertheless. No configuration has
 
   been deployed before for the web server, so we will be creating a new file.
 

	
 
   :file:`~/mysite/group_vars/web.yml`
 
   ::
 

	
 
      ---
 

	
 
      web_default_title: "Welcome to default page!"
 
      web_default_message: "Nothing to see here, move along..."
 

	
 
3. The only thing left now is to create the TLS private key/certificate pair
 
   that should be used for default virtual host.
 

	
 
   1. Create new template for ``certtool``:
 

	
 
      :file:`~/mysite/tls/www.example.com_https.cfg`
 
      ::
 

	
 
         organization = "Example Inc."
roles/php_website/templates/nginx_site.j2
Show inline comments
 
{% if enforce_https -%}
 
server {
 
    # HTTP (plaintext) configuration.
 
    listen 80;
 
    server_name {{ fqdn }};
 

	
 
    # Redirect plaintext connections to HTTPS
 
    return 301 https://$host$request_uri;
 
}
 

	
 
{% endif -%}
 
server {
 
    # Base settings.
 
    root {{ home }}/htdocs/;
 
    index {{ index }};
 
    server_name {{ fqdn }};
 
{% if not enforce_https %}
 

	
 
    # HTTP (plaintext) configuration.
 
    listen 80;
 

	
 
{% endif %}
 
    # HTTPS (TLS) configuration.
 
    listen 443 ssl;
 
    listen [::]:443 ssl;
 
    ssl_certificate_key /etc/ssl/private/{{ fqdn }}_https.key;
 
    ssl_certificate /etc/ssl/certs/{{ fqdn }}_https.pem;
 

	
 
{% if default_enforce_https -%}
 
    # Set-up HSTS header for preventing downgrades for users that visited the
 
    # site via HTTPS at least once.
 
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains";
 
{% endif -%}
 

	
 
    {% for config in additional_nginx_config -%}
 
    # {{ config.comment }}
 
    {{ config.value }}
 
    {% endfor -%}
 

	
 
    {% if rewrites -%}
 
    # Generic URL rewrites.
 
    {% for rewrite in rewrites -%}
 
    rewrite {{ rewrite }};
 
    {% endfor -%}
 
    {% endif %}
 

	
 
    {% if deny_files_regex -%}
 
    # Deny access to user-specified files.
 
    {% for regex in deny_files_regex -%}
 
    location ~ {{ regex }} {
 
        deny all;
 
    }
 
    {% endfor -%}
 
    {% endif %}
 

	
 
    # Interpret PHP files via FastCGI.
 
    location ~ {{ php_file_regex }} {
 
        include snippets/fastcgi-php.conf;
 
        fastcgi_pass unix:/var/run/php5-fpm/{{ fqdn }}.sock;
 
    }
 

	
 
    # Serve the files.
 
    location ~ /(.+) {
 
	try_files $uri $uri/{% if php_rewrite_urls %} @php_rewrite{% else %} =404{% endif %};
 
    }
 

	
 
    {% if php_rewrite_urls -%}
 
    # Apply URL rewrites.
 
    location @php_rewrite {
 
    {% for rewrite in php_rewrite_urls %}
 
    rewrite {{ rewrite }};
 
    {% endfor -%}
 
    }
 
    {% endif -%}
 

	
 
    access_log /var/log/nginx/{{ fqdn }}-access.log;
 
    error_log /var/log/nginx/{{ fqdn }}-error.log;
 
}
roles/web_server/templates/nginx-default.j2
Show inline comments
 
#
 
# Default server (vhost) configuration.
 
#
 
{% if default_enforce_https -%}
 
server {
 
    # HTTP (plaintext) configuration.
 
    listen 80 default_server;
 
    listen [::]:80 default_server;
 

	
 
    # Set server_name to something that won't be matched (for default server).
 
    server_name _;
 

	
 
    # Redirect plaintext connections to HTTPS
 
    return 301 https://$host$request_uri;
 
}
 

	
 
{% endif -%}
 
server {
 
{% if not default_enforce_https %}
 
    # HTTP (plaintext) configuration.
 
    listen 80 default_server;
 
    listen [::]:80 default_server;
 

	
 
{% endif %}
 
    # HTTPS (TLS) configuration.
 
    listen 443 ssl default_server;
 
    listen [::]:443 ssl default_server;
 
    ssl_certificate_key /etc/ssl/private/{{ ansible_fqdn }}_https.key;
 
    ssl_certificate /etc/ssl/certs/{{ ansible_fqdn }}_https.pem;
 

	
 
{% if default_enforce_https %}
 
    # Set-up HSTS header for preventing downgrades for users that visited the
 
    # site via HTTPS at least once.
 
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains";
 
{% endif %}
 

	
 
    # Set-up the serving of default page.
 
    root /var/www/default/;
 
    index index.html;
 

	
 
    # Set server_name to something that won't be matched (for default server).
 
    server_name _;
 

	
 
    location / {
 
        # Always point user to the same index page.
 
        try_files $uri /index.html;
 
    }
 
}
roles/wsgi_website/templates/nginx_site.j2
Show inline comments
 
{% if enforce_https -%}
 
server {
 
    # HTTP (plaintext) configuration.
 
    listen 80;
 
    server_name {{ fqdn }};
 

	
 
    # Redirect plaintext connections to HTTPS
 
    return 301 https://$host$request_uri;
 
}
 

	
 
{% endif -%}
 
server {
 
    # Base settings.
 
    root {{ home }}/htdocs/;
 
    server_name {{ fqdn }};
 
{% if not enforce_https %}
 

	
 
    # HTTP (plaintext) configuration.
 
    listen 80;
 

	
 
{% endif %}
 
    # HTTPS (TLS) configuration.
 
    listen 443 ssl;
 
    listen [::]:443 ssl;
 
    ssl_certificate_key /etc/ssl/private/{{ fqdn }}_https.key;
 
    ssl_certificate /etc/ssl/certs/{{ fqdn }}_https.pem;
 

	
 
{% if default_enforce_https -%}
 
    # Set-up HSTS header for preventing downgrades for users that visited the
 
    # site via HTTPS at least once.
 
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains";
 
{% endif -%}
 

	
 
    {% for config in additional_nginx_config -%}
 
    # {{ config.comment }}
 
    {{ config.value }}
 
    {% endfor -%}
 

	
 
    {% if rewrites -%}
 
    # Site rewrites.
 
    {% for rewrite in rewrites -%}
 
    rewrite {{ rewrite }};
 
    {% endfor -%}
 
    {% endif %}
 

	
 
    {% if static_locations -%}
 
    # Static locations
 
    {% for location in static_locations -%}
 
    location {{ location }} {
 
        try_files $uri $uri/ =404;
 
    }
 
    {% endfor -%}
 
    {% endif %}
 

	
 
    # Pass remaining requests to the WSGI server.
 
    location / {
 
        try_files $uri @proxy_to_app;
 
    }
 

	
 
    location @proxy_to_app {
 
        proxy_set_header X-Forwarded-Proto $scheme;
 
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 
        proxy_set_header Host $http_host;
 
        proxy_redirect off;
 

	
 
        proxy_pass http://unix:/run/wsgi/{{ fqdn }}.sock;
 
    }
 

	
 
    access_log /var/log/nginx/{{ fqdn }}-access.log;
 
    error_log /var/log/nginx/{{ fqdn }}-error.log;
 
}
0 comments (0 inline, 0 general)