.. _installation_iis:

=====================================================================

Installing Kallithea on Microsoft Internet Information Services (IIS)

=====================================================================

The following is documented using IIS 7/8 terminology. There should be nothing

preventing you from applying this on IIS 6 well.

.. note::

    For the best security, it is strongly recommended to only host the site over

    a secure connection, e.g. using TLS.

Prerequisites

-------------

Apart from the normal requirements for Kallithea, it is also necessary to get an

ISAPI-WSGI bridge module, e.g. isapi-wsgi.

Installation

------------

The following assumes that your Kallithea is at ``c:\inetpub\kallithea``, and

will be served from the root of its own website. The changes to serve it in its

own virtual folder will be noted where appropriate.

Application pool

^^^^^^^^^^^^^^^^

Make sure that there is a unique application pool for the Kallithea application

with an identity that has read access to the Kallithea distribution.

The application pool does not need to be able to run any managed code. If you

are using a 32-bit Python installation, then you must enable 32-bit program in

the advanced settings for the application pool; otherwise Python will not be able

to run on the website and neither will Kallithea.

.. note::

    The application pool can be the same as an existing application pool,

    as long as the Kallithea requirements are met by the existing pool.

ISAPI handler

^^^^^^^^^^^^^

The ISAPI handler can be generated using::

This will generate a ``dispatch.py`` file in the current directory that contains

the necessary components to finalize an installation into IIS. Once this file

has been generated, it is necessary to run the following command due to the way

that ISAPI-WSGI is made::

    python2 dispatch.py install

This accomplishes two things: generating an ISAPI compliant DLL file,

``_dispatch.dll``, and installing a script map handler into IIS for the

The ISAPI handler is registered to all file extensions, so it will automatically

the ISAPI handler, it will start a thread pool managed wrapper around the paster

middleware WSGI handler that Kallithea runs within and each HTTP request to the

site will be processed through this logic henceforth.

Authentication with Kallithea using IIS authentication modules

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The recommended way to handle authentication with Kallithea using IIS is to let

IIS handle all the authentication and just pass it to Kallithea.

To move responsibility into IIS from Kallithea, we need to configure Kallithea

to let external systems handle authentication and then let Kallithea create the

user automatically. To do this, access the administration's authentication page

and enable the ``kallithea.lib.auth_modules.auth_container`` plugin. Once it is

added, enable it with the ``REMOTE_USER`` header and check *Clean username*.

Finally, save the changes on this page.

Switch to the administration's permissions page and disable anonymous access,

otherwise Kallithea will not attempt to use the authenticated user name. By

default, Kallithea will populate the list of users lazily as they log in. Either

disable external auth account activation and ensure that you pre-populate the

user database with an external tool, or set it to *Automatic activation of

external account*. Finally, save the changes.

The last necessary step is to enable the relevant authentication in IIS, e.g.

Windows authentication.

Troubleshooting

---------------

Typically, any issues in this setup will either be entirely in IIS or entirely

in Kallithea (or Kallithea's WSGI/paster middleware). Consequently, two

different options for finding issues exist: IIS' failed request tracking which

is great at finding issues until they exist inside Kallithea, at which point the

ISAPI-WSGI wrapper above uses ``win32traceutil``, which is part of ``pywin32``.

In order to dump output from WSGI using ``win32traceutil`` it is sufficient to

type the following in a console window::

    python2 -m win32traceutil

and any exceptions occurring in the WSGI layer and below (i.e. in the Kallithea

application itself) that are uncaught, will be printed here complete with stack

traces, making it a lot easier to identify issues.

repositories. If you choose a location which contains existing

repositories Kallithea will add all of the repositories at the chosen

location to its database.  (Note: make sure you specify the correct

path to the root).

.. note:: the given path for Mercurial_ repositories **must** be write

          accessible for the application. It's very important since

          the Kallithea web interface will work without write access,

          but when trying to do a push it will fail with permission

          denied errors unless it has write access.

You are now ready to use Kallithea. To run it simply execute::

    paster serve my.ini

- This command runs the Kallithea server. The web app should be available at

  http://127.0.0.1:5000. The IP address and port is configurable via the

  configuration file created in the previous step.

- Log in to Kallithea using the admin account created when running ``setup-db``.

- The default permissions on each repository is read, and the owner is admin.

  Remember to update these if needed.

- In the admin panel you can toggle LDAP, anonymous, and permissions

  settings, as well as edit more advanced options on users and

  repositories.

Using Kallithea with SSH

------------------------

Kallithea currently only hosts repositories using http and https. (The addition

of ssh hosting is a planned future feature.) However you can easily use ssh in

parallel with Kallithea. (Repository access via ssh is a standard "out of

the box" feature of Mercurial_ and you can use this to access any of the

repositories that Kallithea is hosting. See PublishingRepositories_)

Kallithea repository structures are kept in directories with the same name

as the project. When using repository groups, each group is a subdirectory.

This allows you to easily use ssh for accessing repositories.

In order to use ssh you need to make sure that your web server and the users'

login accounts have the correct permissions set on the appropriate directories.

.. note:: These permissions are independent of any permissions you

          have set up using the Kallithea web interface.

If your main directory (the same as set in Kallithea settings) is for

example set to ``/srv/repos`` and the repository you are using is

named ``kallithea``, then to clone via ssh you should run::

    hg clone ssh://user@kallithea.example.com/srv/repos/kallithea

Using other external tools such as mercurial-server_ or using ssh key-based

authentication is fully supported.

.. note:: In an advanced setup, in order for your ssh access to use

          the same permissions as set up via the Kallithea web

          interface, you can create an authentication hook to connect

          to the Kallithea db and run check functions for permissions

          against that.

Setting up Whoosh full text search

----------------------------------

Kallithea provides full text search of repositories using `Whoosh`__.

.. __: https://pythonhosted.org/Whoosh/

For an incremental index build, run::

    paster make-index my.ini

For a full index rebuild, run::

    paster make-index my.ini -f

The ``--repo-location`` option allows the location of the repositories to be overriden;

usually, the location is retrieved from the Kallithea database.

The ``--index-only`` option can be used to limit the indexed repositories to a comma-separated list::

    paster make-index my.ini --index-only=vcs,kallithea

To keep your index up-to-date it is necessary to do periodic index builds;

for this, it is recommended to use a crontab entry. Example::

    0  3  *  *  *  /path/to/virtualenv/bin/paster make-index /path/to/kallithea/my.ini

When using incremental mode (the default), Whoosh will check the last

modification date of each file and add it to be reindexed if a newer file is

available. The indexing daemon checks for any removed files and removes them

from index.

If you want to rebuild the index from scratch, you can use the ``-f`` flag as above,

or in the admin panel you can check the "build from scratch" checkbox.

Setting up LDAP support

-----------------------

Kallithea supports LDAP authentication. In order

to use LDAP, you have to install the python-ldap_ package. This package is

available via PyPI, so you can install it by running::

    pip install python-ldap

.. note:: ``python-ldap`` requires some libraries to be installed on

          your system, so before installing it check that you have at

          least the ``openldap`` and ``sasl`` libraries.

Choose *Admin > Authentication*, click the ``kallithea.lib.auth_modules.auth_ldap`` button

and then *Save*, to enable the LDAP plugin and configure its settings.

Here's a typical LDAP setup::

 Connection settings

 Enable LDAP          = checked

 Host                 = host.example.com

 Port                 = 389

 Account              = <account>

 Password             = <password>

 Connection Security  = LDAPS connection

 Certificate Checks   = DEMAND

 Search settings

 Base DN              = CN=users,DC=host,DC=example,DC=org

 LDAP Filter          = (&(objectClass=user)(!(objectClass=computer)))

 LDAP Search Scope    = SUBTREE

 Attribute mappings

 Login Attribute      = uid

 First Name Attribute = firstName

 Last Name Attribute  = lastName

 Email Attribute      = mail

If your user groups are placed in an Organisation Unit (OU) structure, the Search Settings configuration differs::

 Search settings

 Base DN              = DC=host,DC=example,DC=org

 LDAP Filter          = (&(memberOf=CN=your user group,OU=subunit,OU=unit,DC=host,DC=example,DC=org)(objectClass=user))

 LDAP Search Scope    = SUBTREE

.. _enable_ldap:

Enable LDAP : required

    Whether to use LDAP for authenticating users.

.. _ldap_host:

Host : required

    LDAP server hostname or IP address. Can be also a comma separated

    list of servers to support LDAP fail-over.

.. _Port:

Port : required

    389 for un-encrypted LDAP, 636 for SSL-encrypted LDAP.

.. _ldap_account:

Account : optional

    Only required if the LDAP server does not allow anonymous browsing of

    records.  This should be a special account for record browsing.  This

    will require `LDAP Password`_ below.

.. _LDAP Password:

Password : optional

    Only required if the LDAP server does not allow anonymous browsing of

    records.

.. _Enable LDAPS:

Connection Security : required

    Defines the connection to LDAP server

    No encryption

        Plain non encrypted connection

    LDAPS connection

        Enable LDAPS connections. It will likely require `Port`_ to be set to

        a different value (standard LDAPS port is 636). When LDAPS is enabled

        then `Certificate Checks`_ is required.

    START_TLS on LDAP connection

        START TLS connection

.. _Certificate Checks:

Certificate Checks : optional

    How SSL certificates verification is handled -- this is only useful when

    `Enable LDAPS`_ is enabled.  Only DEMAND or HARD offer full SSL security

    proxy_set_header            Host $host;

    ## needed for container auth

    #proxy_set_header            REMOTE_USER $remote_user;

    #proxy_set_header            X-Forwarded-User $remote_user;

    proxy_set_header            X-Url-Scheme $scheme;

    proxy_set_header            X-Host $http_host;

    proxy_set_header            X-Real-IP $remote_addr;

    proxy_set_header            X-Forwarded-For $proxy_add_x_forwarded_for;

    proxy_set_header            Proxy-host $proxy_host;

    proxy_buffering             off;

    proxy_connect_timeout       7200;

    proxy_send_timeout          7200;

    proxy_read_timeout          7200;

    proxy_buffers               8 32k;

    client_max_body_size        1024m;

    client_body_buffer_size     128k;

    large_client_header_buffers 8 64k;

Apache virtual host reverse proxy example

-----------------------------------------

Here is a sample configuration file for Apache using proxy:

.. code-block:: apache

    <VirtualHost *:80>

            ServerName kallithea.example.com

            <Proxy *>

              # For Apache 2.4 and later:

              Require all granted

              # For Apache 2.2 and earlier, instead use:

              # Order allow,deny

              # Allow from all

            </Proxy>

            #important !

            #Directive to properly generate url (clone url) for pylons

            ProxyPreserveHost On

            #kallithea instance

            ProxyPass / http://127.0.0.1:5000/

            ProxyPassReverse / http://127.0.0.1:5000/

            #to enable https use line below

            #SetEnvIf X-Url-Scheme https HTTPS=1

    </VirtualHost>

Additional tutorial

http://pylonsbook.com/en/1.1/deployment.html#using-apache-to-proxy-requests-to-pylons

Apache as subdirectory

----------------------

Apache subdirectory part:

.. code-block:: apache

    <Location /<someprefix> >

      ProxyPass http://127.0.0.1:5000/<someprefix>

      ProxyPassReverse http://127.0.0.1:5000/<someprefix>

      SetEnvIf X-Url-Scheme https HTTPS=1

    </Location>

Besides the regular apache setup you will need to add the following line

into ``[app:main]`` section of your .ini file::

    filter-with = proxy-prefix

Add the following at the end of the .ini file::

    [filter:proxy-prefix]

    use = egg:PasteDeploy#prefix

    prefix = /<someprefix>

then change ``<someprefix>`` into your chosen prefix

Apache with mod_wsgi

--------------------

Alternatively, Kallithea can be set up with Apache under mod_wsgi. For

that, you'll need to:

- Install mod_wsgi. If using a Debian-based distro, you can install

  the package libapache2-mod-wsgi::

    aptitude install libapache2-mod-wsgi

- Enable mod_wsgi::

    a2enmod wsgi

- Create a wsgi dispatch script, like the one below. Make sure you

  check that the paths correctly point to where you installed Kallithea

  and its Python Virtual Environment.

- Enable the ``WSGIScriptAlias`` directive for the WSGI dispatch script,

  as in the following example. Once again, check the paths are

  correctly specified.

Here is a sample excerpt from an Apache Virtual Host configuration file:

.. code-block:: apache

    WSGIDaemonProcess kallithea \

    WSGIScriptAlias / /srv/kallithea/dispatch.wsgi

    WSGIPassAuthorization On

Or if using a dispatcher WSGI script with proper virtualenv activation:

.. code-block:: apache

    WSGIScriptAlias / /srv/kallithea/dispatch.wsgi

    WSGIPassAuthorization On

Example WSGI dispatch script:

.. code-block:: python

    import os

    os.environ["HGENCODING"] = "UTF-8"

    os.environ['PYTHON_EGG_CACHE'] = '/srv/kallithea/.egg-cache'

    # sometimes it's needed to set the curent dir

    os.chdir('/srv/kallithea/')

    import site

    site.addsitedir("/srv/kallithea/venv/lib/python2.7/site-packages")

    from paste.script.util.logging_config import fileConfig

Or using proper virtualenv activation:

.. code-block:: python

    activate_this = '/srv/kallithea/venv/bin/activate_this.py'

    execfile(activate_this, dict(__file__=activate_this))

    import os

    os.environ['HOME'] = '/srv/kallithea'

    ini = '/srv/kallithea/kallithea.ini'

    from paste.script.util.logging_config import fileConfig

    fileConfig(ini)

    from paste.deploy import loadapp

    application = loadapp('config:' + ini)

Other configuration files

-------------------------

A number of `example init.d scripts`__ can be found in

the ``init.d`` directory of the Kallithea source.

.. __: https://kallithea-scm.org/repos/kallithea/files/tip/init.d/ .

.. _virtualenv: http://pypi.python.org/pypi/virtualenv

.. _python: http://www.python.org/

.. _Mercurial: http://mercurial.selenic.com/

.. _Celery: http://celeryproject.org/

.. _Celery documentation: http://docs.celeryproject.org/en/latest/getting-started/index.html

.. _RabbitMQ: http://www.rabbitmq.com/

.. _Redis: http://redis.io/

.. _python-ldap: http://www.python-ldap.org/

.. _mercurial-server: http://www.lshift.net/mercurial-server.html

.. _PublishingRepositories: http://mercurial.selenic.com/wiki/PublishingRepositories

import inspect

import logging

import types

import traceback

import time

from paste.response import replace_header

from pylons.controllers import WSGIController

from webob.exc import HTTPError

from kallithea.model.db import User

from kallithea.model import meta

from kallithea.lib.compat import izip_longest, json

from kallithea.lib.auth import AuthUser

from kallithea.lib.base import _get_ip_addr as _get_ip, _get_access_path

from kallithea.lib.utils2 import safe_unicode, safe_str

log = logging.getLogger('JSONRPC')

class JSONRPCError(BaseException):

    def __init__(self, message):

        self.message = message

        super(JSONRPCError, self).__init__()

    def __str__(self):

        return safe_str(self.message)

def jsonrpc_error(message, retid=None, code=None):

    """

    Generate a Response object with a JSON-RPC error body

    :param code:

    :param retid:

    :param message:

    """

    from pylons.controllers.util import Response

    return Response(

        body=json.dumps(dict(id=retid, result=None, error=message)),

        status=code,

        content_type='application/json'

    )

class JSONRPCController(WSGIController):

    """

     A WSGI-speaking JSON-RPC controller class

     See the specification:

     <http://json-rpc.org/wiki/specification>`.

     Valid controller return values should be json-serializable objects.

     Sub-classes should catch their exceptions and raise JSONRPCError

     if they want to pass meaningful errors to the client.

     """

    def _get_ip_addr(self, environ):

        return _get_ip(environ)

    def _get_method_args(self):

        """

        Return `self._rpc_args` to dispatched controller method

        chosen by __call__

        """

        return self._rpc_args

    def __call__(self, environ, start_response):

        """

        Parse the request body as JSON, look up the method on the

        controller and if it exists, dispatch to it.

        """

        try:

            return self._handle_request(environ, start_response)

        finally:

            meta.Session.remove()

    def _handle_request(self, environ, start_response):

        start = time.time()

        ip_addr = self.ip_addr = self._get_ip_addr(environ)

        self._req_id = None

        if 'CONTENT_LENGTH' not in environ:

            log.debug("No Content-Length")

            return jsonrpc_error(retid=self._req_id,

                                 message="No Content-Length in request")

        else:

            length = environ['CONTENT_LENGTH'] or 0

            length = int(environ['CONTENT_LENGTH'])

            log.debug('Content-Length: %s', length)

        if length == 0:

            return jsonrpc_error(retid=self._req_id,

                                 message="Content-Length is 0")

        raw_body = environ['wsgi.input'].read(length)

        try:

            json_body = json.loads(raw_body)

        except ValueError as e:

            # catch JSON errors Here

            return jsonrpc_error(retid=self._req_id,

                                 message="JSON parse error ERR:%s RAW:%r"

                                 % (e, raw_body))

        # check AUTH based on API key

        try:

            self._req_api_key = json_body['api_key']

            self._req_id = json_body['id']

            self._req_method = json_body['method']

            self._request_params = json_body['args']

            if not isinstance(self._request_params, dict):

                self._request_params = {}

            log.debug(

                'method: %s, params: %s', self._req_method,

                                            self._request_params

            )

        except KeyError as e:

            return jsonrpc_error(retid=self._req_id,

                                 message='Incorrect JSON query missing %s' % e)

        # check if we can find this session using api_key

        try:

            u = User.get_by_api_key(self._req_api_key)

            if u is None:

                return jsonrpc_error(retid=self._req_id,

                                     message='Invalid API key')

            auth_u = AuthUser(dbuser=u)

            if not AuthUser.check_ip_allowed(auth_u, ip_addr):

                return jsonrpc_error(retid=self._req_id,

                        message='request from IP:%s not allowed' % (ip_addr,))

            else:

                log.info('Access for IP:%s allowed', ip_addr)

        except Exception as e:

            return jsonrpc_error(retid=self._req_id,

                                 message='Invalid API key')

        self._error = None

        try:

            self._func = self._find_method()

        except AttributeError as e:

            return jsonrpc_error(retid=self._req_id,

                                 message=str(e))

        # now that we have a method, add self._req_params to

        # self.kargs and dispatch control to WGIController

        argspec = inspect.getargspec(self._func)

        arglist = argspec[0][1:]

        defaults = map(type, argspec[3] or [])

        default_empty = types.NotImplementedType

        # kw arguments required by this method

        func_kwargs = dict(izip_longest(reversed(arglist), reversed(defaults),

                                        fillvalue=default_empty))

        # this is little trick to inject logged in user for

        # perms decorators to work they expect the controller class to have

        # authuser attribute set

        self.authuser = auth_u

        # This attribute will need to be first param of a method that uses

        # api_key, which is translated to instance of user at that name

        USER_SESSION_ATTR = 'apiuser'

        if USER_SESSION_ATTR not in arglist:

            return jsonrpc_error(

                retid=self._req_id,

                message='This method [%s] does not support '

                         'authentication (missing %s param)' % (

                                    self._func.__name__, USER_SESSION_ATTR)

            )

        # get our arglist and check if we provided them as args

        for arg, default in func_kwargs.iteritems():

            if arg == USER_SESSION_ATTR:

                # USER_SESSION_ATTR is something translated from API key and

                # this is checked before so we don't need validate it

                continue

            # skip the required param check if it's default value is

            # NotImplementedType (default_empty)

            if default == default_empty and arg not in self._request_params:

                return jsonrpc_error(

                    retid=self._req_id,

                    message=(

                        'Missing non optional `%s` arg in JSON DATA' % arg

                    )

                )

        self._rpc_args = {USER_SESSION_ATTR: u}

        self._rpc_args.update(self._request_params)

        self._rpc_args['action'] = self._req_method

        self._rpc_args['environ'] = environ

        self._rpc_args['start_response'] = start_response

        status = []

        headers = []

        exc_info = []

        def change_content(new_status, new_headers, new_exc_info=None):

            status.append(new_status)

            headers.extend(new_headers)

            exc_info.append(new_exc_info)

        output = WSGIController.__call__(self, environ, change_content)

        replace_header(headers, 'Content-Type', 'application/json')

        start_response(status[0], headers, exc_info[0])

        log.info('IP: %s Request to %s time: %.3fs' % (

            self._get_ip_addr(environ),

            safe_unicode(_get_access_path(environ)), time.time() - start)

        )

        return output

    def _dispatch_call(self):

        """

        Implement dispatch interface specified by WSGIController

        """

        raw_response = ''

        try:

            raw_response = self._inspect_call(self._func)

            if isinstance(raw_response, HTTPError):

                self._error = str(raw_response)

        except JSONRPCError as e:

            self._error = safe_str(e)

        except Exception as e:

            log.error('Encountered unhandled exception: %s',

                      traceback.format_exc(),)

            json_exc = JSONRPCError('Internal server error')

            self._error = safe_str(json_exc)

        if self._error is not None:

            raw_response = None

        response = dict(id=self._req_id, result=raw_response, error=self._error)

        try:

            return json.dumps(response)

        except TypeError as e:

            log.error('API FAILED. Error encoding response: %s', e)

            return json.dumps(

                dict(

                    id=self._req_id,

                    result=None,

                    error="Error encoding response"

                )

            )

    def _find_method(self):

        """

        Return method named by `self._req_method` in controller if able

        """

        log.debug('Trying to find JSON-RPC method: %s', self._req_method)

        if self._req_method.startswith('_'):

            raise AttributeError("Method not allowed")

        try:

            func = getattr(self, self._req_method, None)

        except UnicodeEncodeError:

            raise AttributeError("Problem decoding unicode in requested "

                                 "method name.")

        if isinstance(func, types.MethodType):

            return func

        else:

            raise AttributeError("No such method: %s" % (self._req_method,))