clients access the application.

When these authentication methods are enabled in Kallithea, it uses the

username that the container/proxy (Apache or Nginx, etc.) provides and doesn't

perform the authentication itself. The authorization, however, is still done by

Kallithea according to its settings.

When a user logs in for the first time using these authentication methods,

a matching user account is created in Kallithea with default permissions. An

administrator can then modify it using Kallithea's admin interface.

It's also possible for an administrator to create accounts and configure their

permissions before the user logs in for the first time, using the :ref:`create-user` API.

Container-based authentication

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

In a container-based authentication setup, Kallithea reads the user name from

the ``REMOTE_USER`` server variable provided by the WSGI container.

After setting up your container (see `Apache with mod_wsgi`_), you'll need

to configure it to require authentication on the location configured for

Kallithea.

Proxy pass-through authentication

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

In a proxy pass-through authentication setup, Kallithea reads the user name

from the ``X-Forwarded-User`` request header, which should be configured to be

sent by the reverse-proxy server.

After setting up your proxy solution (see `Apache virtual host reverse proxy example`_,

`Apache as subdirectory`_ or `Nginx virtual host example`_), you'll need to

configure the authentication and add the username in a request header named

``X-Forwarded-User``.

For example, the following config section for Apache sets a subdirectory in a

reverse-proxy setup with basic auth:

.. code-block:: apache

    <Location /someprefix>

      ProxyPass http://127.0.0.1:5000/someprefix

      ProxyPassReverse http://127.0.0.1:5000/someprefix

      SetEnvIf X-Url-Scheme https HTTPS=1

      AuthType Basic

      AuthName "Kallithea authentication"

      AuthUserFile /srv/kallithea/.htpasswd

      Require valid-user

      RequestHeader unset X-Forwarded-User

      RewriteEngine On

      RewriteCond %{LA-U:REMOTE_USER} (.+)

      RewriteRule .* - [E=RU:%1]

      RequestHeader set X-Forwarded-User %{RU}e

    </Location>

Setting metadata in container/reverse-proxy

"""""""""""""""""""""""""""""""""""""""""""

When a new user account is created on the first login, Kallithea has no information about

the user's email and full name. So you can set some additional request headers like in the

example below. In this example the user is authenticated via Kerberos and an Apache

mod_python fixup handler is used to get the user information from a LDAP server. But you

could set the request headers however you want.

.. code-block:: apache

    <Location /someprefix>

      ProxyPass http://127.0.0.1:5000/someprefix

      ProxyPassReverse http://127.0.0.1:5000/someprefix

      SetEnvIf X-Url-Scheme https HTTPS=1

      AuthName "Kerberos Login"

      AuthType Kerberos

      Krb5Keytab /etc/apache2/http.keytab

      KrbMethodK5Passwd off

      KrbVerifyKDC on

      Require valid-user

      PythonFixupHandler ldapmetadata

      RequestHeader set X_REMOTE_USER %{X_REMOTE_USER}e

      RequestHeader set X_REMOTE_EMAIL %{X_REMOTE_EMAIL}e

      RequestHeader set X_REMOTE_FIRSTNAME %{X_REMOTE_FIRSTNAME}e

      RequestHeader set X_REMOTE_LASTNAME %{X_REMOTE_LASTNAME}e

    </Location>

.. code-block:: python

    from mod_python import apache

    import ldap

    LDAP_SERVER = "ldaps://server.mydomain.com:636"

    LDAP_USER = ""

    LDAP_PASS = ""

    LDAP_ROOT = "dc=mydomain,dc=com"

    LDAP_FILTER = "sAMAccountName=%s"

    LDAP_ATTR_LIST = ['sAMAccountName','givenname','sn','mail']

    def fixuphandler(req):

        if req.user is None:

            # no user to search for

            return apache.OK

        else:

            try:

                if('\\' in req.user):

                    username = req.user.split('\\')[1]

                elif('@' in req.user):

                    username = req.user.split('@')[0]

                else:

                    username = req.user

                l = ldap.initialize(LDAP_SERVER)

                l.simple_bind_s(LDAP_USER, LDAP_PASS)

                r = l.search_s(LDAP_ROOT, ldap.SCOPE_SUBTREE, LDAP_FILTER % username, attrlist=LDAP_ATTR_LIST)

                req.subprocess_env['X_REMOTE_USER'] = username

                req.subprocess_env['X_REMOTE_EMAIL'] = r[0][1]['mail'][0].lower()

                req.subprocess_env['X_REMOTE_FIRSTNAME'] = "%s" % r[0][1]['givenname'][0]

                req.subprocess_env['X_REMOTE_LASTNAME'] = "%s" % r[0][1]['sn'][0]

            except Exception, e:

                apache.log_error("error getting data from ldap %s" % str(e), apache.APLOG_ERR)

            return apache.OK

.. note::

   If you enable proxy pass-through authentication, make sure your server is

   only accessible through the proxy. Otherwise, any client would be able to

   forge the authentication header and could effectively become authenticated

   using any account of their liking.

Integration with issue trackers

-------------------------------

Kallithea provides a simple integration with issue trackers. It's possible

to define a regular expression that will match an issue ID in commit messages,

and have that replaced with a URL to the issue. To enable this simply

uncomment the following variables in the ini file::

    issue_pat = (?:^#|\s#)(\w+)

    issue_server_link = https://issues.example.com/{repo}/issue/{id}

    issue_prefix = #

``issue_pat`` is the regular expression describing which strings in

commit messages will be treated as issue references. A match group in

parentheses should be used to specify the actual issue id.

The default expression matches issues in the format ``#<number>``, e.g., ``#300``.

Matched issue references are replaced with the link specified in

``issue_server_link``. ``{id}`` is replaced with the issue ID, and

``{repo}`` with the repository name.  Since the # is stripped away,

``issue_prefix`` is prepended to the link text.  ``issue_prefix`` doesn't

necessarily need to be ``#``: if you set issue prefix to ``ISSUE-`` this will

generate a URL in the format:

.. code-block:: html

  <a href="https://issues.example.com/example_repo/issue/300">ISSUE-300</a>

If needed, more than one pattern can be specified by appending a unique suffix to

the variables. For example::

    issue_pat_wiki = (?:wiki-)(.+)

    issue_server_link_wiki = https://wiki.example.com/{id}

    issue_prefix_wiki = WIKI-

With these settings, wiki pages can be referenced as wiki-some-id, and every

such reference will be transformed into:

.. code-block:: html

  <a href="https://wiki.example.com/some-id">WIKI-some-id</a>

Hook management

---------------

Hooks can be managed in similar way to that used in ``.hgrc`` files.

To manage hooks, choose *Admin > Settings > Hooks*.

The built-in hooks cannot be modified, though they can be enabled or disabled in the *VCS* section.

To add another custom hook simply fill in the first textbox with

``<name>.<hook_type>`` and the second with the hook path. Example hooks

can be found in ``kallithea.lib.hooks``.

Changing default encoding

-------------------------

By default, Kallithea uses UTF-8 encoding.

This is configurable as ``default_encoding`` in the .ini file.

This affects many parts in Kallithea including user names, filenames, and

encoding of commit messages. In addition Kallithea can detect if the ``chardet``

library is installed. If ``chardet`` is detected Kallithea will fallback to it

when there are encode/decode errors.

Celery configuration

--------------------

Kallithea can use the distributed task queue system Celery_ to run tasks like

cloning repositories or sending emails.

Kallithea will in most setups work perfectly fine out of the box (without

Celery), executing all tasks in the web server process. Some tasks can however

take some time to run and it can be better to run such tasks asynchronously in

a separate process so the web server can focus on serving web requests.

For installation and configuration of Celery, see the `Celery documentation`_.

Note that Celery requires a message broker service like RabbitMQ_ (recommended)

or Redis_.

The use of Celery is configured in the Kallithea ini configuration file.

To enable it, simply set::

  use_celery = true

and add or change the ``celery.*`` and ``broker.*`` configuration variables.

Remember that the ini files use the format with '.' and not with '_' like

Celery. So for example setting `BROKER_HOST` in Celery means setting

`broker.host` in the configuration file.

To start the Celery process, run::

 gearbox celeryd -c <configfile.ini>

Extra options to the Celery worker can be passed after ``--`` - see ``-- -h``

for more info.

.. note::

   Make sure you run this command from the same virtualenv, and with the same

   user that Kallithea runs.

HTTPS support

-------------

Kallithea will by default generate URLs based on the WSGI environment.

Alternatively, you can use some special configuration settings to control

directly which scheme/protocol Kallithea will use when generating URLs:

- With ``https_fixup = true``, the scheme will be taken from the

  ``X-Url-Scheme``, ``X-Forwarded-Scheme`` or ``X-Forwarded-Proto`` HTTP header

  (default ``http``).

- With ``force_https = true`` the default will be ``https``.

- With ``use_htsts = true``, Kallithea will set ``Strict-Transport-Security`` when using https.

Nginx virtual host example

--------------------------

Sample config for Nginx using proxy:

.. code-block:: nginx

    upstream kallithea {

        server 127.0.0.1:5000;

        # add more instances for load balancing

        #server 127.0.0.1:5001;

        #server 127.0.0.1:5002;

    }

    ## gist alias

    server {

       listen          443;

       server_name     gist.example.com;

       access_log      /var/log/nginx/gist.access.log;

       error_log       /var/log/nginx/gist.error.log;

       ssl on;

       ssl_certificate     gist.your.kallithea.server.crt;

       ssl_certificate_key gist.your.kallithea.server.key;

       ssl_session_timeout 5m;

       ssl_protocols SSLv3 TLSv1;

       ssl_ciphers DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:EDH-RSA-DES-CBC3-SHA:AES256-SHA:DES-CBC3-SHA:AES128-SHA:RC4-SHA:RC4-MD5;

       ssl_prefer_server_ciphers on;

       rewrite ^/(.+)$ https://kallithea.example.com/_admin/gists/$1;

       rewrite (.*)    https://kallithea.example.com/_admin/gists;

    }

    server {

       listen          443;

       server_name     kallithea.example.com

       access_log      /var/log/nginx/kallithea.access.log;

       error_log       /var/log/nginx/kallithea.error.log;

       ssl on;

       ssl_certificate     your.kallithea.server.crt;

       ssl_certificate_key your.kallithea.server.key;

       ssl_session_timeout 5m;

       ssl_protocols SSLv3 TLSv1;

       ssl_ciphers DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:EDH-RSA-DES-CBC3-SHA:AES256-SHA:DES-CBC3-SHA:AES128-SHA:RC4-SHA:RC4-MD5;

       ssl_prefer_server_ciphers on;

       ## uncomment root directive if you want to serve static files by nginx

       ## requires static_files = false in .ini file

       #root /srv/kallithea/kallithea/kallithea/public;

       include         /etc/nginx/proxy.conf;

       location / {

            try_files $uri @kallithea;

       }

       location @kallithea {

            proxy_pass      http://127.0.0.1:5000;

       }

    }

Here's the proxy.conf. It's tuned so it will not timeout on long

pushes or large pushes::

    proxy_redirect              off;

    proxy_set_header            Host $host;

    ## needed for container auth

    #proxy_set_header            REMOTE_USER $remote_user;

    #proxy_set_header            X-Forwarded-User $remote_user;

    proxy_set_header            X-Url-Scheme $scheme;

    proxy_set_header            X-Host $http_host;

    proxy_set_header            X-Real-IP $remote_addr;

    proxy_set_header            X-Forwarded-For $proxy_add_x_forwarded_for;

    proxy_set_header            Proxy-host $proxy_host;

    proxy_buffering             off;

    proxy_connect_timeout       7200;

    proxy_send_timeout          7200;

    proxy_read_timeout          7200;

    proxy_buffers               8 32k;

    client_max_body_size        1024m;

    client_body_buffer_size     128k;

    large_client_header_buffers 8 64k;

Apache virtual host reverse proxy example

-----------------------------------------

Here is a sample configuration file for Apache using proxy:

.. code-block:: apache

    <VirtualHost *:80>

            ServerName kallithea.example.com

            <Proxy *>

              # For Apache 2.4 and later:

              Require all granted

              # For Apache 2.2 and earlier, instead use:

              # Order allow,deny

              # Allow from all

            </Proxy>

            #important !

            #Directive to properly generate url (clone url) for Kallithea

            ProxyPreserveHost On

            #kallithea instance

            ProxyPass / http://127.0.0.1:5000/

            ProxyPassReverse / http://127.0.0.1:5000/

            #to enable https use line below

            #SetEnvIf X-Url-Scheme https HTTPS=1

    </VirtualHost>

Additional tutorial

http://pylonsbook.com/en/1.1/deployment.html#using-apache-to-proxy-requests-to-pylons

Apache as subdirectory

----------------------

Apache subdirectory part:

.. code-block:: apache

      SetEnvIf X-Url-Scheme https HTTPS=1

    </Location>

Besides the regular apache setup you will need to add the following line

into ``[app:main]`` section of your .ini file::

    filter-with = proxy-prefix

Add the following at the end of the .ini file::

    [filter:proxy-prefix]

    use = egg:PasteDeploy#prefix

Apache with mod_wsgi

--------------------

Alternatively, Kallithea can be set up with Apache under mod_wsgi. For

that, you'll need to:

- Install mod_wsgi. If using a Debian-based distro, you can install

  the package libapache2-mod-wsgi::

    aptitude install libapache2-mod-wsgi

- Enable mod_wsgi::

    a2enmod wsgi

- Add global Apache configuration to tell mod_wsgi that Python only will be

  used in the WSGI processes and shouldn't be initialized in the Apache

  processes::

    WSGIRestrictEmbedded On

- Create a wsgi dispatch script, like the one below. Make sure you

  check that the paths correctly point to where you installed Kallithea

  and its Python Virtual Environment.

- Enable the ``WSGIScriptAlias`` directive for the WSGI dispatch script,

  as in the following example. Once again, check the paths are

  correctly specified.

Here is a sample excerpt from an Apache Virtual Host configuration file:

.. code-block:: apache

    WSGIDaemonProcess kallithea processes=5 threads=1 maximum-requests=100 \

        python-home=/srv/kallithea/venv

    WSGIProcessGroup kallithea

    WSGIScriptAlias / /srv/kallithea/dispatch.wsgi

    WSGIPassAuthorization On

Or if using a dispatcher WSGI script with proper virtualenv activation:

.. code-block:: apache

    WSGIDaemonProcess kallithea processes=5 threads=1 maximum-requests=100

    WSGIProcessGroup kallithea

    WSGIScriptAlias / /srv/kallithea/dispatch.wsgi

    WSGIPassAuthorization On

Apache will by default run as a special Apache user, on Linux systems

usually ``www-data`` or ``apache``. If you need to have the repositories

directory owned by a different user, use the user and group options to

WSGIDaemonProcess to set the name of the user and group.

Example WSGI dispatch script:

.. code-block:: python

    import os

    os.environ["HGENCODING"] = "UTF-8"

    os.environ['PYTHON_EGG_CACHE'] = '/srv/kallithea/.egg-cache'

    # sometimes it's needed to set the current dir

    os.chdir('/srv/kallithea/')

    import site

    site.addsitedir("/srv/kallithea/venv/lib/python2.7/site-packages")

    ini = '/srv/kallithea/my.ini'

    from paste.script.util.logging_config import fileConfig

    fileConfig(ini)

    from paste.deploy import loadapp

    application = loadapp('config:' + ini)

Or using proper virtualenv activation:

.. code-block:: python

    activate_this = '/srv/kallithea/venv/bin/activate_this.py'

    execfile(activate_this, dict(__file__=activate_this))

    import os

    os.environ['HOME'] = '/srv/kallithea'

    ini = '/srv/kallithea/kallithea.ini'

    from paste.script.util.logging_config import fileConfig

    fileConfig(ini)

    from paste.deploy import loadapp

    application = loadapp('config:' + ini)

Other configuration files

-------------------------

A number of `example init.d scripts`__ can be found in

the ``init.d`` directory of the Kallithea source.

.. __: https://kallithea-scm.org/repos/kallithea/files/tip/init.d/ .

.. _virtualenv: http://pypi.python.org/pypi/virtualenv

.. _python: http://www.python.org/

.. _Mercurial: https://www.mercurial-scm.org/

.. _Celery: http://celeryproject.org/

.. _Celery documentation: http://docs.celeryproject.org/en/latest/getting-started/index.html

.. _RabbitMQ: http://www.rabbitmq.com/

.. _Redis: http://redis.io/

.. _python-ldap: http://www.python-ldap.org/

.. _mercurial-server: http://www.lshift.net/mercurial-server.html

.. _PublishingRepositories: https://www.mercurial-scm.org/wiki/PublishingRepositories

.. _vcs_support:

===============================

Version control systems support

===============================

Kallithea supports Git and Mercurial repositories out-of-the-box.

For Git, you do need the ``git`` command line client installed on the server.

You can always disable Git or Mercurial support by editing the

file ``kallithea/__init__.py`` and commenting out the backend.

.. code-block:: python

   BACKENDS = {

       'hg': 'Mercurial repository',

       #'git': 'Git repository',

   }

Git support

-----------

Web server with chunked encoding

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Large Git pushes require an HTTP server with support for

chunked encoding for POST. The Python web servers waitress_ and

gunicorn_ (Linux only) can be used. By default, Kallithea uses

waitress_ for `gearbox serve` instead of the built-in `paste` WSGI

server.

The web server used by gearbox is controlled in the .ini file::

    use = egg:waitress#main

or::

    use = egg:gunicorn#main

Also make sure to comment out the following options::

    threadpool_workers =

    threadpool_max_requests =

    use_threadpool =

Mercurial support

-----------------

Working with Mercurial subrepositories

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

This section explains how to use Mercurial subrepositories_ in Kallithea.

Example usage::

    ## init a simple repo

    hg init mainrepo

    cd mainrepo

    echo "file" > file

    hg add file

    hg ci --message "initial file"

    # clone subrepo we want to add from Kallithea

    hg clone http://kallithea.local/subrepo

    ## specify URL to existing repo in Kallithea as subrepository path

    echo "subrepo = http://kallithea.local/subrepo" > .hgsub

    hg add .hgsub

    hg ci --message "added remote subrepo"

In the file list of a clone of ``mainrepo`` you will see a connected

subrepository at the revision it was cloned with. Clicking on the

subrepository link sends you to the proper repository in Kallithea.

Cloning ``mainrepo`` will also clone the attached subrepository.

Next we can edit the subrepository data, and push back to Kallithea. This will

update both repositories.

.. _importing:

Importing existing repositories

-------------------------------

There are two main methods to import repositories in Kallithea: via the web

interface or via the filesystem. If you have a large number of repositories to

import, importing them via the filesystem is more convenient.

Importing via web interface

^^^^^^^^^^^^^^^^^^^^^^^^^^^

For a small number of repositories, it may be easier to create the target

repositories through the Kallithea web interface, via *Admin > Repositories* or

via the *Add Repository* button on the entry page of the web interface.

Repositories can be nested in repository groups by first creating the group (via

*Admin > Repository Groups* or via the *Add Repository Group* button on the

entry page of the web interface) and then selecting the appropriate group when

adding the repository.

After creation of the (empty) repository, push the existing commits to the

*Clone URL* displayed on the repository summary page. For Git repositories,

first add the *Clone URL* as remote, then push the commits to that remote.  The

specific commands to execute are shown under the *Existing repository?* section

of the new repository's summary page.

A benefit of this method particular for Git repositories, is that the

Kallithea-specific Git hooks are installed automatically.  For Mercurial, no

hooks are required anyway.

Importing via the filesystem

^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The alternative method of importing repositories consists of creating the

repositories in the desired hierarchy on the filesystem and letting Kallithea

scan that location.

All repositories are stored in a central location on the filesystem. This

location is specified during installation (via ``setup-db``) and can be reviewed

at *Admin > Settings > VCS > Location of repositories*. Repository groups

(defined in *Admin > Repository Groups*) are represented by a directory in that

repository location. Repositories of the repository group are nested under that

directory.

To import a set of repositories and organize them in a certain repository group

structure, first place clones in the desired hierarchy at the configured

repository location.

These clones should be created without working directory. For Mercurial, this is

done with ``hg clone -U``, for Git with ``git clone --bare``.

When the repositories are added correctly on the filesystem:

* go to *Admin > Settings > Remap and Rescan* in the Kallithea web interface

* select the *Install Git hooks* checkbox when importing Git repositories

* click *Rescan Repositories*

This step will scan the filesystem and create the appropriate repository groups

and repositories in Kallithea.

*Note*: Once repository groups have been created this way, manage their access

permissions through the Kallithea web interface.

.. _waitress: http://pypi.python.org/pypi/waitress

.. _gunicorn: http://pypi.python.org/pypi/gunicorn

.. _subrepositories: http://mercurial.aragost.com/kick-start/en/subrepositories/

============

Translations

============

Translations are available on Hosted Weblate at the following URL:

    https://hosted.weblate.org/projects/kallithea/kallithea/

Registered users may contribute to the existing languages, or request a new

language translations.

Translating using Weblate

-------------------------

Weblate_ offers a simple and easy to use interface featuring glossary, machine

translation, suggestions based on similar translations in other projects,

automatic checks etc. Weblate imports the source code tree directly from

the version control system, and commits edits back from time to time.

When registering at Weblate, make sure you name and email address you prefer to

be used when your changes are committed. We can and probably will amend changesets

coming from Weblate, but having things right from the beginning makes things easier.

Weblate performs sanity checks all the time and tries to prevent you from ignoring

them. Most common mistakes are inconsistent punctuation, whitespaces, missing or extra

format parameters, untranslated strings copied into the translation. Please perform

necessary corrections when they're needed, or override the false positives.

Merging translations from Weblate

---------------------------------

Weblate rebases its changes every time it pulls from our repository. Pulls are triggered

by a web hook from Our Own Kallithea every time it receives new commits. Usually merging

the new translations is a straightforward process consisting of a pull from Weblate-hosted

repository which is available under Data Exports tab in Weblate interface.

Weblate tries to minimise the number of commits, but that's not always work, especially

when two translators work with different languages at more or less the same time.

It makes sense sometimes to re-order or fold commits by the same author when they touch

just the same language translation. That, however, may confuse Weblate sometimes, in

which case it should be manually convinced it has to discard the commits it created by

using its administrative interface.

Manual creation of a new language translation

---------------------------------------------

In the prepared development environment, run the following to ensure

all translation strings are extracted and up-to-date::

    python2 setup.py extract_messages

Create new language by executing following command::

    python2 setup.py init_catalog -l <new_language_code>

This creates a new translation under directory `kallithea/i18n/<new_language_code>`

based on the translation template file, `kallithea/i18n/kallithea.pot`.

Edit the new PO file located in `LC_MESSAGES` directory with poedit or your

favorite PO files editor. After you finished with the translations, check the

translation file for errors by executing::

    msgfmt -f -c kallithea/i18n/<new_language_code>/LC_MESSAGES/<updated_file.po>

Finally, compile the translations::

    python2 setup.py compile_catalog -l <new_language_code>

Updating translations

---------------------

Extract the latest versions of strings for translation by running::

    python2 setup.py extract_messages

Update the PO file by doing::

    python2 setup.py update_catalog -l <new_language_code>

Edit the new updated translation file. Repeat all steps after `init_catalog` step from

new translation instructions

Testing translations

--------------------

Edit kallithea/tests/test.ini file and set lang attribute to::

    lang=<new_language_code>

Run Kallithea tests by executing::

    py.test