.. _api:

===

API

===

Kallithea has a simple JSON RPC API with a single schema for calling all API

methods. Everything is available by sending JSON encoded http(s) requests to

``<your_server>/_admin/api``.

API access for web views

++++++++++++++++++++++++

API access can also be turned on for each web view in Kallithea that is

decorated with the ``@LoginRequired`` decorator. Some views use

``@LoginRequired(api_access=True)`` and are always available. By default only

RSS/Atom feed views are enabled. Other views are

only available if they have been whitelisted. Edit the

``api_access_controllers_whitelist`` option in your .ini file and define views

that should have API access enabled.

For example, to enable API access to patch/diff, raw file and archive::

    api_access_controllers_whitelist =

        ChangesetController:changeset_patch,

        ChangesetController:changeset_raw,

        FilesController:raw,

        FilesController:archivefile

After this change, a Kallithea view can be accessed without login by adding a

GET parameter ``?api_key=<api_key>`` to the URL.

Exposing raw diffs is a good way to integrate with

third-party services like code review, or build farms that can download archives.

API access

++++++++++

Clients must send JSON encoded JSON-RPC requests::

    {

        "id: "<id>",

        "api_key": "<api_key>",

        "method": "<method_name>",

        "args": {"<arg_key>": "<arg_val>"}

    }

For example, to pull to a local "CPython" mirror using curl::

    curl https://example.com/_admin/api -X POST -H 'content-type:text/plain' \

        --data-binary '{"id":1,"api_key":"xe7cdb2v278e4evbdf5vs04v832v0efvcbcve4a3","method":"pull","args":{"repo":"CPython"}}'

In general, provide

 - *id*, a value of any type, can be used to match the response with the request that it is replying to.

 - *api_key*, for authentication and permission validation.

 - *method*, the name of the method to call -- a list of available methods can be found below.

 - *args*, the arguments to pass to the method.

.. note::

    api_key can be found or set on the user account page.

The response to the JSON-RPC API call will always be a JSON structure::

    {

        "id": <id>,  # the id that was used in the request

        "result": <result>|null,  # JSON formatted result (null on error)

        "error": null|<error_message>  # JSON formatted error (null on success)

    }

All responses from the API will be ``HTTP/1.0 200 OK``. If an error occurs,

the reponse will have a failure description in *error* and

*result* will be null.

API client

++++++++++

Kallithea comes with a ``kallithea-api`` command line tool, providing a convenient

way to call the JSON-RPC API.

For example, to call ``get_repo``::

Oops, looks like we forgot to add an argument. Let's try again, now

providing the ``repoid`` as a parameter::

To avoid specifying ``apihost`` and ``apikey`` every time, run::

so you don't have to specify them every time.

API methods

+++++++++++

pull

----

Pull the given repo from remote location. Can be used to automatically keep

remote repos up to date.

This command can only be executed using the api_key of a user with admin rights.

INPUT::

    id : <id_for_response>

    api_key : "<api_key>"

    method :  "pull"

    args :    {

                "repoid" : "<reponame or repo_id>"

              }

OUTPUT::

    id : <id_given_in_input>

    result : "Pulled from `<reponame>`"

    error :  null

rescan_repos

------------

Rescan repositories. If ``remove_obsolete`` is set,

Kallithea will delete repos that are in the database but not in the filesystem.

This command can only be executed using the api_key of a user with admin rights.

INPUT::

    id : <id_for_response>

    api_key : "<api_key>"

    method :  "rescan_repos"

    args :    {

                "remove_obsolete" : "<boolean = Optional(False)>"

              }

OUTPUT::

    id : <id_given_in_input>

    result : "{'added': [<list of names of added repos>],

               'removed': [<list of names of removed repos>]}"

    error :  null

invalidate_cache

----------------

Invalidate the cache for a repository.

This command can only be executed using the api_key of a user with admin rights,

or that of a regular user with admin or write access to the repository.

INPUT::

    id : <id_for_response>

    api_key : "<api_key>"

    method :  "invalidate_cache"

    args :    {

                "repoid" : "<reponame or repo_id>"

              }

OUTPUT::

    id : <id_given_in_input>

    result : "Caches of repository `<reponame>`"

    error :  null

lock

----

Set the locking state on the given repository by the given user.

If the param ``userid`` is skipped, it is set to the ID of the user who is calling this method.

If param ``locked`` is skipped, the current lock state of the repository is returned.

This command can only be executed using the api_key of a user with admin rights, or that of a regular user with admin or write access to the repository.

INPUT::

    id : <id_for_response>

    api_key : "<api_key>"

    method :  "lock"

    args :    {

                "repoid" : "<reponame or repo_id>"

                "userid" : "<user_id or username = Optional(=apiuser)>",

                "locked" : "<bool true|false = Optional(=None)>"

              }

OUTPUT::

    id : <id_given_in_input>

    result : {

                 "repo": "<reponame>",

                 "locked": "<bool true|false>",

                 "locked_since": "<float lock_time>",

                 "locked_by": "<username>",

                 "msg": "User `<username>` set lock state for repo `<reponame>` to `<false|true>`"

             }

    error :  null

get_ip

------

Return IP address as seen from Kallithea server, together with all

defined IP addresses for given user.

This command can only be executed using the api_key of a user with admin rights.

INPUT::

    id : <id_for_response>

    api_key : "<api_key>"

    method :  "get_ip"

    args :    {

                "userid" : "<user_id or username>",

              }

OUTPUT::

    id : <id_given_in_input>

    result : {

                 "ip_addr_server": <ip_from_clien>",

                 "user_ips": [

                                {

                                   "ip_addr": "<ip_with_mask>",

                                   "ip_range": ["<start_ip>", "<end_ip>"],

                                },

                                ...

                             ]

             }

    error :  null

get_user

--------

Get a user by username or userid. The result is empty if user can't be found.

If userid param is skipped, it is set to id of user who is calling this method.

Any userid can be specified when the command is executed using the api_key of a user with admin rights.

Regular users can only speicy their own userid.

INPUT::

    id : <id_for_response>

    api_key : "<api_key>"

    method :  "get_user"

    args :    {

                "userid" : "<username or user_id Optional(=apiuser)>"

              }

OUTPUT::

    id : <id_given_in_input>

    result: None if user does not exist or

            {

                "user_id" :     "<user_id>",

                "api_key" :     "<api_key>",

                "username" :    "<username>",

                "firstname":    "<firstname>",

                "lastname" :    "<lastname>",

                "email" :       "<email>",

                "emails":       "<list_of_all_additional_emails>",

                "ip_addresses": "<list_of_ip_addresses_for_user>",

                "active" :      "<bool>",

                "admin" :       "<bool>",

                "ldap_dn" :     "<ldap_dn>",

                "last_login":   "<last_login>",

                "permissions": {

                    "global": ["hg.create.repository",

                               "repository.read",

                               "hg.register.manual_activate"],

                    "repositories": {"repo1": "repository.none"},

                    "repositories_groups": {"Group1": "group.read"}

                 },

            }

    error:  null

get_users

---------

List all existing users.

This command can only be executed using the api_key of a user with admin rights.

INPUT::

    id : <id_for_response>

    api_key : "<api_key>"

    method :  "get_users"

      RewriteEngine On

      RewriteCond %{LA-U:REMOTE_USER} (.+)

      RewriteRule .* - [E=RU:%1]

      RequestHeader set X-Forwarded-User %{RU}e

    </Location>

.. note::

   If you enable proxy pass-through authentication, make sure your server is

   only accessible through the proxy. Otherwise, any client would be able to

   forge the authentication header and could effectively become authenticated

   using any account of their liking.

Integration with issue trackers

-------------------------------

Kallithea provides a simple integration with issue trackers. It's possible

to define a regular expression that will match an issue ID in commit messages,

and have that replaced with a URL to the issue. To enable this simply

uncomment the following variables in the ini file::

    issue_pat = (?:^#|\s#)(\w+)

    issue_server_link = https://myissueserver.com/{repo}/issue/{id}

    issue_prefix = #

``issue_pat`` is the regular expression describing which strings in

commit messages will be treated as issue references. A match group in

parentheses should be used to specify the actual issue id.

The default expression matches issues in the format ``#<number>``, e.g., ``#300``.

Matched issue references are replaced with the link specified in

``issue_server_link``. ``{id}`` is replaced with the issue ID, and

``{repo}`` with the repository name.  Since the # is stripped away,

``issue_prefix`` is prepended to the link text.  ``issue_prefix`` doesn't

necessarily need to be ``#``: if you set issue prefix to ``ISSUE-`` this will

generate a URL in the format:

.. code-block:: html

  <a href="https://myissueserver.com/example_repo/issue/300">ISSUE-300</a>

If needed, more than one pattern can be specified by appending a unique suffix to

the variables. For example::

    issue_pat_wiki = (?:wiki-)(.+)

    issue_server_link_wiki = https://mywiki.com/{id}

    issue_prefix_wiki = WIKI-

With these settings, wiki pages can be referenced as wiki-some-id, and every

such reference will be transformed into:

.. code-block:: html

  <a href="https://mywiki.com/some-id">WIKI-some-id</a>

Hook management

---------------

Hooks can be managed in similar way to that used in ``.hgrc`` files.

To manage hooks, choose *Admin > Settings > Hooks*.

The built-in hooks cannot be modified, though they can be enabled or disabled in the *VCS* section.

To add another custom hook simply fill in the first textbox with

``<name>.<hook_type>`` and the second with the hook path. Example hooks

can be found in ``kallithea.lib.hooks``.

Changing default encoding

-------------------------

By default, Kallithea uses UTF-8 encoding.

This is configurable as ``default_encoding`` in the .ini file.

This affects many parts in Kallithea including user names, filenames, and

encoding of commit messages. In addition Kallithea can detect if the ``chardet``

library is installed. If ``chardet`` is detected Kallithea will fallback to it

when there are encode/decode errors.

Celery configuration

--------------------

Kallithea can use the distributed task queue system Celery_ to run tasks like

cloning repositories or sending emails.

Kallithea will in most setups work perfectly fine out of the box (without

Celery), executing all tasks in the web server process. Some tasks can however

take some time to run and it can be better to run such tasks asynchronously in

a separate process so the web server can focus on serving web requests.

For installation and configuration of Celery, see the `Celery documentation`_.

Note that Celery requires a message broker service like RabbitMQ_ (recommended)

or Redis_.

The use of Celery is configured in the Kallithea ini configuration file.

To enable it, simply set::

  use_celery = true

and add or change the ``celery.*`` and ``broker.*`` configuration variables.

Remember that the ini files use the format with '.' and not with '_' like

Celery. So for example setting `BROKER_HOST` in Celery means setting

`broker.host` in the configuration file.

To start the Celery process, run::

 paster celeryd <configfile.ini>

.. note::

   Make sure you run this command from the same virtualenv, and with the same

   user that Kallithea runs.

HTTPS support

-------------

Kallithea will by default generate URLs based on the WSGI environment.

Alternatively, you can use some special configuration settings to control

directly which scheme/protocol Kallithea will use when generating URLs:

- With ``https_fixup = true``, the scheme will be taken from the

  ``X-Url-Scheme``, ``X-Forwarded-Scheme`` or ``X-Forwarded-Proto`` HTTP header

  (default ``http``).

- With ``force_https = true`` the default will be ``https``.

- With ``use_htsts = true``, Kallithea will set ``Strict-Transport-Security`` when using https.

Nginx virtual host example

--------------------------

Sample config for Nginx using proxy:

.. code-block:: nginx

    upstream kallithea {

        server 127.0.0.1:5000;

        # add more instances for load balancing

        #server 127.0.0.1:5001;

        #server 127.0.0.1:5002;

    }

    ## gist alias

    server {

       listen          443;

       server_name     gist.myserver.com;

       access_log      /var/log/nginx/gist.access.log;

       error_log       /var/log/nginx/gist.error.log;

       ssl on;

       ssl_certificate     gist.your.kallithea.server.crt;

       ssl_certificate_key gist.your.kallithea.server.key;

       ssl_session_timeout 5m;

       ssl_protocols SSLv3 TLSv1;

       ssl_ciphers DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:EDH-RSA-DES-CBC3-SHA:AES256-SHA:DES-CBC3-SHA:AES128-SHA:RC4-SHA:RC4-MD5;

       ssl_prefer_server_ciphers on;

       rewrite ^/(.+)$ https://your.kallithea.server/_admin/gists/$1;

       rewrite (.*)    https://your.kallithea.server/_admin/gists;

    }

    server {

       listen          443;

       server_name     your.kallithea.server;

       access_log      /var/log/nginx/kallithea.access.log;

       error_log       /var/log/nginx/kallithea.error.log;

       ssl on;

       ssl_certificate     your.kallithea.server.crt;

       ssl_certificate_key your.kallithea.server.key;

       ssl_session_timeout 5m;

       ssl_protocols SSLv3 TLSv1;

       ssl_ciphers DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:EDH-RSA-DES-CBC3-SHA:AES256-SHA:DES-CBC3-SHA:AES128-SHA:RC4-SHA:RC4-MD5;

       ssl_prefer_server_ciphers on;

       ## uncomment root directive if you want to serve static files by nginx

       ## requires static_files = false in .ini file

       #root /path/to/installation/kallithea/public;

       include         /etc/nginx/proxy.conf;

       location / {

            try_files $uri @kallithea;

       }

       location @kallithea {

       }

    }

Here's the proxy.conf. It's tuned so it will not timeout on long

pushes or large pushes::

    proxy_redirect              off;

    proxy_set_header            Host $host;

    ## needed for container auth

    #proxy_set_header            REMOTE_USER $remote_user;

    #proxy_set_header            X-Forwarded-User $remote_user;

    proxy_set_header            X-Url-Scheme $scheme;

    proxy_set_header            X-Host $http_host;

    proxy_set_header            X-Real-IP $remote_addr;

    proxy_set_header            X-Forwarded-For $proxy_add_x_forwarded_for;

    proxy_set_header            Proxy-host $proxy_host;

    proxy_buffering             off;

    proxy_connect_timeout       7200;

    proxy_send_timeout          7200;

    proxy_read_timeout          7200;

    proxy_buffers               8 32k;

    client_max_body_size        1024m;

    client_body_buffer_size     128k;

    large_client_header_buffers 8 64k;

Apache virtual host reverse proxy example

-----------------------------------------

Here is a sample configuration file for Apache using proxy:

.. code-block:: apache

    <VirtualHost *:80>

            ServerName hg.myserver.com

            <Proxy *>

              # For Apache 2.4 and later:

              Require all granted

              # For Apache 2.2 and earlier, instead use:

              # Order allow,deny

              # Allow from all

            </Proxy>

            #important !

            #Directive to properly generate url (clone url) for pylons

            ProxyPreserveHost On

            #kallithea instance

            ProxyPass / http://127.0.0.1:5000/

            ProxyPassReverse / http://127.0.0.1:5000/

            #to enable https use line below

            #SetEnvIf X-Url-Scheme https HTTPS=1

    </VirtualHost>

Additional tutorial

http://pylonsbook.com/en/1.1/deployment.html#using-apache-to-proxy-requests-to-pylons

Apache as subdirectory

----------------------

Apache subdirectory part:

.. code-block:: apache

    <Location /<someprefix> >

      ProxyPass http://127.0.0.1:5000/<someprefix>

      ProxyPassReverse http://127.0.0.1:5000/<someprefix>

      SetEnvIf X-Url-Scheme https HTTPS=1

    </Location>

Besides the regular apache setup you will need to add the following line

into ``[app:main]`` section of your .ini file::

    filter-with = proxy-prefix

Add the following at the end of the .ini file::

    [filter:proxy-prefix]

    use = egg:PasteDeploy#prefix

    prefix = /<someprefix>

then change ``<someprefix>`` into your chosen prefix

Apache with mod_wsgi

--------------------

Alternatively, Kallithea can be set up with Apache under mod_wsgi. For

that, you'll need to:

- Install mod_wsgi. If using a Debian-based distro, you can install

  the package libapache2-mod-wsgi::

    aptitude install libapache2-mod-wsgi

- Enable mod_wsgi::

    a2enmod wsgi

- Create a wsgi dispatch script, like the one below. Make sure you

  check that the paths correctly point to where you installed Kallithea

  and its Python Virtual Environment.

- Enable the ``WSGIScriptAlias`` directive for the WSGI dispatch script,

  as in the following example. Once again, check the paths are

  correctly specified.

Here is a sample excerpt from an Apache Virtual Host configuration file:

.. code-block:: apache

    WSGIDaemonProcess kallithea \

        processes=1 threads=4 \

        python-path=/srv/kallithea/pyenv/lib/python2.7/site-packages

    WSGIScriptAlias / /srv/kallithea/dispatch.wsgi

    WSGIPassAuthorization On

Or if using a dispatcher WSGI script with proper virtualenv activation:

.. code-block:: apache

    WSGIDaemonProcess kallithea processes=1 threads=4

    WSGIScriptAlias / /srv/kallithea/dispatch.wsgi

    WSGIPassAuthorization On

.. note::

   When running apache as root, please make sure it doesn't run Kallithea as

   root, for examply by adding: ``user=www-data group=www-data`` to the configuration.

.. note::

   If running Kallithea in multiprocess mode,

   make sure you set ``instance_id = *`` in the configuration so each process

   gets it's own cache invalidation key.

Example WSGI dispatch script:

.. code-block:: python

    import os

    os.environ["HGENCODING"] = "UTF-8"

    os.environ['PYTHON_EGG_CACHE'] = '/srv/kallithea/.egg-cache'

    # sometimes it's needed to set the curent dir

    os.chdir('/srv/kallithea/')

    import site

    site.addsitedir("/srv/kallithea/pyenv/lib/python2.7/site-packages")

    from paste.deploy import loadapp

    from paste.script.util.logging_config import fileConfig

    fileConfig('/srv/kallithea/my.ini')

    application = loadapp('config:/srv/kallithea/my.ini')

Or using proper virtualenv activation:

.. code-block:: python

    activate_this = '/srv/kallithea/venv/bin/activate_this.py'

    execfile(activate_this, dict(__file__=activate_this))

    import os

    os.environ['HOME'] = '/srv/kallithea'

    ini = '/srv/kallithea/kallithea.ini'

    from paste.script.util.logging_config import fileConfig

    fileConfig(ini)

    from paste.deploy import loadapp

    application = loadapp('config:' + ini)

Other configuration files

-------------------------

A number of `example init.d scripts`__ can be found in

the ``init.d`` directory of the Kallithea source.

.. __: https://kallithea-scm.org/repos/kallithea/files/tip/init.d/ .

.. _virtualenv: http://pypi.python.org/pypi/virtualenv

.. _python: http://www.python.org/

.. _Mercurial: http://mercurial.selenic.com/

.. _Celery: http://celeryproject.org/

.. _Celery documentation: http://docs.celeryproject.org/en/latest/getting-started/index.html

.. _RabbitMQ: http://www.rabbitmq.com/

.. _Redis: http://redis.io/

.. _python-ldap: http://www.python-ldap.org/

.. _mercurial-server: http://www.lshift.net/mercurial-server.html

.. _PublishingRepositories: http://mercurial.selenic.com/wiki/PublishingRepositories