syntax: glob

*.pyc

*.swp

*.sqlite

*.tox

*.egg-info

*.egg

*.mo

.eggs/

tarballcache/

node_modules/

syntax: regexp

^rcextensions

^build

^dist/

^docs/build/

^docs/_build/

^data$

^sql_dumps/

^\.settings$

^\.project$

^\.pydevproject$

^\.coverage$

^kallithea\.db$

^test\.db$

^Kallithea\.egg-info$

^my\.ini$

^fabfile.py

^\.idea$

^\.cache$

/__pycache__$

.. _contributing:

=========================

Contributing to Kallithea

=========================

Kallithea is developed and maintained by its users. Please join us and scratch

your own itch.

Infrastructure

--------------

The main repository is hosted on Our Own Kallithea (aka OOK) at

https://kallithea-scm.org/repos/kallithea/, our self-hosted instance

of Kallithea.

For now, we use Bitbucket_ for `pull requests`_ and `issue tracking`_. The

issue tracker is for tracking bugs, not for support, discussion, or ideas --

please use the `mailing list`_ or :ref:`IRC <readme>` to reach the community.

We use Weblate_ to translate the user interface messages into languages other

than English. Join our project on `Hosted Weblate`_ to help us.

To register, you can use your Bitbucket or GitHub account. See :ref:`translations`

for more details.

Getting started

---------------

To get started with Kallithea development::

        hg clone https://kallithea-scm.org/repos/kallithea

        cd kallithea

        virtualenv ../kallithea-venv

        source ../kallithea-venv/bin/activate

        pip install --upgrade pip setuptools

        pip install --upgrade -e .

        pip install --upgrade -r dev_requirements.txt

        gearbox make-config my.ini

        gearbox setup-db -c my.ini --user=user --email=user@example.com --password=password --repos=/tmp

        gearbox serve -c my.ini --reload &

        firefox http://127.0.0.1:5000/

If you plan to use Bitbucket_ for sending contributions, you can also fork

Kallithea on Bitbucket_ first (https://bitbucket.org/conservancy/kallithea) and

then replace the clone step above by a clone of your fork. In this case, please

see :ref:`contributing-guidelines` below for configuring your fork correctly.

Contribution flow

-----------------

Starting from an existing Kallithea clone, make sure it is up to date with the

latest upstream changes::

        hg pull

        hg update

Review the :ref:`contributing-guidelines` and :ref:`coding-guidelines`.

If you are new to Mercurial, refer to Mercurial `Quick Start`_ and `Beginners

Guide`_ on the Mercurial wiki.

Now, make some changes and test them (see :ref:`contributing-tests`). Don't

forget to add new tests to cover new functionality or bug fixes.

For documentation changes, run ``make html`` from the ``docs`` directory to

generate the HTML result, then review them in your browser.

Before submitting any changes, run the cleanup script::

        ./scripts/run-all-cleanup

When you are completely ready, you can send your changes to the community for

review and inclusion. Most commonly used methods are sending patches to the

mailing list (via ``hg email``) or by creating a pull request on Bitbucket_.

.. _contributing-tests:

Running tests

-------------

After finishing your changes make sure all tests pass cleanly. Run the testsuite

by invoking ``py.test`` from the project root::

environment used for running Kallithea.

- Packages *could* be installed in Python's ``site-packages`` directory ... but

  that would require running pip_ as root and it would be hard to uninstall or

  upgrade and is probably not a good idea unless using a package manager.

- Packages could also be installed in ``~/.local`` ... but that is probably

  only a good idea if using a dedicated user per application or instance.

- Finally, it can be installed in a virtualenv_. That is a very lightweight

  "container" where each Kallithea instance can get its own dedicated and

  self-contained virtual environment.

We recommend using virtualenv for installing Kallithea.

Installation methods

--------------------

Kallithea must be installed on a server. Kallithea is installed in a Python

environment so it can use packages that are installed there and make itself

available for other packages.

Two different cases will pretty much cover the options for how it can be

installed.

- The Kallithea source repository can be cloned and used -- it is kept stable and

  can be used in production. The Kallithea maintainers use the development

  branch in production. The advantage of installation from source and regularly

  updating it is that you take advantage of the most recent improvements. Using

  it directly from a DVCS also means that it is easy to track local customizations.

  Running ``pip install -e .`` in the source will use pip to install the

  necessary dependencies in the Python environment and create a

  ``.../site-packages/Kallithea.egg-link`` file there that points at the Kallithea

  source.

- Kallithea can also be installed from ready-made packages using a package manager.

  The official released versions are available on PyPI_ and can be downloaded and

  installed with all dependencies using ``pip install kallithea``.

  With this method, Kallithea is installed in the Python environment as any

  other package, usually as a ``.../site-packages/Kallithea-X-py2.7.egg/``

  directory with Python files and everything else that is needed.

  (``pip install kallithea`` from a source tree will do pretty much the same

  but build the Kallithea package itself locally instead of downloading it.)

Web server

----------

Kallithea is (primarily) a WSGI_ application that must be run from a web

server that serves WSGI applications over HTTP.

Kallithea itself is not serving HTTP (or HTTPS); that is the web server's

responsibility. Kallithea does however need to know its own user facing URL

(protocol, address, port and path) for each HTTP request. Kallithea will

usually use its own HTML/cookie based authentication but can also be configured

to use web server authentication.

There are several web server options:

- Kallithea uses the Gearbox_ tool as command line interface. Gearbox provides

  ``gearbox serve`` as a convenient way to launch a Python WSGI / web server

  from the command line. That is perfect for development and evaluation.

  Actual use in production might have different requirements and need extra

  work to make it manageable as a scalable system service.

  Gearbox comes with its own built-in web server but Kallithea defaults to use

  Waitress_. Gunicorn_ is also an option. These web servers have different

  limited feature sets.

  The web server used by ``gearbox`` is configured in the ``.ini`` file passed

  to it. The entry point for the WSGI application is configured

  in ``setup.py`` as ``kallithea.config.middleware:make_app``.

- `Apache httpd`_ can serve WSGI applications directly using mod_wsgi_ and a

  simple Python file with the necessary configuration. This is a good option if

  Apache is an option.

- uWSGI_ is also a full web server with built-in WSGI module.

- IIS_ can also server WSGI applications directly using isapi-wsgi_.

- A `reverse HTTP proxy <https://en.wikipedia.org/wiki/Reverse_proxy>`_

  can be put in front of another web server which has WSGI support.

  Such a layered setup can be complex but might in some cases be the right

  option, for example to standardize on one internet-facing web server, to add

  encryption or special authentication or for other security reasons, to

  provide caching of static files, or to provide load balancing or fail-over.

  Nginx_, Varnish_ and HAProxy_ are often used for this purpose, often in front

  of a ``gearbox serve`` that somehow is wrapped as a service.

The best option depends on what you are familiar with and the requirements for

performance and stability. Also, keep in mind that Kallithea mainly is serving

.. _setup:

=====

Setup

=====

Setting up Kallithea

--------------------

First, you will need to create a Kallithea configuration file. Run the

following command to do so::

    gearbox make-config my.ini

This will create the file ``my.ini`` in the current directory. This

configuration file contains the various settings for Kallithea, e.g.

proxy port, email settings, usage of static files, cache, Celery

settings, and logging. Extra settings can be specified like::

    gearbox make-config my.ini host=8.8.8.8 "[handler_console]" formatter=color_formatter

Next, you need to create the databases used by Kallithea. It is recommended to

use PostgreSQL or SQLite (default). If you choose a database other than the

default, ensure you properly adjust the database URL in your ``my.ini``

configuration file to use this other database. Kallithea currently supports

PostgreSQL, SQLite and MySQL databases. Create the database by running

the following command::

    gearbox setup-db -c my.ini

This will prompt you for a "root" path. This "root" path is the location where

Kallithea will store all of its repositories on the current machine. After

entering this "root" path ``setup-db`` will also prompt you for a username

and password for the initial admin account which ``setup-db`` sets

up for you.

The ``setup-db`` values can also be given on the command line.

Example::

    gearbox setup-db -c my.ini --user=nn --password=secret --email=nn@example.com --repos=/srv/repos

The ``setup-db`` command will create all needed tables and an

admin account. When choosing a root path you can either use a new

empty location, or a location which already contains existing

repositories. If you choose a location which contains existing

repositories Kallithea will add all of the repositories at the chosen

location to its database.  (Note: make sure you specify the correct

path to the root).

.. note:: the given path for Mercurial_ repositories **must** be write

          accessible for the application. It's very important since

          the Kallithea web interface will work without write access,

          but when trying to do a push it will fail with permission

          denied errors unless it has write access.

If you are using :ref:`rcextensions <customization>`, you should also

make a copy of the entire ``rcextensions`` directory.

Back up your database

^^^^^^^^^^^^^^^^^^^^^

If using SQLite, simply make a copy of the Kallithea database (``.db``)

file.

If using PostgreSQL, please consult the documentation for the ``pg_dump``

utility.

If using MySQL, please consult the documentation for the ``mysqldump``

utility.

Look for ``sqlalchemy.url`` in your configuration file to determine

database type, settings, location, etc.

3. Activate the Kallithea virtual environment (if any)

------------------------------------------------------

Verify that you are using the Python environment that you originally

installed Kallithea in by running::

    pip freeze

This will list all packages installed in the current environment. If

Kallithea isn't listed, activate the correct virtual environment.

See the appropriate installation page for details.

4. Install new version of Kallithea

-----------------------------------

Please refer to the instructions for the installation method you

originally used to install Kallithea.

If you originally installed using pip, it is as simple as::

    pip install --upgrade kallithea

If you originally installed from version control, it is as simple as::

    cd my-kallithea-clone

    hg pull -u

    pip install --upgrade -e .

5. Upgrade your configuration

-----------------------------

Run the following command to create a new configuration (``.ini``) file::

    gearbox make-config new.ini

Then compare it with your old config file and see what changed.

.. note::

    Please always make sure your ``.ini`` files are up to date. Errors

    can often be caused by missing parameters added in new versions.

.. _upgrade_db:

6. Upgrade your database

------------------------

.. note::

    If you are *downgrading* Kallithea, you should perform the database

    migration step *before* installing the older version. (That is,

    always perform migrations using the most recent of the two versions

    you're migrating between.)

First, run the following command to see your current database version::

    alembic -c my.ini current

Typical output will be something like "9358dc3d6828 (head)", which is

the current Alembic database "revision ID". Write down the entire output

for troubleshooting purposes.

The output will be empty if you're upgrading from Kallithea 0.3.x or

older. That's expected. If you get an error that the config file was not

found or has no ``[alembic]`` section, see the next section.

Next, if you are performing an *upgrade*: Run the following command to

upgrade your database to the current Kallithea version::

    alembic -c my.ini upgrade head

If you are performing a *downgrade*: Run the following command to

downgrade your database to the given version::

    alembic -c my.ini downgrade 0.4