Kallithea is developed and maintained by its users. Please join us and scratch

your own itch.

Infrastructure

--------------

The main repository is hosted on Our Own Kallithea (aka OOK) at

https://kallithea-scm.org/repos/kallithea/, our self-hosted instance

of Kallithea.

For now, we use Bitbucket_ for `pull requests`_ and `issue tracking`_. The

issue tracker is for tracking bugs, not for support, discussion, or ideas --

please use the `mailing list`_ or :ref:`IRC <readme>` to reach the community.

We use Weblate_ to translate the user interface messages into languages other

than English. Join our project on `Hosted Weblate`_ to help us.

To register, you can use your Bitbucket or GitHub account. See :ref:`translations`

for more details.

Getting started

---------------

To get started with Kallithea development::

        hg clone https://kallithea-scm.org/repos/kallithea

        cd kallithea

        virtualenv ../kallithea-venv

        source ../kallithea-venv/bin/activate

        pip install --upgrade pip setuptools

        pip install --upgrade -e .

        pip install --upgrade -r dev_requirements.txt

        npm install     # install dependencies - both tools and data

        npm run less    # for generating css from less

        gearbox make-config my.ini

        gearbox setup-db -c my.ini --user=user --email=user@example.com --password=password --repos=/tmp

        gearbox serve -c my.ini --reload &

        firefox http://127.0.0.1:5000/

If you plan to use Bitbucket_ for sending contributions, you can also fork

Kallithea on Bitbucket_ first (https://bitbucket.org/conservancy/kallithea) and

then replace the clone step above by a clone of your fork. In this case, please

see :ref:`contributing-guidelines` below for configuring your fork correctly.

Contribution flow

-----------------

Starting from an existing Kallithea clone, make sure it is up to date with the

latest upstream changes::

        hg pull

        hg update

Review the :ref:`contributing-guidelines` and :ref:`coding-guidelines`.

If you are new to Mercurial, refer to Mercurial `Quick Start`_ and `Beginners

Guide`_ on the Mercurial wiki.

Now, make some changes and test them (see :ref:`contributing-tests`). Don't

forget to add new tests to cover new functionality or bug fixes.

For documentation changes, run ``make html`` from the ``docs`` directory to

generate the HTML result, then review them in your browser.

Before submitting any changes, run the cleanup script::

        ./scripts/run-all-cleanup

When you are completely ready, you can send your changes to the community for

review and inclusion. Most commonly used methods are sending patches to the

mailing list (via ``hg email``) or by creating a pull request on Bitbucket_.

.. _contributing-tests:

Running tests

-------------

After finishing your changes make sure all tests pass cleanly. Run the testsuite

by invoking ``py.test`` from the project root::

    py.test

Note that testing on Python 2.6 also requires ``unittest2``.

Note that on unix systems, the temporary directory (``/tmp`` or where

``$TMPDIR`` points) must allow executable files; Git hooks must be executable,

and the test suite creates repositories in the temporary directory. Linux

systems with /tmp mounted noexec will thus fail.

You can also use ``tox`` to run the tests with all supported Python versions

(currently Python 2.6--2.7).

It is possible to avoid recreating the full test database on each invocation of

the tests, thus eliminating the initial delay. To achieve this, run the tests as::

    KALLITHEA_WHOOSH_TEST_DISABLE=1 KALLITHEA_NO_TMP_PATH=1 py.test

    kill -9 $(cat test.pid)

In these commands, the following variables are used::

    KALLITHEA_WHOOSH_TEST_DISABLE=1 - skip whoosh index building and tests

    KALLITHEA_NO_TMP_PATH=1 - disable new temp path for tests, used mostly for testing_vcs_operations

You can run individual tests by specifying their path as argument to py.test.

py.test also has many more options, see `py.test -h`. Some useful options

are::

    -k EXPRESSION         only run tests which match the given substring

                          expression. An expression is a python evaluable

                          expression where all names are substring-matched

                          against test names and their parent classes. Example:

    -x, --exitfirst       exit instantly on first error or failed test.

    --lf                  rerun only the tests that failed at the last run (or

                          all if none failed)

    --ff                  run all tests but run the last failures first. This

                          may re-order tests and thus lead to repeated fixture

                          setup/teardown

    --pdb                 start the interactive Python debugger on errors.

    -s, --capture=no      don't capture stdout (any stdout output will be

                          printed immediately)

Performance tests

^^^^^^^^^^^^^^^^^

A number of performance tests are present in the test suite, but they are

not run in a standard test run. These tests are useful to

evaluate the impact of certain code changes with respect to performance.

To run these tests::

    env TEST_PERFORMANCE=1 py.test kallithea/tests/performance

To analyze performance, you could install pytest-profiling_, which enables the

--profile and --profile-svg options to py.test.

.. _pytest-profiling: https://github.com/manahl/pytest-plugins/tree/master/pytest-profiling

.. _contributing-guidelines:

Contribution guidelines

-----------------------

Kallithea is GPLv3 and we assume all contributions are made by the

committer/contributor and under GPLv3 unless explicitly stated. We do care a

lot about preservation of copyright and license information for existing code

that is brought into the project.

Contributions will be accepted in most formats -- such as pull requests on

Bitbucket, something hosted on your own Kallithea instance, or patches sent by

email to the `kallithea-general`_ mailing list.

When contributing via Bitbucket, please make your fork of

https://bitbucket.org/conservancy/kallithea/ `non-publishing`_ -- it is one of

the settings on "Repository details" page. This ensures your commits are in

"draft" phase and makes it easier for you to address feedback and for project

maintainers to integrate your changes.

.. _non-publishing: https://www.mercurial-scm.org/wiki/Phases#Publishing_Repository

Make sure to test your changes both manually and with the automatic tests

before posting.

We care about quality and review and keeping a clean repository history. We

might give feedback that requests polishing contributions until they are

"perfect". We might also rebase and collapse and make minor adjustments to your

changes when we apply them.

We try to make sure we have consensus on the direction the project is taking.

Everything non-sensitive should be discussed in public -- preferably on the

mailing list.  We aim at having all non-trivial changes reviewed by at least

one other core developer before pushing. Obvious non-controversial changes will

be handled more casually.

There is a main development branch ("default") which is generally stable so that

it can be (and is) used in production. There is also a "stable" branch that is

almost exclusively reserved for bug fixes or trivial changes. Experimental

changes should live elsewhere (for example in a pull request) until they are

ready.

.. _coding-guidelines:

Coding guidelines

-----------------

We don't have a formal coding/formatting standard. We are currently using a mix

of Mercurial's (https://www.mercurial-scm.org/wiki/CodingStyle), pep8, and

consistency with existing code. Run ``scripts/run-all-cleanup`` before

committing to ensure some basic code formatting consistency.

============

Translations

============

Translations are available on Hosted Weblate at the following URL:

    https://hosted.weblate.org/projects/kallithea/kallithea/

Registered users may contribute to the existing languages, or request a new

language translations.

Translating using Weblate

-------------------------

Weblate_ offers a simple and easy to use interface featuring glossary, machine

translation, suggestions based on similar translations in other projects,

automatic checks etc. Weblate imports the source code tree directly from

the version control system, and commits edits back from time to time.

When registering at Weblate, make sure you name and email address you prefer to

be used when your changes are committed. We can and probably will amend changesets

coming from Weblate, but having things right from the beginning makes things easier.

Weblate performs sanity checks all the time and tries to prevent you from ignoring

them. Most common mistakes are inconsistent punctuation, whitespaces, missing or extra

format parameters, untranslated strings copied into the translation. Please perform

necessary corrections when they're needed, or override the false positives.

Merging translations from Weblate

---------------------------------

Weblate rebases its changes every time it pulls from our repository. Pulls are triggered

by a web hook from Our Own Kallithea every time it receives new commits. Usually merging

the new translations is a straightforward process consisting of a pull from Weblate-hosted

repository which is available under Data Exports tab in Weblate interface.

Weblate tries to minimise the number of commits, but that's not always work, especially

when two translators work with different languages at more or less the same time.

It makes sense sometimes to re-order or fold commits by the same author when they touch

just the same language translation. That, however, may confuse Weblate sometimes, in

which case it should be manually convinced it has to discard the commits it created by

using its administrative interface.

Manual creation of a new language translation

---------------------------------------------

In the prepared development environment, run the following to ensure

all translation strings are extracted and up-to-date::

    python2 setup.py extract_messages

Create new language by executing following command::

    python2 setup.py init_catalog -l <new_language_code>

This creates a new translation under directory `kallithea/i18n/<new_language_code>`

based on the translation template file, `kallithea/i18n/kallithea.pot`.

Edit the new PO file located in `LC_MESSAGES` directory with poedit or your

favorite PO files editor. After you finished with the translations, check the

translation file for errors by executing::

    msgfmt -f -c kallithea/i18n/<new_language_code>/LC_MESSAGES/<updated_file.po>

Finally, compile the translations::

    python2 setup.py compile_catalog -l <new_language_code>

Updating translations

---------------------

Extract the latest versions of strings for translation by running::

    python2 setup.py extract_messages

Update the PO file by doing::

    python2 setup.py update_catalog -l <new_language_code>

Edit the new updated translation file. Repeat all steps after `init_catalog` step from

new translation instructions

Testing translations

--------------------

    py.test

.. _Weblate: http://weblate.org/

import os

import re

import sys

import logging

import pkg_resources

import time

import formencode

from paste.deploy import loadwsgi

from routes.util import URLGenerator

import pytest

from pytest_localserver.http import WSGIServer

from kallithea.controllers.root import RootController

from kallithea.lib import inifile

from kallithea.lib.utils import repo2db_mapper

from kallithea.model.user import UserModel

from kallithea.model.meta import Session

from kallithea.model.db import Setting, User, UserIpMap

from kallithea.model.scm import ScmModel

from kallithea.tests.base import invalidate_all_caches, TEST_USER_REGULAR_LOGIN, TESTS_TMP_PATH, \

    TEST_USER_ADMIN_LOGIN, TEST_USER_ADMIN_PASS

import kallithea.tests.base # FIXME: needed for setting testapp instance!!!

from tg.util.webtest import test_context

def pytest_configure():

    os.environ['TZ'] = 'UTC'

    if not kallithea.is_windows:

        time.tzset() # only available on Unix

    path = os.getcwd()

    sys.path.insert(0, path)

    pkg_resources.working_set.add_entry(path)

    # Disable INFO logging of test database creation, restore with NOTSET

    logging.disable(logging.INFO)

    ini_settings = {

        '[server:main]': {

            'port': '4999',

        },

        '[app:main]': {

            'app_instance_uuid': 'test',

            'show_revision_number': 'true',

            'beaker.cache.sql_cache_short.expire': '1',

            'beaker.session.secret': '{74e0cd75-b339-478b-b129-07dd221def1f}',

        },

        '[handler_console]': {

            'formatter': 'color_formatter',

        },

        # The 'handler_console_sql' block is very similar to the one in

        # development.ini, but without the explicit 'level=DEBUG' setting:

        # it causes duplicate sqlalchemy debug logs, one through

        # handler_console_sql and another through another path.

        '[handler_console_sql]': {

            'formatter': 'color_formatter_sql',

        },

    }

    if os.environ.get('TEST_DB'):

        ini_settings['[app:main]']['sqlalchemy.url'] = os.environ.get('TEST_DB')

    test_ini_file = os.path.join(TESTS_TMP_PATH, 'test.ini')

    inifile.create(test_ini_file, None, ini_settings)

    context = loadwsgi.loadcontext(loadwsgi.APP, 'config:%s' % test_ini_file)

    from kallithea.tests.fixture import create_test_env, create_test_index

    # set KALLITHEA_NO_TMP_PATH=1 to disable re-creating the database and test repos

    if not int(os.environ.get('KALLITHEA_NO_TMP_PATH', 0)):

        create_test_env(TESTS_TMP_PATH, context.config())

    # set KALLITHEA_WHOOSH_TEST_DISABLE=1 to disable whoosh index during tests

    if not int(os.environ.get('KALLITHEA_WHOOSH_TEST_DISABLE', 0)):

        create_test_index(TESTS_TMP_PATH, context.config(), True)

    kallithea.tests.base.testapp = context.create()

    # do initial repo scan

    repo2db_mapper(ScmModel().repo_scan(TESTS_TMP_PATH))

    logging.disable(logging.NOTSET)

    kallithea.tests.base.url = URLGenerator(RootController().mapper, {'HTTP_HOST': 'example.com'})

    # set fixed language for form messages, regardless of environment settings

    formencode.api.set_stdtranslation(languages=[])

@pytest.fixture

def create_test_user():

    """Provide users that automatically disappear after test is over."""

    test_user_ids = []

    def _create_test_user(user_form):

        user = UserModel().create(user_form)

        test_user_ids.append(user.user_id)

        return user

    yield _create_test_user

    for user_id in test_user_ids:

        UserModel().delete(user_id)

    Session().commit()

def _set_settings(*kvtseq):

    session = Session()

    for kvt in kvtseq:

        assert len(kvt) in (2, 3)

        k = kvt[0]

        v = kvt[1]

        t = kvt[2] if len(kvt) == 3 else 'unicode'

        Setting.create_or_update(k, v, t)

    session.commit()

@pytest.fixture

def set_test_settings():

    """Restore settings after test is over."""

    # Save settings.

    settings_snapshot = [

        (s.app_settings_name, s.app_settings_value, s.app_settings_type)

        for s in Setting.query().all()]

    yield _set_settings

    # Restore settings.

    session = Session()

    keys = frozenset(k for (k, v, t) in settings_snapshot)

    for s in Setting.query().all():

        if s.app_settings_name not in keys:

            session.delete(s)

    for k, v, t in settings_snapshot:

        if t == 'list' and hasattr(v, '__iter__'):

            v = ','.join(v) # Quirk: must format list value manually.

        Setting.create_or_update(k, v, t)

    session.commit()

@pytest.fixture

def auto_clear_ip_permissions():

    """Fixture that provides nothing but clearing IP permissions upon test

    exit. This clearing is needed to avoid other test failing to make fake http

    accesses."""

    yield

    # cleanup

    user_model = UserModel()

#!/usr/bin/env python2

"""

"""

import re

from kallithea.lib import inifile

# files to be generated from the mako template

ini_files = [

    ('development.ini',

        {

            '[server:main]': {

                'host': '0.0.0.0',

            },

            '[app:main]': {

                'debug': 'true',

                'app_instance_uuid': 'development-not-secret',

                'beaker.session.secret': 'development-not-secret',

            },

            '[handler_console]': {

                'formatter': 'color_formatter',

            },

            '[handler_console_sql]': {

                'formatter': 'color_formatter_sql',

            },

            '[logger_routes]': {

                'level': 'DEBUG',

            },

            '[logger_beaker]': {

                'level': 'DEBUG',

            },

            '[logger_templates]': {

                'level': 'INFO',

            },

            '[logger_kallithea]': {

                'level': 'DEBUG',

            },

            '[logger_tg]': {

                'level': 'DEBUG',

            },

            '[logger_gearbox]': {

                'level': 'DEBUG',

            },

            '[logger_whoosh_indexer]': {

                'level': 'DEBUG',

            },

        },

    ),

]

def main():

    # make sure all mako lines starting with '#' (the '##' comments) are marked up as <text>

    makofile = inifile.template_file

    print 'reading:', makofile

    mako_org = open(makofile).read()

    mako_no_text_markup = re.sub(r'</?%text>', '', mako_org)

    mako_marked_up = re.sub(r'\n(##.*)', r'\n<%text>\1</%text>', mako_no_text_markup, flags=re.MULTILINE)

    if mako_marked_up != mako_org:

        print 'writing:', makofile

        open(makofile, 'w').write(mako_marked_up)

    # create ini files

    for fn, settings in ini_files:

        print 'updating:', fn

        inifile.create(fn, None, settings)

if __name__ == '__main__':

    main()