.. _contributing:

=========================

Contributing to Kallithea

=========================

Kallithea is developed and maintained by its users. Please join us and scratch

your own itch.

Infrastructure

--------------

The main repository is hosted on Our Own Kallithea (aka OOK) at

https://kallithea-scm.org/repos/kallithea/, our self-hosted instance

of Kallithea.

For now, we use Bitbucket_ for `pull requests`_ and `issue tracking`_. The

issue tracker is for tracking bugs, not for support, discussion, or ideas --

please use the `mailing list`_ or :ref:`IRC <readme>` to reach the community.

We use Weblate_ to translate the user interface messages into languages other

than English. Join our project on `Hosted Weblate`_ to help us.

To register, you can use your Bitbucket or GitHub account. See :ref:`translations`

for more details.

Getting started

---------------

To get started with development::

        hg clone https://kallithea-scm.org/repos/kallithea

        cd kallithea

        virtualenv ../kallithea-venv

        source ../kallithea-venv/bin/activate

        pip install --upgrade pip setuptools

        pip install -e .

        gearbox make-config my.ini

        gearbox setup-db -c my.ini --user=user --email=user@example.com --password=password --repos=/tmp

        gearbox serve -c my.ini --reload &

        firefox http://127.0.0.1:5000/

You can also start out by forking https://bitbucket.org/conservancy/kallithea

on Bitbucket_ and create a local clone of your own fork.

Running tests

-------------

After finishing your changes make sure all tests pass cleanly. Install the test

dependencies, then run the testsuite by invoking ``py.test`` from the

project root::

    pip install -r dev_requirements.txt

    py.test

Note that testing on Python 2.6 also requires ``unittest2``.

You can also use ``tox`` to run the tests with all supported Python versions

(currently Python 2.6--2.7).

When running tests, Kallithea uses `kallithea/tests/test.ini` and populates the

SQLite database specified there.

It is possible to avoid recreating the full test database on each invocation of

the tests, thus eliminating the initial delay. To achieve this, run the tests as::

    gearbox serve -c kallithea/tests/test.ini --pid-file=test.pid --daemon

    KALLITHEA_WHOOSH_TEST_DISABLE=1 KALLITHEA_NO_TMP_PATH=1 py.test

    kill -9 $(cat test.pid)

In these commands, the following variables are used::

    KALLITHEA_WHOOSH_TEST_DISABLE=1 - skip whoosh index building and tests

    KALLITHEA_NO_TMP_PATH=1 - disable new temp path for tests, used mostly for testing_vcs_operations

You can run individual tests by specifying their path as argument to py.test.

py.test also has many more options, see `py.test -h`. Some useful options

are::

    -k EXPRESSION         only run tests which match the given substring

                          expression. An expression is a python evaluable

                          expression where all names are substring-matched

                          against test names and their parent classes. Example:

    -x, --exitfirst       exit instantly on first error or failed test.

    --lf                  rerun only the tests that failed at the last run (or

                          all if none failed)

    --ff                  run all tests but run the last failures first. This

                          may re-order tests and thus lead to repeated fixture

                          setup/teardown

    --pdb                 start the interactive Python debugger on errors.

    -s, --capture=no      don't capture stdout (any stdout output will be

                          printed immediately)

Contribution guidelines

-----------------------

Kallithea is GPLv3 and we assume all contributions are made by the

committer/contributor and under GPLv3 unless explicitly stated. We do care a

lot about preservation of copyright and license information for existing code

that is brought into the project.

Contributions will be accepted in most formats -- such as pull requests on

Bitbucket, something hosted on your own Kallithea instance, or patches sent by

email to the `kallithea-general`_ mailing list.

When contributing via Bitbucket, please make your fork of

https://bitbucket.org/conservancy/kallithea/ `non-publishing`_ -- it is one of

the settings on "Repository details" page. This ensures your commits are in

"draft" phase and makes it easier for you to address feedback and for project

maintainers to integrate your changes.

.. _non-publishing: https://www.mercurial-scm.org/wiki/Phases#Publishing_Repository

Make sure to test your changes both manually and with the automatic tests

before posting.

We care about quality and review and keeping a clean repository history. We

might give feedback that requests polishing contributions until they are

"perfect". We might also rebase and collapse and make minor adjustments to your

changes when we apply them.

We try to make sure we have consensus on the direction the project is taking.

Everything non-sensitive should be discussed in public -- preferably on the

mailing list.  We aim at having all non-trivial changes reviewed by at least

one other core developer before pushing. Obvious non-controversial changes will

be handled more casually.

For now we just have one official branch ("default") and will keep it so stable

that it can be (and is) used in production. Experimental changes should live

elsewhere (for example in a pull request) until they are ready.

Coding guidelines

-----------------

We don't have a formal coding/formatting standard. We are currently using a mix

of Mercurial's (https://www.mercurial-scm.org/wiki/CodingStyle), pep8, and

consistency with existing code. Run ``scripts/run-all-cleanup`` before

committing to ensure some basic code formatting consistency.

We support both Python 2.6.x and 2.7.x and nothing else. For now we don't care

about Python 3 compatibility.

We try to support the most common modern web browsers. IE9 is still supported

to the extent it is feasible, IE8 is not.

We primarily support Linux and OS X on the server side but Windows should also work.

HTML templates should use 2 spaces for indentation ... but be pragmatic. We

should use templates cleverly and avoid duplication. We should use reasonable

semantic markup with element classes and IDs that can be used for styling and testing.

We should only use inline styles in places where it really is semantic (such as

        assert 'fatal: Authentication failed' in stderr

    def test_push_back_to_wrong_url_hg(self):

        DEST = _get_tmp_dir()

        clone_url = _construct_url(HG_REPO)

        stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, DEST)

        stdout, stderr = _add_files_and_push('hg', DEST,

                                    clone_url='http://%s/tmp' % HOST,

                                    ignoreReturnCode = True)

        assert 'HTTP Error 404: Not Found' in stderr

    def test_push_back_to_wrong_url_git(self):

        DEST = _get_tmp_dir()

        clone_url = _construct_url(GIT_REPO)

        stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, DEST)

        stdout, stderr = _add_files_and_push('git', DEST,

                                    clone_url='http://%s/tmp' % HOST,

                                    ignoreReturnCode = True)

        assert 'not found' in stderr

    def test_clone_and_create_lock_hg(self):

        # enable locking

        r = Repository.get_by_repo_name(HG_REPO)

        r.enable_locking = True

        Session().commit()

        # clone

        clone_url = _construct_url(HG_REPO)

        stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, _get_tmp_dir())

        #check if lock was made

        r = Repository.get_by_repo_name(HG_REPO)

        assert r.locked[0] == User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id

    def test_clone_and_create_lock_git(self):

        # enable locking

        r = Repository.get_by_repo_name(GIT_REPO)

        r.enable_locking = True

        Session().commit()

        # clone

        clone_url = _construct_url(GIT_REPO)

        stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, _get_tmp_dir())

        #check if lock was made

        r = Repository.get_by_repo_name(GIT_REPO)

        assert r.locked[0] == User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id

    def test_clone_after_repo_was_locked_hg(self):

        #lock repo

        r = Repository.get_by_repo_name(HG_REPO)

        Repository.lock(r, User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id)

        #pull fails since repo is locked

        clone_url = _construct_url(HG_REPO)

        stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, _get_tmp_dir(), ignoreReturnCode=True)

        msg = ("""abort: HTTP Error 423: Repository `%s` locked by user `%s`"""

                % (HG_REPO, TEST_USER_ADMIN_LOGIN))

        assert msg in stderr

    def test_clone_after_repo_was_locked_git(self):

        #lock repo

        r = Repository.get_by_repo_name(GIT_REPO)

        Repository.lock(r, User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id)

        #pull fails since repo is locked

        clone_url = _construct_url(GIT_REPO)

        stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, _get_tmp_dir(), ignoreReturnCode=True)

        msg = ("""The requested URL returned error: 423""")

        assert msg in stderr

    def test_push_on_locked_repo_by_other_user_hg(self):

        #clone some temp

        DEST = _get_tmp_dir()

        clone_url = _construct_url(HG_REPO)

        stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, DEST)

        #lock repo

        r = Repository.get_by_repo_name(HG_REPO)

        # let this user actually push !

        RepoModel().grant_user_permission(repo=r, user=TEST_USER_REGULAR_LOGIN,

                                          perm='repository.write')

        Session().commit()

        Repository.lock(r, User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id)

        #push fails repo is locked by other user !

        stdout, stderr = _add_files_and_push('hg', DEST,

                                             user=TEST_USER_REGULAR_LOGIN,

                                             passwd=TEST_USER_REGULAR_PASS,

                                             ignoreReturnCode=True)

        msg = ("""abort: HTTP Error 423: Repository `%s` locked by user `%s`"""

                % (HG_REPO, TEST_USER_ADMIN_LOGIN))

        assert msg in stderr

    def test_push_on_locked_repo_by_other_user_git(self):

        #clone some temp

        DEST = _get_tmp_dir()

        clone_url = _construct_url(GIT_REPO)

        stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, DEST)

        #lock repo

        r = Repository.get_by_repo_name(GIT_REPO)

        # let this user actually push !

        RepoModel().grant_user_permission(repo=r, user=TEST_USER_REGULAR_LOGIN,

                                          perm='repository.write')

        Session().commit()

        Repository.lock(r, User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id)

        #push fails repo is locked by other user !

        stdout, stderr = _add_files_and_push('git', DEST,

                                             user=TEST_USER_REGULAR_LOGIN,

                                             passwd=TEST_USER_REGULAR_PASS,

                                             ignoreReturnCode=True)

        err = 'Repository `%s` locked by user `%s`' % (GIT_REPO, TEST_USER_ADMIN_LOGIN)

        assert err in stderr

        #TODO: fix this somehow later on Git, Git is stupid and even if we throw

        #back 423 to it, it makes ANOTHER request and we fail there with 405 :/

        msg = ("""abort: HTTP Error 423: Repository `%s` locked by user `%s`"""

                % (GIT_REPO, TEST_USER_ADMIN_LOGIN))

        #msg = "405 Method Not Allowed"

        #assert msg in stderr

    def test_push_unlocks_repository_hg(self):

        # enable locking

        fork_name = '%s_fork%s' % (HG_REPO, _RandomNameSequence().next())

        fixture.create_fork(HG_REPO, fork_name)

        r = Repository.get_by_repo_name(fork_name)

        r.enable_locking = True

        Session().commit()

        #clone some temp

        DEST = _get_tmp_dir()

        clone_url = _construct_url(fork_name)

        stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, DEST)

        #check for lock repo after clone

        r = Repository.get_by_repo_name(fork_name)

        uid = User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id

        assert r.locked[0] == uid

        #push is ok and repo is now unlocked

        stdout, stderr = _add_files_and_push('hg', DEST, clone_url=clone_url.split()[0])

        assert str('remote: Released lock on repo `%s`' % fork_name) in stdout

        #we need to cleanup the Session Here !

        Session.remove()

        r = Repository.get_by_repo_name(fork_name)

        assert r.locked == [None, None]

    #TODO: fix me ! somehow during tests hooks don't get called on Git

    def test_push_unlocks_repository_git(self):

        # enable locking

        fork_name = '%s_fork%s' % (GIT_REPO, _RandomNameSequence().next())

        fixture.create_fork(GIT_REPO, fork_name)

        r = Repository.get_by_repo_name(fork_name)

        r.enable_locking = True

        Session().commit()

        #clone some temp

        DEST = _get_tmp_dir()

        clone_url = _construct_url(fork_name)

        stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, DEST)

        #check for lock repo after clone

        r = Repository.get_by_repo_name(fork_name)

        assert r.locked[0] == User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id

        #push is ok and repo is now unlocked

        stdout, stderr = _add_files_and_push('git', DEST, clone_url=clone_url.split()[0])

        _check_proper_git_push(stdout, stderr)

        assert ('remote: Released lock on repo `%s`' % fork_name) in stderr

        #we need to cleanup the Session Here !

        Session.remove()

        r = Repository.get_by_repo_name(fork_name)

        assert r.locked == [None, None]

    def test_ip_restriction_hg(self):

        user_model = UserModel()

        try:

            user_model.add_extra_ip(TEST_USER_ADMIN_LOGIN, '10.10.10.10/32')

            Session().commit()

            clone_url = _construct_url(HG_REPO)

            stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, _get_tmp_dir(), ignoreReturnCode=True)

            assert 'abort: HTTP Error 403: Forbidden' in stderr

        finally:

            #release IP restrictions

            for ip in UserIpMap.query():

                UserIpMap.delete(ip.ip_id)

            Session().commit()

        # IP permissions are cached, need to wait for the cache in the server process to expire