configuration file to use this other database. Kallithea currently supports

PostgreSQL, SQLite and MySQL databases. Create the database by running

the following command::

    paster setup-db my.ini

This will prompt you for a "root" path. This "root" path is the location where

Kallithea will store all of its repositories on the current machine. After

entering this "root" path ``setup-db`` will also prompt you for a username

and password for the initial admin account which ``setup-db`` sets

up for you.

The ``setup-db`` values can also be given on the command line.

Example::

    paster setup-db my.ini --user=nn --password=secret --email=nn@example.com --repos=/srv/repos

The ``setup-db`` command will create all needed tables and an

admin account. When choosing a root path you can either use a new

empty location, or a location which already contains existing

repositories. If you choose a location which contains existing

repositories Kallithea will add all of the repositories at the chosen

location to its database.  (Note: make sure you specify the correct

path to the root).

.. note:: the given path for Mercurial_ repositories **must** be write

          accessible for the application. It's very important since

          the Kallithea web interface will work without write access,

          but when trying to do a push it will fail with permission

          denied errors unless it has write access.

You are now ready to use Kallithea. To run it simply execute::

    paster serve my.ini

- This command runs the Kallithea server. The web app should be available at

  http://127.0.0.1:5000. The IP address and port is configurable via the

  configuration file created in the previous step.

- Log in to Kallithea using the admin account created when running ``setup-db``.

- The default permissions on each repository is read, and the owner is admin.

  Remember to update these if needed.

- In the admin panel you can toggle LDAP, anonymous, and permissions

  settings, as well as edit more advanced options on users and

  repositories.

Using Kallithea with SSH

------------------------

Kallithea currently only hosts repositories using http and https. (The addition

of ssh hosting is a planned future feature.) However you can easily use ssh in

parallel with Kallithea. (Repository access via ssh is a standard "out of

the box" feature of Mercurial_ and you can use this to access any of the

repositories that Kallithea is hosting. See PublishingRepositories_)

Kallithea repository structures are kept in directories with the same name

as the project. When using repository groups, each group is a subdirectory.

This allows you to easily use ssh for accessing repositories.

In order to use ssh you need to make sure that your web server and the users'

login accounts have the correct permissions set on the appropriate directories.

.. note:: These permissions are independent of any permissions you

          have set up using the Kallithea web interface.

If your main directory (the same as set in Kallithea settings) is for

example set to ``/srv/repos`` and the repository you are using is

named ``kallithea``, then to clone via ssh you should run::

    hg clone ssh://user@kallithea.example.com/srv/repos/kallithea

Using other external tools such as mercurial-server_ or using ssh key-based

authentication is fully supported.

.. note:: In an advanced setup, in order for your ssh access to use

          the same permissions as set up via the Kallithea web

          interface, you can create an authentication hook to connect

          to the Kallithea db and run check functions for permissions

          against that.

Setting up Whoosh full text search

----------------------------------

Kallithea provides full text search of repositories using `Whoosh`__.

.. __: https://pythonhosted.org/Whoosh/

For an incremental index build, run::

    paster make-index my.ini

For a full index rebuild, run::

    paster make-index my.ini -f

usually, the location is retrieved from the Kallithea database.

The ``--index-only`` option can be used to limit the indexed repositories to a comma-separated list::

    paster make-index my.ini --index-only=vcs,kallithea

To keep your index up-to-date it is necessary to do periodic index builds;

for this, it is recommended to use a crontab entry. Example::

    0  3  *  *  *  /path/to/virtualenv/bin/paster make-index /path/to/kallithea/my.ini

When using incremental mode (the default), Whoosh will check the last

modification date of each file and add it to be reindexed if a newer file is

available. The indexing daemon checks for any removed files and removes them

from index.

If you want to rebuild the index from scratch, you can use the ``-f`` flag as above,

or in the admin panel you can check the "build from scratch" checkbox.

.. _ldap-setup:

Setting up LDAP support

-----------------------

Kallithea supports LDAP authentication. In order

to use LDAP, you have to install the python-ldap_ package. This package is

available via PyPI, so you can install it by running::

    pip install python-ldap

.. note:: ``python-ldap`` requires some libraries to be installed on

          your system, so before installing it check that you have at

          least the ``openldap`` and ``sasl`` libraries.

Choose *Admin > Authentication*, click the ``kallithea.lib.auth_modules.auth_ldap`` button

and then *Save*, to enable the LDAP plugin and configure its settings.

Here's a typical LDAP setup::

 Connection settings

 Enable LDAP          = checked

 Host                 = host.example.com

 Port                 = 389

 Account              = <account>

 Password             = <password>

 Connection Security  = LDAPS connection

 Certificate Checks   = DEMAND

 Search settings

 Base DN              = CN=users,DC=host,DC=example,DC=org

 LDAP Filter          = (&(objectClass=user)(!(objectClass=computer)))

 LDAP Search Scope    = SUBTREE

 Attribute mappings

 Login Attribute      = uid

 First Name Attribute = firstName

 Last Name Attribute  = lastName

 Email Attribute      = mail

If your user groups are placed in an Organisation Unit (OU) structure, the Search Settings configuration differs::

 Search settings

 Base DN              = DC=host,DC=example,DC=org

 LDAP Filter          = (&(memberOf=CN=your user group,OU=subunit,OU=unit,DC=host,DC=example,DC=org)(objectClass=user))

 LDAP Search Scope    = SUBTREE

.. _enable_ldap:

Enable LDAP : required

    Whether to use LDAP for authenticating users.

.. _ldap_host:

Host : required

    LDAP server hostname or IP address. Can be also a comma separated

    list of servers to support LDAP fail-over.

.. _Port:

Port : required

    389 for un-encrypted LDAP, 636 for SSL-encrypted LDAP.

.. _ldap_account:

Account : optional

    Only required if the LDAP server does not allow anonymous browsing of

    records.  This should be a special account for record browsing.  This

    will require `LDAP Password`_ below.

.. _LDAP Password:

Password : optional

    Only required if the LDAP server does not allow anonymous browsing of

    records.

        if not explicit:

            p = _choose_perm(p, cur_perm)

        permissions[GK][rg_k] = p

    #======================================================================

    # !! PERMISSIONS FOR USER GROUPS !!

    #======================================================================

    # user group for user group permissions

    user_group_user_groups_perms = \

     Session().query(UserGroupUserGroupToPerm, Permission, UserGroup) \

     .join((UserGroup, UserGroupUserGroupToPerm.target_user_group_id

            == UserGroup.users_group_id)) \

     .join((Permission, UserGroupUserGroupToPerm.permission_id

            == Permission.permission_id)) \

     .join((UserGroupMember, UserGroupUserGroupToPerm.user_group_id

            == UserGroupMember.users_group_id)) \

     .filter(UserGroupMember.user_id == user_id) \

     .join((UserGroup, UserGroupMember.users_group_id ==

            UserGroup.users_group_id), aliased=True, from_joinpoint=True) \

     .filter(UserGroup.users_group_active == True) \

     .all()

    multiple_counter = collections.defaultdict(int)

    for perm in user_group_user_groups_perms:

        g_k = perm.UserGroupUserGroupToPerm.target_user_group.users_group_name

        multiple_counter[g_k] += 1

        p = perm.Permission.permission_name

        cur_perm = permissions[UK][g_k]

        if multiple_counter[g_k] > 1:

            p = _choose_perm(p, cur_perm)

        permissions[UK][g_k] = p

    #user explicit permission for user groups

    user_user_groups_perms = Permission.get_default_user_group_perms(user_id)

    for perm in user_user_groups_perms:

        u_k = perm.UserUserGroupToPerm.user_group.users_group_name

        p = perm.Permission.permission_name

        cur_perm = permissions[UK][u_k]

        if not explicit:

            p = _choose_perm(p, cur_perm)

        permissions[UK][u_k] = p

    return permissions

def allowed_api_access(controller_name, whitelist=None, api_key=None):

    """

    Check if given controller_name is in whitelist API access

    """

    if not whitelist:

        from kallithea import CONFIG

        whitelist = aslist(CONFIG.get('api_access_controllers_whitelist'),

                           sep=',')

        log.debug('whitelist of API access is: %s', whitelist)

    api_access_valid = controller_name in whitelist

    if api_access_valid:

        log.debug('controller:%s is in API whitelist', controller_name)

    else:

        msg = 'controller: %s is *NOT* in API whitelist' % (controller_name)

        if api_key:

            #if we use API key and don't have access it's a warning

            log.warning(msg)

        else:

            log.debug(msg)

    return api_access_valid

class AuthUser(object):

    """

    Represents a Kallithea user, including various authentication and

    authorization information. Typically used to store the current user,

    but is also used as a generic user information data structure in

    parts of the code, e.g. user management.

    Constructed from a database `User` object, a user ID or cookie dict,

    it looks up the user (if needed) and copies all attributes to itself,

    adding various non-persistent data. If lookup fails but anonymous

    access to Kallithea is enabled, the default user is loaded instead.

    `AuthUser` does not by itself authenticate users and the constructor

    sets the `is_authenticated` field to False. It's up to other parts

    of the code to check e.g. if a supplied password is correct, and if

    so, set `is_authenticated` to True.

    However, `AuthUser` does refuse to load a user that is not `active`.

    """

    def __init__(self, user_id=None, dbuser=None,

            is_external_auth=False):

        self.is_authenticated = False

        self.is_external_auth = is_external_auth

        user_model = UserModel()

        self.anonymous_user = User.get_default_user(cache=True)

        # requested user cannot be found and the default anonymous user is

        # not enabled.

        self.user_id = None

        self.username = None

        self.api_key = None

        self.name = ''

        self.lastname = ''

        self.email = ''

        self.admin = False

        self.inherit_default_permissions = False

        # Look up database user, if necessary.

        if user_id is not None:

            log.debug('Auth User lookup by USER ID %s', user_id)

            dbuser = user_model.get(user_id)

        else:

            # Note: dbuser is allowed to be None.

            log.debug('Auth User lookup by database user %s', dbuser)

        is_user_loaded = self._fill_data(dbuser)

        # If user cannot be found, try falling back to anonymous.

        if not is_user_loaded:

            is_user_loaded =  self._fill_data(self.anonymous_user)

        self.is_default_user = (self.user_id == self.anonymous_user.user_id)

        if not self.username:

            self.username = 'None'

        log.debug('Auth User is now %s', self)

    def _fill_data(self, dbuser):

        """

        Copies database fields from a `db.User` to this `AuthUser`. Does

        not copy `api_keys` and `permissions` attributes.

        Checks that `dbuser` is `active` (and not None) before copying;

        returns True on success.

        """

        if dbuser is not None and dbuser.active:

            log.debug('filling %s data', dbuser)

            for k, v in dbuser.get_dict().iteritems():

                assert k not in ['api_keys', 'permissions']

                setattr(self, k, v)

            return True

        return False

    @LazyProperty

    def permissions(self):

        return self.__get_perms(user=self, cache=False)

    @property

    def api_keys(self):

        return self._get_api_keys()

    def __get_perms(self, user, explicit=True, algo='higherwin', cache=False):

        """

        Fills user permission attribute with permissions taken from database

        works for permissions given for repositories, and for permissions that

        are granted to groups

        :param user: `AuthUser` instance

        :param explicit: In case there are permissions both for user and a group

            that user is part of, explicit flag will define if user will

            explicitly override permissions from group, if it's False it will

            make decision based on the algo

        :param algo: algorithm to decide what permission should be choose if

            it's multiple defined, eg user in two different groups. It also

            decides if explicit flag is turned off how to specify the permission

            for case when user is in a group + have defined separate permission

        """

        user_id = user.user_id

        user_is_admin = user.is_admin

        user_inherit_default_permissions = user.inherit_default_permissions

        log.debug('Getting PERMISSION tree')

        compute = conditional_cache('short_term', 'cache_desc',

                                    condition=cache, func=_cached_perms_data)

        return compute(user_id, user_is_admin,

                       user_inherit_default_permissions, explicit, algo)

    def _get_api_keys(self):

        api_keys = [self.api_key]

        for api_key in UserApiKeys.query() \

                .filter(UserApiKeys.user_id == self.user_id) \

                .filter(or_(UserApiKeys.expires == -1,

                            UserApiKeys.expires >= time.time())).all():

            api_keys.append(api_key.api_key)

        return api_keys

    @property

    def is_admin(self):

        return self.admin

                                             'firstname': 'test',

                                             'lastname': 'test'})

        msg = validators.UniqSystemEmail()()._messages['email_taken']

        response.mustcontain(msg)

    def test_register_err_same_email_case_sensitive(self):

        response = self.app.post(url(controller='login', action='register'),

                                            {'username': 'test_admin_1',

                                             'password': 'test12',

                                             'password_confirmation': 'test12',

                                             'email': TEST_USER_ADMIN_EMAIL.title(),

                                             'firstname': 'test',

                                             'lastname': 'test'})

        msg = validators.UniqSystemEmail()()._messages['email_taken']

        response.mustcontain(msg)

    def test_register_err_wrong_data(self):

        response = self.app.post(url(controller='login', action='register'),

                                            {'username': 'xs',

                                             'password': 'test',

                                             'password_confirmation': 'test',

                                             'email': 'goodmailm',

                                             'firstname': 'test',

                                             'lastname': 'test'})

        self.assertEqual(response.status, '200 OK')

        response.mustcontain('An email address must contain a single @')

        response.mustcontain('Enter a value 6 characters long or more')

    def test_register_err_username(self):

        response = self.app.post(url(controller='login', action='register'),

                                            {'username': 'error user',

                                             'password': 'test12',

                                             'password_confirmation': 'test12',

                                             'email': 'goodmailm',

                                             'firstname': 'test',

                                             'lastname': 'test'})

        response.mustcontain('An email address must contain a single @')

        response.mustcontain('Username may only contain '

                'alphanumeric characters underscores, '

                'periods or dashes and must begin with an '

                'alphanumeric character')

    def test_register_err_case_sensitive(self):

        usr = TEST_USER_ADMIN_LOGIN.title()

        response = self.app.post(url(controller='login', action='register'),

                                            {'username': usr,

                                             'password': 'test12',

                                             'password_confirmation': 'test12',

                                             'email': 'goodmailm',

                                             'firstname': 'test',

                                             'lastname': 'test'})

        response.mustcontain('An email address must contain a single @')

        msg = validators.ValidUsername()._messages['username_exists']

        msg = h.html_escape(msg % {'username': usr})

        response.mustcontain(msg)

    def test_register_special_chars(self):

        response = self.app.post(url(controller='login', action='register'),

                                        {'username': 'xxxaxn',

                                         'password': 'ąćźżąśśśś',

                                         'password_confirmation': 'ąćźżąśśśś',

                                         'email': 'goodmailm@test.plx',

                                         'firstname': 'test',

                                         'lastname': 'test'})

        msg = validators.ValidPassword()._messages['invalid_password']

        response.mustcontain(msg)

    def test_register_password_mismatch(self):

        response = self.app.post(url(controller='login', action='register'),

                                            {'username': 'xs',

                                             'password': '123qwe',

                                             'password_confirmation': 'qwe123',

                                             'email': 'goodmailm@test.plxa',

                                             'firstname': 'test',

                                             'lastname': 'test'})

        msg = validators.ValidPasswordsMatch('password', 'password_confirmation')._messages['password_mismatch']

        response.mustcontain(msg)

    def test_register_ok(self):

        username = 'test_regular4'

        password = 'qweqwe'

        email = 'user4@example.com'

        name = 'testname'

        lastname = 'testlastname'

        response = self.app.post(url(controller='login', action='register'),

                                            {'username': username,

                                             'password': password,

                                             'password_confirmation': password,

                                             'email': email,

                                             'firstname': name,

                                             'lastname': lastname,

        self.assertEqual(response.status, '302 Found')

        self.checkSessionFlash(response, 'You have successfully registered into Kallithea')

        ret = Session().query(User).filter(User.username == 'test_regular4').one()

        self.assertEqual(ret.username, username)

        self.assertEqual(check_password(password, ret.password), True)

        self.assertEqual(ret.email, email)

        self.assertEqual(ret.name, name)

        self.assertEqual(ret.lastname, lastname)

        self.assertNotEqual(ret.api_key, None)

        self.assertEqual(ret.admin, False)

    #==========================================================================

    # PASSWORD RESET

    #==========================================================================

    def test_forgot_password_wrong_mail(self):

        bad_email = 'username%wrongmail.org'

        response = self.app.post(

                        url(controller='login', action='password_reset'),

                            {'email': bad_email, }

        )

        response.mustcontain('An email address must contain a single @')

    def test_forgot_password(self):

        response = self.app.get(url(controller='login',

                                    action='password_reset'))

        self.assertEqual(response.status, '200 OK')

        username = 'test_password_reset_1'

        password = 'qweqwe'

        email = 'username@example.com'

        name = u'passwd'

        lastname = u'reset'

        timestamp = int(time.time())

        new = User()

        new.username = username

        new.password = password

        new.email = email

        new.name = name

        new.lastname = lastname

        new.api_key = generate_api_key()

        Session().add(new)

        Session().commit()

        response = self.app.post(url(controller='login',

                                     action='password_reset'),

                                 {'email': email, })

        self.checkSessionFlash(response, 'A password reset confirmation code has been sent')

        response = response.follow()

        # BAD TOKEN

        token = "bad"

        response = self.app.post(url(controller='login',

                                     action='password_reset_confirmation'),

                                 {'email': email,

                                  'timestamp': timestamp,

                                  'password': "p@ssw0rd",

                                  'password_confirm': "p@ssw0rd",

                                  'token': token,

                                 })

        self.assertEqual(response.status, '200 OK')

        response.mustcontain('Invalid password reset token')

        # GOOD TOKEN

        # TODO: The token should ideally be taken from the mail sent

        # above, instead of being recalculated.

        token = UserModel().get_reset_password_token(

            User.get_by_username(username), timestamp, self.authentication_token())

        response = self.app.get(url(controller='login',

                                    action='password_reset_confirmation',

                                    email=email,

                                    timestamp=timestamp,

                                    token=token))

        self.assertEqual(response.status, '200 OK')

        response.mustcontain("You are about to set a new password for the email address %s" % email)

        response = self.app.post(url(controller='login',

                                     action='password_reset_confirmation'),

                                 {'email': email,

                                  'timestamp': timestamp,

                                  'password': "p@ssw0rd",

                                  'password_confirm': "p@ssw0rd",

                                  'token': token,

                                 })

        self.assertEqual(response.status, '302 Found')

        self.checkSessionFlash(response, 'Successfully updated password')