For a full index rebuild, run::

    paster make-index my.ini -f

The ``--repo-location`` option allows the location of the repositories to be overridden;

usually, the location is retrieved from the Kallithea database.

The ``--index-only`` option can be used to limit the indexed repositories to a comma-separated list::

    paster make-index my.ini --index-only=vcs,kallithea

To keep your index up-to-date it is necessary to do periodic index builds;

for this, it is recommended to use a crontab entry. Example::

    0  3  *  *  *  /path/to/virtualenv/bin/paster make-index /path/to/kallithea/my.ini

When using incremental mode (the default), Whoosh will check the last

modification date of each file and add it to be reindexed if a newer file is

available. The indexing daemon checks for any removed files and removes them

from index.

If you want to rebuild the index from scratch, you can use the ``-f`` flag as above,

or in the admin panel you can check the "build from scratch" checkbox.

.. _ldap-setup:

Setting up LDAP support

-----------------------

Kallithea supports LDAP authentication. In order

to use LDAP, you have to install the python-ldap_ package. This package is

available via PyPI, so you can install it by running::

    pip install python-ldap

.. note:: ``python-ldap`` requires some libraries to be installed on

          your system, so before installing it check that you have at

          least the ``openldap`` and ``sasl`` libraries.

Choose *Admin > Authentication*, click the ``kallithea.lib.auth_modules.auth_ldap`` button

and then *Save*, to enable the LDAP plugin and configure its settings.

Here's a typical LDAP setup::

 Connection settings

 Enable LDAP          = checked

 Host                 = host.example.com

 Account              = <account>

 Password             = <password>

 Connection Security  = LDAPS connection

 Certificate Checks   = DEMAND

 Search settings

 Base DN              = CN=users,DC=host,DC=example,DC=org

 LDAP Filter          = (&(objectClass=user)(!(objectClass=computer)))

 LDAP Search Scope    = SUBTREE

 Attribute mappings

 Login Attribute      = uid

 First Name Attribute = firstName

 Last Name Attribute  = lastName

 Email Attribute      = mail

If your user groups are placed in an Organisation Unit (OU) structure, the Search Settings configuration differs::

 Search settings

 Base DN              = DC=host,DC=example,DC=org

 LDAP Filter          = (&(memberOf=CN=your user group,OU=subunit,OU=unit,DC=host,DC=example,DC=org)(objectClass=user))

 LDAP Search Scope    = SUBTREE

.. _enable_ldap:

Enable LDAP : required

    Whether to use LDAP for authenticating users.

.. _ldap_host:

Host : required

    LDAP server hostname or IP address. Can be also a comma separated

    list of servers to support LDAP fail-over.

.. _Port:

.. _ldap_account:

Account : optional

    Only required if the LDAP server does not allow anonymous browsing of

    records.  This should be a special account for record browsing.  This

    will require `LDAP Password`_ below.

.. _LDAP Password:

Password : optional

    Only required if the LDAP server does not allow anonymous browsing of

    records.

.. _Enable LDAPS:

Connection Security : required

    Defines the connection to LDAP server

.. _Certificate Checks:

Certificate Checks : optional

    How SSL certificates verification is handled -- this is only useful when

    `Enable LDAPS`_ is enabled.  Only DEMAND or HARD offer full SSL security

    with mandatory certificate validation, while the other options are

    susceptible to man-in-the-middle attacks.

    NEVER

        A serve certificate will never be requested or checked.

    ALLOW

        A server certificate is requested.  Failure to provide a

        certificate or providing a bad certificate will not terminate the

        session.

    TRY

        A server certificate is requested.  Failure to provide a

        certificate does not halt the session; providing a bad certificate

        halts the session.

    DEMAND

        A server certificate is requested and must be provided and

        authenticated for the session to proceed.

    HARD

        The same as DEMAND.

.. _Custom CA Certificates:

Custom CA Certificates : optional

    Directory used by OpenSSL to find CAs for validating the LDAP server certificate.

    Python 2.7.10 and later default to using the system certificate store, and

    this should thus not be necessary when using certificates signed by a CA

    trusted by the system.

    It can be set to something like `/etc/openldap/cacerts` on older systems or

    if using self-signed certificates.

.. _Base DN:

Base DN : required

    The Distinguished Name (DN) where searches for users will be performed.

    Searches can be controlled by `LDAP Filter`_ and `LDAP Search Scope`_.

.. _LDAP Filter:

LDAP Filter : optional

# it under the terms of the GNU General Public License as published by

# the Free Software Foundation, either version 3 of the License, or

# (at your option) any later version.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program.  If not, see <http://www.gnu.org/licenses/>.

"""

kallithea.lib.auth_modules.auth_ldap

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Kallithea authentication plugin for LDAP

This file was forked by the Kallithea project in July 2014.

Original author and date, and relevant copyright and licensing information is below:

:created_on: Created on Nov 17, 2010

:author: marcink

:copyright: (c) 2013 RhodeCode GmbH, and others.

:license: GPLv3, see LICENSE.md for more details.

"""

import logging

import traceback

from kallithea.lib import auth_modules

from kallithea.lib.compat import hybrid_property

from kallithea.lib.utils2 import safe_unicode, safe_str

from kallithea.lib.exceptions import (

    LdapConnectionError, LdapUsernameError, LdapPasswordError, LdapImportError

)

from kallithea.model.db import User

log = logging.getLogger(__name__)

try:

    import ldap

except ImportError:

    # means that python-ldap is not installed

    ldap = None

class AuthLdap(object):

                 tls_kind='PLAIN', tls_reqcert='DEMAND', cacertdir=None, ldap_version=3,

                 ldap_filter='(&(objectClass=user)(!(objectClass=computer)))',

                 search_scope='SUBTREE', attr_login='uid'):

        if ldap is None:

            raise LdapImportError

        self.ldap_version = ldap_version

        self.TLS_KIND = tls_kind

        OPT_X_TLS_DEMAND = 2

        self.TLS_REQCERT = getattr(ldap, 'OPT_X_TLS_%s' % tls_reqcert,

                                   OPT_X_TLS_DEMAND)

        self.cacertdir = cacertdir

        self.LDAP_BIND_DN = safe_str(bind_dn)

        self.LDAP_BIND_PASS = safe_str(bind_pass)

        self.BASE_DN = safe_str(base_dn)

        self.LDAP_FILTER = safe_str(ldap_filter)

        self.SEARCH_SCOPE = getattr(ldap, 'SCOPE_%s' % search_scope)

        self.attr_login = attr_login

    def authenticate_ldap(self, username, password):

        """

        Authenticate a user via LDAP and return his/her LDAP properties.

        Raises AuthenticationError if the credentials are rejected, or

        EnvironmentError if the LDAP server can't be reached.

        :param username: username

        :param password: password

        """

        if not password:

            log.debug("Attempt to authenticate LDAP user "

                      "with blank password rejected.")

            raise LdapPasswordError()

        if "," in username:

            raise LdapUsernameError("invalid character in username: ,")

        try:

            if self.cacertdir:

                if hasattr(ldap, 'OPT_X_TLS_CACERTDIR'):

                    ldap.set_option(ldap.OPT_X_TLS_CACERTDIR, self.cacertdir)

                else:

                    log.debug("OPT_X_TLS_CACERTDIR is not available - can't set %s", self.cacertdir)

            ldap.set_option(ldap.OPT_REFERRALS, ldap.OPT_OFF)

            ldap.set_option(ldap.OPT_RESTART, ldap.OPT_ON)

            ldap.set_option(ldap.OPT_TIMEOUT, 20)

            ldap.set_option(ldap.OPT_NETWORK_TIMEOUT, 10)

            ldap.set_option(ldap.OPT_TIMELIMIT, 15)

            if self.TLS_KIND != 'PLAIN':

                ldap.set_option(ldap.OPT_X_TLS_REQUIRE_CERT, self.TLS_REQCERT)

            server = ldap.initialize(self.LDAP_SERVER)

            if self.ldap_version == 2:

                server.protocol = ldap.VERSION2

            else:

                server.protocol = ldap.VERSION3

            if self.TLS_KIND == 'START_TLS':

                server.start_tls_s()

            if self.LDAP_BIND_DN and self.LDAP_BIND_PASS:

                log.debug('Trying simple_bind with password and given DN: %s',

                          self.LDAP_BIND_DN)

                server.simple_bind_s(self.LDAP_BIND_DN, self.LDAP_BIND_PASS)

            filter_ = '(&%s(%s=%s))' % (self.LDAP_FILTER, self.attr_login,

                                        username)

            log.debug("Authenticating %r filter %s at %s", self.BASE_DN,

                      filter_, self.LDAP_SERVER)

            lobjects = server.search_ext_s(self.BASE_DN, self.SEARCH_SCOPE,

                                           filter_)

            if not lobjects:

                raise ldap.NO_SUCH_OBJECT()

            for (dn, _attrs) in lobjects:

                if dn is None:

                    continue

                try:

                    log.debug('Trying simple bind with %s', dn)

                    server.simple_bind_s(dn, safe_str(password))

                    results = server.search_ext_s(dn, ldap.SCOPE_BASE,

                                                  '(objectClass=*)')

                    if len(results) == 1:

                        dn_, attrs = results[0]

                        assert dn_ == dn

                        return dn, attrs

                except ldap.INVALID_CREDENTIALS:

                    continue # accept authentication as another ldap user with same username

            log.debug("No matching LDAP objects for authentication "

            raise LdapPasswordError()

        except ldap.NO_SUCH_OBJECT:

            raise LdapUsernameError()

        except ldap.SERVER_DOWN:

            # [0] might be {'info': "TLS error -8179:Peer's Certificate issuer is not recognized.", 'desc': "Can't contact LDAP server"}

            raise LdapConnectionError("LDAP can't connect to authentication server")

class KallitheaAuthPlugin(auth_modules.KallitheaExternalAuthPlugin):

    def __init__(self):

        self._logger = logging.getLogger(__name__)

        self._tls_kind_values = ["PLAIN", "LDAPS", "START_TLS"]

        self._tls_reqcert_values = ["NEVER", "ALLOW", "TRY", "DEMAND", "HARD"]

        self._search_scopes = ["BASE", "ONELEVEL", "SUBTREE"]

    @hybrid_property

    def name(self):

        return "ldap"

    def settings(self):

        settings = [

            {

                "name": "host",

                "validator": self.validators.UnicodeString(strip=True),

                "type": "string",

                "description": "Host of the LDAP Server",

                "formname": "LDAP Host"

            },

            {

                "name": "port",

                "type": "string",

            },

            {

                "name": "dn_user",

                "validator": self.validators.UnicodeString(strip=True),

                "type": "string",

                "description": "User to connect to LDAP",

                "formname": "Account"

            },

            {

                "name": "dn_pass",

                "validator": self.validators.UnicodeString(strip=True),

                "type": "password",

                "description": "Password to connect to LDAP",

                "formname": "Password"

            },

            {

                "name": "tls_kind",

                "validator": self.validators.OneOf(self._tls_kind_values),

                "type": "select",

                "values": self._tls_kind_values,

                "description": "TLS Type",

                "default": 'PLAIN',

                "formname": "Connection Security"

            },

            {

                "name": "tls_reqcert",

                "validator": self.validators.OneOf(self._tls_reqcert_values),

                "type": "select",

                "values": self._tls_reqcert_values,

                "description": "Require Cert over TLS?",

                "formname": "Certificate Checks"

            },

            {

                "name": "cacertdir",

                "validator": self.validators.UnicodeString(strip=True),

                "type": "string",

                "description": "Optional: Custom CA certificate directory for validating LDAPS",

                "formname": "Custom CA Certificates"

            },

            {

                "name": "base_dn",

                "validator": self.validators.UnicodeString(strip=True),

                "type": "string",

                "description": "Base DN to search (e.g., dc=mydomain,dc=com)",

                "formname": "Base DN"

            },

            {

                "name": "filter",