diff --git a/docs/setup.rst b/docs/administrator_guide/auth.rst copy from docs/setup.rst copy to docs/administrator_guide/auth.rst --- a/docs/setup.rst +++ b/docs/administrator_guide/auth.rst @@ -1,164 +1,18 @@ -.. _setup: - -===== -Setup -===== - - -Setting up Kallithea --------------------- - -First, you will need to create a Kallithea configuration file. Run the -following command to do so:: - - paster make-config Kallithea my.ini - -This will create the file ``my.ini`` in the current directory. This -configuration file contains the various settings for Kallithea, e.g. -proxy port, email settings, usage of static files, cache, Celery -settings, and logging. - -Next, you need to create the databases used by Kallithea. It is recommended to -use PostgreSQL or SQLite (default). If you choose a database other than the -default, ensure you properly adjust the database URL in your ``my.ini`` -configuration file to use this other database. Kallithea currently supports -PostgreSQL, SQLite and MySQL databases. Create the database by running -the following command:: - - paster setup-db my.ini - -This will prompt you for a "root" path. This "root" path is the location where -Kallithea will store all of its repositories on the current machine. After -entering this "root" path ``setup-db`` will also prompt you for a username -and password for the initial admin account which ``setup-db`` sets -up for you. - -The ``setup-db`` values can also be given on the command line. -Example:: - - paster setup-db my.ini --user=nn --password=secret --email=nn@example.com --repos=/srv/repos - -The ``setup-db`` command will create all needed tables and an -admin account. When choosing a root path you can either use a new -empty location, or a location which already contains existing -repositories. If you choose a location which contains existing -repositories Kallithea will add all of the repositories at the chosen -location to its database. (Note: make sure you specify the correct -path to the root). - -.. note:: the given path for Mercurial_ repositories **must** be write - accessible for the application. It's very important since - the Kallithea web interface will work without write access, - but when trying to do a push it will fail with permission - denied errors unless it has write access. - -You are now ready to use Kallithea. To run it simply execute:: - - paster serve my.ini - -- This command runs the Kallithea server. The web app should be available at - http://127.0.0.1:5000. The IP address and port is configurable via the - configuration file created in the previous step. -- Log in to Kallithea using the admin account created when running ``setup-db``. -- The default permissions on each repository is read, and the owner is admin. - Remember to update these if needed. -- In the admin panel you can toggle LDAP, anonymous, and permissions - settings, as well as edit more advanced options on users and - repositories. - - -Extensions ----------- - -Optionally one can create an ``rcextensions`` package that extends Kallithea -functionality. -To generate a skeleton extensions package, run:: - - paster make-rcext my.ini +.. _authentication: -This will create an ``rcextensions`` package next to the specified ``ini`` file. -With ``rcextensions`` it's possible to add additional mapping for whoosh, -stats and add additional code into the push/pull/create/delete repo hooks, -for example for sending signals to build-bots such as Jenkins. - -See the ``__init__.py`` file inside the generated ``rcextensions`` package -for more details. - - -Using Kallithea with SSH ------------------------- - -Kallithea currently only hosts repositories using http and https. (The addition -of ssh hosting is a planned future feature.) However you can easily use ssh in -parallel with Kallithea. (Repository access via ssh is a standard "out of -the box" feature of Mercurial_ and you can use this to access any of the -repositories that Kallithea is hosting. See PublishingRepositories_) - -Kallithea repository structures are kept in directories with the same name -as the project. When using repository groups, each group is a subdirectory. -This allows you to easily use ssh for accessing repositories. - -In order to use ssh you need to make sure that your web server and the users' -login accounts have the correct permissions set on the appropriate directories. - -.. note:: These permissions are independent of any permissions you - have set up using the Kallithea web interface. - -If your main directory (the same as set in Kallithea settings) is for -example set to ``/srv/repos`` and the repository you are using is -named ``kallithea``, then to clone via ssh you should run:: - - hg clone ssh://user@kallithea.example.com/srv/repos/kallithea - -Using other external tools such as mercurial-server_ or using ssh key-based -authentication is fully supported. +==================== +Authentication setup +==================== -.. note:: In an advanced setup, in order for your ssh access to use - the same permissions as set up via the Kallithea web - interface, you can create an authentication hook to connect - to the Kallithea db and run check functions for permissions - against that. - - -Setting up Whoosh full text search ----------------------------------- - -Kallithea provides full text search of repositories using `Whoosh`__. - -.. __: https://whoosh.readthedocs.io/en/latest/ - -For an incremental index build, run:: - - paster make-index my.ini - -For a full index rebuild, run:: - - paster make-index my.ini -f - -The ``--repo-location`` option allows the location of the repositories to be overriden; -usually, the location is retrieved from the Kallithea database. - -The ``--index-only`` option can be used to limit the indexed repositories to a comma-separated list:: - - paster make-index my.ini --index-only=vcs,kallithea - -To keep your index up-to-date it is necessary to do periodic index builds; -for this, it is recommended to use a crontab entry. Example:: - - 0 3 * * * /path/to/virtualenv/bin/paster make-index /path/to/kallithea/my.ini - -When using incremental mode (the default), Whoosh will check the last -modification date of each file and add it to be reindexed if a newer file is -available. The indexing daemon checks for any removed files and removes them -from index. - -If you want to rebuild the index from scratch, you can use the ``-f`` flag as above, -or in the admin panel you can check the "build from scratch" checkbox. +Users can be authenticated in different ways. By default, Kallithea +uses its internal user database. Alternative authentication +methods include LDAP, PAM, Crowd, and container-based authentication. .. _ldap-setup: -Setting up LDAP support ------------------------ + +LDAP Authentication +------------------- Kallithea supports LDAP authentication. In order to use LDAP, you have to install the python-ldap_ package. This package is @@ -178,10 +32,9 @@ Here's a typical LDAP setup:: Connection settings Enable LDAP = checked Host = host.example.com - Port = 389 Account = Password = - Connection Security = LDAPS connection + Connection Security = LDAPS Certificate Checks = DEMAND Search settings @@ -215,8 +68,9 @@ Host : required .. _Port: -Port : required - 389 for un-encrypted LDAP, 636 for SSL-encrypted LDAP. +Port : optional + Defaults to 389 for PLAIN un-encrypted LDAP and START_TLS. + Defaults to 636 for LDAPS. .. _ldap_account: @@ -236,26 +90,27 @@ Password : optional Connection Security : required Defines the connection to LDAP server - No encryption - Plain non encrypted connection + PLAIN + Plain unencrypted LDAP connection. + This will by default use `Port`_ 389. - LDAPS connection - Enable LDAPS connections. It will likely require `Port`_ to be set to - a different value (standard LDAPS port is 636). When LDAPS is enabled - then `Certificate Checks`_ is required. + LDAPS + Use secure LDAPS connections according to `Certificate + Checks`_ configuration. + This will by default use `Port`_ 636. - START_TLS on LDAP connection - START TLS connection + START_TLS + Use START TLS according to `Certificate Checks`_ configuration on an + apparently "plain" LDAP connection. + This will by default use `Port`_ 389. .. _Certificate Checks: Certificate Checks : optional How SSL certificates verification is handled -- this is only useful when `Enable LDAPS`_ is enabled. Only DEMAND or HARD offer full SSL security - while the other options are susceptible to man-in-the-middle attacks. SSL - certificates can be installed to /etc/openldap/cacerts so that the - DEMAND or HARD options can be used with self-signed certificates or - certificates that do not have traceable certificates of authority. + with mandatory certificate validation, while the other options are + susceptible to man-in-the-middle attacks. NEVER A serve certificate will never be requested or checked. @@ -277,6 +132,16 @@ Certificate Checks : optional HARD The same as DEMAND. +.. _Custom CA Certificates: + +Custom CA Certificates : optional + Directory used by OpenSSL to find CAs for validating the LDAP server certificate. + Python 2.7.10 and later default to using the system certificate store, and + this should thus not be necessary when using certificates signed by a CA + trusted by the system. + It can be set to something like `/etc/openldap/cacerts` on older systems or + if using self-signed certificates. + .. _Base DN: Base DN : required @@ -347,7 +212,7 @@ information check out the Kallithea logs will be saved there. Active Directory -'''''''''''''''' +^^^^^^^^^^^^^^^^ Kallithea can use Microsoft Active Directory for user authentication. This is done through an LDAP or LDAPS connection to Active Directory. The @@ -384,24 +249,24 @@ It's also possible for an administrator permissions before the user logs in for the first time, using the :ref:`create-user` API. Container-based authentication -'''''''''''''''''''''''''''''' +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ In a container-based authentication setup, Kallithea reads the user name from the ``REMOTE_USER`` server variable provided by the WSGI container. -After setting up your container (see `Apache with mod_wsgi`_), you'll need +After setting up your container (see :ref:`apache_mod_wsgi`), you'll need to configure it to require authentication on the location configured for Kallithea. Proxy pass-through authentication -''''''''''''''''''''''''''''''''' +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ In a proxy pass-through authentication setup, Kallithea reads the user name from the ``X-Forwarded-User`` request header, which should be configured to be sent by the reverse-proxy server. -After setting up your proxy solution (see `Apache virtual host reverse proxy example`_, -`Apache as subdirectory`_ or `Nginx virtual host example`_), you'll need to +After setting up your proxy solution (see :ref:`apache_virtual_host_reverse_proxy`, +:ref:`apache_subdirectory` or :ref:`nginx_virtual_host`), you'll need to configure the authentication and add the username in a request header named ``X-Forwarded-User``. @@ -428,6 +293,73 @@ reverse-proxy setup with basic auth: RequestHeader set X-Forwarded-User %{RU}e +Setting metadata in container/reverse-proxy +""""""""""""""""""""""""""""""""""""""""""" +When a new user account is created on the first login, Kallithea has no information about +the user's email and full name. So you can set some additional request headers like in the +example below. In this example the user is authenticated via Kerberos and an Apache +mod_python fixup handler is used to get the user information from a LDAP server. But you +could set the request headers however you want. + +.. code-block:: apache + + + ProxyPass http://127.0.0.1:5000/someprefix + ProxyPassReverse http://127.0.0.1:5000/someprefix + SetEnvIf X-Url-Scheme https HTTPS=1 + + AuthName "Kerberos Login" + AuthType Kerberos + Krb5Keytab /etc/apache2/http.keytab + KrbMethodK5Passwd off + KrbVerifyKDC on + Require valid-user + + PythonFixupHandler ldapmetadata + + RequestHeader set X_REMOTE_USER %{X_REMOTE_USER}e + RequestHeader set X_REMOTE_EMAIL %{X_REMOTE_EMAIL}e + RequestHeader set X_REMOTE_FIRSTNAME %{X_REMOTE_FIRSTNAME}e + RequestHeader set X_REMOTE_LASTNAME %{X_REMOTE_LASTNAME}e + + +.. code-block:: python + + from mod_python import apache + import ldap + + LDAP_SERVER = "ldaps://server.mydomain.com:636" + LDAP_USER = "" + LDAP_PASS = "" + LDAP_ROOT = "dc=mydomain,dc=com" + LDAP_FILTER = "sAMAccountName=%s" + LDAP_ATTR_LIST = ['sAMAccountName','givenname','sn','mail'] + + def fixuphandler(req): + if req.user is None: + # no user to search for + return apache.OK + else: + try: + if('\\' in req.user): + username = req.user.split('\\')[1] + elif('@' in req.user): + username = req.user.split('@')[0] + else: + username = req.user + l = ldap.initialize(LDAP_SERVER) + l.simple_bind_s(LDAP_USER, LDAP_PASS) + r = l.search_s(LDAP_ROOT, ldap.SCOPE_SUBTREE, LDAP_FILTER % username, attrlist=LDAP_ATTR_LIST) + + req.subprocess_env['X_REMOTE_USER'] = username + req.subprocess_env['X_REMOTE_EMAIL'] = r[0][1]['mail'][0].lower() + req.subprocess_env['X_REMOTE_FIRSTNAME'] = "%s" % r[0][1]['givenname'][0] + req.subprocess_env['X_REMOTE_LASTNAME'] = "%s" % r[0][1]['sn'][0] + except Exception, e: + apache.log_error("error getting data from ldap %s" % str(e), apache.APLOG_ERR) + + return apache.OK + .. note:: If you enable proxy pass-through authentication, make sure your server is only accessible through the proxy. Otherwise, any client would be able to @@ -435,384 +367,4 @@ reverse-proxy setup with basic auth: using any account of their liking. -Integration with issue trackers -------------------------------- - -Kallithea provides a simple integration with issue trackers. It's possible -to define a regular expression that will match an issue ID in commit messages, -and have that replaced with a URL to the issue. To enable this simply -uncomment the following variables in the ini file:: - - issue_pat = (?:^#|\s#)(\w+) - issue_server_link = https://issues.example.com/{repo}/issue/{id} - issue_prefix = # - -``issue_pat`` is the regular expression describing which strings in -commit messages will be treated as issue references. A match group in -parentheses should be used to specify the actual issue id. - -The default expression matches issues in the format ``#``, e.g., ``#300``. - -Matched issue references are replaced with the link specified in -``issue_server_link``. ``{id}`` is replaced with the issue ID, and -``{repo}`` with the repository name. Since the # is stripped away, -``issue_prefix`` is prepended to the link text. ``issue_prefix`` doesn't -necessarily need to be ``#``: if you set issue prefix to ``ISSUE-`` this will -generate a URL in the format: - -.. code-block:: html - - ISSUE-300 - -If needed, more than one pattern can be specified by appending a unique suffix to -the variables. For example:: - - issue_pat_wiki = (?:wiki-)(.+) - issue_server_link_wiki = https://wiki.example.com/{id} - issue_prefix_wiki = WIKI- - -With these settings, wiki pages can be referenced as wiki-some-id, and every -such reference will be transformed into: - -.. code-block:: html - - WIKI-some-id - - -Hook management ---------------- - -Hooks can be managed in similar way to that used in ``.hgrc`` files. -To manage hooks, choose *Admin > Settings > Hooks*. - -The built-in hooks cannot be modified, though they can be enabled or disabled in the *VCS* section. - -To add another custom hook simply fill in the first textbox with -``.`` and the second with the hook path. Example hooks -can be found in ``kallithea.lib.hooks``. - - -Changing default encoding -------------------------- - -By default, Kallithea uses UTF-8 encoding. -This is configurable as ``default_encoding`` in the .ini file. -This affects many parts in Kallithea including user names, filenames, and -encoding of commit messages. In addition Kallithea can detect if the ``chardet`` -library is installed. If ``chardet`` is detected Kallithea will fallback to it -when there are encode/decode errors. - - -Celery configuration --------------------- - -Kallithea can use the distributed task queue system Celery_ to run tasks like -cloning repositories or sending emails. - -Kallithea will in most setups work perfectly fine out of the box (without -Celery), executing all tasks in the web server process. Some tasks can however -take some time to run and it can be better to run such tasks asynchronously in -a separate process so the web server can focus on serving web requests. - -For installation and configuration of Celery, see the `Celery documentation`_. -Note that Celery requires a message broker service like RabbitMQ_ (recommended) -or Redis_. - -The use of Celery is configured in the Kallithea ini configuration file. -To enable it, simply set:: - - use_celery = true - -and add or change the ``celery.*`` and ``broker.*`` configuration variables. - -Remember that the ini files use the format with '.' and not with '_' like -Celery. So for example setting `BROKER_HOST` in Celery means setting -`broker.host` in the configuration file. - -To start the Celery process, run:: - - paster celeryd - -.. note:: - Make sure you run this command from the same virtualenv, and with the same - user that Kallithea runs. - - -HTTPS support -------------- - -Kallithea will by default generate URLs based on the WSGI environment. - -Alternatively, you can use some special configuration settings to control -directly which scheme/protocol Kallithea will use when generating URLs: - -- With ``https_fixup = true``, the scheme will be taken from the - ``X-Url-Scheme``, ``X-Forwarded-Scheme`` or ``X-Forwarded-Proto`` HTTP header - (default ``http``). -- With ``force_https = true`` the default will be ``https``. -- With ``use_htsts = true``, Kallithea will set ``Strict-Transport-Security`` when using https. - - -Nginx virtual host example --------------------------- - -Sample config for Nginx using proxy: - -.. code-block:: nginx - - upstream kallithea { - server 127.0.0.1:5000; - # add more instances for load balancing - #server 127.0.0.1:5001; - #server 127.0.0.1:5002; - } - - ## gist alias - server { - listen 443; - server_name gist.example.com; - access_log /var/log/nginx/gist.access.log; - error_log /var/log/nginx/gist.error.log; - - ssl on; - ssl_certificate gist.your.kallithea.server.crt; - ssl_certificate_key gist.your.kallithea.server.key; - - ssl_session_timeout 5m; - - ssl_protocols SSLv3 TLSv1; - ssl_ciphers DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:EDH-RSA-DES-CBC3-SHA:AES256-SHA:DES-CBC3-SHA:AES128-SHA:RC4-SHA:RC4-MD5; - ssl_prefer_server_ciphers on; - - rewrite ^/(.+)$ https://kallithea.example.com/_admin/gists/$1; - rewrite (.*) https://kallithea.example.com/_admin/gists; - } - - server { - listen 443; - server_name kallithea.example.com - access_log /var/log/nginx/kallithea.access.log; - error_log /var/log/nginx/kallithea.error.log; - - ssl on; - ssl_certificate your.kallithea.server.crt; - ssl_certificate_key your.kallithea.server.key; - - ssl_session_timeout 5m; - - ssl_protocols SSLv3 TLSv1; - ssl_ciphers DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:EDH-RSA-DES-CBC3-SHA:AES256-SHA:DES-CBC3-SHA:AES128-SHA:RC4-SHA:RC4-MD5; - ssl_prefer_server_ciphers on; - - ## uncomment root directive if you want to serve static files by nginx - ## requires static_files = false in .ini file - #root /path/to/installation/kallithea/public; - include /etc/nginx/proxy.conf; - location / { - try_files $uri @kallithea; - } - - location @kallithea { - proxy_pass http://127.0.0.1:5000; - } - - } - -Here's the proxy.conf. It's tuned so it will not timeout on long -pushes or large pushes:: - - proxy_redirect off; - proxy_set_header Host $host; - ## needed for container auth - #proxy_set_header REMOTE_USER $remote_user; - #proxy_set_header X-Forwarded-User $remote_user; - proxy_set_header X-Url-Scheme $scheme; - proxy_set_header X-Host $http_host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header Proxy-host $proxy_host; - proxy_buffering off; - proxy_connect_timeout 7200; - proxy_send_timeout 7200; - proxy_read_timeout 7200; - proxy_buffers 8 32k; - client_max_body_size 1024m; - client_body_buffer_size 128k; - large_client_header_buffers 8 64k; - - -Apache virtual host reverse proxy example ------------------------------------------ - -Here is a sample configuration file for Apache using proxy: - -.. code-block:: apache - - - ServerName kallithea.example.com - - - # For Apache 2.4 and later: - Require all granted - - # For Apache 2.2 and earlier, instead use: - # Order allow,deny - # Allow from all - - - #important ! - #Directive to properly generate url (clone url) for pylons - ProxyPreserveHost On - - #kallithea instance - ProxyPass / http://127.0.0.1:5000/ - ProxyPassReverse / http://127.0.0.1:5000/ - - #to enable https use line below - #SetEnvIf X-Url-Scheme https HTTPS=1 - - -Additional tutorial -http://pylonsbook.com/en/1.1/deployment.html#using-apache-to-proxy-requests-to-pylons - - -Apache as subdirectory ----------------------- - -Apache subdirectory part: - -.. code-block:: apache - - > - ProxyPass http://127.0.0.1:5000/ - ProxyPassReverse http://127.0.0.1:5000/ - SetEnvIf X-Url-Scheme https HTTPS=1 - - -Besides the regular apache setup you will need to add the following line -into ``[app:main]`` section of your .ini file:: - - filter-with = proxy-prefix - -Add the following at the end of the .ini file:: - - [filter:proxy-prefix] - use = egg:PasteDeploy#prefix - prefix = / - -then change ```` into your chosen prefix - - -Apache with mod_wsgi --------------------- - -Alternatively, Kallithea can be set up with Apache under mod_wsgi. For -that, you'll need to: - -- Install mod_wsgi. If using a Debian-based distro, you can install - the package libapache2-mod-wsgi:: - - aptitude install libapache2-mod-wsgi - -- Enable mod_wsgi:: - - a2enmod wsgi - -- Add global Apache configuration to tell mod_wsgi that Python only will be - used in the WSGI processes and shouldn't be initialized in the Apache - processes:: - - WSGIRestrictEmbedded On - -- Create a wsgi dispatch script, like the one below. Make sure you - check that the paths correctly point to where you installed Kallithea - and its Python Virtual Environment. -- Enable the ``WSGIScriptAlias`` directive for the WSGI dispatch script, - as in the following example. Once again, check the paths are - correctly specified. - -Here is a sample excerpt from an Apache Virtual Host configuration file: - -.. code-block:: apache - - WSGIDaemonProcess kallithea processes=5 threads=1 maximum-requests=100 \ - python-home=/srv/kallithea/venv - WSGIProcessGroup kallithea - WSGIScriptAlias / /srv/kallithea/dispatch.wsgi - WSGIPassAuthorization On - -Or if using a dispatcher WSGI script with proper virtualenv activation: - -.. code-block:: apache - - WSGIDaemonProcess kallithea processes=5 threads=1 maximum-requests=100 - WSGIProcessGroup kallithea - WSGIScriptAlias / /srv/kallithea/dispatch.wsgi - WSGIPassAuthorization On - -Apache will by default run as a special Apache user, on Linux systems -usually ``www-data`` or ``apache``. If you need to have the repositories -directory owned by a different user, use the user and group options to -WSGIDaemonProcess to set the name of the user and group. - -.. note:: - If running Kallithea in multiprocess mode, - make sure you set ``instance_id = *`` in the configuration so each process - gets it's own cache invalidation key. - -Example WSGI dispatch script: - -.. code-block:: python - - import os - os.environ["HGENCODING"] = "UTF-8" - os.environ['PYTHON_EGG_CACHE'] = '/srv/kallithea/.egg-cache' - - # sometimes it's needed to set the curent dir - os.chdir('/srv/kallithea/') - - import site - site.addsitedir("/srv/kallithea/venv/lib/python2.7/site-packages") - - ini = '/srv/kallithea/my.ini' - from paste.script.util.logging_config import fileConfig - fileConfig(ini) - from paste.deploy import loadapp - application = loadapp('config:' + ini) - -Or using proper virtualenv activation: - -.. code-block:: python - - activate_this = '/srv/kallithea/venv/bin/activate_this.py' - execfile(activate_this, dict(__file__=activate_this)) - - import os - os.environ['HOME'] = '/srv/kallithea' - - ini = '/srv/kallithea/kallithea.ini' - from paste.script.util.logging_config import fileConfig - fileConfig(ini) - from paste.deploy import loadapp - application = loadapp('config:' + ini) - - -Other configuration files -------------------------- - -A number of `example init.d scripts`__ can be found in -the ``init.d`` directory of the Kallithea source. - -.. __: https://kallithea-scm.org/repos/kallithea/files/tip/init.d/ . - - -.. _virtualenv: http://pypi.python.org/pypi/virtualenv -.. _python: http://www.python.org/ -.. _Mercurial: https://www.mercurial-scm.org/ -.. _Celery: http://celeryproject.org/ -.. _Celery documentation: http://docs.celeryproject.org/en/latest/getting-started/index.html -.. _RabbitMQ: http://www.rabbitmq.com/ -.. _Redis: http://redis.io/ .. _python-ldap: http://www.python-ldap.org/ -.. _mercurial-server: http://www.lshift.net/mercurial-server.html -.. _PublishingRepositories: https://www.mercurial-scm.org/wiki/PublishingRepositories