language: python

python:

  - "2.6"

  - "2.7"

env:  

  - TEST_DB=sqlite:////tmp/kallithea_test.sqlite

  - TEST_DB=mysql://root@127.0.0.1/kallithea_test

  - TEST_DB=postgresql://postgres@127.0.0.1/kallithea_test

services:

  - mysql

  - postgresql

# command to install dependencies

before_script:

  - mysql -e 'create database kallithea_test;'

  - psql -c 'create database kallithea_test;' -U postgres

  - git --version

before_install:

  - sudo apt-get remove git

  - sudo add-apt-repository ppa:pdoes/ppa -y

  - sudo apt-get update -y

  - sudo apt-get install git -y

install:

  - pip install mysql-python psycopg2 mock unittest2

  - pip install . --use-mirrors

# command to run tests

script: nosetests

notifications:

    email:

        - ci@kallithea-scm.org

    irc: "irc.freenode.org#kallithea"

branches:

  only:

    - master

.. _contributing:

=========================

Contributing to Kallithea

=========================

If you would like to contribute to Kallithea, please contact us, any help is

greatly appreciated!

Could I request that you make your source contributions by first forking the

Kallithea repository on bitbucket_

https://bitbucket.org/conservancy/kallithea and then make your changes to

your forked repository. Please post all fixes into **dev** bookmark since your

change might be already fixed there and i try to merge all fixes from dev into

stable, and not the other way. Finally, when you are finished with your changes,

please send us a pull request.

To run Kallithea in a development version you always need to install the latest

required libs. Simply clone Kallithea and switch to beta branch::

    hg clone https://kallithea-scm.org/repos/kallithea

after downloading/pulling Kallithea make sure you run::

    python setup.py develop

command to install/verify all required packages, and prepare development

enviroment.

There are two files in the directory production.ini and developement.ini copy

the `development.ini` file as rc.ini (which is excluded from version controll)

and put all your changes like db connection or server port in there.

After finishing your changes make sure all tests passes ok. You can run

the testsuite running ``nosetest`` from the project root, or if you use tox

test.ini file is used and by default it uses sqlite for tests, edit this file

to change your testing enviroment.

There's a special set of tests for push/pull operations, you can runn them using::

    paster serve test.ini --pid-file=test.pid --daemon

    KALLITHEA_WHOOSH_TEST_DISABLE=1 KALLITHEA_NO_TMP_PATH=1 nosetests -x kallithea/tests/other/test_vcs_operations.py

    kill -9 $(cat test.pid)

| Thank you for any contributions!

.. _bitbucket: http://bitbucket.org/

.. _installation:

============

Installation

============

``Kallithea`` is written entirely in Python. Before posting any issues make

sure, your not missing any system libraries and using right version of

libraries required by Kallithea. There's also restriction in terms of mercurial

clients. Minimal version of hg client known working fine with Kallithea is

**1.6**. If you're using older client, please upgrade.

Installing Kallithea from PyPI (aka "Cheeseshop")

-------------------------------------------------

The easiest way to install ``kallithea`` is to run::

    easy_install kallithea

Or::

    pip install kallithea

If you prefer to install Kallithea manually simply grab latest release from

http://pypi.python.org/pypi/Kallithea, decompress the archive and run::

    python setup.py install

Step by step installation example for Windows

---------------------------------------------

:ref:`installation_win`

Step by step installation example for Linux

-------------------------------------------

For installing Kallithea i highly recommend using separate virtualenv_. This

way many required by Kallithea libraries will remain sandboxed from your main

python and making things less problematic when doing system python updates.

Alternative very detailed installation instructions for Ubuntu Server with

celery, indexer and daemon scripts: https://gist.github.com/4546398

- Assuming you have installed virtualenv_ create a new virtual environment

  using virtualenv command::

    virtualenv --no-site-packages /opt/kallithea-venv

.. note:: Using ``--no-site-packages`` when generating your

   virtualenv is **very important**. This flag provides the necessary

   isolation for running the set of packages required by

   Kallithea.  If you do not specify ``--no-site-packages``,

   it's possible that Kallithea will not install properly into

   the virtualenv, or, even if it does, may not run properly,

   depending on the packages you've already got installed into your

   Python's "main" site-packages dir.

- this will install new virtualenv_ into `/opt/kallithea-venv`.

- Activate the virtualenv_ by running::

    source /opt/kallithea-venv/bin/activate

.. note:: If you're using UNIX, *do not* use ``sudo`` to run the

   ``virtualenv`` script.  It's perfectly acceptable (and desirable)

   to create a virtualenv as a normal user.

- Make a folder for Kallithea data files, and configuration somewhere on the

  filesystem. For example::

    mkdir /opt/kallithea

- Go into the created directory run this command to install kallithea::

    easy_install kallithea

  or::

    pip install kallithea

- This will install Kallithea together with pylons and all other required

  python libraries into activated virtualenv

Requirements for Celery (optional)

----------------------------------

In order to gain maximum performance

there are some third-party you must install. When Kallithea is used

together with celery you have to install some kind of message broker,

recommended one is rabbitmq_ to make the async tasks work.

Of course Kallithea works in sync mode also and then you do not have to install

any third party applications. However, using Celery_ will give you a large

speed improvement when using many big repositories. If you plan to use

Kallithea for say 7 to 10 repositories, Kallithea will perform perfectly well

without celery running.

If you make the decision to run Kallithea with celery make sure you run

celeryd using paster and message broker together with the application.

.. note::

   Installing message broker and using celery is optional, Kallithea will

   work perfectly fine without them.

**Message Broker**

- preferred is `RabbitMq <http://www.rabbitmq.com/>`_

- A possible alternative is `Redis <http://code.google.com/p/redis/>`_

For installation instructions you can visit:

http://ask.github.com/celery/getting-started/index.html.

This is a very nice tutorial on how to start using celery_ with rabbitmq_

You can now proceed to :ref:`setup`

-----------------------------------

.. _virtualenv: http://pypi.python.org/pypi/virtualenv

.. _python: http://www.python.org/

.. _mercurial: http://mercurial.selenic.com/

.. _celery: http://celeryproject.org/

.. _rabbitmq: http://www.rabbitmq.com/

.. _installation_win:

Step by step Installation for Windows

=====================================

Kallithea step-by-step install Guide for Windows

Target OS: Windows XP SP3 32bit English (Clean installation)

+ All Windows Updates until 24-may-2012

.. note::

   This installation is for 32bit systems, for 64bit windows you might need

   to download proper 64bit versions of the different packages(Windows Installer, Win32py extensions)

   plus some extra tweaks.

   These extra steps haven been marked as "64bit".

   Tested on Windows Server 2008 R2 SP1, 9-feb-2013.

   If you run into any 64bit related problems, please check these pages:

   - http://blog.victorjabur.com/2011/06/05/compiling-python-2-7-modules-on-windows-32-and-64-using-msvc-2008-express/

   - http://bugs.python.org/issue7511

Step1 - Install Visual Studio 2008 Express

------------------------------------------

Download "Visual C++ 2008 Express Edition with SP1" from:

http://www.microsoft.com/visualstudio/en-us/products/2008-editions/express

(if not found or relocated, google for "visual studio 2008 express" for

updated link)

You can also download full ISO file for offline installation, just

choose "All - Offline Install ISO image file" in the previous page and

choose "Visual C++ 2008 Express" when installing.

.. note::

   Using other versions of Visual Studio will lead to random crashes.

   You must use Visual Studio 2008!"

.. note::

   Silverlight Runtime and SQL Server 2008 Express Edition are not

   required, you can uncheck them

.. note::

   64bit: You also need to install the Microsoft Windows SDK for .NET 3.5 SP1 (.NET 4.0 won't work).

   Download from: http://www.microsoft.com/en-us/download/details.aspx?id=3138

.. note::

   64bit: You also need to copy and rename a .bat file to make the Visual C++ compiler work.

   I am not sure why this is not necessary for 32bit.

   Copy C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin\vcvars64.bat to C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin\amd64\vcvarsamd64.bat

Step2 - Install Python

----------------------

Download Python 2.x.y from:

http://www.python.org/download/

Choose "Windows Installer" (32bit version) not "Windows X86-64

Installer". While writing this guide, the latest version was v2.7.3.

Remember the specific major and minor version installed, because it will

be needed in the next step. In this case, it is "2.7".

.. note::

   64bit: Just download and install the 64bit version of python.

Step3 - Install Win32py extensions

----------------------------------

Download pywin32 from:

http://sourceforge.net/projects/pywin32/files/

- Click on "pywin32" folder

- Click on the first folder (in this case, Build 217, maybe newer when you try)

- Choose the file ending with ".win32-py2.x.exe" -> x being the minor

  version of Python you installed (in this case, 7)

  When writing this guide, the file was:

  http://sourceforge.net/projects/pywin32/files/pywin32/Build%20217/pywin32-217.win32-py2.7.exe/download

  .. note::

     64bit: Download and install the 64bit version.

     At the time of writing you can find this at:

     http://sourceforge.net/projects/pywin32/files/pywin32/Build%20218/pywin32-218.win-amd64-py2.7.exe/download

Step4 - Python BIN

------------------

Add Python BIN folder to the path

You have to add the Python folder to the path, you can do it manually

(editing "PATH" environment variable) or using Windows Support Tools

that came preinstalled in Vista/7 and can be installed in Windows XP.

- Using support tools on WINDOWS XP:

  If you use Windows XP you can install them using Windows XP CD and

  navigating to \SUPPORT\TOOLS. There, execute Setup.EXE (not MSI).

  Afterwards, open a CMD and type::

    SETX PATH "%PATH%;[your-python-path]" -M

  Close CMD (the path variable will be updated then)

- Using support tools on WINDOWS Vista/7:

  Open a CMD and type::

    SETX PATH "%PATH%;[your-python-path]" /M

  Please substitute [your-python-path] with your Python installation path.

  Typically: C:\\Python27

Step5 - Kallithea folder structure

----------------------------------

Create a Kallithea folder structure

This is only a example to install Kallithea, you can of course change

it. However, this guide will follow the proposed structure, so please

later adapt the paths if you change them. My recommendation is to use

folders with NO SPACES. But you can try if you are brave...

Create the following folder structure::

  C:\Kallithea

  C:\Kallithea\Bin

  C:\Kallithea\Env

  C:\Kallithea\Repos

Step6 - Install virtualenv

---------------------------

Install Virtual Env for Python

Navigate to: http://www.virtualenv.org/en/latest/index.html#installation

Right click on "virtualenv.py" file and choose "Save link as...".

Download to C:\\Kallithea (or whatever you want)

(the file is located at

https://raw.github.com/pypa/virtualenv/master/virtualenv.py)

Create a virtual Python environment in C:\\Kallithea\\Env (or similar). To

do so, open a CMD (Python Path should be included in Step3), navigate

where you downloaded "virtualenv.py", and write::

 python virtualenv.py C:\Kallithea\Env

(--no-site-packages is now the default behaviour of virtualenv, no need

to include it)

Step7 - Install Kallithea

-------------------------

Finally, install Kallithea

Close previously opened command prompt/s, and open a Visual Studio 2008

Command Prompt (**IMPORTANT!!**). To do so, go to Start Menu, and then open

"Microsoft Visual C++ 2008 Express Edition" -> "Visual Studio Tools" ->

"Visual Studio 2008 Command Prompt"

.. note::

   64bit: For 64bit you need to modify the shortcut that is used to start the

   Visual Studio 2008 Command Prompt. Use right-mouse click to open properties.

Change commandline from::

%comspec% /k ""C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\vcvarsall.bat"" x86

to::

%comspec% /k ""C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\vcvarsall.bat"" amd64

In that CMD (loaded with VS2008 PATHs) type::

  cd C:\Kallithea\Env\Scripts (or similar)

  activate

The prompt will change into "(Env) C:\\Kallithea\\Env\\Scripts" or similar

(depending of your folder structure). Then type::

 pip install kallithea

(long step, please wait until fully complete)

Some warnings will appear, don't worry as they are normal.

Step8 - Configuring Kallithea

-----------------------------

steps taken from http://packages.python.org/Kallithea/setup.html

You have to use the same Visual Studio 2008 command prompt as Step7, so

if you closed it reopen it following the same commands (including the

"activate" one). When ready, just type::

  cd C:\Kallithea\Bin

  paster make-config Kallithea production.ini

Then, you must edit production.ini to fit your needs (ip address, ip

port, mail settings, database, whatever). I recommend using NotePad++

(free) or similar text editor, as it handles well the EndOfLine

character differences between Unix and Windows

(http://notepad-plus-plus.org/)

For the sake of simplicity lets run it with the default settings. After

your edits (if any), in the previous Command Prompt, type::

 paster setup-db production.ini

(this time a NEW database will be installed, you must follow a different

step to later UPGRADE to a newer Kallithea version)

The script will ask you for confirmation about creating a NEW database,

answer yes (y)

The script will ask you for repository path, answer C:\\Kallithea\\Repos

(or similar)

The script will ask you for admin username and password, answer "admin"

+ "123456" (or whatever you want)

The script will ask you for admin mail, answer "admin@xxxx.com" (or

whatever you want)

If you make some mistake and the script does not end, don't worry, start

it again.

Step9 - Running Kallithea

-------------------------

In the previous command prompt, being in the C:\\Kallithea\\Bin folder,

just type::

 paster serve production.ini

Open yout web server, and go to http://127.0.0.1:5000

It works!! :-)

Remark:

If it does not work first time, just Ctrl-C the CMD process and start it

again. Don't forget the "http://" in Internet Explorer

What this Guide does not cover:

- Installing Celery

- Running Kallithea as Windows Service. You can investigate here:

  - http://pypi.python.org/pypi/wsgisvc

  - http://ryrobes.com/python/running-python-scripts-as-a-windows-service/

  - http://wiki.pylonshq.com/display/pylonscookbook/How+to+run+Pylons+as+a+Windows+service

- Using Apache. You can investigate here:

  - https://groups.google.com/group/rhodecode/msg/c433074e813ffdc4

Upgrading

=========

Stop running Kallithea

Open a CommandPrompt like in Step7 (VS2008 path + activate) and type::

 easy_install -U kallithea

 cd \Kallithea\Bin

{ backup your production.ini file now} ::

 paster make-config Kallithea production.ini

(check changes and update your production.ini accordingly) ::

 paster upgrade-db production.ini (update database)

Full steps in http://packages.python.org/Kallithea/upgrade.html

.. _setup:

=====

Setup

=====

Setting up Kallithea

--------------------

First, you will need to create a Kallithea configuration file. Run the

following command to do this::

    paster make-config Kallithea production.ini

- This will create the file `production.ini` in the current directory. This

  configuration file contains the various settings for Kallithea, e.g proxy

  port, email settings, usage of static files, cache, celery settings and

  logging.

Next, you need to create the databases used by Kallithea. I recommend that you

use postgresql or sqlite (default). If you choose a database other than the

default ensure you properly adjust the db url in your production.ini

configuration file to use this other database. Kallithea currently supports

postgresql, sqlite and mysql databases. Create the database by running

the following command::

    paster setup-db production.ini

This will prompt you for a "root" path. This "root" path is the location where

Kallithea will store all of its repositories on the current machine. After

entering this "root" path ``setup-db`` will also prompt you for a username

and password for the initial admin account which ``setup-db`` sets

up for you.

setup process can be fully automated, example for lazy::

    paster setup-db production.ini --user=nn --password=secret --email=nn@your.kallithea.server --repos=/home/nn/my_repos

- The ``setup-db`` command will create all of the needed tables and an

  admin account. When choosing a root path you can either use a new empty

  location, or a location which already contains existing repositories. If you

  choose a location which contains existing repositories Kallithea will simply

  add all of the repositories at the chosen location to it's database.

  (Note: make sure you specify the correct path to the root).

- Note: the given path for mercurial_ repositories **must** be write accessible

  for the application. It's very important since the Kallithea web interface

  will work without write access, but when trying to do a push it will

  eventually fail with permission denied errors unless it has write access.

You are now ready to use Kallithea, to run it simply execute::

    paster serve production.ini

- This command runs the Kallithea server. The web app should be available at the

  127.0.0.1:5000. This ip and port is configurable via the production.ini

  file created in previous step

- Use the admin account you created above when running ``setup-db``

  to login to the web app.

- The default permissions on each repository is read, and the owner is admin.

  Remember to update these if needed.

- In the admin panel you can toggle ldap, anonymous, permissions settings. As

  well as edit more advanced options on users and repositories

Optionally users can create `rcextensions` package that extends Kallithea

functionality. To do this simply execute::

    paster make-rcext production.ini

This will create `rcextensions` package in the same place that your `ini` file

lives. With `rcextensions` it's possible to add additional mapping for whoosh,

stats and add additional code into the push/pull/create/delete repo hooks.

For example for sending signals to build-bots such as jenkins.

Please see the `__init__.py` file inside `rcextensions` package

for more details.

Using Kallithea with SSH

------------------------

Kallithea currently only hosts repositories using http and https. (The addition

of ssh hosting is a planned future feature.) However you can easily use ssh in

parallel with Kallithea. (Repository access via ssh is a standard "out of

the box" feature of mercurial_ and you can use this to access any of the

repositories that Kallithea is hosting. See PublishingRepositories_)

Kallithea repository structures are kept in directories with the same name

as the project. When using repository groups, each group is a subdirectory.

This allows you to easily use ssh for accessing repositories.

In order to use ssh you need to make sure that your web-server and the users

login accounts have the correct permissions set on the appropriate directories.

(Note that these permissions are independent of any permissions you have set up

using the Kallithea web interface.)

If your main directory (the same as set in Kallithea settings) is for example

set to **/home/hg** and the repository you are using is named `kallithea`, then

to clone via ssh you should run::

    hg clone ssh://user@server.com/home/hg/kallithea

Using other external tools such as mercurial-server_ or using ssh key based

authentication is fully supported.

Note: In an advanced setup, in order for your ssh access to use the same

permissions as set up via the Kallithea web interface, you can create an

authentication hook to connect to the Kallithea db and runs check functions for

permissions against that.

Setting up Whoosh full text search

----------------------------------

Starting from version 1.1 the whoosh index can be build by using the paster

command ``make-index``. To use ``make-index`` you must specify the configuration

file that stores the location of the index. You may specify the location of the

repositories (`--repo-location`).  If not specified, this value is retrieved

from the Kallithea database.  This was required prior to 1.2.  Starting from

version 1.2 it is also possible to specify a comma separated list of

repositories (`--index-only`) to build index only on chooses repositories

skipping any other found in repos location

You may optionally pass the option `-f` to enable a full index rebuild. Without

the `-f` option, indexing will run always in "incremental" mode.

For an incremental index build use::

    paster make-index production.ini

For a full index rebuild use::

    paster make-index production.ini -f

building index just for chosen repositories is possible with such command::

 paster make-index production.ini --index-only=vcs,kallithea

In order to do periodical index builds and keep your index always up to date.

It's recommended to do a crontab entry for incremental indexing.

An example entry might look like this::

    /path/to/python/bin/paster make-index /path/to/kallithea/production.ini

When using incremental mode (the default) whoosh will check the last

modification date of each file and add it to be reindexed if a newer file is

available. The indexing daemon checks for any removed files and removes them

from index.

If you want to rebuild index from scratch, you can use the `-f` flag as above,

or in the admin panel you can check `build from scratch` flag.

Setting up LDAP support

-----------------------

Kallithea starting from version 1.1 supports ldap authentication. In order

to use LDAP, you have to install the python-ldap_ package. This package is

available via pypi, so you can install it by running

using easy_install::

    easy_install python-ldap

using pip::

    pip install python-ldap

.. note::

   python-ldap requires some certain libs on your system, so before installing

   it check that you have at least `openldap`, and `sasl` libraries.

LDAP settings are located in admin->ldap section,

Here's a typical ldap setup::

 Connection settings

 Enable LDAP          = checked

 Host                 = host.example.org

 Port                 = 389

 Account              = <account>

 Password             = <password>

 Connection Security  = LDAPS connection

 Certificate Checks   = DEMAND

 Search settings

 Base DN              = CN=users,DC=host,DC=example,DC=org

 LDAP Filter          = (&(objectClass=user)(!(objectClass=computer)))

 LDAP Search Scope    = SUBTREE

 Attribute mappings

 Login Attribute      = uid

 First Name Attribute = firstName

 Last Name Attribute  = lastName

 E-mail Attribute     = mail

If your user groups are placed in a Organisation Unit (OU) structure the Search Settings configuration differs::

 Search settings

 Base DN              = DC=host,DC=example,DC=org

 LDAP Filter          = (&(memberOf=CN=your user group,OU=subunit,OU=unit,DC=host,DC=example,DC=org)(objectClass=user))

 LDAP Search Scope    = SUBTREE

.. _enable_ldap:

Enable LDAP : required

    Whether to use LDAP for authenticating users.

.. _ldap_host:

Host : required

    LDAP server hostname or IP address. Can be also a comma separated

    list of servers to support LDAP fail-over.

.. _Port:

Port : required

    389 for un-encrypted LDAP, 636 for SSL-encrypted LDAP.

.. _ldap_account:

Account : optional

    Only required if the LDAP server does not allow anonymous browsing of

    records.  This should be a special account for record browsing.  This

    will require `LDAP Password`_ below.

.. _LDAP Password:

Password : optional

    Only required if the LDAP server does not allow anonymous browsing of

    records.

.. _Enable LDAPS:

Connection Security : required

    Defines the connection to LDAP server

    No encryption

        Plain non encrypted connection

    LDAPS connection

        Enable ldaps connection. It will likely require `Port`_ to be set to

        a different value (standard LDAPS port is 636). When LDAPS is enabled

        then `Certificate Checks`_ is required.

    START_TLS on LDAP connection

        START TLS connection

.. _Certificate Checks:

Certificate Checks : optional

    How SSL certificates verification is handled - this is only useful when

    `Enable LDAPS`_ is enabled.  Only DEMAND or HARD offer full SSL security

    while the other options are susceptible to man-in-the-middle attacks.  SSL

    certificates can be installed to /etc/openldap/cacerts so that the

    DEMAND or HARD options can be used with self-signed certificates or

    certificates that do not have traceable certificates of authority.

    NEVER

        A serve certificate will never be requested or checked.

    ALLOW

        A server certificate is requested.  Failure to provide a

        certificate or providing a bad certificate will not terminate the

        session.

    TRY

        A server certificate is requested.  Failure to provide a

        certificate does not halt the session; providing a bad certificate

        halts the session.

    DEMAND

        A server certificate is requested and must be provided and

        authenticated for the session to proceed.

    HARD

        The same as DEMAND.

.. _Base DN:

Base DN : required

    The Distinguished Name (DN) where searches for users will be performed.

    Searches can be controlled by `LDAP Filter`_ and `LDAP Search Scope`_.

.. _LDAP Filter:

LDAP Filter : optional

    A LDAP filter defined by RFC 2254.  This is more useful when `LDAP

    Search Scope`_ is set to SUBTREE.  The filter is useful for limiting

    which LDAP objects are identified as representing Users for

    authentication.  The filter is augmented by `Login Attribute`_ below.

    This can commonly be left blank.

.. _LDAP Search Scope:

LDAP Search Scope : required

    This limits how far LDAP will search for a matching object.

    BASE

        Only allows searching of `Base DN`_ and is usually not what you

        want.

    ONELEVEL

        Searches all entries under `Base DN`_, but not Base DN itself.

    SUBTREE

        Searches all entries below `Base DN`_, but not Base DN itself.

        When using SUBTREE `LDAP Filter`_ is useful to limit object

        location.

.. _Login Attribute:

Login Attribute : required

    The LDAP record attribute that will be matched as the USERNAME or

    ACCOUNT used to connect to Kallithea.  This will be added to `LDAP

    Filter`_ for locating the User object.  If `LDAP Filter`_ is specified as

    "LDAPFILTER", `Login Attribute`_ is specified as "uid" and the user has

    connected as "jsmith" then the `LDAP Filter`_ will be augmented as below

    ::

        (&(LDAPFILTER)(uid=jsmith))

.. _ldap_attr_firstname:

First Name Attribute : required

    The LDAP record attribute which represents the user's first name.

.. _ldap_attr_lastname:

Last Name Attribute : required

    The LDAP record attribute which represents the user's last name.

.. _ldap_attr_email:

Email Attribute : required