#smtp_password =

#smtp_port = 25

#smtp_use_ssl = false

#smtp_use_tls = false

[server:main]

## nr of worker threads to spawn

#threadpool_workers = 1

## max request before thread respawn

#threadpool_max_requests = 100

## option to use threads of process

#use_threadpool = true

## WAITRESS ##

use = egg:waitress#main

## number of worker threads

threads = 1

## MAX BODY SIZE 100GB

max_request_body_size = 107374182400

################################

### LOGGING CONFIGURATION   ####

################################

[loggers]

[handlers]

keys = console, console_sql

[formatters]

keys = generic, color_formatter, color_formatter_sql

[logger_kallithea]

level = DEBUG

handlers =

qualname = kallithea

propagate = 1

[logger_sqlalchemy]

level = INFO

handlers = console_sql

qualname = sqlalchemy.engine

propagate = 0

        hg clone https://kallithea-scm.org/repos/kallithea

        cd kallithea

        virtualenv ../kallithea-venv

        source ../kallithea-venv/bin/activate

        pip install --upgrade pip setuptools

        pip install -e .

        firefox http://127.0.0.1:5000/

You can also start out by forking https://bitbucket.org/conservancy/kallithea

on Bitbucket_ and create a local clone of your own fork.

When running tests, Kallithea uses `kallithea/tests/test.ini` and populates the

SQLite database specified there.

It is possible to avoid recreating the full test database on each invocation of

the tests, thus eliminating the initial delay. To achieve this, run the tests as::

    KALLITHEA_WHOOSH_TEST_DISABLE=1 KALLITHEA_NO_TMP_PATH=1 py.test

    kill -9 $(cat test.pid)

In these commands, the following variables are used::

    KALLITHEA_WHOOSH_TEST_DISABLE=1 - skip whoosh index building and tests

   schema *before* the changes you're currently developing. (Do not

   create the database while your new schema changes are applied.)

2. Create a separate throwaway configuration for iterating on the actual

   database changes::

   Edit the file to change database settings. SQLite is typically fine,

   but make sure to change the path to e.g. ``temp.db``, to avoid

   clobbering any existing database file.

3. Make your code changes (including database schema changes in ``db.py``).

4. After every database schema change, recreate the throwaway database

   to test the changes::

    rm temp.db

5. Once satisfied with the schema changes, auto-generate a draft Alembic

   script using the development database that has *not* been upgraded.

   (The generated script will upgrade the database to match the code.)

   ::

ISAPI handler

^^^^^^^^^^^^^

The ISAPI handler can be generated using::

This will generate a ``dispatch.py`` file in the current directory that contains

the necessary components to finalize an installation into IIS. Once this file

has been generated, it is necessary to run the following command due to the way

that ISAPI-WSGI is made::

This accomplishes two things: generating an ISAPI compliant DLL file,

``_dispatch.dll``, and installing a script map handler into IIS for the

``--virtualdir`` specified above pointing to ``_dispatch.dll``.

The ISAPI handler is registered to all file extensions, so it will automatically

be the one handling all requests to the specified virtual directory. When the website starts

middleware WSGI handler that Kallithea runs within and each HTTP request to the

site will be processed through this logic henceforth.

Authentication with Kallithea using IIS authentication modules

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Troubleshooting

---------------

Typically, any issues in this setup will either be entirely in IIS or entirely

different options for finding issues exist: IIS' failed request tracking which

is great at finding issues until they exist inside Kallithea, at which point the

ISAPI-WSGI wrapper above uses ``win32traceutil``, which is part of ``pywin32``.

In order to dump output from WSGI using ``win32traceutil`` it is sufficient to

type the following in a console window::

You have to use the same command prompt as in Step 7, so if you closed

it, reopen it following the same commands (including the "activate"

one). When ready, type::

  cd C:\Kallithea\Bin

Then you must edit production.ini to fit your needs (IP address, IP

port, mail settings, database, etc.). `NotePad++`__ or a similar text

editor is recommended to properly handle the newline character

differences between Unix and Windows.

__ http://notepad-plus-plus.org/

For the sake of simplicity, run it with the default settings. After your edits (if any) in the previous command prompt, type::

.. warning:: This time a *new* database will be installed. You must

             follow a different process to later :ref:`upgrade <upgrade>`

             to a newer Kallithea version.

The script will ask you for confirmation about creating a new database, answer yes (y)

Step 10 -- Running Kallithea

^^^^^^^^^^^^^^^^^^^^^^^^^^^^

In the previous command prompt, being in the C:\\Kallithea\\Bin folder, type::

Open your web server, and go to http://127.0.0.1:5000

It works!! :-)

Remark:

You have to use the same Visual Studio 2008 command prompt as Step7, so

if you closed it reopen it following the same commands (including the

"activate" one). When ready, just type::

  cd C:\Kallithea\Bin

Then, you must edit production.ini to fit your needs (network address and

port, mail settings, database, whatever). I recommend using NotePad++

(free) or similar text editor, as it handles well the EndOfLine

character differences between Unix and Windows

(http://notepad-plus-plus.org/)

For the sake of simplicity lets run it with the default settings. After

your edits (if any), in the previous Command Prompt, type::

.. warning:: This time a *new* database will be installed. You must

             follow a different process to later :ref:`upgrade <upgrade>`

             to a newer Kallithea version.

The script will ask you for confirmation about creating a NEW database,

Step 9 -- Running Kallithea

^^^^^^^^^^^^^^^^^^^^^^^^^^^

In the previous command prompt, being in the C:\\Kallithea\\Bin folder,

just type::

Open yout web server, and go to http://127.0.0.1:5000

It works!! :-)

Remark:

(protocol, address, port and path) for each HTTP request. Kallithea will

usually use its own HTML/cookie based authentication but can also be configured

to use web server authentication.

There are several web server options:

  from the command line. That is perfect for development and evaluation.

  Actual use in production might have different requirements and need extra

  work to make it manageable as a scalable system service.

  Waitress_. Gunicorn_ is also an option. These web servers have different

  limited feature sets.

  to it. The entry point for the WSGI application is configured

  in ``setup.py`` as ``kallithea.config.middleware:make_app``.

- `Apache httpd`_ can serve WSGI applications directly using mod_wsgi_ and a

  simple Python file with the necessary configuration. This is a good option if

  Apache is an option.

  can be put in front of another web server which has WSGI support.

  Such a layered setup can be complex but might in some cases be the right

  option, for example to standardize on one internet-facing web server, to add

  encryption or special authentication or for other security reasons, to

  provide caching of static files, or to provide load balancing or fail-over.

  Nginx_, Varnish_ and HAProxy_ are often used for this purpose, often in front

The best option depends on what you are familiar with and the requirements for

performance and stability. Also, keep in mind that Kallithea mainly is serving

dynamically generated pages from a relatively slow Python process. Kallithea is

also often used inside organizations with a limited amount of users and thus no

continuous hammering from the internet.

.. _Python: http://www.python.org/

.. _Gunicorn: http://gunicorn.org/

.. _Waitress: http://waitress.readthedocs.org/en/latest/

.. _virtualenv: http://pypi.python.org/pypi/virtualenv

.. _PyPI: https://pypi.python.org/pypi

.. _Apache httpd: http://httpd.apache.org/

.. _mod_wsgi: https://code.google.com/p/modwsgi/

.. _isapi-wsgi: https://github.com/hexdump42/isapi-wsgi

.. _uWSGI: https://uwsgi-docs.readthedocs.org/en/latest/

.. _nginx: http://nginx.org/en/

Setting up Kallithea

--------------------

First, you will need to create a Kallithea configuration file. Run the

following command to do so::

This will create the file ``my.ini`` in the current directory. This

configuration file contains the various settings for Kallithea, e.g.

proxy port, email settings, usage of static files, cache, Celery

settings, and logging.

use PostgreSQL or SQLite (default). If you choose a database other than the

default, ensure you properly adjust the database URL in your ``my.ini``

configuration file to use this other database. Kallithea currently supports

PostgreSQL, SQLite and MySQL databases. Create the database by running

the following command::

This will prompt you for a "root" path. This "root" path is the location where

Kallithea will store all of its repositories on the current machine. After

entering this "root" path ``setup-db`` will also prompt you for a username

and password for the initial admin account which ``setup-db`` sets

up for you.

The ``setup-db`` values can also be given on the command line.

Example::

The ``setup-db`` command will create all needed tables and an

admin account. When choosing a root path you can either use a new

empty location, or a location which already contains existing

repositories. If you choose a location which contains existing

repositories Kallithea will add all of the repositories at the chosen

          the Kallithea web interface will work without write access,

          but when trying to do a push it will fail with permission

          denied errors unless it has write access.

You are now ready to use Kallithea. To run it simply execute::

- This command runs the Kallithea server. The web app should be available at

  http://127.0.0.1:5000. The IP address and port is configurable via the

  configuration file created in the previous step.

- Log in to Kallithea using the admin account created when running ``setup-db``.

- The default permissions on each repository is read, and the owner is admin.

Kallithea provides full text search of repositories using `Whoosh`__.

.. __: https://pythonhosted.org/Whoosh/

For an incremental index build, run::

For a full index rebuild, run::

The ``--repo-location`` option allows the location of the repositories to be overridden;

usually, the location is retrieved from the Kallithea database.

The ``--index-only`` option can be used to limit the indexed repositories to a comma-separated list::

To keep your index up-to-date it is necessary to do periodic index builds;

for this, it is recommended to use a crontab entry. Example::

When using incremental mode (the default), Whoosh will check the last

modification date of each file and add it to be reindexed if a newer file is

available. The indexing daemon checks for any removed files and removes them

from index.

Remember that the ini files use the format with '.' and not with '_' like

Celery. So for example setting `BROKER_HOST` in Celery means setting

`broker.host` in the configuration file.

To start the Celery process, run::

.. note::

   Make sure you run this command from the same virtualenv, and with the same

   user that Kallithea runs.

5. Upgrade your configuration

-----------------------------

Run the following command to upgrade your configuration (``.ini``) file::

This will display any changes made by the new version of Kallithea to your

current configuration, and attempt an automatic merge. It is recommended

that you check the contents after the merge.

.. note::

repository hooks (for example to send signals to build bots such as Jenkins) and

even to monkey-patch certain parts of the Kallithea source code (for example

overwrite an entire function, change a global variable, ...).

To generate a skeleton extensions package, run::

This will create an ``rcextensions`` package next to the specified ``ini`` file.

See the ``__init__.py`` file inside the generated ``rcextensions`` package

for more details.

not physically delete said repository from the filesystem, but instead

renames it in a special way so that it is not possible to push, clone

or access the repository.

There is a special command for cleaning up such archived repositories::

This command scans for archived repositories that are older than

30 days, displays them, and asks if you want to delete them (unless given

the ``--dont-ask`` flag). If you host a large amount of repositories with

forks that are constantly being deleted, it is recommended that you run this

command via crontab.

Web server with chunked encoding

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Large Git pushes require an HTTP server with support for

chunked encoding for POST. The Python web servers waitress_ and

gunicorn_ (Linux only) can be used. By default, Kallithea uses

server.

    use = egg:waitress#main

or::

    use = egg:gunicorn#main

env USER=hg

# To use group (if different from user), you must edit sudoers file and change

# root's entry from (ALL) to (ALL:ALL)

# env GROUP=hg

script

    if [ -z "$GROUP" ]; then

        exec sudo -u $USER $COMMAND

    else

        exec sudo -u $USER -g $GROUP $COMMAND

    fi

end script

DAEMON=kallithea

APP_HOMEDIR="/srv"

APP_PATH="$APP_HOMEDIR/$DAEMON"

CONF_NAME="production.ini"

LOG_FILE="/var/log/$DAEMON.log"

PID_FILE="/run/daemons/$DAEMON"

RUN_AS="*****"

ARGS="serve --daemon \

--user=$RUN_AS \

--group=$RUN_AS \

--pid-file=$PID_FILE \

--log-file=$LOG_FILE \

[ -r /etc/conf.d/$DAEMON ] && . /etc/conf.d/$DAEMON

if [[ -r $PID_FILE ]]; then

    read -r PID < "$PID_FILE"

    if [[ $PID && ! -d /proc/$PID ]]; then

LOG_PATH="$APP_PATH/$APP_NAME.log"

PYTHON_PATH="/$APP_HOMEDIR/$APP_NAME-venv"

RUN_AS="root"

DAEMON_OPTS="serve --daemon \

 --user=$RUN_AS \

 --group=$RUN_AS \

 --pid-file=$PID_PATH \

start() {

  echo "Starting $APP_NAME"

  PYTHON_EGG_CACHE="/tmp" start-stop-daemon -d $APP_PATH \

      --start --quiet \

LOG_PATH="$APP_PATH/$APP_NAME.log"

PYTHON_PATH="/home/$APP_HOMEDIR/v-env"

RUN_AS="username"

DAEMON_OPTS="serve --daemon \

--user=$RUN_AS \

--group=$RUN_AS \

--pid-file=$PID_PATH \

#extra options

opts="${opts} restartdelay"

depend() {

    need nginx

# since this is a web app, it should go in /var/www

APP_PATH="/var/www/$APP_NAME"

CONF_NAME="production.ini"

# write to wherever the PID should be stored, just ensure

# same goes for the log file

PID_PATH="/var/run/kallithea/pid"

LOG_PATH="/var/log/kallithea/kallithea.log"

# replace this with the path to the virtual environment you

# made for Kallithea

PYTHON_PATH="/opt/python_virtualenvironments/kallithea-venv"

RUN_AS="kallithea"

DAEMON_OPTS="serve --daemon \

    --user=$RUN_AS \

    --group=$RUN_AS \

    --pid-file=$PID_PATH \

DESC="kallithea-server"

LOCK_FILE="/var/lock/subsys/$APP_NAME"

# source CentOS init functions

. /etc/init.d/functions

env LOGFILE=/var/hg/kallithea/log/kallithea.log

env APPINI=/var/hg/kallithea/production.ini

env HOME=/var/hg

env USER=hg

env GROUP=hg

post-stop script

    rm -f $PIDFILE

end script

; restart with supervisorctl restart kallithea:*

[program:kallithea]

numprocs = 1

numprocs_start = 5000 # possible should match ports

directory=/srv/kallithea

process_name = %(program_name)s_%(process_num)04d

redirect_stderr=true

stdout_logfile=/%(here)s/kallithea.log

# Configure default logging for Alembic. (This can be overriden by the

# config file, but usually isn't.)

logging.getLogger('alembic').setLevel(logging.INFO)

# Setup Python loggers based on the config file provided to the alembic

# command. If we're being invoked via the Alembic API (presumably for

# and loggers are assumed to already have been configured.

if config.config_file_name:

    fileConfig(config.config_file_name, disable_existing_loggers=False)

def include_in_autogeneration(object, name, type, reflected, compare_to):

#smtp_password =

#smtp_port = 25

#smtp_use_ssl = false

#smtp_use_tls = false

[server:main]

<%text>## nr of worker threads to spawn</%text>

threadpool_workers = 1

<%text>## max request before thread respawn</%text>

threadpool_max_requests = 100

<%text>## option to use threads of process</%text>

use_threadpool = true

%elif http_server == 'waitress':

<%text>## WAITRESS ##</%text>

use = egg:waitress#main

<%text>## number of worker threads</%text>

threads = 1

<%text>## MAX BODY SIZE 100GB</%text>

<%text>################################</%text>

<%text>### LOGGING CONFIGURATION   ####</%text>

<%text>################################</%text>

[loggers]

[handlers]

keys = console, console_sql

[formatters]

keys = generic, color_formatter, color_formatter_sql

[logger_kallithea]

level = DEBUG

handlers =

qualname = kallithea

propagate = 1

[logger_sqlalchemy]

level = INFO

handlers = console_sql

qualname = sqlalchemy.engine

propagate = 0

#smtp_password =

#smtp_port = 25

#smtp_use_ssl = false

#smtp_use_tls = false

[server:main]

## nr of worker threads to spawn

#threadpool_workers = 1

## max request before thread respawn

#threadpool_max_requests = 100

## option to use threads of process

#use_threadpool = true

## WAITRESS ##

use = egg:waitress#main

## number of worker threads

threads = 1

## MAX BODY SIZE 100GB

max_request_body_size = 107374182400

################################

### LOGGING CONFIGURATION   ####

################################

[loggers]

[handlers]

keys = console, console_sql

[formatters]

keys = generic, color_formatter, color_formatter_sql

[logger_kallithea]

level = DEBUG

handlers =

qualname = kallithea

propagate = 1

[logger_sqlalchemy]

level = INFO

handlers = console_sql

qualname = sqlalchemy.engine

propagate = 0