.. _setup:

Setup

=====

Setting up the application

--------------------------

First You'll ned to create RhodeCode config file. Run the following command 

to do this

::

 paster make-config RhodeCode production.ini

- This will create `production.ini` config inside the directory

  this config contains various settings for RhodeCode, e.g proxy port, 

  email settings, usage of static files, cache, celery settings and logging.

Next we need to create the database.

::

 paster setup-app production.ini

- This command will create all needed tables and an admin account. 

  When asked for a path You can either use a new location of one with already 

  existing ones. RhodeCode will simply add all new found repositories to 

  it's database. Also make sure You specify correct path to repositories.

- Remember that the given path for mercurial_ repositories must be write 

  accessible for the application. It's very important since RhodeCode web 

  interface will work even without such an access but, when trying to do a 

  push it'll eventually fail with permission denied errors. 

You are ready to use rhodecode, to run it simply execute

::

 paster serve production.ini

- This command runs the RhodeCode server the app should be available at the 

  127.0.0.1:5000. This ip and port is configurable via the production.ini 

  file created in previous step

- Use admin account you created to login.

- Default permissions on each repository is read, and owner is admin. So 

  remember to update these if needed. In the admin panel You can toggle ldap,

  anonymous, permissions settings. As well as edit more advanced options on 

  users and repositories

Using RhodeCode with SSH

------------------------

RhodeCode repository structures are kept in directories with the same name 

as the project, when using repository groups, each group is a a subdirectory.

This will allow You to use ssh for accessing repositories quite easy. There

are some exceptions when using ssh for accessing repositories.

You have to make sure that the webserver as well as the ssh users have unix

permission for directories. Secondly when using ssh rhodecode will not 

authenticate those requests and permissions set by the web interface will not

work on the repositories accessed via ssh. There is a solution to this to use 

auth hooks, that connects to rhodecode db, and runs check functions for

permissions.

TODO: post more info on this !

if Your main directory (the same as set in RhodeCode settings) is set to

for example `\home\hg` and repository You are using is `rhodecode`

The command runned should look like this::

 hg clone ssh://user@server.com/home/hg/rhodecode

Using external tools such as mercurial server or using ssh key based auth is

fully supported.

Setting up Whoosh full text search

----------------------------------

Starting from version 1.1 whoosh index can be build using paster command.

You have to specify the config file that stores location of index, and

location of repositories (`--repo-location`). Starting from version 1.2 it is 

also possible to specify a comma separated list of repositories (`--index-only`)

to build index only on chooses repositories skipping any other found in repos

location

There is possible also to pass `-f` to the options

to enable full index rebuild. Without that indexing will run always in in

incremental mode.

incremental mode::

 paster make-index production.ini --repo-location=<location for repos> 

for full index rebuild You can use::

 paster make-index production.ini -f --repo-location=<location for repos>

building index just for chosen repositories is possible with such command::

 paster make-index production.ini --repo-location=<location for repos> --index-only=vcs,rhodecode

In order to do periodical index builds and keep Your index always up to date.

It's recommended to do a crontab entry for incremental indexing. 

An example entry might look like this

::

 /path/to/python/bin/paster /path/to/rhodecode/production.ini --repo-location=<location for repos> 

When using incremental (default) mode whoosh will check last modification date 

of each file and add it to reindex if newer file is available. Also indexing 

daemon checks for removed files and removes them from index. 

Sometime You might want to rebuild index from scratch. You can do that using 

the `-f` flag passed to paster command or, in admin panel You can check 

`build from scratch` flag.

Setting up LDAP support

-----------------------

RhodeCode starting from version 1.1 supports ldap authentication. In order

via pypi, so You can install it by running

::

 easy_install python-ldap

::

 pip install python-ldap

.. note::

   python-ldap requires some certain libs on Your system, so before installing 

   it check that You have at least `openldap`, and `sasl` libraries.

Setting Up Celery

-----------------

Since version 1.1 celery is configured by the rhodecode ini configuration files

simply set use_celery=true in the ini file then add / change the configuration 

variables inside the ini file.

Remember that the ini files uses format with '.' not with '_' like celery

so for example setting `BROKER_HOST` in celery means setting `broker.host` in

the config file.

In order to make start using celery run::

 paster celeryd <configfile.ini>

.. note::

   Make sure You run this command from same virtualenv, and with the same user

   that rhodecode runs.

Nginx virtual host example

--------------------------

Sample config for nginx using proxy::

 server {

    listen          80;

    server_name     hg.myserver.com;

    access_log      /var/log/nginx/rhodecode.access.log;

    error_log       /var/log/nginx/rhodecode.error.log;

    location / {

            root /var/www/rhodecode/rhodecode/public/;

            if (!-f $request_filename){

                proxy_pass      http://127.0.0.1:5000;

            }

            #this is important if You want to use https !!!

            proxy_set_header X-Url-Scheme $scheme;

            include         /etc/nginx/proxy.conf;  

    }

 }  

Here's the proxy.conf. It's tuned so it'll not timeout on long

pushes and also on large pushes::

    proxy_redirect              off;

    proxy_set_header            Host $host;

    proxy_set_header            X-Host $http_host;

    proxy_set_header            X-Real-IP $remote_addr;

    proxy_set_header            X-Forwarded-For $proxy_add_x_forwarded_for;

    proxy_set_header            Proxy-host $proxy_host;

    client_max_body_size        400m;

    client_body_buffer_size     128k;

    proxy_buffering             off;

    proxy_connect_timeout       3600;

    proxy_send_timeout          3600;

    proxy_read_timeout          3600;

    proxy_buffer_size           16k;

    proxy_buffers               4 16k;

    proxy_busy_buffers_size     64k;

    proxy_temp_file_write_size  64k;

Also when using root path with nginx You might set the static files to false

in production.ini file::

  [app:main]

    use = egg:rhodecode

    full_stack = true

    static_files = false

    lang=en

    cache_dir = %(here)s/data

To not have the statics served by the application. And improve speed.

Apache virtual host example

---------------------------

Sample config for apache using proxy::

    <VirtualHost *:80>

            ServerName hg.myserver.com

            ServerAlias hg.myserver.com

            <Proxy *>

              Order allow,deny

              Allow from all

            </Proxy>

            #important !

            #Directive to properly generate url (clone url) for pylons

            ProxyPreserveHost On

            #rhodecode instance

            ProxyPass / http://127.0.0.1:5000/

            ProxyPassReverse / http://127.0.0.1:5000/

            #to enable https use line below

            #SetEnvIf X-Url-Scheme https HTTPS=1

    </VirtualHost> 

Additional tutorial

http://wiki.pylonshq.com/display/pylonscookbook/Apache+as+a+reverse+proxy+for+Pylons

Apache's example FCGI config

----------------------------

TODO !

Other configuration files

-------------------------

Some example init.d script can be found here, for debian and gentoo:

https://rhodeocode.org/rhodecode/files/tip/init.d

Troubleshooting

---------------

- missing static files ?

 - make sure either to set the `static_files = true` in the .ini file or

   double check the root path for Your http setup. It should point to 

   for example:

   /home/my-virtual-python/lib/python2.6/site-packages/rhodecode/public

- can't install celery/rabbitmq

 - don't worry RhodeCode works without them too. No extra setup required

- long lasting push timeouts ?

 - make sure You set a longer timeouts in Your proxy/fcgi settings, timeouts

   are caused by https server and not RhodeCode

- large pushes timeouts ?

 - make sure You set a proper max_body_size for the http server

.. _virtualenv: http://pypi.python.org/pypi/virtualenv

.. _python: http://www.python.org/

.. _mercurial: http://mercurial.selenic.com/

.. _celery: http://celeryproject.org/

# -*- coding: utf-8 -*-

"""

    rhodecode.controllers.admin.ldap_settings

    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    ldap controller for RhodeCode

    :created_on: Nov 26, 2010

    :author: marcink

    :copyright: (C) 2009-2011 Marcin Kuzminski <marcin@python-works.com>    

    :license: GPLv3, see COPYING for more details.

"""

# This program is free software; you can redistribute it and/or

# modify it under the terms of the GNU General Public License

# as published by the Free Software Foundation; version 2

# of the License or (at your opinion) any later version of the license.

# 

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

# 

# You should have received a copy of the GNU General Public License

# along with this program; if not, write to the Free Software

# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,

# MA  02110-1301, USA.

import logging

import formencode

import traceback

from formencode import htmlfill

from pylons import request, response, session, tmpl_context as c, url

from pylons.controllers.util import abort, redirect

from pylons.i18n.translation import _

from rhodecode.lib.base import BaseController, render

from rhodecode.lib import helpers as h

from rhodecode.lib.auth import LoginRequired, HasPermissionAllDecorator

from rhodecode.lib.auth_ldap import LdapImportError

from rhodecode.model.settings import SettingsModel

from rhodecode.model.forms import LdapSettingsForm

from sqlalchemy.exc import DatabaseError

log = logging.getLogger(__name__)

class LdapSettingsController(BaseController):

    @LoginRequired()

    @HasPermissionAllDecorator('hg.admin')

    def __before__(self):

        c.admin_user = session.get('admin_user')

        c.admin_username = session.get('admin_username')

        super(LdapSettingsController, self).__before__()

    def index(self):

        defaults = SettingsModel().get_ldap_settings()

        return htmlfill.render(

                    render('admin/ldap/ldap.html'),

                    defaults=defaults,

                    encoding="UTF-8",

                    force_defaults=True,)

    def ldap_settings(self):

        """POST ldap create and store ldap settings"""

        settings_model = SettingsModel()

        try:

            form_result = _form.to_python(dict(request.POST))

            try:

                for k, v in form_result.items():

                    if k.startswith('ldap_'):

                        setting = settings_model.get(k)

                        setting.app_settings_value = v

                        self.sa.add(setting)

                self.sa.commit()

                h.flash(_('Ldap settings updated successfully'),

                    category='success')

            except (DatabaseError,):

                raise

        except LdapImportError:

            h.flash(_('Unable to activate ldap. The "python-ldap" library '

                      'is missing.'), category='warning')

        except formencode.Invalid, errors:

            return htmlfill.render(

                render('admin/ldap/ldap.html'),

                defaults=errors.value,

                errors=errors.error_dict or {},

                prefix_error=False,

                encoding="UTF-8")

        except Exception:

            log.error(traceback.format_exc())

            h.flash(_('error occurred during update of ldap settings'),

                    category='error')

        return redirect(url('ldap_home'))

# -*- coding: utf-8 -*-

"""

    rhodecode.lib.auth

    ~~~~~~~~~~~~~~~~~~

    authentication and permission libraries

    :created_on: Apr 4, 2010

    :copyright: (c) 2010 by marcink.

    :license: LICENSE_NAME, see LICENSE_FILE for more details.

"""

# This program is free software; you can redistribute it and/or

# modify it under the terms of the GNU General Public License

# as published by the Free Software Foundation; version 2

# of the License or (at your opinion) any later version of the license.

# 

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

# 

# You should have received a copy of the GNU General Public License

# along with this program; if not, write to the Free Software

# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,

# MA  02110-1301, USA.

import bcrypt

import random

import logging

import traceback

from decorator import decorator

from pylons import config, session, url, request

from pylons.controllers.util import abort, redirect

from rhodecode.lib.exceptions import LdapPasswordError, LdapUsernameError

from rhodecode.lib.utils import get_repo_slug

from rhodecode.lib.auth_ldap import AuthLdap

from rhodecode.model import meta

from rhodecode.model.user import UserModel

from rhodecode.model.db import User, RepoToPerm, Repository, Permission, \

    UserToPerm

log = logging.getLogger(__name__)

class PasswordGenerator(object):

    """This is a simple class for generating password from

        different sets of characters

        usage:

        passwd_gen = PasswordGenerator()

        #print 8-letter password containing only big and small letters of alphabet

        print passwd_gen.gen_password(8, passwd_gen.ALPHABETS_BIG_SMALL)        

    """

    ALPHABETS_NUM = r'''1234567890'''#[0]

    ALPHABETS_SMALL = r'''qwertyuiopasdfghjklzxcvbnm'''#[1]

    ALPHABETS_BIG = r'''QWERTYUIOPASDFGHJKLZXCVBNM'''#[2]

    ALPHABETS_SPECIAL = r'''`-=[]\;',./~!@#$%^&*()_+{}|:"<>?'''    #[3]

    ALPHABETS_FULL = ALPHABETS_BIG + ALPHABETS_SMALL + ALPHABETS_NUM + ALPHABETS_SPECIAL#[4]

    ALPHABETS_ALPHANUM = ALPHABETS_BIG + ALPHABETS_SMALL + ALPHABETS_NUM#[5]

    ALPHABETS_BIG_SMALL = ALPHABETS_BIG + ALPHABETS_SMALL

    ALPHABETS_ALPHANUM_BIG = ALPHABETS_BIG + ALPHABETS_NUM#[6]

    ALPHABETS_ALPHANUM_SMALL = ALPHABETS_SMALL + ALPHABETS_NUM#[7]

    def __init__(self, passwd=''):

        self.passwd = passwd

    def gen_password(self, len, type):

        self.passwd = ''.join([random.choice(type) for _ in xrange(len)])

        return self.passwd

def get_crypt_password(password):

    """Cryptographic function used for password hashing based on sha1

    :param password: password to hash

    """

    return bcrypt.hashpw(password, bcrypt.gensalt(10))

def check_password(password, hashed):

    return bcrypt.hashpw(password, hashed) == hashed

def authfunc(environ, username, password):

    """

    Dummy authentication function used in Mercurial/Git/ and access control,

    :param environ: needed only for using in Basic auth

    """

    return authenticate(username, password)

def authenticate(username, password):

    """

    Authentication function used for access control,

    firstly checks for db authentication then if ldap is enabled for ldap

    authentication, also creates ldap user if not in database

    :param username: username

    :param password: password

    """

    user_model = UserModel()

    user = user_model.get_by_username(username, cache=False)

    log.debug('Authenticating user using RhodeCode account')

        if user.active:

            if user.username == 'default' and user.active:

                log.info('user %s authenticated correctly as anonymous user',

                         username)

                return True

            elif user.username == username and check_password(password, user.password):

                log.info('user %s authenticated correctly', username)

                return True

        else:

            log.warning('user %s is disabled', username)

    else:

        log.debug('Regular authentication failed')

        user_obj = user_model.get_by_username(username, cache=False,

                                            case_insensitive=True)

            log.debug('this user already exists as non ldap')

            return False

        from rhodecode.model.settings import SettingsModel

        ldap_settings = SettingsModel().get_ldap_settings()

        #======================================================================

        # FALLBACK TO LDAP AUTH IN ENABLE                

        #======================================================================

        if ldap_settings.get('ldap_active', False):

            log.debug("Authenticating user using ldap")

            kwargs = {

                  'server':ldap_settings.get('ldap_host', ''),

                  'base_dn':ldap_settings.get('ldap_base_dn', ''),

                  'port':ldap_settings.get('ldap_port'),

                  'bind_dn':ldap_settings.get('ldap_dn_user'),

                  'bind_pass':ldap_settings.get('ldap_dn_pass'),

                  'use_ldaps':ldap_settings.get('ldap_ldaps'),

                  'ldap_version':3,

                  }

            log.debug('Checking for ldap authentication')

            try:

                aldap = AuthLdap(**kwargs)

                    log.info('created new ldap user')

                return True

            except (LdapUsernameError, LdapPasswordError,):

                pass

            except (Exception,):

                log.error(traceback.format_exc())

                pass

    return False

class  AuthUser(object):

    """

    A simple object that handles a mercurial username for authentication

    """

    def __init__(self):

        self.username = 'None'

        self.name = ''

        self.lastname = ''

        self.email = ''

        self.user_id = None

        self.is_authenticated = False

        self.is_admin = False

        self.permissions = {}

    def __repr__(self):

        return "<AuthUser('id:%s:%s')>" % (self.user_id, self.username)

def set_available_permissions(config):

    """This function will propagate pylons globals with all available defined

    permission given in db. We don't wannt to check each time from db for new 

    permissions since adding a new permission also requires application restart

    ie. to decorate new views with the newly created permission

    :param config: current pylons config instance

    """

    log.info('getting information about all available permissions')

    try:

        sa = meta.Session()

        all_perms = sa.query(Permission).all()

    except:

        pass

    finally:

        meta.Session.remove()

    config['available_permissions'] = [x.permission_name for x in all_perms]

def set_base_path(config):

    config['base_path'] = config['pylons.app_globals'].base_path

def fill_perms(user):

    """Fills user permission attribute with permissions taken from database

    :param user:

    """