List of contributors to RhodeCode project:

    Marcin Kuźmiński <marcin@python-works.com>

    Lukasz Balcerzak <lukaszbalcerzak@gmail.com>

    Jason Harris <jason@jasonfharris.com>

    Thayne Harbaugh  <thayne@fusionio.com>

    cejones <>

    Thomas Waldmann <tw-public@gmx.de>

    Lorenzo M. Catucci <lorenzo@sancho.ccd.uniroma2.it>

    Dmitri Kuznetsov <>

    Jared Bunting <jared.bunting@peachjean.com>

    Steve Romanow <slestak989@gmail.com>

    Augosto Hermann <augusto.herrmann@planejamento.gov.br>    

    Ankit Solanki <ankit.solanki@gmail.com>    

    Liad Shani <liadff@gmail.com>

    Les Peabody <lpeabody@gmail.com>

    Jonas Oberschweiber <jonas.oberschweiber@d-velop.de>

    Matt Zuba <matt.zuba@goodwillaz.org>

    Aras Pranckevicius <aras@unity3d.com>

    Tony Bussieres <t.bussieres@gmail.com>

    Erwin Kroon <e.kroon@smartmetersolutions.nl>

.. _changelog:

=========

Changelog

=========

1.4.0 (**2012-XX-XX**)

----------------------

:status: in-progress

:branch: beta

news

++++

fixes

+++++

----------------------

news

++++

- chinese traditional translation

fixes

+++++

- fixed no scm found warning

- fixed __future__ import error on rcextensions

- made simplejson required lib for speedup on JSON encoding

- fixes #449 bad regex could get more than revisions from parsing history

1.3.5 (**2012-05-10**)

----------------------

news

++++

- use ext_json for json module

- unified annotation view with file source view

- notification improvements, better inbox + css

- #419 don't strip passwords for login forms, make rhodecode 

  more compatible with LDAP servers

- Added HTTP_X_FORWARDED_FOR as another method of extracting 

  IP for pull/push logs. - moved all to base controller  

- #415: Adding comment to changeset causes reload. 

  Comments are now added via ajax and doesn't reload the page

- #374 LDAP config is discarded when LDAP can't be activated

- limited push/pull operations are now logged for git in the journal

- bumped mercurial to 2.2.X series

- added support for displaying submodules in file-browser

- #421 added bookmarks in changelog view

fixes

+++++

- fixed dev-version marker for stable when served from source codes

- fixed missing permission checks on show forks page

- #418 cast to unicode fixes in notification objects

- #426 fixed mention extracting regex

- fixed remote-pulling for git remotes remopositories

- fixed #434: Error when accessing files or changesets of a git repository 

  with submodules

- fixed issue with empty APIKEYS for users after registration ref. #438

- fixed issue with getting README files from git repositories

1.3.4 (**2012-03-28**)

----------------------

news

++++

- Whoosh logging is now controlled by the .ini files logging setup

- added clone-url into edit form on /settings page

- added help text into repo add/edit forms

- created rcextensions module with additional mappings (ref #322) and

  post push/pull/create repo hooks callbacks

- implemented #377 Users view for his own permissions on account page

- #399 added inheritance of permissions for users group on repos groups

- #401 repository group is automatically pre-selected when adding repos 

  inside a repository group

- added alternative HTTP 403 response when client failed to authenticate. Helps 

  solving issues with Mercurial and LDAP

- #402 removed group prefix from repository name when listing repositories 

  inside a group

- added gravatars into permission view and permissions autocomplete

- #347 when running multiple RhodeCode instances, properly invalidates cache 

  for all registered servers

fixes

+++++

- fixed #390 cache invalidation problems on repos inside group

- fixed #385 clone by ID url was loosing proxy prefix in URL

- fixed some unicode problems with waitress

- fixed issue with escaping < and > in changeset commits

- fixed error occurring during recursive group creation in API 

  create_repo function

- fixed #393 py2.5 fixes for routes url generator

- fixed #397 Private repository groups shows up before login

- fixed #396 fixed problems with revoking users in nested groups

- fixed mysql unicode issues + specified InnoDB as default engine with 

  utf8 charset

- #406 trim long branch/tag names in changelog to not break UI

1.3.3 (**2012-03-02**)

----------------------

news

++++

fixes

+++++

- fixed some python2.5 compatibility issues 

- fixed issues with removed repos was accidentally added as groups, after

  full rescan of paths

- fixes #376 Cannot edit user (using container auth)

- fixes #378 Invalid image urls on changeset screen with proxy-prefix 

  configuration

- fixed initial sorting of repos inside repo group

- fixes issue when user tried to resubmit same permission into user/user_groups

- bumped beaker version that fixes #375 leap error bug

- fixed raw_changeset for git. It was generated with hg patch headers

- fixed vcs issue with last_changeset for filenodes

- fixed missing commit after hook delete

.. _git_support:

===========

GIT support

===========

Although There are some limitations on git usage.

- large pushes needs http server with chunked encoding support.

if you plan to use git you need to run RhodeCode with some

http server that supports chunked encoding which git http protocol uses, 

i recommend using waitress_ or gunicorn_ (linux only) for `paste` wsgi app 

replacement.

    use = egg:Paste#http

    use = egg:waitress#main

And comment out bellow options::

    threadpool_workers = 

    threadpool_max_requests = 

    use_threadpool = 

You can simply run `paster serve` as usual.

You can always disable git/hg support by editing a 

file **rhodecode/__init__.py** and commenting out backends

.. code-block:: python

   BACKENDS = {

       'hg': 'Mercurial repository',

       #'git': 'Git repository',

   }

.. _waitress: http://pypi.python.org/pypi/waitress

.. _gunicorn: http://pypi.python.org/pypi/gunicorn

# -*- coding: utf-8 -*-

"""

    rhodecode.__init__

    ~~~~~~~~~~~~~~~~~~

    RhodeCode, a web based repository management based on pylons

    versioning implementation: http://www.python.org/dev/peps/pep-0386/

    :created_on: Apr 9, 2010

    :author: marcink

    :copyright: (C) 2010-2012 Marcin Kuzminski <marcin@python-works.com>

    :license: GPLv3, see COPYING for more details.

"""

# This program is free software: you can redistribute it and/or modify

# it under the terms of the GNU General Public License as published by

# the Free Software Foundation, either version 3 of the License, or

# (at your option) any later version.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program.  If not, see <http://www.gnu.org/licenses/>.

import sys

import platform

VERSION = (1, 4, 0, 'b')

try:

    from rhodecode.lib import get_current_revision

    _rev = get_current_revision()

    if _rev and len(VERSION) > 3:

        VERSION += ('dev%s' % _rev[0],)

except ImportError:

    pass

__version__ = ('.'.join((str(each) for each in VERSION[:3])) +

               '.'.join(VERSION[3:]))

__dbversion__ = 6  # defines current db version for migrations

__platform__ = platform.system()

__license__ = 'GPLv3'

__py_version__ = sys.version_info

PLATFORM_WIN = ('Windows')

PLATFORM_OTHERS = ('Linux', 'Darwin', 'FreeBSD', 'OpenBSD', 'SunOS')

is_windows = __platform__ in PLATFORM_WIN

is_unix = __platform__ in PLATFORM_OTHERS

requirements = [

    "Pylons==1.0.0",

    "Beaker==1.6.3",

    "WebHelpers==1.3",

    "formencode==1.2.4",

    "SQLAlchemy==0.7.6",

    "Mako==0.7.0",

    "pygments>=1.4",

    "whoosh>=2.4.0,<2.5",

    "celery>=2.2.5,<2.3",

    "babel",

    "python-dateutil>=1.5.0,<2.0.0",

    "dulwich>=0.8.5,<0.9.0",

    "webob==1.0.8",

    "markdown==2.1.1",

    "docutils==0.8.1",

    "simplejson==2.5.2",

]

if __py_version__ < (2, 6):

    requirements.append("pysqlite")

if is_windows:

    requirements.append("mercurial>=2.2.1,<2.3")

else:

    requirements.append("py-bcrypt")

    requirements.append("mercurial>=2.2.1,<2.3")

def get_version():

    """Returns shorter version (digit parts only) as string."""

    return '.'.join((str(each) for each in VERSION[:3]))

BACKENDS = {

    'hg': 'Mercurial repository',

    'git': 'Git repository',

}

CELERY_ON = False

# link to config for pylons

CONFIG = {}

# Linked module for extensions

EXTENSIONS = {}

"""Pylons environment configuration"""

import os

import logging

import rhodecode

from mako.lookup import TemplateLookup

from pylons.configuration import PylonsConfig

from pylons.error import handle_mako_error

# don't remove this import it does magic for celery

from rhodecode.lib import celerypylons

import rhodecode.lib.app_globals as app_globals

from rhodecode.config.routing import make_map

from rhodecode.lib import helpers

from rhodecode.lib.auth import set_available_permissions

from rhodecode.lib.utils import repo2db_mapper, make_ui, set_rhodecode_config,\

    load_rcextensions

from rhodecode.lib.utils2 import engine_from_config, str2bool

from rhodecode.model import init_model

from rhodecode.model.scm import ScmModel

log = logging.getLogger(__name__)

def load_environment(global_conf, app_conf, initial=False):

    """

    Configure the Pylons environment via the ``pylons.config``

    object

    """

    config = PylonsConfig()

    # Pylons paths

    root = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))

    paths = dict(

        root=root,

        controllers=os.path.join(root, 'controllers'),

        static_files=os.path.join(root, 'public'),

        templates=[os.path.join(root, 'templates')]

    )

    # Initialize config with the basic options

    config.init_app(global_conf, app_conf, package='rhodecode', paths=paths)

    # store some globals into rhodecode

    rhodecode.CELERY_ON = str2bool(config['app_conf'].get('use_celery'))

    config['routes.map'] = make_map(config)

    config['pylons.app_globals'] = app_globals.Globals(config)

    config['pylons.h'] = helpers

    rhodecode.CONFIG = config

    load_rcextensions(root_path=config['here'])

    # Setup cache object as early as possible

    import pylons

    pylons.cache._push_object(config['pylons.app_globals'].cache)

    # Create the Mako TemplateLookup, with the default auto-escaping

    config['pylons.app_globals'].mako_lookup = TemplateLookup(

        directories=paths['templates'],

        error_handler=handle_mako_error,

        module_directory=os.path.join(app_conf['cache_dir'], 'templates'),

        input_encoding='utf-8', default_filters=['escape'],

        imports=['from webhelpers.html import escape'])

    # sets the c attribute access when don't existing attribute are accessed

    config['pylons.strict_tmpl_context'] = True

    test = os.path.split(config['__file__'])[-1] == 'test.ini'

    if test:

        from rhodecode.lib.utils import create_test_env, create_test_index

        from rhodecode.tests import  TESTS_TMP_PATH

        create_test_env(TESTS_TMP_PATH, config)

        create_test_index(TESTS_TMP_PATH, config, True)

    # MULTIPLE DB configs

    # Setup the SQLAlchemy database engine

    sa_engine_db1 = engine_from_config(config, 'sqlalchemy.db1.')

    init_model(sa_engine_db1)

    repos_path = make_ui('db').configitems('paths')[0][1]

    repo2db_mapper(ScmModel().repo_scan(repos_path))

    set_available_permissions(config)

    config['base_path'] = repos_path

    set_rhodecode_config(config)

    # CONFIGURATION OPTIONS HERE (note: all config options will override

    # any Pylons config options)

    # store config reference into our module to skip import magic of

    # pylons

    rhodecode.CONFIG.update(config)

    return config

import os

from paste.script.appinstall import AbstractInstallCommand

from paste.script.command import BadCommand

from paste.deploy import appconfig

class SetupCommand(AbstractInstallCommand):

    default_verbosity = 1

    max_args = 1

    min_args = 1

    summary = "Setup an application, given a config file"

    usage = "CONFIG_FILE"

    description = """\

    Note: this is an experimental command, and it will probably change

    in several ways by the next release.

    Setup an application according to its configuration file.  This is

    the second part of a two-phase web application installation

    process (the first phase is prepare-app).  The setup process may

    consist of things like creating directories and setting up

    databases.

    """

    parser = AbstractInstallCommand.standard_parser(

        simulate=True, quiet=True, interactive=True)

    parser.add_option('--user',

                      action='store',

                      dest='username',

                      default=None,

                      help='Admin Username')

    parser.add_option('--email',

                      action='store',

                      dest='email',

                      default=None,

                      help='Admin Email')

    parser.add_option('--password',

                      action='store',

                      dest='password',

                      default=None,

                      help='Admin password min 6 chars')

    parser.add_option('--repos',

                      action='store',

                      dest='repos_location',

                      default=None,

                      help='Absolute path to repositories location')

    parser.add_option('--name',

                      action='store',

                      dest='section_name',

                      default=None,

                      help='The name of the section to set up (default: app:main)')

    def command(self):

        config_spec = self.args[0]

        section = self.options.section_name

        if section is None:

            if '#' in config_spec:

                config_spec, section = config_spec.split('#', 1)

            else:

                section = 'main'

        if not ':' in section:

            plain_section = section

            section = 'app:'+section

        else:

            plain_section = section.split(':', 1)[0]

        if not config_spec.startswith('config:'):

            config_spec = 'config:' + config_spec

        if plain_section != 'main':

            config_spec += '#' + plain_section

        config_file = config_spec[len('config:'):].split('#', 1)[0]

        config_file = os.path.join(os.getcwd(), config_file)

        self.logging_file_config(config_file)

        conf = appconfig(config_spec, relative_to=os.getcwd())

        ep_name = conf.context.entry_point_name

        ep_group = conf.context.protocol

        dist = conf.context.distribution

        if dist is None:

            raise BadCommand(

                "The section %r is not the application (probably a filter).  "

                "You should add #section_name, where section_name is the "

                "section that configures your application" % plain_section)

        installer = self.get_installer(dist, ep_group, ep_name)

        installer.setup_config(

            self, config_file, section, self.sysconfig_install_vars(installer))

        self.call_sysconfig_functions(

# -*- coding: utf-8 -*-

"""

    rhodecode.controllers.files

    ~~~~~~~~~~~~~~~~~~~~~~~~~~~

    Files controller for RhodeCode

    :created_on: Apr 21, 2010

    :author: marcink

    :copyright: (C) 2010-2012 Marcin Kuzminski <marcin@python-works.com>

    :license: GPLv3, see COPYING for more details.

"""

# This program is free software: you can redistribute it and/or modify

# it under the terms of the GNU General Public License as published by

# the Free Software Foundation, either version 3 of the License, or

# (at your option) any later version.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program.  If not, see <http://www.gnu.org/licenses/>.

import os

import logging

import traceback

import tempfile

from pylons import request, response, tmpl_context as c, url

from pylons.i18n.translation import _

from pylons.controllers.util import redirect

from pylons.decorators import jsonify

from rhodecode.lib import diffs

from rhodecode.lib import helpers as h

from rhodecode.lib.compat import OrderedDict

from rhodecode.lib.utils2 import convert_line_endings, detect_mode, safe_str

from rhodecode.lib.auth import LoginRequired, HasRepoPermissionAnyDecorator

from rhodecode.lib.base import BaseRepoController, render

from rhodecode.lib.utils import EmptyChangeset

from rhodecode.lib.vcs.conf import settings

from rhodecode.lib.vcs.exceptions import RepositoryError, \

    ChangesetDoesNotExistError, EmptyRepositoryError, \

    ImproperArchiveTypeError, VCSError, NodeAlreadyExistsError

from rhodecode.lib.vcs.nodes import FileNode

from rhodecode.model.repo import RepoModel

from rhodecode.model.scm import ScmModel

from rhodecode.model.db import Repository

from rhodecode.controllers.changeset import anchor_url, _ignorews_url,\

    _context_url, get_line_ctx, get_ignore_ws

log = logging.getLogger(__name__)

class FilesController(BaseRepoController):

    @LoginRequired()

    def __before__(self):

        super(FilesController, self).__before__()

        c.cut_off_limit = self.cut_off_limit

    def __get_cs_or_redirect(self, rev, repo_name, redirect_after=True):

        """

        Safe way to get changeset if error occur it redirects to tip with

        proper message

        :param rev: revision to fetch

        :param repo_name: repo name to redirect after

        """

        try:

            return c.rhodecode_repo.get_changeset(rev)

        except EmptyRepositoryError, e:

            if not redirect_after:

                return None

            url_ = url('files_add_home',

                       repo_name=c.repo_name,

                       revision=0, f_path='')

            add_new = '<a href="%s">[%s]</a>' % (url_, _('add new'))

            h.flash(h.literal(_('There are no files yet %s' % add_new)),

                    category='warning')

            redirect(h.url('summary_home', repo_name=repo_name))

        except RepositoryError, e:

            h.flash(str(e), category='warning')

            redirect(h.url('files_home', repo_name=repo_name, revision='tip'))

    def __get_filenode_or_redirect(self, repo_name, cs, path):

        """

        Returns file_node, if error occurs or given path is directory,

        it'll redirect to top level path

        :param repo_name: repo_name

        :param cs: given changeset

        :param path: path to lookup

        """

        try:

            file_node = cs.get_node(path)

            if file_node.is_dir():

                raise RepositoryError('given path is a directory')

        except RepositoryError, e:

            h.flash(str(e), category='warning')

            redirect(h.url('files_home', repo_name=repo_name,

                           revision=cs.raw_id))

        return file_node

    @HasRepoPermissionAnyDecorator('repository.read', 'repository.write',

                                   'repository.admin')

    def index(self, repo_name, revision, f_path, annotate=False):

        # redirect to given revision from form if given

        post_revision = request.POST.get('at_rev', None)

        if post_revision:

            cs = self.__get_cs_or_redirect(post_revision, repo_name)

            redirect(url('files_home', repo_name=c.repo_name,

                         revision=cs.raw_id, f_path=f_path))

        c.changeset = self.__get_cs_or_redirect(revision, repo_name)

        c.branch = request.GET.get('branch', None)

        c.f_path = f_path

        c.annotate = annotate

        cur_rev = c.changeset.revision

                h.flash(_('Error occurred during commit'), category='error')

            return redirect(url('changeset_home',

                                repo_name=c.repo_name, revision='tip'))

        return render('files/files_edit.html')

    @HasRepoPermissionAnyDecorator('repository.write', 'repository.admin')

    def add(self, repo_name, revision, f_path):

        r_post = request.POST

        c.cs = self.__get_cs_or_redirect(revision, repo_name,

                                         redirect_after=False)

        if c.cs is None:

            c.cs = EmptyChangeset(alias=c.rhodecode_repo.alias)

        c.f_path = f_path

        if r_post:

            unix_mode = 0

            content = convert_line_endings(r_post.get('content'), unix_mode)

            message = r_post.get('message') or (_('Added %s via RhodeCode')

                                                % (f_path))

            location = r_post.get('location')

            filename = r_post.get('filename')

            file_obj = r_post.get('upload_file', None)

            if file_obj is not None and hasattr(file_obj, 'filename'):

                filename = file_obj.filename

                content = file_obj.file

            node_path = os.path.join(location, filename)

            author = self.rhodecode_user.full_contact

            if not content:

                h.flash(_('No content'), category='warning')

                return redirect(url('changeset_home', repo_name=c.repo_name,

                                    revision='tip'))

            if not filename:

                h.flash(_('No filename'), category='warning')

                return redirect(url('changeset_home', repo_name=c.repo_name,

                                    revision='tip'))

            try:

                self.scm_model.create_node(repo=c.rhodecode_repo,

                                           repo_name=repo_name, cs=c.cs,

                                           user=self.rhodecode_user,

                                           author=author, message=message,

                                           content=content, f_path=node_path)

                h.flash(_('Successfully committed to %s' % node_path),

                        category='success')

            except NodeAlreadyExistsError, e:

                h.flash(_(e), category='error')

            except Exception:

                log.error(traceback.format_exc())

                h.flash(_('Error occurred during commit'), category='error')

            return redirect(url('changeset_home',

                                repo_name=c.repo_name, revision='tip'))

        return render('files/files_add.html')

    @HasRepoPermissionAnyDecorator('repository.read', 'repository.write',

                                   'repository.admin')

    def archivefile(self, repo_name, fname):

        fileformat = None

        revision = None

        ext = None

        subrepos = request.GET.get('subrepos') == 'true'

        for a_type, ext_data in settings.ARCHIVE_SPECS.items():

            archive_spec = fname.split(ext_data[1])

            if len(archive_spec) == 2 and archive_spec[1] == '':

                fileformat = a_type or ext_data[1]

                revision = archive_spec[0]

                ext = ext_data[1]

        try:

            dbrepo = RepoModel().get_by_repo_name(repo_name)

            if dbrepo.enable_downloads is False:

                return _('downloads disabled')

            if c.rhodecode_repo.alias == 'hg':

                # patch and reset hooks section of UI config to not run any

                # hooks on fetching archives with subrepos

                for k, v in c.rhodecode_repo._repo.ui.configitems('hooks'):

                    c.rhodecode_repo._repo.ui.setconfig('hooks', k, None)

            cs = c.rhodecode_repo.get_changeset(revision)

            content_type = settings.ARCHIVE_SPECS[fileformat][0]

        except ChangesetDoesNotExistError:

            return _('Unknown revision %s') % revision

        except EmptyRepositoryError:

            return _('Empty repository')

        except (ImproperArchiveTypeError, KeyError):

            return _('Unknown archive type')

            while True:

                if not data:

                    break

                yield data

    @HasRepoPermissionAnyDecorator('repository.read', 'repository.write',

                                   'repository.admin')

    def diff(self, repo_name, f_path):

        ignore_whitespace = request.GET.get('ignorews') == '1'

        line_context = request.GET.get('context', 3)

        diff1 = request.GET.get('diff1', '')

        diff2 = request.GET.get('diff2', '')

        c.action = request.GET.get('diff')

        c.no_changes = diff1 == diff2

        c.f_path = f_path

        c.big_diff = False

        c.anchor_url = anchor_url

        c.ignorews_url = _ignorews_url

        c.context_url = _context_url

        c.changes = OrderedDict()

        c.changes[diff2] = []

        try:

            if diff1 not in ['', None, 'None', '0' * 12, '0' * 40]:

                c.changeset_1 = c.rhodecode_repo.get_changeset(diff1)

                node1 = c.changeset_1.get_node(f_path)

            else:

                c.changeset_1 = EmptyChangeset(repo=c.rhodecode_repo)

                node1 = FileNode('.', '', changeset=c.changeset_1)

            if diff2 not in ['', None, 'None', '0' * 12, '0' * 40]:

                c.changeset_2 = c.rhodecode_repo.get_changeset(diff2)

                node2 = c.changeset_2.get_node(f_path)

            else:

                c.changeset_2 = EmptyChangeset(repo=c.rhodecode_repo)

                node2 = FileNode('.', '', changeset=c.changeset_2)

        except RepositoryError:

            return redirect(url('files_home', repo_name=c.repo_name,

                                f_path=f_path))

        if c.action == 'download':

            _diff = diffs.get_gitdiff(node1, node2,

                                      ignore_whitespace=ignore_whitespace,

                                      context=line_context)

            diff = diffs.DiffProcessor(_diff, format='gitdiff')

            diff_name = '%s_vs_%s.diff' % (diff1, diff2)

            response.content_type = 'text/plain'

            response.content_disposition = (

                'attachment; filename=%s' % diff_name

            )

            return diff.raw_diff()

        elif c.action == 'raw':

            _diff = diffs.get_gitdiff(node1, node2,

                                      ignore_whitespace=ignore_whitespace,