.. _changelog:

=========

Changelog

=========

1.4.0 (**2012-09-03**)

----------------------

news

++++

- new codereview system

- email map, allowing users to have multiple email addresses mapped into

  their accounts

- improved git-hook system. Now all actions for git are logged into journal

  including pushed revisions, user and IP address

- changed setup-app into setup-rhodecode and added default options to it.

- new git repos are created as bare now by default

- #464 added links to groups in permission box

- #465 mentions autocomplete inside comments boxes

- #469 added --update-only option to whoosh to re-index only given list

  of repos in index 

- rhodecode-api CLI client

- new git http protocol replaced buggy dulwich implementation.

  Now based on pygrack & gitweb

- Improved RSS/ATOM feeds. Discoverable by browsers using proper headers, and 

  reformated based on user suggestions. Additional rss/atom feeds for user

  journal

- various i18n improvements

- #478 permissions overview for admin in user edit view

- File view now displays small gravatars off all authors of given file

- Implemented landing revisions. Each repository will get landing_rev attribute

  that defines 'default' revision/branch for generating readme files

- Implemented #509, RhodeCode enforces SSL for push/pulling if requested at 

  earliest possible call.

- Import remote svn repositories to mercurial using hgsubversion.

- Fixed #508 RhodeCode now has a option to explicitly set forking permissions

- RhodeCode can use alternative server for generating avatar icons

- implemented repositories locking. Pull locks, push unlocks. Also can be done

  via API calls

- #538 form for permissions can handle multiple users at once 

fixes

+++++

- improved translations

- fixes issue #455 Creating an archive generates an exception on Windows

- fixes #448 Download ZIP archive keeps file in /tmp open and results 

  in out of disk space

- fixes issue #454 Search results under Windows include proceeding

  backslash

- fixed issue #450. Rhodecode no longer will crash when bad revision is

  present in journal data.

- fix for issue #417, git execution was broken on windows for certain

  commands.

- fixed #413. Don't disable .git directory for bare repos on deleting

- fixed issue #459. Changed the way of obtaining logger in reindex task.

- fixed #453 added ID field in whoosh SCHEMA that solves the issue of

  reindexing modified files

- fixed #481 rhodecode emails are sent without Date header 

- fixed #458 wrong count when no repos are present

- fixed issue #492 missing `\ No newline at end of file` test at the end of 

  new chunk in html diff

- full text search now works also for commit messages

1.3.6 (**2012-05-17**)

----------------------

news

++++

- chinese traditional translation

- changed setup-app into setup-rhodecode and added arguments for auto-setup 

  mode that doesn't need user interaction 

fixes

+++++

- fixed no scm found warning

- fixed __future__ import error on rcextensions

- made simplejson required lib for speedup on JSON encoding

- fixes #449 bad regex could get more than revisions from parsing history

- don't clear DB session when CELERY_EAGER is turned ON

1.3.5 (**2012-05-10**)

----------------------

news

++++

- use ext_json for json module

- unified annotation view with file source view

- notification improvements, better inbox + css

- #419 don't strip passwords for login forms, make rhodecode 

  more compatible with LDAP servers

- Added HTTP_X_FORWARDED_FOR as another method of extracting 

  IP for pull/push logs. - moved all to base controller  

- #415: Adding comment to changeset causes reload. 

  Comments are now added via ajax and doesn't reload the page

- #374 LDAP config is discarded when LDAP can't be activated

- limited push/pull operations are now logged for git in the journal

.. _installation:

============

Installation

============

``RhodeCode`` is written entirely in Python. Before posting any issues make 

sure, your not missing any system libraries and using right version of 

libraries required by RhodeCode. There's also restriction in terms of mercurial

clients. Minimal version of hg client known working fine with RhodeCode is

**1.6**. If you're using older client, please upgrade.

Rhodecode requires python version 2.5 or higher.

The easiest way to install ``rhodecode`` is to run::

    easy_install rhodecode

Or::

    pip install rhodecode

If you prefer to install RhodeCode manually simply grab latest release from

http://pypi.python.org/pypi/RhodeCode, decompress the archive and run::

    python setup.py install

Step by step installation example for Windows

---------------------------------------------

:ref:`installation_win`

Step by step installation example for Linux

-------------------------------------------

For installing RhodeCode i highly recommend using separate virtualenv_. This

way many required by RhodeCode libraries will remain sandboxed from your main

python and making things less problematic when doing system python updates. 

- Assuming you have installed virtualenv_ create a new virtual environment 

  using virtualenv command:: 

    virtualenv --no-site-packages /opt/rhodecode-venv

.. note:: Using ``--no-site-packages`` when generating your

   virtualenv is **very important**. This flag provides the necessary

   isolation for running the set of packages required by

   RhodeCode.  If you do not specify ``--no-site-packages``,

   it's possible that RhodeCode will not install properly into

   the virtualenv, or, even if it does, may not run properly,

   depending on the packages you've already got installed into your

   Python's "main" site-packages dir.

- this will install new virtualenv_ into `/opt/rhodecode-venv`. 

- Activate the virtualenv_ by running::

    source /opt/rhodecode-venv/bin/activate

.. note:: If you're using UNIX, *do not* use ``sudo`` to run the

   ``virtualenv`` script.  It's perfectly acceptable (and desirable)

   to create a virtualenv as a normal user.

- Make a folder for rhodecode data files, and configuration somewhere on the 

  filesystem. For example::

    mkdir /opt/rhodecode

- Go into the created directory run this command to install rhodecode::

    easy_install rhodecode

  or::

    pip install rhodecode 

- This will install rhodecode together with pylons and all other required 

  python libraries into activated virtualenv

Requirements for Celery (optional)

----------------------------------

In order to gain maximum performance

there are some third-party you must install. When RhodeCode is used 

together with celery you have to install some kind of message broker,

recommended one is rabbitmq_ to make the async tasks work.

Of course RhodeCode works in sync mode also and then you do not have to install

any third party applications. However, using Celery_ will give you a large 

speed improvement when using many big repositories. If you plan to use 

RhodeCode for say 7 to 10 repositories, RhodeCode will perform perfectly well 

without celery running.

If you make the decision to run RhodeCode with celery make sure you run 

celeryd using paster and message broker together with the application. 

.. note::

   Installing message broker and using celery is optional, RhodeCode will

   work perfectly fine without them.

**Message Broker** 

- preferred is `RabbitMq <http://www.rabbitmq.com/>`_

- A possible alternative is `Redis <http://code.google.com/p/redis/>`_

For installation instructions you can visit: 

http://ask.github.com/celery/getting-started/index.html.

This is a very nice tutorial on how to start using celery_ with rabbitmq_

You can now proceed to :ref:`setup`

-----------------------------------

.. _virtualenv: http://pypi.python.org/pypi/virtualenv  

.. _python: http://www.python.org/

.. _mercurial: http://mercurial.selenic.com/

.. _celery: http://celeryproject.org/

    proxy_buffers               8 32k;

Also, when using root path with nginx you might set the static files to false

in the production.ini file::

    [app:main]

      use = egg:rhodecode

      full_stack = true

      static_files = false

      lang=en

      cache_dir = %(here)s/data

In order to not have the statics served by the application. This improves speed.

Apache virtual host reverse proxy example

-----------------------------------------

Here is a sample configuration file for apache using proxy::

    <VirtualHost *:80>

            ServerName hg.myserver.com

            ServerAlias hg.myserver.com

            <Proxy *>

              Order allow,deny

              Allow from all

            </Proxy>

            #important !

            #Directive to properly generate url (clone url) for pylons

            ProxyPreserveHost On

            #rhodecode instance

            ProxyPass / http://127.0.0.1:5000/

            ProxyPassReverse / http://127.0.0.1:5000/

            #to enable https use line below

            #SetEnvIf X-Url-Scheme https HTTPS=1

    </VirtualHost> 

Additional tutorial

http://wiki.pylonshq.com/display/pylonscookbook/Apache+as+a+reverse+proxy+for+Pylons

Apache as subdirectory

----------------------

Apache subdirectory part::

    <Location /<someprefix> >

      ProxyPass http://127.0.0.1:5000/<someprefix>

      ProxyPassReverse http://127.0.0.1:5000/<someprefix>

      SetEnvIf X-Url-Scheme https HTTPS=1

    </Location> 

Besides the regular apache setup you will need to add the following line

into [app:main] section of your .ini file::

    filter-with = proxy-prefix

Add the following at the end of the .ini file::

    [filter:proxy-prefix]

    use = egg:PasteDeploy#prefix

    prefix = /<someprefix> 

then change <someprefix> into your choosen prefix

Apache's WSGI config

--------------------

Alternatively, RhodeCode can be set up with Apache under mod_wsgi. For

that, you'll need to:

- Install mod_wsgi. If using a Debian-based distro, you can install

  the package libapache2-mod-wsgi::

    aptitude install libapache2-mod-wsgi

- Enable mod_wsgi::

    a2enmod wsgi

- Create a wsgi dispatch script, like the one below. Make sure you

  check the paths correctly point to where you installed RhodeCode

  and its Python Virtual Environment.

- Enable the WSGIScriptAlias directive for the wsgi dispatch script,

  as in the following example. Once again, check the paths are

  correctly specified.

Here is a sample excerpt from an Apache Virtual Host configuration file::

        threads=4 \

        python-path=/home/web/rhodecode/pyenv/lib/python2.6/site-packages

    WSGIScriptAlias / /home/web/rhodecode/dispatch.wsgi

    WSGIPassAuthorization On

Example wsgi dispatch script::

    import os

    os.environ["HGENCODING"] = "UTF-8"

    os.environ['PYTHON_EGG_CACHE'] = '/home/web/rhodecode/.egg-cache'

    # sometimes it's needed to set the curent dir

    os.chdir('/home/web/rhodecode/') 

    import site

    site.addsitedir("/home/web/rhodecode/pyenv/lib/python2.6/site-packages")

    from paste.deploy import loadapp

    from paste.script.util.logging_config import fileConfig

    fileConfig('/home/web/rhodecode/production.ini')

    application = loadapp('config:/home/web/rhodecode/production.ini')

Note: when using mod_wsgi you'll need to install the same version of

Mercurial that's inside RhodeCode's virtualenv also on the system's Python

environment.

Other configuration files

-------------------------

Some example init.d scripts can be found in init.d directory::

  https://secure.rhodecode.org/rhodecode/files/beta/init.d

.. _virtualenv: http://pypi.python.org/pypi/virtualenv

.. _python: http://www.python.org/

.. _mercurial: http://mercurial.selenic.com/

.. _celery: http://celeryproject.org/

.. _rabbitmq: http://www.rabbitmq.com/

.. _python-ldap: http://www.python-ldap.org/

.. _mercurial-server: http://www.lshift.net/mercurial-server.html

.. _PublishingRepositories: http://mercurial.selenic.com/wiki/PublishingRepositories

.. _Issues tracker: https://bitbucket.org/marcinkuzminski/rhodecode/issues

.. _google group rhodecode: http://groups.google.com/group/rhodecode

.. _upgrade:

=======

Upgrade

=======

.. note::

 easy_install -U rhodecode

Or::

 pip install --upgrade rhodecode

 paster make-config RhodeCode production.ini

This will display any changes made by the new version of RhodeCode to your

.. note::

   Please always make sure your .ini files are up to date. Often errors are

   caused by missing params added in new versions.

It is also recommended that you rebuild the whoosh index after upgrading since 

the new whoosh version could introduce some incompatible index changes. Please

Read the changelog to see if there were any changes to whoosh.

The final step is to upgrade the database. To do this simply run::

This will upgrade the schema and update some of the defaults in the database,

and will always recheck the settings of the application, if there are no new 

options that need to be set.

.. note::

   If you're using Celery, make sure you restart all instances of it after

   upgrade.

.. _virtualenv: http://pypi.python.org/pypi/virtualenv  

.. _python: http://www.python.org/

.. _mercurial: http://mercurial.selenic.com/

.. _celery: http://celeryproject.org/

# -*- coding: utf-8 -*-

"""

    rhodecode.__init__

    ~~~~~~~~~~~~~~~~~~

    RhodeCode, a web based repository management based on pylons

    versioning implementation: http://www.python.org/dev/peps/pep-0386/

    :created_on: Apr 9, 2010

    :author: marcink

    :copyright: (C) 2010-2012 Marcin Kuzminski <marcin@python-works.com>

    :license: GPLv3, see COPYING for more details.

"""

# This program is free software: you can redistribute it and/or modify

# it under the terms of the GNU General Public License as published by

# the Free Software Foundation, either version 3 of the License, or

# (at your option) any later version.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program.  If not, see <http://www.gnu.org/licenses/>.

import sys

import platform

try:

    from rhodecode.lib import get_current_revision

    _rev = get_current_revision(quiet=True)

    if _rev and len(VERSION) > 3:

        VERSION += ('dev%s' % _rev[0],)

except ImportError:

    pass

__version__ = ('.'.join((str(each) for each in VERSION[:3])) +

               '.'.join(VERSION[3:]))

__platform__ = platform.system()

__license__ = 'GPLv3'

__py_version__ = sys.version_info

__author__ = 'Marcin Kuzminski'

__url__ = 'http://rhodecode.org'

PLATFORM_WIN = ('Windows')

PLATFORM_OTHERS = ('Linux', 'Darwin', 'FreeBSD', 'OpenBSD', 'SunOS') #depracated

is_windows = __platform__ in PLATFORM_WIN

is_unix = not is_windows

BACKENDS = {

    'hg': 'Mercurial repository',

    'git': 'Git repository',

}

CELERY_ON = False

CELERY_EAGER = False

# link to config for pylons

CONFIG = {}

# Linked module for extensions

EXTENSIONS = {}

"""

Routes configuration

The more specific and detailed routes should be defined first so they

may take precedent over the more generic routes. For more information

refer to the routes manual at http://routes.groovie.org/docs/

"""

from __future__ import with_statement

from routes import Mapper

# prefix for non repository related links needs to be prefixed with `/`

ADMIN_PREFIX = '/_admin'

def make_map(config):

    """Create, configure and return the routes Mapper"""

    rmap = Mapper(directory=config['pylons.paths']['controllers'],

                 always_scan=config['debug'])

    rmap.minimization = False

    rmap.explicit = False

    from rhodecode.lib.utils import is_valid_repo

    from rhodecode.lib.utils import is_valid_repos_group

    def check_repo(environ, match_dict):

        """

        check for valid repository for proper 404 handling

        :param environ:

        :param match_dict:

        """

        from rhodecode.model.db import Repository

        repo_name = match_dict.get('repo_name')

        try:

            by_id = repo_name.split('_')

                repo_name = Repository.get(by_id[1]).repo_name

                match_dict['repo_name'] = repo_name

        except:

            pass

        return is_valid_repo(repo_name, config['base_path'])

    def check_group(environ, match_dict):

        """

        check for valid repositories group for proper 404 handling

        :param environ:

        :param match_dict:

        """

        repos_group_name = match_dict.get('group_name')

        return is_valid_repos_group(repos_group_name, config['base_path'])

    def check_int(environ, match_dict):

        return match_dict.get('id').isdigit()

    # The ErrorController route (handles 404/500 error pages); it should

    # likely stay at the top, ensuring it can always be resolved

    rmap.connect('/error/{action}', controller='error')

    rmap.connect('/error/{action}/{id}', controller='error')

    #==========================================================================

    # CUSTOM ROUTES HERE

    #==========================================================================

    #MAIN PAGE

    rmap.connect('home', '/', controller='home', action='index')

    rmap.connect('repo_switcher', '/repos', controller='home',

                 action='repo_switcher')

    rmap.connect('branch_tag_switcher', '/branches-tags/{repo_name:.*?}',

                 controller='home', action='branch_tag_switcher')

    rmap.connect('bugtracker',

                 "http://bitbucket.org/marcinkuzminski/rhodecode/issues",

                 _static=True)

    rmap.connect('rst_help',

                 "http://docutils.sourceforge.net/docs/user/rst/quickref.html",

                 _static=True)

    rmap.connect('rhodecode_official', "http://rhodecode.org", _static=True)

    #ADMIN REPOSITORY REST ROUTES

    with rmap.submapper(path_prefix=ADMIN_PREFIX,

                        controller='admin/repos') as m:

        m.connect("repos", "/repos",

             action="create", conditions=dict(method=["POST"]))

        m.connect("repos", "/repos",

             action="index", conditions=dict(method=["GET"]))

        m.connect("formatted_repos", "/repos.{format}",

             action="index",

            conditions=dict(method=["GET"]))

        m.connect("new_repo", "/repos/new",

             action="new", conditions=dict(method=["GET"]))

        m.connect("formatted_new_repo", "/repos/new.{format}",

             action="new", conditions=dict(method=["GET"]))

        m.connect("/repos/{repo_name:.*?}",

             action="update", conditions=dict(method=["PUT"],

                                              function=check_repo))

        m.connect("/repos/{repo_name:.*?}",

             action="delete", conditions=dict(method=["DELETE"],

                                              function=check_repo))

        m.connect("edit_repo", "/repos/{repo_name:.*?}/edit",

             action="edit", conditions=dict(method=["GET"],

                                            function=check_repo))

        m.connect("formatted_edit_repo", "/repos/{repo_name:.*?}.{format}/edit",

             action="edit", conditions=dict(method=["GET"],

                                            function=check_repo))

        m.connect("repo", "/repos/{repo_name:.*?}",

             action="show", conditions=dict(method=["GET"],

                                            function=check_repo))

        m.connect("formatted_repo", "/repos/{repo_name:.*?}.{format}",

             action="show", conditions=dict(method=["GET"],

                                            function=check_repo))

        #ajax delete repo perm user

        m.connect('delete_repo_user', "/repos_delete_user/{repo_name:.*?}",

             action="delete_perm_user",

             conditions=dict(method=["DELETE"], function=check_repo))

        #ajax delete repo perm users_group

        m.connect('delete_repo_users_group',

                  "/repos_delete_users_group/{repo_name:.*?}",

                  action="delete_perm_users_group",

                  conditions=dict(method=["DELETE"], function=check_repo))

        #settings actions

        m.connect('repo_stats', "/repos_stats/{repo_name:.*?}",

                  action="repo_stats", conditions=dict(method=["DELETE"],

                                                       function=check_repo))

        m.connect('repo_cache', "/repos_cache/{repo_name:.*?}",

                  action="repo_cache", conditions=dict(method=["DELETE"],

                                                       function=check_repo))

        m.connect('repo_public_journal', "/repos_public_journal/{repo_name:.*?}",

                  action="repo_public_journal", conditions=dict(method=["PUT"],

# -*- coding: utf-8 -*-

"""

    rhodecode.controllers.admin.repos_groups

    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    Repositories groups controller for RhodeCode

    :created_on: Mar 23, 2010

    :author: marcink

    :copyright: (C) 2010-2012 Marcin Kuzminski <marcin@python-works.com>

    :license: GPLv3, see COPYING for more details.

"""

# This program is free software: you can redistribute it and/or modify

# it under the terms of the GNU General Public License as published by

# the Free Software Foundation, either version 3 of the License, or

# (at your option) any later version.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program.  If not, see <http://www.gnu.org/licenses/>.

import logging

import traceback

import formencode

from formencode import htmlfill

from pylons import request, tmpl_context as c, url

from pylons.controllers.util import redirect

from pylons.i18n.translation import _

from sqlalchemy.exc import IntegrityError

from rhodecode.lib import helpers as h

from rhodecode.lib.auth import LoginRequired, HasPermissionAnyDecorator,\

    HasReposGroupPermissionAnyDecorator

from rhodecode.lib.base import BaseController, render

from rhodecode.model.db import RepoGroup

from rhodecode.model.repos_group import ReposGroupModel

from rhodecode.model.forms import ReposGroupForm

from rhodecode.model.meta import Session

from rhodecode.model.repo import RepoModel

from webob.exc import HTTPInternalServerError, HTTPNotFound

log = logging.getLogger(__name__)

class ReposGroupsController(BaseController):

    """REST Controller styled on the Atom Publishing Protocol"""

    # To properly map this controller, ensure your config/routing.py

    # file has a resource setup:

    #     map.resource('repos_group', 'repos_groups')

    @LoginRequired()

    def __before__(self):

        super(ReposGroupsController, self).__before__()

    def __load_defaults(self):

        c.repo_groups = RepoGroup.groups_choices()

        c.repo_groups_choices = map(lambda k: unicode(k[0]), c.repo_groups)

        repo_model = RepoModel()

        c.users_array = repo_model.get_users_js()

        c.users_groups_array = repo_model.get_users_groups_js()

    def __load_data(self, group_id):

        """

        Load defaults settings for edit, and update

        :param group_id:

        """

        self.__load_defaults()

        repo_group = RepoGroup.get_or_404(group_id)

        data = repo_group.get_dict()

        data['group_name'] = repo_group.name

        # fill repository users

        for p in repo_group.repo_group_to_perm:

            data.update({'u_perm_%s' % p.user.username: