.. _changelog:

Changelog

=========

1.3.0 (**2012-02-26**)

----------------------

news

++++

- code review, inspired by github code-comments 

- #215 rst and markdown README files support

- #252 Container-based and proxy pass-through authentication support

- #44 branch browser. Filtering of changelog by branches

- mercurial bookmarks support

- new hover top menu, optimized to add maximum size for important views

- configurable clone url template with possibility to specify  protocol like 

  ssh:// or http:// and also manually alter other parts of clone_url.

- enabled largefiles extension by default

- optimized summary file pages and saved a lot of unused space in them

- #239 option to manually mark repository as fork

- #320 mapping of commit authors to RhodeCode users

- #304 hashes are displayed using monospace font    

- diff configuration, toggle white lines and context lines

- #307 configurable diffs, whitespace toggle, increasing context lines

- sorting on branches, tags and bookmarks using YUI datatable

- improved file filter on files page

- implements #330 api method for listing nodes ar particular revision

- #73 added linking issues in commit messages to chosen issue tracker url

  based on user defined regular expression

- added linking of changesets in commit messages  

- new compact changelog with expandable commit messages

- firstname and lastname are optional in user creation

- #348 added post-create repository hook

- #212 global encoding settings is now configurable from .ini files 

- #227 added repository groups permissions

- markdown gets codehilite extensions

- new API methods, delete_repositories, grante/revoke permissions for groups 

  and repos

fixes

+++++

- rewrote dbsession management for atomic operations, and better error handling

- fixed sorting of repo tables

- #326 escape of special html entities in diffs

- normalized user_name => username in api attributes

- fixes #298 ldap created users with mixed case emails created conflicts 

  on saving a form

- fixes issue when owner of a repo couldn't revoke permissions for users 

  and groups

- fixes #271 rare JSON serialization problem with statistics

- fixes #337 missing validation check for conflicting names of a group with a

  repositories group

- #340 fixed session problem for mysql and celery tasks

- fixed #331 RhodeCode mangles repository names if the a repository group 

  contains the "full path" to the repositories

- #355 RhodeCode doesn't store encrypted LDAP passwords

1.2.5 (**2012-01-28**)

----------------------

news

++++

fixes

+++++

- #340 Celery complains about MySQL server gone away, added session cleanup

  for celery tasks

- #341 "scanning for repositories in None" log message during Rescan was missing

  a parameter

- fixed creating archives with subrepos. Some hooks were triggered during that

  operation leading to crash.

- fixed missing email in account page.

- Reverted Mercurial to 2.0.1 for windows due to bug in Mercurial that makes

  forking on windows impossible 

1.2.4 (**2012-01-19**)

----------------------

news

++++

- RhodeCode is bundled with mercurial series 2.0.X by default, with

  full support to largefiles extension. Enabled by default in new installations

- #329 Ability to Add/Remove Groups to/from a Repository via AP

- added requires.txt file with requirements

fixes

+++++

- fixes db session issues with celery when emailing admins

- #331 RhodeCode mangles repository names if the a repository group 

  contains the "full path" to the repositories

- #298 Conflicting e-mail addresses for LDAP and RhodeCode users

- DB session cleanup after hg protocol operations, fixes issues with

  `mysql has gone away` errors

- #333 doc fixes for get_repo api function

            % (repo_name, revision, ext)

        import tempfile

        archive = tempfile.mkstemp()[1]

        t = open(archive, 'wb')

        cs.fill_archive(stream=t, kind=fileformat, subrepos=subrepos)

        def get_chunked_archive(archive):

            stream = open(archive, 'rb')

            while True:

                data = stream.read(4096)

                if not data:

                    os.remove(archive)

                    break

                yield data

        return get_chunked_archive(archive)

    @HasRepoPermissionAnyDecorator('repository.read', 'repository.write',

                                   'repository.admin')

    def diff(self, repo_name, f_path):

        ignore_whitespace = request.GET.get('ignorews') == '1'

        line_context = request.GET.get('context', 3)

        diff1 = request.GET.get('diff1', '')

        diff2 = request.GET.get('diff2', '')

        c.action = request.GET.get('diff')

        c.no_changes = diff1 == diff2

        c.f_path = f_path

        c.big_diff = False

        c.anchor_url = anchor_url

        c.ignorews_url = _ignorews_url

        c.context_url = _context_url

        c.changes = OrderedDict()

        c.changes[diff2] = []

        try:

            if diff1 not in ['', None, 'None', '0' * 12, '0' * 40]:

                c.changeset_1 = c.rhodecode_repo.get_changeset(diff1)

                node1 = c.changeset_1.get_node(f_path)

            else:

                c.changeset_1 = EmptyChangeset(repo=c.rhodecode_repo)

                node1 = FileNode('.', '', changeset=c.changeset_1)

            if diff2 not in ['', None, 'None', '0' * 12, '0' * 40]:

                c.changeset_2 = c.rhodecode_repo.get_changeset(diff2)

                node2 = c.changeset_2.get_node(f_path)

            else:

                c.changeset_2 = EmptyChangeset(repo=c.rhodecode_repo)

                node2 = FileNode('.', '', changeset=c.changeset_2)

        except RepositoryError:

            return redirect(url('files_home', repo_name=c.repo_name,

                                f_path=f_path))

        if c.action == 'download':

            _diff = diffs.get_gitdiff(node1, node2,

                                      ignore_whitespace=ignore_whitespace,

                                      context=line_context)

            diff = diffs.DiffProcessor(_diff, format='gitdiff')

            diff_name = '%s_vs_%s.diff' % (diff1, diff2)

            response.content_type = 'text/plain'

            response.content_disposition = 'attachment; filename=%s' \

                                                    % diff_name

            return diff.raw_diff()

        elif c.action == 'raw':

            _diff = diffs.get_gitdiff(node1, node2,

                                      ignore_whitespace=ignore_whitespace,

                                      context=line_context)

            diff = diffs.DiffProcessor(_diff, format='gitdiff')

            response.content_type = 'text/plain'

            return diff.raw_diff()

        else:

            fid = h.FID(diff2, node2.path)

            line_context_lcl = get_line_ctx(fid, request.GET)

            ign_whitespace_lcl = get_ignore_ws(fid, request.GET)

            lim = request.GET.get('fulldiff') or self.cut_off_limit

            _, cs1, cs2, diff, st = wrapped_diff(filenode_old=node1,

                                         filenode_new=node2,

                                         cut_off_limit=lim,

                                         ignore_whitespace=ign_whitespace_lcl,

                                         line_context=line_context_lcl,

                                         enable_comments=False)

            c.changes = [('', node2, diff, cs1, cs2, st,)]

        return render('files/file_diff.html')

    def _get_node_history(self, cs, f_path):

        changesets = cs.get_file_history(f_path)

        hist_l = []

        changesets_group = ([], _("Changesets"))

        branches_group = ([], _("Branches"))

        tags_group = ([], _("Tags"))

        for chs in changesets:

            changesets_group[0].append((chs.raw_id, n_desc,))

        hist_l.append(changesets_group)

        for name, chs in c.rhodecode_repo.branches.items():

            branches_group[0].append((chs, name),)

        hist_l.append(branches_group)

        for name, chs in c.rhodecode_repo.tags.items():

            tags_group[0].append((chs, name),)

        hist_l.append(tags_group)

        return hist_l

    @jsonify

    @HasRepoPermissionAnyDecorator('repository.read', 'repository.write',

                                   'repository.admin')

    def nodelist(self, repo_name, revision, f_path):

        if request.environ.get('HTTP_X_PARTIAL_XHR'):

            cs = self.__get_cs_or_redirect(revision, repo_name)

            _d, _f = ScmModel().get_nodes(repo_name, cs.raw_id, f_path,

                                          flat=False)

            return _d + _f

# -*- coding: utf-8 -*-

"""

    rhodecode.controllers.login

    ~~~~~~~~~~~~~~~~~~~~~~~~~~~

    Login controller for rhodeocode

    :created_on: Apr 22, 2010

    :author: marcink

    :copyright: (C) 2010-2012 Marcin Kuzminski <marcin@python-works.com>

    :license: GPLv3, see COPYING for more details.

"""

# This program is free software: you can redistribute it and/or modify

# it under the terms of the GNU General Public License as published by

# the Free Software Foundation, either version 3 of the License, or

# (at your option) any later version.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program.  If not, see <http://www.gnu.org/licenses/>.

import logging

import formencode

from formencode import htmlfill

from pylons.i18n.translation import _

from pylons.controllers.util import abort, redirect

from pylons import request, response, session, tmpl_context as c, url

import rhodecode.lib.helpers as h

from rhodecode.lib.auth import AuthUser, HasPermissionAnyDecorator

from rhodecode.lib.base import BaseController, render

from rhodecode.model.db import User

from rhodecode.model.forms import LoginForm, RegisterForm, PasswordResetForm

from rhodecode.model.user import UserModel

from rhodecode.model.meta import Session

log = logging.getLogger(__name__)

class LoginController(BaseController):

    def __before__(self):

        super(LoginController, self).__before__()

    def index(self):

        # redirect if already logged in

        c.came_from = request.GET.get('came_from', None)

        if self.rhodecode_user.is_authenticated \

                            and self.rhodecode_user.username != 'default':

            return redirect(url('home'))

        if request.POST:

            # import Login Form validator class

            login_form = LoginForm()

            try:

                c.form_result = login_form.to_python(dict(request.POST))

                # form checks for username/password, now we're authenticated

                username = c.form_result['username']

                user = User.get_by_username(username, case_insensitive=True)

                auth_user = AuthUser(user.user_id)

                auth_user.set_authenticated()

                cs = auth_user.get_cookie_store()

                session['rhodecode_user'] = cs

                # If they want to be remembered, update the cookie

                if c.form_result['remember'] is not False:

                    session.cookie_expires = False

                session._update_cookie_out()

                session.save()

                log.info('user %s is now authenticated and stored in '

                         'session, session attrs %s' % (username, cs))

                user.update_lastlogin()

                Session.commit()

                if c.came_from:

                    return redirect(c.came_from)

                else:

                    return redirect(url('home'))

            except formencode.Invalid, errors:

                return htmlfill.render(

                    render('/login.html'),

                    defaults=errors.value,

                    errors=errors.error_dict or {},

                    prefix_error=False,

                    encoding="UTF-8")

        return render('/login.html')

    @HasPermissionAnyDecorator('hg.admin', 'hg.register.auto_activate',

                               'hg.register.manual_activate')

    def register(self):

        c.auto_active = False

        for perm in User.get_by_username('default').user_perms:

            if perm.permission.permission_name == 'hg.register.auto_activate':

                c.auto_active = True

                break

        if request.POST:

            register_form = RegisterForm()()

            try:

                form_result = register_form.to_python(dict(request.POST))

                form_result['active'] = c.auto_active

                UserModel().create_registration(form_result)

                h.flash(_('You have successfully registered into rhodecode'),

                            category='success')

                Session.commit()

                return redirect(url('login_home'))

            except formencode.Invalid, errors:

                return htmlfill.render(

                    render('/register.html'),

                    defaults=errors.value,

                    errors=errors.error_dict or {},

                    prefix_error=False,

                    encoding="UTF-8")

        return render('/register.html')

    def password_reset(self):

        if request.POST:

            password_reset_form = PasswordResetForm()()

            try:

                form_result = password_reset_form.to_python(dict(request.POST))

                UserModel().reset_password_link(form_result)

                h.flash(_('Your password reset link was sent'),

                            category='success')

                return redirect(url('login_home'))

            except formencode.Invalid, errors:

                return htmlfill.render(

                    render('/password_reset.html'),

                    defaults=errors.value,

                    errors=errors.error_dict or {},

                    prefix_error=False,

                    encoding="UTF-8")

        return render('/password_reset.html')

    def password_reset_confirmation(self):

        if request.GET and request.GET.get('key'):

            try:

                user = User.get_by_api_key(request.GET.get('key'))

                data = dict(email=user.email)

                UserModel().reset_password(data)

                h.flash(_('Your password reset was successful, '

                          'new password has been sent to your email'),

                            category='success')

            except Exception, e:

                log.error(e)

                return redirect(url('reset_password'))

        return redirect(url('login_home'))

    def logout(self):

        session.delete()

        log.info('Logging out and deleting session for user')

        redirect(url('home'))

    if not user.active:

        return None

    user.update_lastlogin()

    Session.commit()

    log.debug('User %s is now logged in by container authentication',

              user.username)

    return user

def get_container_username(environ, config):

    username = None

    if str2bool(config.get('container_auth_enabled', False)):

        from paste.httpheaders import REMOTE_USER

        username = REMOTE_USER(environ)

    if not username and str2bool(config.get('proxypass_auth_enabled', False)):

        username = environ.get('HTTP_X_FORWARDED_USER')

    if username:

        # Removing realm and domain from username

        username = username.partition('@')[0]

        username = username.rpartition('\\')[2]

        log.debug('Received username %s from container' % username)

    return username

class CookieStoreWrapper(object):

    def __init__(self, cookie_store):

        self.cookie_store = cookie_store

    def __repr__(self):

        return 'CookieStore<%s>' % (self.cookie_store)

    def get(self, key, other=None):

        if isinstance(self.cookie_store, dict):

            return self.cookie_store.get(key, other)

        elif isinstance(self.cookie_store, AuthUser):

            return self.cookie_store.__dict__.get(key, other)

class  AuthUser(object):

    """

    A simple object that handles all attributes of user in RhodeCode

    It does lookup based on API key,given user, or user present in session

    Then it fills all required information for such user. It also checks if

    anonymous access is enabled and if so, it returns default user as logged

    in

    """

    def __init__(self, user_id=None, api_key=None, username=None):

        self.user_id = user_id

        self.api_key = None

        self.username = username

        self.name = ''

        self.lastname = ''

        self.email = ''

        self.is_authenticated = False

        self.admin = False

        self.permissions = {}

        self._api_key = api_key

        self.propagate_data()

        self._instance = None

    def propagate_data(self):

        user_model = UserModel()

        self.anonymous_user = User.get_by_username('default', cache=True)

        is_user_loaded = False

        # try go get user by api key

        if self._api_key and self._api_key != self.anonymous_user.api_key:

            log.debug('Auth User lookup by API KEY %s' % self._api_key)

            is_user_loaded = user_model.fill_data(self, api_key=self._api_key)

        # lookup by userid

        elif (self.user_id is not None and

              self.user_id != self.anonymous_user.user_id):

            log.debug('Auth User lookup by USER ID %s' % self.user_id)

            is_user_loaded = user_model.fill_data(self, user_id=self.user_id)

        # lookup by username

        elif self.username and \

            str2bool(config.get('container_auth_enabled', False)):

            log.debug('Auth User lookup by USER NAME %s' % self.username)

            dbuser = login_container_auth(self.username)

            if dbuser is not None:

                for k, v in dbuser.get_dict().items():

                    setattr(self, k, v)

                self.set_authenticated()

                is_user_loaded = True

        if not is_user_loaded:

            # if we cannot authenticate user try anonymous

            if self.anonymous_user.active is True:

                user_model.fill_data(self, user_id=self.anonymous_user.user_id)

                # then we set this user is logged in

                self.is_authenticated = True

            else:

                self.user_id = None

                self.username = None

                self.is_authenticated = False

        if not self.username:

            self.username = 'None'

        log.debug('Auth User is now %s' % self)

        user_model.fill_perms(self)

    @property

    def is_admin(self):

        return self.admin

    def __repr__(self):

        return "<AuthUser('id:%s:%s|%s')>" % (self.user_id, self.username,

                                              self.is_authenticated)

    def set_authenticated(self, authenticated=True):

        if self.user_id != self.anonymous_user.user_id:

            self.is_authenticated = authenticated

    def get_cookie_store(self):

        return {'username': self.username,

                'user_id': self.user_id,

                'is_authenticated': self.is_authenticated}

    @classmethod

    def from_cookie_store(cls, cookie_store):

        """

        Creates AuthUser from a cookie store

        :param cls:

        :param cookie_store:

        """

        user_id = cookie_store.get('user_id')

        username = cookie_store.get('username')

        api_key = cookie_store.get('api_key')

        return AuthUser(user_id, api_key, username)

def set_available_permissions(config):

    """

    This function will propagate pylons globals with all available defined

    permission given in db. We don't want to check each time from db for new

    permissions since adding a new permission also requires application restart

    ie. to decorate new views with the newly created permission

    :param config: current pylons config instance

    """

    log.info('getting information about all available permissions')

    try:

        sa = meta.Session

        all_perms = sa.query(Permission).all()

    except Exception:

        pass

    finally:

        meta.Session.remove()

    config['available_permissions'] = [x.permission_name for x in all_perms]

#==============================================================================

# CHECK DECORATORS

#==============================================================================

class LoginRequired(object):

    """

    Must be logged in to execute this function else

    redirect to login page

    :param api_access: if enabled this checks only for valid auth token

        and grants access based on valid token

    """

    def __init__(self, api_access=False):

        self.api_access = api_access

    def __call__(self, func):

        return decorator(self.__wrapper, func)

    def __wrapper(self, func, *fargs, **fkwargs):

        cls = fargs[0]

        user = cls.rhodecode_user

        api_access_ok = False

        if self.api_access:

            log.debug('Checking API KEY access for %s' % cls)

    def check_permissions(self):

        if self.required_perms.intersection(self.user_perms.get('global')):

            return True

        return False

class HasRepoPermissionAllDecorator(PermsDecorator):

    """

    Checks for access permission for all given predicates for specific

    repository. All of them have to be meet in order to fulfill the request

    """

    def check_permissions(self):

        repo_name = get_repo_slug(request)

        try:

            user_perms = set([self.user_perms['repositories'][repo_name]])

        except KeyError:

            return False

        if self.required_perms.issubset(user_perms):

            return True

        return False

class HasRepoPermissionAnyDecorator(PermsDecorator):

    """

    Checks for access permission for any of given predicates for specific

    repository. In order to fulfill the request any of predicates must be meet

    """

    def check_permissions(self):

        repo_name = get_repo_slug(request)

        try:

            user_perms = set([self.user_perms['repositories'][repo_name]])

        except KeyError:

            return False

        if self.required_perms.intersection(user_perms):

            return True

        return False

class HasReposGroupPermissionAllDecorator(PermsDecorator):

    """

    Checks for access permission for all given predicates for specific

    repository. All of them have to be meet in order to fulfill the request

    """

    def check_permissions(self):

        group_name = get_repos_group_slug(request)

        try:

            user_perms = set([self.user_perms['repositories_groups'][group_name]])

        except KeyError:

            return False

        if self.required_perms.issubset(user_perms):

            return True

        return False

class HasReposGroupPermissionAnyDecorator(PermsDecorator):

    """

    Checks for access permission for any of given predicates for specific

    repository. In order to fulfill the request any of predicates must be meet

    """

    def check_permissions(self):

        group_name = get_repos_group_slug(request)

        try:

            user_perms = set([self.user_perms['repositories_groups'][group_name]])

        except KeyError:

            return False

        if self.required_perms.intersection(user_perms):

            return True

        return False

#==============================================================================

# CHECK FUNCTIONS

#==============================================================================

class PermsFunction(object):

    """Base function for other check functions"""

    def __init__(self, *perms):

        available_perms = config['available_permissions']

        for perm in perms:

            if perm not in available_perms:

                raise Exception("'%s' permission in not defined" % perm)

        self.required_perms = set(perms)

        self.user_perms = None

        self.granted_for = ''

        self.repo_name = None

    def __call__(self, check_Location=''):

        user = request.user

        if not user:

            return False

        self.user_perms = user.permissions

        self.granted_for = user

        if self.check_permissions():

            log.debug('Permission granted %s @ %s', self.granted_for,

                      check_Location or 'unspecified location')

            return True

        else:

            log.debug('Permission denied for %s @ %s', self.granted_for,

                        check_Location or 'unspecified location')

            return False

    def check_permissions(self):

        """Dummy function for overriding"""

        raise Exception('You have to write this function in child class')

class HasPermissionAll(PermsFunction):

    def check_permissions(self):

        if self.required_perms.issubset(self.user_perms.get('global')):

            return True

        return False

class HasPermissionAny(PermsFunction):

    def check_permissions(self):

        if self.required_perms.intersection(self.user_perms.get('global')):

            return True

        return False

class HasRepoPermissionAll(PermsFunction):

    def __call__(self, repo_name=None, check_Location=''):

        self.repo_name = repo_name

        return super(HasRepoPermissionAll, self).__call__(check_Location)

    def check_permissions(self):

        if not self.repo_name:

            self.repo_name = get_repo_slug(request)

        try:

            self.user_perms = set(

                [self.user_perms['repositories'][self.repo_name]]

            )

        except KeyError:

            return False

        self.granted_for = self.repo_name

        if self.required_perms.issubset(self.user_perms):

            return True

        return False

class HasRepoPermissionAny(PermsFunction):

    def __call__(self, repo_name=None, check_Location=''):

        self.repo_name = repo_name

        return super(HasRepoPermissionAny, self).__call__(check_Location)

    def check_permissions(self):

        if not self.repo_name:

            self.repo_name = get_repo_slug(request)

        try:

            self.user_perms = set(

                [self.user_perms['repositories'][self.repo_name]]

            )

        except KeyError:

            return False

        self.granted_for = self.repo_name

        if self.required_perms.intersection(self.user_perms):

            return True

        return False

class HasReposGroupPermissionAny(PermsFunction):

    def __call__(self, group_name=None, check_Location=''):

        self.group_name = group_name

        return super(HasReposGroupPermissionAny, self).__call__(check_Location)

    def check_permissions(self):

        try:

            self.user_perms = set(

                [self.user_perms['repositories_groups'][self.group_name]]

            )

        except KeyError:

            return False

        self.granted_for = self.repo_name

        if self.required_perms.intersection(self.user_perms):

            return True

        return False

class HasReposGroupPermissionAll(PermsFunction):

    def __call__(self, group_name=None, check_Location=''):

        self.group_name = group_name

        return super(HasReposGroupPermissionAny, self).__call__(check_Location)

        raise NotImplementedError()

    def _get_by_id(self, repo_name):

        """

        Get's a special pattern _<ID> from clone url and tries to replace it

        with a repository_name for support of _<ID> non changable urls

        :param repo_name:

        """

        try:

            data = repo_name.split('/')

            if len(data) >= 2:

                by_id = data[1].split('_')

                if len(by_id) == 2 and by_id[1].isdigit():

                    _repo_name = Repository.get(by_id[1]).repo_name

                    data[1] = _repo_name

        except:

            log.debug('Failed to extract repo_name from id %s' % (

                      traceback.format_exc()

                      )

            )

        return '/'.join(data)

    def _invalidate_cache(self, repo_name):

        """

        Set's cache for this repository for invalidation on next access

        :param repo_name: full repo name, also a cache key

        """

        invalidate_cache('get_repo_cached_%s' % repo_name)

    def _check_permission(self, action, user, repo_name):

        """

        Checks permissions using action (push/pull) user and repository

        name

        :param action: push or pull action

        :param user: user instance

        :param repo_name: repository name

        """

        if action == 'push':

            if not HasPermissionAnyMiddleware('repository.write',

                                              'repository.admin')(user,

                                                                  repo_name):

                return False

        else:

            #any other action need at least read permission