.. _changelog:

Changelog

=========

1.3.0 (**XXXX-XX-XX**)

======================

:status: in-progress

:branch: beta

news

----

- code review, inspired by github code-comments 

- #215 rst and markdown README files support

- #252 Container-based and proxy pass-through authentication support

- #44 branch browser. Filtering of changelog by branches

- mercurial bookmarks support

- new hover top menu, optimized to add maximum size for important views

- configurable clone url template with possibility to specify  protocol like 

  ssh:// or http:// and also manually alter other parts of clone_url.

- enabled largefiles extension by default

- optimized summary file pages and saved a lot of unused space in them

- #239 option to manually mark repository as fork

- #320 mapping of commit authors to RhodeCode users

- #304 hashes are displayed using monospace font    

- diff configuration, toggle white lines and context lines

- #307 configurable diffs, whitespace toggle, increasing context lines

- sorting on branches, tags and bookmarks using YUI datatable

- improved file filter on files page

- implements #330 api method for listing nodes ar particular revision

- #73 added linking issues in commit messages to chosen issue tracker url

  based on user defined regular expression

- added linking of changesets in commit messages  

- new compact changelog with expandable commit messages

- firstname and lastname are optional in user creation

fixes

-----

- rewrote dbsession management for atomic operations, and better error handling

- fixed sorting of repo tables

- #326 escape of special html entities in diffs

- normalized user_name => username in api attributes

- fixes #298 ldap created users with mixed case emails created conflicts 

  on saving a form

- fixes issue when owner of a repo couldn't revoke permissions for users 

  and groups

- fixes #271 rare JSON serialization problem with statistics

- fixes #337 missing validation check for conflicting names of a group with a

  repositories group

- #340 fixed session problem for mysql and celery tasks

- fixed #331 RhodeCode mangles repository names if the a repository group 

  contains the "full path" to the repositories

1.2.5 (**2012-01-28**)

======================

news

----

fixes

-----

- #340 Celery complains about MySQL server gone away, added session cleanup

  for celery tasks

- #341 "scanning for repositories in None" log message during Rescan was missing

  a parameter

- fixed creating archives with subrepos. Some hooks were triggered during that

  operation leading to crash.

- fixed missing email in account page.

- Reverted Mercurial to 2.0.1 for windows due to bug in Mercurial that makes

  forking on windows impossible 

1.2.4 (**2012-01-19**)

======================

news

----

- RhodeCode is bundled with mercurial series 2.0.X by default, with

  full support to largefiles extension. Enabled by default in new installations

- #329 Ability to Add/Remove Groups to/from a Repository via AP

- added requires.txt file with requirements

fixes

-----

- fixes db session issues with celery when emailing admins

- #331 RhodeCode mangles repository names if the a repository group 

  contains the "full path" to the repositories

- #298 Conflicting e-mail addresses for LDAP and RhodeCode users

- DB session cleanup after hg protocol operations, fixes issues with

  `mysql has gone away` errors

- #333 doc fixes for get_repo api function

- #271 rare JSON serialization problem with statistics enabled

- #337 Fixes issues with validation of repository name conflicting with 

  a group name. A proper message is now displayed.

- #292 made ldap_dn in user edit readonly, to get rid of confusion that field

  doesn't work   

- #316 fixes issues with web description in hgrc files 

1.2.3 (**2011-11-02**)

======================

news

----

- added option to manage repos group for non admin users

- added following API methods for get_users, create_user, get_users_groups, 

  get_users_group, create_users_group, add_user_to_users_groups, get_repos, 

  get_repo, create_repo, add_user_to_repo

- implements #237 added password confirmation for my account 

  and admin edit user.

- implements #291 email notification for global events are now sent to all

  administrator users, and global config email.

fixes

-----

- added option for passing auth method for smtp mailer

- #276 issue with adding a single user with id>10 to usergroups

- #277 fixes windows LDAP settings in which missing values breaks the ldap auth 

- #288 fixes managing of repos in a group for non admin user

1.2.2 (**2011-10-17**)

======================

news

----

- #226 repo groups are available by path instead of numerical id

from rhodecode.lib import helpers as h

from rhodecode.lib.utils import action_logger

def repo_size(ui, repo, hooktype=None, **kwargs):

    """

    Presents size of repository after push

    :param ui:

    :param repo:

    :param hooktype:

    """

    size_hg, size_root = 0, 0

    for path, dirs, files in os.walk(repo.root):

        if path.find('.hg') != -1:

            for f in files:

                try:

                    size_hg += os.path.getsize(os.path.join(path, f))

                except OSError:

                    pass

        else:

            for f in files:

                try:

                    size_root += os.path.getsize(os.path.join(path, f))

                except OSError:

                    pass

    size_hg_f = h.format_byte_size(size_hg)

    size_root_f = h.format_byte_size(size_root)

    size_total_f = h.format_byte_size(size_root + size_hg)

    last_cs = repo[len(repo) - 1]

    msg = ('Repository size .hg:%s repo:%s total:%s\n'

           'Last revision is now r%s:%s\n') % (

        size_hg_f, size_root_f, size_total_f, last_cs.rev(), last_cs.hex()[:12]

    )

    sys.stdout.write(msg)

def log_pull_action(ui, repo, **kwargs):

    """

    Logs user last pull action

    :param ui:

    :param repo:

    """

    extra_params = dict(repo.ui.configitems('rhodecode_extras'))

    username = extra_params['username']

    repository = extra_params['repository']

    action = 'pull'

    action_logger(username, action, repository, extra_params['ip'],

                  commit=True)

    return 0

def log_push_action(ui, repo, **kwargs):

    """

    Maps user last push action to new changeset id, from mercurial

    :param ui:

    :param repo:

    """

    extra_params = dict(repo.ui.configitems('rhodecode_extras'))

    username = extra_params['username']

    repository = extra_params['repository']

    action = extra_params['action'] + ':%s'

    node = kwargs['node']

    def get_revs(repo, rev_opt):

        if rev_opt:

            revs = revrange(repo, rev_opt)

            if len(revs) == 0:

                return (nullrev, nullrev)

            return (max(revs), min(revs))

        else:

            return (len(repo) - 1, 0)

    stop, start = get_revs(repo, [node + ':'])

    revs = (str(repo[r]) for r in xrange(start, stop + 1))

    action = action % ','.join(revs)

    action_logger(username, action, repository, extra_params['ip'],

                  commit=True)

    return 0

# -*- coding: utf-8 -*-

"""

    rhodecode.model.repo

    ~~~~~~~~~~~~~~~~~~~~

    Repository model for rhodecode

    :created_on: Jun 5, 2010

    :author: marcink

    :copyright: (C) 2010-2012 Marcin Kuzminski <marcin@python-works.com>

    :license: GPLv3, see COPYING for more details.

"""

# This program is free software: you can redistribute it and/or modify

# it under the terms of the GNU General Public License as published by

# the Free Software Foundation, either version 3 of the License, or

# (at your option) any later version.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program.  If not, see <http://www.gnu.org/licenses/>.

import os

import shutil

import logging

import traceback

from datetime import datetime

from vcs.utils.lazy import LazyProperty

from vcs.backends import get_backend

from rhodecode.lib import safe_str

from rhodecode.lib.caching_query import FromCache

from rhodecode.model import BaseModel

from rhodecode.model.db import Repository, UserRepoToPerm, User, Permission, \

    Statistics, UsersGroup, UsersGroupRepoToPerm, RhodeCodeUi, RepoGroup

log = logging.getLogger(__name__)

class RepoModel(BaseModel):

    @LazyProperty

    def repos_path(self):

        """

        Get's the repositories root path from database

        """

        q = self.sa.query(RhodeCodeUi).filter(RhodeCodeUi.ui_key == '/').one()

        return q.ui_value

    def get(self, repo_id, cache=False):

        repo = self.sa.query(Repository)\

            .filter(Repository.repo_id == repo_id)

        if cache:

            repo = repo.options(FromCache("sql_cache_short",

                                          "get_repo_%s" % repo_id))

        return repo.scalar()

    def get_by_repo_name(self, repo_name, cache=False):

        repo = self.sa.query(Repository)\

            .filter(Repository.repo_name == repo_name)

        if cache:

            repo = repo.options(FromCache("sql_cache_short",

                                          "get_repo_%s" % repo_name))

        return repo.scalar()

    def get_users_js(self):

        users = self.sa.query(User).filter(User.active == True).all()

        u_tmpl = '''{id:%s, fname:"%s", lname:"%s", nname:"%s"},'''

        users_array = '[%s]' % '\n'.join([u_tmpl % (u.user_id, u.name,

                                                    u.lastname, u.username)

                                        for u in users])

        return users_array

    def get_users_groups_js(self):

        users_groups = self.sa.query(UsersGroup)\

            .filter(UsersGroup.users_group_active == True).all()

        g_tmpl = '''{id:%s, grname:"%s",grmembers:"%s"},'''

        users_groups_array = '[%s]' % '\n'.join([g_tmpl % \

                                    (gr.users_group_id, gr.users_group_name,

                                     len(gr.members))

                                        for gr in users_groups])

        return users_groups_array

    def _get_defaults(self, repo_name):

        """

        Get's information about repository, and returns a dict for

        usage in forms

        :param repo_name:

        """

        repo_info = Repository.get_by_repo_name(repo_name)

        if repo_info is None:

            return None

        defaults = repo_info.get_dict()

        group, repo_name = repo_info.groups_and_repo

        defaults['repo_name'] = repo_name

        defaults['repo_group'] = getattr(group[-1] if group else None,

                                         'group_id', None)

        # fill owner

        if repo_info.user:

            defaults.update({'user': repo_info.user.username})

        else:

            replacement_user = User.query().filter(User.admin ==

                                                   True).first().username

            defaults.update({'user': replacement_user})

        # fill repository users

        for p in repo_info.repo_to_perm:

            defaults.update({'u_perm_%s' % p.user.username:

                             p.permission.permission_name})

        # fill repository groups

        for p in repo_info.users_group_to_perm:

            defaults.update({'g_perm_%s' % p.users_group.users_group_name:

                             p.permission.permission_name})

        return defaults

    def update(self, repo_name, form_data):

        try:

            cur_repo = self.get_by_repo_name(repo_name, cache=False)

            # update permissions

            for member, perm, member_type in form_data['perms_updates']:

                if member_type == 'user':

                    _member = User.get_by_username(member)

                    r2p = self.sa.query(UserRepoToPerm)\

                        .filter(UserRepoToPerm.user == _member)\

                        .filter(UserRepoToPerm.repository == cur_repo)\

                        .one()

                    r2p.permission = self.sa.query(Permission)\

                                        .filter(Permission.permission_name ==

                                                perm).scalar()

                    self.sa.add(r2p)

                else:

                    g2p = self.sa.query(UsersGroupRepoToPerm)\

                            .filter(UsersGroupRepoToPerm.users_group ==

                                    UsersGroup.get_by_group_name(member))\

                            .filter(UsersGroupRepoToPerm.repository ==

                                    cur_repo).one()

                    g2p.permission = self.sa.query(Permission)\

                                        .filter(Permission.permission_name ==

                                                perm).scalar()

                    self.sa.add(g2p)

            # set new permissions

            for member, perm, member_type in form_data['perms_new']:

                if member_type == 'user':

                    r2p = UserRepoToPerm()

                    r2p.repository = cur_repo

                    r2p.user = User.get_by_username(member)

                    r2p.permission = self.sa.query(Permission)\

                                        .filter(Permission.

                                                permission_name == perm)\

                                                .scalar()

                    self.sa.add(r2p)

                else:

                    g2p = UsersGroupRepoToPerm()

                    g2p.repository = cur_repo

                    g2p.users_group = UsersGroup.get_by_group_name(member)

                    g2p.permission = self.sa.query(Permission)\

                                        .filter(Permission.

                                                permission_name == perm)\

                                                .scalar()

                    self.sa.add(g2p)

            # update current repo

            for k, v in form_data.items():

                if k == 'user':

                    cur_repo.user = User.get_by_username(v)

                elif k == 'repo_name':

                    pass

                elif k == 'repo_group':

                    cur_repo.group = RepoGroup.get(v)

                else:

                    setattr(cur_repo, k, v)

            new_name = cur_repo.get_new_name(form_data['repo_name'])

            cur_repo.repo_name = new_name

            self.sa.add(cur_repo)

            if repo_name != new_name:

                # rename repository

                self.__rename_repo(old=repo_name, new=new_name)

            return cur_repo

        except:

            log.error(traceback.format_exc())

            raise

    def create(self, form_data, cur_user, just_db=False, fork=False):

        from rhodecode.model.scm import ScmModel

        try:

            if fork:

                fork_parent_id = form_data['fork_parent_id']

            # repo name is just a name of repository

            # while repo_name_full is a full qualified name that is combined

            # with name and path of group

            repo_name = form_data['repo_name']

            repo_name_full = form_data['repo_name_full']

            new_repo = Repository()

            new_repo.enable_statistics = False

            for k, v in form_data.items():

                if k == 'repo_name':

                    v = repo_name_full

                if k == 'repo_group':

                    k = 'group_id'

                if k == 'description':

                    v = v or repo_name

                setattr(new_repo, k, v)

            if fork:

                parent_repo = Repository.get(fork_parent_id)

                new_repo.fork = parent_repo

            new_repo.user_id = cur_user.user_id

            self.sa.add(new_repo)

            def _create_default_perms():

                # create default permission

                repo_to_perm = UserRepoToPerm()

                default = 'repository.read'

                for p in User.get_by_username('default').user_perms:

                    if p.permission.permission_name.startswith('repository.'):

                        default = p.permission.permission_name

                        break

                default_perm = 'repository.none' if form_data['private'] else default

                repo_to_perm.permission_id = self.sa.query(Permission)\

                        .filter(Permission.permission_name == default_perm)\

                        .one().permission_id

                repo_to_perm.repository = new_repo

                repo_to_perm.user_id = User.get_by_username('default').user_id

                self.sa.add(repo_to_perm)

            if fork:

                if form_data.get('copy_permissions'):

                    repo = Repository.get(fork_parent_id)

                    user_perms = UserRepoToPerm.query()\

                        .filter(UserRepoToPerm.repository == repo).all()

                    group_perms = UsersGroupRepoToPerm.query()\

                        .filter(UsersGroupRepoToPerm.repository == repo).all()

                    for perm in user_perms:

                        UserRepoToPerm.create(perm.user, new_repo,

                                              perm.permission)

                    for perm in group_perms:

                        UsersGroupRepoToPerm.create(perm.users_group, new_repo,

                                                    perm.permission)

                else:

                    _create_default_perms()

            else:

                _create_default_perms()

            if not just_db:

                self.__create_repo(repo_name, form_data['repo_type'],

                                   form_data['repo_group'],

                                   form_data['clone_uri'])

            # now automatically start following this repository as owner

            ScmModel(self.sa).toggle_following_repo(new_repo.repo_id,

                                                    cur_user.user_id)

            return new_repo

        except:

            log.error(traceback.format_exc())

            raise

    def create_fork(self, form_data, cur_user):

        """

        Simple wrapper into executing celery task for fork creation

        :param form_data:

        :param cur_user:

        """

        from rhodecode.lib.celerylib import tasks, run_task

        run_task(tasks.create_repo_fork, form_data, cur_user)

    def delete(self, repo):

        try:

            self.sa.delete(repo)

            self.__delete_repo(repo)

        except:

            log.error(traceback.format_exc())

            raise

    def delete_perm_user(self, form_data, repo_name):

        try:

            obj = self.sa.query(UserRepoToPerm)\

                .filter(UserRepoToPerm.repository \

                        == self.get_by_repo_name(repo_name))\

                .filter(UserRepoToPerm.user_id == form_data['user_id']).one()

            self.sa.delete(obj)

        except:

            log.error(traceback.format_exc())

            raise

    def delete_perm_users_group(self, form_data, repo_name):

        try:

            obj = self.sa.query(UsersGroupRepoToPerm)\

                .filter(UsersGroupRepoToPerm.repository \

                        == self.get_by_repo_name(repo_name))\

                .filter(UsersGroupRepoToPerm.users_group_id

                        == form_data['users_group_id']).one()

            self.sa.delete(obj)

        except:

            log.error(traceback.format_exc())

            raise

    def delete_stats(self, repo_name):

        """

        removes stats for given repo

        :param repo_name:

        """

        try:

            obj = self.sa.query(Statistics)\

                    .filter(Statistics.repository == \

                        self.get_by_repo_name(repo_name)).one()

            self.sa.delete(obj)

        except:

            log.error(traceback.format_exc())

            raise

    def __create_repo(self, repo_name, alias, new_parent_id, clone_uri=False):

        """

        makes repository on filesystem. It's group aware means it'll create

        a repository within a group, and alter the paths accordingly of

        group location

        :param repo_name:

        :param alias:

        :param parent_id:

        :param clone_uri:

        """

        from rhodecode.lib.utils import is_valid_repo, is_valid_repos_group

        if new_parent_id:

            paths = RepoGroup.get(new_parent_id)\

                .full_path.split(RepoGroup.url_sep())

            new_parent_path = os.sep.join(paths)

        else:

            new_parent_path = ''

        repo_path = os.path.join(*map(lambda x:safe_str(x),

                                [self.repos_path, new_parent_path, repo_name]))

        # check if this path is not a repository

        if is_valid_repo(repo_path, self.repos_path):

            raise Exception('This path %s is a valid repository' % repo_path)

        # check if this path is a group

        if is_valid_repos_group(repo_path, self.repos_path):

            raise Exception('This path %s is a valid group' % repo_path)

        log.info('creating repo %s in %s @ %s', repo_name, repo_path,

                 clone_uri)

        backend = get_backend(alias)