.. _changelog:

=========

Changelog

=========

1.3.4 (**2012-XX-XX**)

----------------------

:status: in-progress

:branch: beta

news

++++

- Whoosh logging is now controlled by the .ini files logging setup

- added clone-url into edit form on /settings page

- added help text into repo add/edit forms

- created rcextensions module with additional mappings (ref #322) and

  post push/pull/create repo hooks callbacks

fixes

+++++

- fixed #390 cache invalidation problems on repos inside group

- fixed #385 clone by ID url was loosing proxy prefix in URL

- fixed some unicode problems with waitress

- fixed issue with escaping < and > in changeset commits

- fixed error occurring during recursive group creation in API 

  create_repo function

- fixed #393 py2.5 fixes for routes url generator

1.3.3 (**2012-03-02**)

----------------------

news

++++

fixes

+++++

- fixed some python2.5 compatibility issues 

- fixed issues with removed repos was accidentally added as groups, after

  full rescan of paths

- fixes #376 Cannot edit user (using container auth)

- fixes #378 Invalid image urls on changeset screen with proxy-prefix 

  configuration

- fixed initial sorting of repos inside repo group

- fixes issue when user tried to resubmit same permission into user/user_groups

- bumped beaker version that fixes #375 leap error bug

- fixed raw_changeset for git. It was generated with hg patch headers

- fixed vcs issue with last_changeset for filenodes

- fixed missing commit after hook delete

- fixed #372 issues with git operation detection that caused a security issue 

  for git repos

1.3.2 (**2012-02-28**)

----------------------

news

++++

fixes

+++++

- fixed git protocol issues with repos-groups

- fixed git remote repos validator that prevented from cloning remote git repos

- fixes #370 ending slashes fixes for repo and groups

- fixes #368 improved git-protocol detection to handle other clients

- fixes #366 When Setting Repository Group To Blank Repo Group Wont Be 

  Moved To Root

- fixes #371 fixed issues with beaker/sqlalchemy and non-ascii cache keys 

- fixed #373 missing cascade drop on user_group_to_perm table

1.3.1 (**2012-02-27**)

----------------------

news

++++

fixes

+++++

- redirection loop occurs when remember-me wasn't checked during login

- fixes issues with git blob history generation 

- don't fetch branch for git in file history dropdown. Causes unneeded slowness

1.3.0 (**2012-02-26**)

----------------------

news

++++

- code review, inspired by github code-comments 

- #215 rst and markdown README files support

- #252 Container-based and proxy pass-through authentication support

- #44 branch browser. Filtering of changelog by branches

- mercurial bookmarks support

- new hover top menu, optimized to add maximum size for important views

- configurable clone url template with possibility to specify  protocol like 

  ssh:// or http:// and also manually alter other parts of clone_url.

- enabled largefiles extension by default

- optimized summary file pages and saved a lot of unused space in them

- #239 option to manually mark repository as fork

- #320 mapping of commit authors to RhodeCode users

- #304 hashes are displayed using monospace font    

- diff configuration, toggle white lines and context lines

- #307 configurable diffs, whitespace toggle, increasing context lines

- sorting on branches, tags and bookmarks using YUI datatable

- improved file filter on files page

- implements #330 api method for listing nodes ar particular revision

- #73 added linking issues in commit messages to chosen issue tracker url

  based on user defined regular expression

- added linking of changesets in commit messages  

- new compact changelog with expandable commit messages

- firstname and lastname are optional in user creation

- #348 added post-create repository hook

- #212 global encoding settings is now configurable from .ini files 

- #227 added repository groups permissions

- markdown gets codehilite extensions

- new API methods, delete_repositories, grante/revoke permissions for groups 

  and repos

                new_user.password = get_crypt_password(password)

                new_user.api_key = generate_api_key(username)

                new_user.email = attrs['email'] or generate_email(username)

                new_user.active = attrs.get('active', True)

                new_user.ldap_dn = safe_unicode(user_dn)

                new_user.name = attrs['name']

                new_user.lastname = attrs['lastname']

                self.sa.add(new_user)

                return new_user

            except (DatabaseError,):

                log.error(traceback.format_exc())

                self.sa.rollback()

                raise

        log.debug('this %s user exists skipping creation of ldap account',

                  username)

        return None

    def create_registration(self, form_data):

        from rhodecode.model.notification import NotificationModel

        try:

            new_user = User()

            for k, v in form_data.items():

                if k != 'admin':

                    setattr(new_user, k, v)

            self.sa.add(new_user)

            self.sa.flush()

            # notification to admins

            subject = _('new user registration')

            body = ('New user registration\n'

                    '---------------------\n'

                    '- Username: %s\n'

                    '- Full Name: %s\n'

                    '- Email: %s\n')

            body = body % (new_user.username, new_user.full_name,

                           new_user.email)

            edit_url = url('edit_user', id=new_user.user_id, qualified=True)

            kw = {'registered_user_url': edit_url}

            NotificationModel().create(created_by=new_user, subject=subject,

                                       body=body, recipients=None,

                                       type_=Notification.TYPE_REGISTRATION,

                                       email_kwargs=kw)

        except:

            log.error(traceback.format_exc())

            raise

    def update(self, user_id, form_data):

        try:

            user = self.get(user_id, cache=False)

            if user.username == 'default':

                raise DefaultUserException(

                                _("You can't Edit this user since it's"

                                  " crucial for entire application"))

            for k, v in form_data.items():

                if k == 'new_password' and v != '':

                    user.password = v

                    user.api_key = generate_api_key(user.username)

                else:

                    setattr(user, k, v)

            self.sa.add(user)

        except:

            log.error(traceback.format_exc())

            raise

    def update_my_account(self, user_id, form_data):

        try:

            user = self.get(user_id, cache=False)

            if user.username == 'default':

                raise DefaultUserException(

                                _("You can't Edit this user since it's"

                                  " crucial for entire application"))

            for k, v in form_data.items():

                if k == 'new_password' and v != '':

                    user.password = v

                    user.api_key = generate_api_key(user.username)

                else:

                    if k not in ['admin', 'active']:

                        setattr(user, k, v)

            self.sa.add(user)

        except:

            log.error(traceback.format_exc())

            raise

    def delete(self, user):

        user = self.__get_user(user)

        try:

            if user.username == 'default':

                raise DefaultUserException(

            if user.repositories:

            self.sa.delete(user)

        except:

            log.error(traceback.format_exc())

            raise

    def reset_password_link(self, data):

        from rhodecode.lib.celerylib import tasks, run_task

        run_task(tasks.send_password_link, data['email'])

    def reset_password(self, data):

        from rhodecode.lib.celerylib import tasks, run_task

        run_task(tasks.reset_user_password, data['email'])

    def fill_data(self, auth_user, user_id=None, api_key=None):

        """

        Fetches auth_user by user_id,or api_key if present.

        Fills auth_user attributes with those taken from database.

        Additionally set's is_authenitated if lookup fails

        present in database

        :param auth_user: instance of user to set attributes

        :param user_id: user id to fetch by

        :param api_key: api key to fetch by

        """

        if user_id is None and api_key is None:

            raise Exception('You need to pass user_id or api_key')

        try:

            if api_key:

                dbuser = self.get_by_api_key(api_key)

            else:

                dbuser = self.get(user_id)

            if dbuser is not None and dbuser.active:

                log.debug('filling %s data' % dbuser)

                for k, v in dbuser.get_dict().items():

                    setattr(auth_user, k, v)

            else:

                return False

        except:

            log.error(traceback.format_exc())

            auth_user.is_authenticated = False

            return False

        return True

    def fill_perms(self, user):

        """

        Fills user permission attribute with permissions taken from database

        works for permissions given for repositories, and for permissions that

        are granted to groups

        :param user: user instance to fill his perms

        """

        RK = 'repositories'

        GK = 'repositories_groups'

        GLOBAL = 'global'

        user.permissions[RK] = {}

        user.permissions[GK] = {}

        user.permissions[GLOBAL] = set()

        #======================================================================

        # fetch default permissions

        #======================================================================

        default_user = User.get_by_username('default', cache=True)

        default_user_id = default_user.user_id

        default_repo_perms = Permission.get_default_perms(default_user_id)

        default_repo_groups_perms = Permission.get_default_group_perms(default_user_id)

        if user.is_admin:

            #==================================================================

            # admin user have all default rights for repositories

            # and groups set to admin

            #==================================================================

            user.permissions[GLOBAL].add('hg.admin')

            # repositories

            for perm in default_repo_perms:

                r_k = perm.UserRepoToPerm.repository.repo_name

                p = 'repository.admin'

                user.permissions[RK][r_k] = p

            # repositories groups

            for perm in default_repo_groups_perms:

                rg_k = perm.UserRepoGroupToPerm.group.group_name

                p = 'group.admin'

                user.permissions[GK][rg_k] = p

        else:

            #==================================================================

            # set default permissions first for repositories and groups

            #==================================================================

            uid = user.user_id

                .filter(UserToPerm.user_id == default_user_id)

            for perm in default_global_perms:

                user.permissions[GLOBAL].add(perm.permission.permission_name)

            # default for repositories

            for perm in default_repo_perms:

                r_k = perm.UserRepoToPerm.repository.repo_name

                if perm.Repository.private and not (perm.Repository.user_id == uid):

                    # disable defaults for private repos,

                    p = 'repository.none'

                elif perm.Repository.user_id == uid:

                    # set admin if owner

                    p = 'repository.admin'

                else:

                    p = perm.Permission.permission_name

                user.permissions[RK][r_k] = p

            # default for repositories groups

            for perm in default_repo_groups_perms:

                rg_k = perm.UserRepoGroupToPerm.group.group_name

                p = perm.Permission.permission_name

                user.permissions[GK][rg_k] = p

            #==================================================================

            # overwrite default with user permissions if any

            #==================================================================

            # user global

            user_perms = self.sa.query(UserToPerm)\

                    .options(joinedload(UserToPerm.permission))\

                    .filter(UserToPerm.user_id == uid).all()

            for perm in user_perms:

                user.permissions[GLOBAL].add(perm.permission.permission_name)

            # user repositories

            user_repo_perms = \

             self.sa.query(UserRepoToPerm, Permission, Repository)\

             .join((Repository, UserRepoToPerm.repository_id == Repository.repo_id))\

             .join((Permission, UserRepoToPerm.permission_id == Permission.permission_id))\

             .filter(UserRepoToPerm.user_id == uid)\

             .all()

            for perm in user_repo_perms:

                # set admin if owner

                r_k = perm.UserRepoToPerm.repository.repo_name

                if perm.Repository.user_id == uid:

                    p = 'repository.admin'

                else:

                    p = perm.Permission.permission_name

                user.permissions[RK][r_k] = p

            #==================================================================

            # check if user is part of groups for this repository and fill in

            # (or replace with higher) permissions

            #==================================================================

            # users group global

            user_perms_from_users_groups = self.sa.query(UsersGroupToPerm)\

                .options(joinedload(UsersGroupToPerm.permission))\

                .join((UsersGroupMember, UsersGroupToPerm.users_group_id ==

                       UsersGroupMember.users_group_id))\

                .filter(UsersGroupMember.user_id == uid).all()

            for perm in user_perms_from_users_groups:

                user.permissions[GLOBAL].add(perm.permission.permission_name)

            # users group repositories

            user_repo_perms_from_users_groups = \

             self.sa.query(UsersGroupRepoToPerm, Permission, Repository,)\

             .join((Repository, UsersGroupRepoToPerm.repository_id == Repository.repo_id))\

             .join((Permission, UsersGroupRepoToPerm.permission_id == Permission.permission_id))\

             .join((UsersGroupMember, UsersGroupRepoToPerm.users_group_id == UsersGroupMember.users_group_id))\

             .filter(UsersGroupMember.user_id == uid)\

             .all()

            for perm in user_repo_perms_from_users_groups:

                r_k = perm.UsersGroupRepoToPerm.repository.repo_name

                p = perm.Permission.permission_name

                cur_perm = user.permissions[RK][r_k]

                # overwrite permission only if it's greater than permission

                # given from other sources

                if PERM_WEIGHTS[p] > PERM_WEIGHTS[cur_perm]:

                    user.permissions[RK][r_k] = p

            #==================================================================

            # get access for this user for repos group and override defaults

            #==================================================================

            # user repositories groups

            user_repo_groups_perms = \

             self.sa.query(UserRepoGroupToPerm, Permission, RepoGroup)\

             .join((RepoGroup, UserRepoGroupToPerm.group_id == RepoGroup.group_id))\

             .join((Permission, UserRepoGroupToPerm.permission_id == Permission.permission_id))\

             .all()

            for perm in user_repo_groups_perms:

                rg_k = perm.UserRepoGroupToPerm.group.group_name

                p = perm.Permission.permission_name

                cur_perm = user.permissions[GK][rg_k]

                if PERM_WEIGHTS[p] > PERM_WEIGHTS[cur_perm]:

                    user.permissions[GK][rg_k] = p

        return user

    def has_perm(self, user, perm):

        if not isinstance(perm, Permission):

            raise Exception('perm needs to be an instance of Permission class '

                            'got %s instead' % type(perm))

        user = self.__get_user(user)

        return UserToPerm.query().filter(UserToPerm.user == user)\

            .filter(UserToPerm.permission == perm).scalar() is not None

    def grant_perm(self, user, perm):

        """

        Grant user global permissions

        :param user:

        :param perm:

        """

        user = self.__get_user(user)

        perm = self.__get_perm(perm)

        # if this permission is already granted skip it

        _perm = UserToPerm.query()\

            .filter(UserToPerm.user == user)\

            .filter(UserToPerm.permission == perm)\

            .scalar()

        if _perm:

            return

        new = UserToPerm()

        new.user = user

        new.permission = perm

        self.sa.add(new)

    def revoke_perm(self, user, perm):

        """

        Revoke users global permissions

        :param user:

        :param perm:

        """

        user = self.__get_user(user)

        perm = self.__get_perm(perm)

        obj = UserToPerm.query()\

                .filter(UserToPerm.user == user)\

                .filter(UserToPerm.permission == perm)\

                .scalar()

        if obj:

            self.sa.delete(obj)

        #u1 and u2 still have assignments

        u1notification = UserNotification.query()\

                            .filter(UserNotification.notification ==

                                    notification)\

                            .filter(UserNotification.user_id == self.u1)\

                            .scalar()

        self.assertNotEqual(u1notification, None)

        u2notification = UserNotification.query()\

                            .filter(UserNotification.notification ==

                                    notification)\

                            .filter(UserNotification.user_id == self.u2)\

                            .scalar()

        self.assertNotEqual(u2notification, None)

    def test_notification_counter(self):

        self._clean_notifications()

        self.assertEqual([], Notification.query().all())

        self.assertEqual([], UserNotification.query().all())

        NotificationModel().create(created_by=self.u1,

                            subject=u'title', body=u'hi there_delete',

                            recipients=[self.u3, self.u1])

        Session.commit()

        self.assertEqual(NotificationModel()

                         .get_unread_cnt_for_user(self.u1), 1)

        self.assertEqual(NotificationModel()

                         .get_unread_cnt_for_user(self.u2), 0)

        self.assertEqual(NotificationModel()

                         .get_unread_cnt_for_user(self.u3), 1)

        notification = NotificationModel().create(created_by=self.u1,

                                           subject=u'title', body=u'hi there3',

                                    recipients=[self.u3, self.u1, self.u2])

        Session.commit()

        self.assertEqual(NotificationModel()

                         .get_unread_cnt_for_user(self.u1), 2)

        self.assertEqual(NotificationModel()

                         .get_unread_cnt_for_user(self.u2), 1)

        self.assertEqual(NotificationModel()

                         .get_unread_cnt_for_user(self.u3), 2)

class TestUsers(unittest.TestCase):

    def __init__(self, methodName='runTest'):

        super(TestUsers, self).__init__(methodName=methodName)

    def setUp(self):

        self.u1 = UserModel().create_or_update(username=u'u1',

                                        password=u'qweqwe',

                                        email=u'u1@rhodecode.org',

                                        name=u'u1', lastname=u'u1')

    def tearDown(self):

        perm = Permission.query().all()

        for p in perm:

            UserModel().revoke_perm(self.u1, p)

        UserModel().delete(self.u1)

        Session.commit()

    def test_add_perm(self):

        perm = Permission.query().all()[0]

        UserModel().grant_perm(self.u1, perm)

        Session.commit()

        self.assertEqual(UserModel().has_perm(self.u1, perm), True)

    def test_has_perm(self):

        perm = Permission.query().all()

        for p in perm:

            has_p = UserModel().has_perm(self.u1, p)

            self.assertEqual(False, has_p)

    def test_revoke_perm(self):

        perm = Permission.query().all()[0]

        UserModel().grant_perm(self.u1, perm)

        Session.commit()

        self.assertEqual(UserModel().has_perm(self.u1, perm), True)

        #revoke

        UserModel().revoke_perm(self.u1, perm)

        Session.commit()

        self.assertEqual(UserModel().has_perm(self.u1, perm), False)

class TestPermissions(unittest.TestCase):

    def __init__(self, methodName='runTest'):

        super(TestPermissions, self).__init__(methodName=methodName)

    def setUp(self):

        self.u1 = UserModel().create_or_update(

            username=u'u1', password=u'qweqwe',

            email=u'u1@rhodecode.org', name=u'u1', lastname=u'u1'

        )

        self.a1 = UserModel().create_or_update(

            username=u'a1', password=u'qweqwe',

            email=u'a1@rhodecode.org', name=u'a1', lastname=u'a1', admin=True

        )

        Session.commit()

    def tearDown(self):

        UserModel().delete(self.u1)

        UserModel().delete(self.a1)

        if hasattr(self, 'g1'):

            ReposGroupModel().delete(self.g1.group_id)

        if hasattr(self, 'g2'):

            ReposGroupModel().delete(self.g2.group_id)

        if hasattr(self, 'ug1'):

            UsersGroupModel().delete(self.ug1, force=True)

        Session.commit()

    def test_default_perms_set(self):

        u1_auth = AuthUser(user_id=self.u1.user_id)

        perms = {

            'repositories_groups': {},

            'global': set([u'hg.create.repository', u'repository.read',

                           u'hg.register.manual_activate']),

            'repositories': {u'vcs_test_hg': u'repository.read'}

        }

        self.assertEqual(u1_auth.permissions['repositories'][HG_REPO],

                         perms['repositories'][HG_REPO])

        new_perm = 'repository.write'

        RepoModel().grant_user_permission(repo=HG_REPO, user=self.u1, perm=new_perm)

        Session.commit()

        u1_auth = AuthUser(user_id=self.u1.user_id)

        self.assertEqual(u1_auth.permissions['repositories'][HG_REPO], new_perm)

    def test_default_admin_perms_set(self):

        a1_auth = AuthUser(user_id=self.a1.user_id)

        perms = {

            'repositories_groups': {},

            'global': set([u'hg.admin']),

            'repositories': {u'vcs_test_hg': u'repository.admin'}

        }

        self.assertEqual(a1_auth.permissions['repositories'][HG_REPO],

                         perms['repositories'][HG_REPO])

        new_perm = 'repository.write'

        RepoModel().grant_user_permission(repo=HG_REPO, user=self.a1, perm=new_perm)

        Session.commit()

        # cannot really downgrade admins permissions !? they still get's set as

        # admin !

        u1_auth = AuthUser(user_id=self.a1.user_id)

        self.assertEqual(u1_auth.permissions['repositories'][HG_REPO],

                         perms['repositories'][HG_REPO])

    def test_default_group_perms(self):

        self.g1 = _make_group('test1', skip_if_exists=True)

        self.g2 = _make_group('test2', skip_if_exists=True)

        u1_auth = AuthUser(user_id=self.u1.user_id)

        perms = {

            'repositories_groups': {u'test1': 'group.read', u'test2': 'group.read'},

            'global': set([u'hg.create.repository', u'repository.read', u'hg.register.manual_activate']),

            'repositories': {u'vcs_test_hg': u'repository.read'}

        }

        self.assertEqual(u1_auth.permissions['repositories'][HG_REPO],

                         perms['repositories'][HG_REPO])

        self.assertEqual(u1_auth.permissions['repositories_groups'],

                         perms['repositories_groups'])

    def test_default_admin_group_perms(self):

        self.g1 = _make_group('test1', skip_if_exists=True)

        self.g2 = _make_group('test2', skip_if_exists=True)

        a1_auth = AuthUser(user_id=self.a1.user_id)

        perms = {

            'repositories_groups': {u'test1': 'group.admin', u'test2': 'group.admin'},

            'global': set(['hg.admin']),

            'repositories': {u'vcs_test_hg': 'repository.admin'}

        }

        self.assertEqual(a1_auth.permissions['repositories'][HG_REPO],

                         perms['repositories'][HG_REPO])

        self.assertEqual(a1_auth.permissions['repositories_groups'],

                         perms['repositories_groups'])

    def test_propagated_permission_from_users_group(self):

        # make group

        self.ug1 = UsersGroupModel().create('G1')

        # add user to group

        UsersGroupModel().add_user_to_group(self.ug1, self.u1)

        # set permission to lower

        new_perm = 'repository.none'

        RepoModel().grant_user_permission(repo=HG_REPO, user=self.u1, perm=new_perm)

        Session.commit()

        u1_auth = AuthUser(user_id=self.u1.user_id)

        self.assertEqual(u1_auth.permissions['repositories'][HG_REPO],

                         new_perm)

        # grant perm for group this should override permission from user

        new_perm = 'repository.write'

        RepoModel().grant_users_group_permission(repo=HG_REPO,

                                                 group_name=self.ug1,

                                                 perm=new_perm)

        # check perms

        u1_auth = AuthUser(user_id=self.u1.user_id)

        perms = {

            'repositories_groups': {},

            'global': set([u'hg.create.repository', u'repository.read',

                           u'hg.register.manual_activate']),

            'repositories': {u'vcs_test_hg': u'repository.read'}

        }

        self.assertEqual(u1_auth.permissions['repositories'][HG_REPO],

                         new_perm)

        self.assertEqual(u1_auth.permissions['repositories_groups'],

                         perms['repositories_groups'])

    def test_propagated_permission_from_users_group_lower_weight(self):

        # make group

        self.ug1 = UsersGroupModel().create('G1')

        # add user to group

        UsersGroupModel().add_user_to_group(self.ug1, self.u1)

        # set permission to lower

        new_perm_h = 'repository.write'

        RepoModel().grant_user_permission(repo=HG_REPO, user=self.u1,

                                          perm=new_perm_h)

        Session.commit()

        u1_auth = AuthUser(user_id=self.u1.user_id)

        self.assertEqual(u1_auth.permissions['repositories'][HG_REPO],

                         new_perm_h)

        # grant perm for group this should NOT override permission from user

        # since it's lower than granted

        new_perm_l = 'repository.read'

        RepoModel().grant_users_group_permission(repo=HG_REPO,

                                                 group_name=self.ug1,

                                                 perm=new_perm_l)

        # check perms

        u1_auth = AuthUser(user_id=self.u1.user_id)

        perms = {

            'repositories_groups': {},

            'global': set([u'hg.create.repository', u'repository.read',

                           u'hg.register.manual_activate']),

            'repositories': {u'vcs_test_hg': u'repository.write'}

        }

        self.assertEqual(u1_auth.permissions['repositories'][HG_REPO],

                         new_perm_h)

        self.assertEqual(u1_auth.permissions['repositories_groups'],

                         perms['repositories_groups'])