.. _changelog:

=========

Changelog

=========

1.3.4 (**2012-XX-XX**)

----------------------

:status: in-progress

:branch: beta

news

++++

- Whoosh logging is now controlled by the .ini files logging setup

- added clone-url into edit form on /settings page

- added help text into repo add/edit forms

- created rcextensions module with additional mappings (ref #322) and

  post push/pull/create repo hooks callbacks

- implemented #377 Users view for his own permissions on account page

fixes

+++++

- fixed #390 cache invalidation problems on repos inside group

- fixed #385 clone by ID url was loosing proxy prefix in URL

- fixed some unicode problems with waitress

- fixed issue with escaping < and > in changeset commits

- fixed error occurring during recursive group creation in API 

  create_repo function

- fixed #393 py2.5 fixes for routes url generator

- fixed #397 Private repository groups shows up before login

- fixed #396 fixed problems with revoking users in nested groups

1.3.3 (**2012-03-02**)

----------------------

news

++++

fixes

+++++

- fixed some python2.5 compatibility issues 

- fixed issues with removed repos was accidentally added as groups, after

  full rescan of paths

- fixes #376 Cannot edit user (using container auth)

- fixes #378 Invalid image urls on changeset screen with proxy-prefix 

  configuration

- fixed initial sorting of repos inside repo group

- fixes issue when user tried to resubmit same permission into user/user_groups

- bumped beaker version that fixes #375 leap error bug

- fixed raw_changeset for git. It was generated with hg patch headers

- fixed vcs issue with last_changeset for filenodes

- fixed missing commit after hook delete

- fixed #372 issues with git operation detection that caused a security issue 

  for git repos

1.3.2 (**2012-02-28**)

----------------------

news

++++

fixes

+++++

- fixed git protocol issues with repos-groups

- fixed git remote repos validator that prevented from cloning remote git repos

- fixes #370 ending slashes fixes for repo and groups

- fixes #368 improved git-protocol detection to handle other clients

- fixes #366 When Setting Repository Group To Blank Repo Group Wont Be 

  Moved To Root

- fixes #371 fixed issues with beaker/sqlalchemy and non-ascii cache keys 

- fixed #373 missing cascade drop on user_group_to_perm table

1.3.1 (**2012-02-27**)

----------------------

news

++++

fixes

+++++

- redirection loop occurs when remember-me wasn't checked during login

- fixes issues with git blob history generation 

- don't fetch branch for git in file history dropdown. Causes unneeded slowness

1.3.0 (**2012-02-26**)

----------------------

news

++++

- code review, inspired by github code-comments 

- #215 rst and markdown README files support

- #252 Container-based and proxy pass-through authentication support

- #44 branch browser. Filtering of changelog by branches

- mercurial bookmarks support

- new hover top menu, optimized to add maximum size for important views

- configurable clone url template with possibility to specify  protocol like 

  ssh:// or http:// and also manually alter other parts of clone_url.

- enabled largefiles extension by default

- optimized summary file pages and saved a lot of unused space in them

- #239 option to manually mark repository as fork

- #320 mapping of commit authors to RhodeCode users

- #304 hashes are displayed using monospace font    

- diff configuration, toggle white lines and context lines

- #307 configurable diffs, whitespace toggle, increasing context lines

- sorting on branches, tags and bookmarks using YUI datatable

- improved file filter on files page

- implements #330 api method for listing nodes ar particular revision

- #73 added linking issues in commit messages to chosen issue tracker url

  based on user defined regular expression

- added linking of changesets in commit messages  

- new compact changelog with expandable commit messages

- firstname and lastname are optional in user creation

- #348 added post-create repository hook

- #212 global encoding settings is now configurable from .ini files 

- #227 added repository groups permissions

- markdown gets codehilite extensions

- new API methods, delete_repositories, grante/revoke permissions for groups 

  and repos

fixes

+++++

- rewrote dbsession management for atomic operations, and better error handling

- fixed sorting of repo tables

- #326 escape of special html entities in diffs

- normalized user_name => username in api attributes

- fixes #298 ldap created users with mixed case emails created conflicts 

  on saving a form

- fixes issue when owner of a repo couldn't revoke permissions for users 

  and groups

- fixes #271 rare JSON serialization problem with statistics

- fixes #337 missing validation check for conflicting names of a group with a

  repositories group

- #340 fixed session problem for mysql and celery tasks

- fixed #331 RhodeCode mangles repository names if the a repository group 

  contains the "full path" to the repositories

- #355 RhodeCode doesn't store encrypted LDAP passwords

1.2.5 (**2012-01-28**)

----------------------

news

++++

fixes

+++++

- #340 Celery complains about MySQL server gone away, added session cleanup

  for celery tasks

- #341 "scanning for repositories in None" log message during Rescan was missing

  a parameter

- fixed creating archives with subrepos. Some hooks were triggered during that

  operation leading to crash.

- fixed missing email in account page.

- Reverted Mercurial to 2.0.1 for windows due to bug in Mercurial that makes

  forking on windows impossible 

1.2.4 (**2012-01-19**)

----------------------

news

++++

- RhodeCode is bundled with mercurial series 2.0.X by default, with

  full support to largefiles extension. Enabled by default in new installations

- #329 Ability to Add/Remove Groups to/from a Repository via AP

- added requires.txt file with requirements

fixes

+++++

- fixes db session issues with celery when emailing admins

- #331 RhodeCode mangles repository names if the a repository group 

  contains the "full path" to the repositories

- #298 Conflicting e-mail addresses for LDAP and RhodeCode users

- DB session cleanup after hg protocol operations, fixes issues with

  `mysql has gone away` errors

- #333 doc fixes for get_repo api function

- #271 rare JSON serialization problem with statistics enabled

- #337 Fixes issues with validation of repository name conflicting with 

  a group name. A proper message is now displayed.

- #292 made ldap_dn in user edit readonly, to get rid of confusion that field

  doesn't work   

- #316 fixes issues with web description in hgrc files 

1.2.3 (**2011-11-02**)

----------------------

news

++++

- added option to manage repos group for non admin users

- added following API methods for get_users, create_user, get_users_groups, 

  get_users_group, create_users_group, add_user_to_users_groups, get_repos, 

  get_repo, create_repo, add_user_to_repo

- implements #237 added password confirmation for my account 

  and admin edit user.

- implements #291 email notification for global events are now sent to all

  administrator users, and global config email.

fixes

+++++

- added option for passing auth method for smtp mailer

- #276 issue with adding a single user with id>10 to usergroups

- #277 fixes windows LDAP settings in which missing values breaks the ldap auth 

- #288 fixes managing of repos in a group for non admin user

1.2.2 (**2011-10-17**)

----------------------

news

++++

- #226 repo groups are available by path instead of numerical id

fixes

+++++

- #259 Groups with the same name but with different parent group

- #260 Put repo in group, then move group to another group -> repo becomes unavailable

- #258 RhodeCode 1.2 assumes egg folder is writable (lockfiles problems)

- #265 ldap save fails sometimes on converting attributes to booleans, 

  added getter and setter into model that will prevent from this on db model level

- fixed problems with timestamps issues #251 and #213

- fixes #266 RhodeCode allows to create repo with the same name and in 

  the same parent as group

- fixes #245 Rescan of the repositories on Windows

- fixes #248 cannot edit repos inside a group on windows

- fixes #219 forking problems on windows

1.2.1 (**2011-10-08**)

----------------------

news

++++

fixes

+++++

- fixed problems with basic auth and push problems 

- gui fixes

- fixed logger

1.2.0 (**2011-10-07**)

----------------------

news

++++

- implemented #47 repository groups

- implemented #89 Can setup google analytics code from settings menu

- implemented #91 added nicer looking archive urls with more download options

  like tags, branches

- implemented #44 into file browsing, and added follow branch option

- implemented #84 downloads can be enabled/disabled for each repository

- anonymous repository can be cloned without having to pass default:default

  into clone url

- fixed #90 whoosh indexer can index chooses repositories passed in command 

  line

- extended journal with day aggregates and paging

- implemented #107 source code lines highlight ranges

- implemented #93 customizable changelog on combined revision ranges - 

  equivalent of githubs compare view 

- implemented #108 extended and more powerful LDAP configuration

- implemented #56 users groups

- major code rewrites optimized codes for speed and memory usage

- raw and diff downloads are now in git format

- setup command checks for write access to given path

- fixed many issues with international characters and unicode. It uses utf8

  decode with replace to provide less errors even with non utf8 encoded strings

- #125 added API KEY access to feeds

- #109 Repository can be created from external Mercurial link (aka. remote 

  repository, and manually updated (via pull) from admin panel

- beta git support - push/pull server + basic view for git repos

- added followers page and forks page

- server side file creation (with binary file upload interface) 

  and edition with commits powered by codemirror 

- #111 file browser file finder, quick lookup files on whole file tree 

- added quick login sliding menu into main page

- changelog uses lazy loading of affected files details, in some scenarios 

  this can improve speed of changelog page dramatically especially for 

  larger repositories.

- implements #214 added support for downloading subrepos in download menu.

- Added basic API for direct operations on rhodecode via JSON

- Implemented advanced hook management

fixes

+++++

- fixed file browser bug, when switching into given form revision the url was 

  not changing

- fixed propagation to error controller on simplehg and simplegit middlewares

- fixed error when trying to make a download on empty repository

- fixed problem with '[' chars in commit messages in journal

- fixed #99 Unicode errors, on file node paths with non utf-8 characters

- journal fork fixes

- removed issue with space inside renamed repository after deletion

- fixed strange issue on formencode imports

- fixed #126 Deleting repository on Windows, rename used incompatible chars. 

- #150 fixes for errors on repositories mapped in db but corrupted in 

  filesystem

- fixed problem with ascendant characters in realm #181

- fixed problem with sqlite file based database connection pool

- whoosh indexer and code stats share the same dynamic extensions map

- fixes #188 - relationship delete of repo_to_perm entry on user removal

- fixes issue #189 Trending source files shows "show more" when no more exist

- fixes issue #197 Relative paths for pidlocks

- fixes issue #198 password will require only 3 chars now for login form

- fixes issue #199 wrong redirection for non admin users after creating a repository

- fixes issues #202, bad db constraint made impossible to attach same group 

  more than one time. Affects only mysql/postgres

- fixes #218 os.kill patch for windows was missing sig param

- improved rendering of dag (they are not trimmed anymore when number of 

  heads exceeds 5)

1.1.8 (**2011-04-12**)

----------------------

news

++++

- improved windows support

fixes

+++++

- fixed #140 freeze of python dateutil library, since new version is python2.x

  incompatible

- setup-app will check for write permission in given path

- cleaned up license info issue #149

- fixes for issues #137,#116 and problems with unicode and accented characters.

- fixes crashes on gravatar, when passed in email as unicode

- fixed tooltip flickering problems

- fixed came_from redirection on windows

- fixed logging modules, and sql formatters

- windows fixes for os.kill issue #133

- fixes path splitting for windows issues #148

- fixed issue #143 wrong import on migration to 1.1.X

- fixed problems with displaying binary files, thanks to Thomas Waldmann

- removed name from archive files since it's breaking ui for long repo names

- fixed issue with archive headers sent to browser, thanks to Thomas Waldmann

- fixed compatibility for 1024px displays, and larger dpi settings, thanks to 

  Thomas Waldmann

- fixed issue #166 summary pager was skipping 10 revisions on second page

1.1.7 (**2011-03-23**)

----------------------

news

++++

fixes

+++++

- fixed (again) #136 installation support for FreeBSD

1.1.6 (**2011-03-21**)

----------------------

news

++++

fixes

+++++

- fixed #136 installation support for FreeBSD

- RhodeCode will check for python version during installation

1.1.5 (**2011-03-17**)

----------------------

news

++++

- basic windows support, by exchanging pybcrypt into sha256 for windows only

  highly inspired by idea of mantis406

fixes

+++++

- fixed sorting by author in main page

- fixed crashes with diffs on binary files

- fixed #131 problem with boolean values for LDAP

- fixed #122 mysql problems thanks to striker69 

- fixed problem with errors on calling raw/raw_files/annotate functions 

  with unknown revisions

- fixed returned rawfiles attachment names with international character

- cleaned out docs, big thanks to Jason Harris

1.1.4 (**2011-02-19**)

# -*- coding: utf-8 -*-

"""

    rhodecode.model.user

    ~~~~~~~~~~~~~~~~~~~~

    users model for RhodeCode

    :created_on: Apr 9, 2010

    :author: marcink

    :copyright: (C) 2010-2012 Marcin Kuzminski <marcin@python-works.com>

    :license: GPLv3, see COPYING for more details.

"""

# This program is free software: you can redistribute it and/or modify

# it under the terms of the GNU General Public License as published by

# the Free Software Foundation, either version 3 of the License, or

# (at your option) any later version.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program.  If not, see <http://www.gnu.org/licenses/>.

import logging

import traceback

from pylons import url

from pylons.i18n.translation import _

from rhodecode.lib.utils2 import safe_unicode, generate_api_key

from rhodecode.lib.caching_query import FromCache

from rhodecode.model import BaseModel

from rhodecode.model.db import User, UserRepoToPerm, Repository, Permission, \

    UserToPerm, UsersGroupRepoToPerm, UsersGroupToPerm, UsersGroupMember, \

from rhodecode.lib.exceptions import DefaultUserException, \

    UserOwnsReposException

from sqlalchemy.exc import DatabaseError

from sqlalchemy.orm import joinedload

log = logging.getLogger(__name__)

PERM_WEIGHTS = {

    'repository.none': 0,

    'repository.read': 1,

    'repository.write': 3,

    'repository.admin': 4,

    'group.none': 0,

    'group.read': 1,

    'group.write': 3,

    'group.admin': 4,

}

class UserModel(BaseModel):

    def __get_user(self, user):

        return self._get_instance(User, user, callback=User.get_by_username)

    def __get_perm(self, permission):

        return self._get_instance(Permission, permission,

                                  callback=Permission.get_by_key)

    def get(self, user_id, cache=False):

        user = self.sa.query(User)

        if cache:

            user = user.options(FromCache("sql_cache_short",

                                          "get_user_%s" % user_id))

        return user.get(user_id)

    def get_user(self, user):

        return self.__get_user(user)

    def get_by_username(self, username, cache=False, case_insensitive=False):

        if case_insensitive:

            user = self.sa.query(User).filter(User.username.ilike(username))

        else:

            user = self.sa.query(User)\

                .filter(User.username == username)

        if cache:

            user = user.options(FromCache("sql_cache_short",

                                          "get_user_%s" % username))

        return user.scalar()

    def get_by_api_key(self, api_key, cache=False):

        return User.get_by_api_key(api_key, cache)

    def create(self, form_data):

        try:

            new_user = User()

            for k, v in form_data.items():

                setattr(new_user, k, v)

            new_user.api_key = generate_api_key(form_data['username'])

            self.sa.add(new_user)

            return new_user

        except:

            log.error(traceback.format_exc())

            raise

    def create_or_update(self, username, password, email, name, lastname,

                         active=True, admin=False, ldap_dn=None):

        """

        Creates a new instance if not found, or updates current one

        :param username:

        :param password:

        :param email:

        :param active:

        :param name:

        :param lastname:

        :param active:

        :param admin:

        :param ldap_dn:

        """

        from rhodecode.lib.auth import get_crypt_password

        log.debug('Checking for %s account in RhodeCode database' % username)

        user = User.get_by_username(username, case_insensitive=True)

        if user is None:

            log.debug('creating new user %s' % username)

            new_user = User()

        else:

            log.debug('updating user %s' % username)

            new_user = user

        try:

            new_user.username = username

            new_user.admin = admin

            new_user.password = get_crypt_password(password)

            new_user.api_key = generate_api_key(username)

            new_user.email = email

            new_user.active = active

            new_user.ldap_dn = safe_unicode(ldap_dn) if ldap_dn else None

            new_user.name = name

            new_user.lastname = lastname

            self.sa.add(new_user)

            return new_user

        except (DatabaseError,):

            log.error(traceback.format_exc())

            raise

    def create_for_container_auth(self, username, attrs):

        """

        Creates the given user if it's not already in the database

        :param username:

        :param attrs:

        """

        if self.get_by_username(username, case_insensitive=True) is None:

            # autogenerate email for container account without one

            generate_email = lambda usr: '%s@container_auth.account' % usr

            try:

                new_user = User()

                new_user.username = username

                new_user.password = None

                new_user.api_key = generate_api_key(username)

                new_user.email = attrs['email']

                new_user.active = attrs.get('active', True)

                new_user.name = attrs['name'] or generate_email(username)

                new_user.lastname = attrs['lastname']

                self.sa.add(new_user)

                return new_user

            except (DatabaseError,):

                log.error(traceback.format_exc())

                self.sa.rollback()

                raise

        log.debug('User %s already exists. Skipping creation of account'

                  ' for container auth.', username)

        return None

    def create_ldap(self, username, password, user_dn, attrs):

        """

        Checks if user is in database, if not creates this user marked

        as ldap user

        :param username:

        :param password:

        :param user_dn:

        :param attrs:

        """

        from rhodecode.lib.auth import get_crypt_password

        log.debug('Checking for such ldap account in RhodeCode database')

        if self.get_by_username(username, case_insensitive=True) is None:

            # autogenerate email for ldap account without one

            generate_email = lambda usr: '%s@ldap.account' % usr

            try:

                new_user = User()

                username = username.lower()

                # add ldap account always lowercase

                new_user.username = username

                new_user.password = get_crypt_password(password)

                new_user.api_key = generate_api_key(username)

                new_user.email = attrs['email'] or generate_email(username)

                new_user.active = attrs.get('active', True)

                new_user.ldap_dn = safe_unicode(user_dn)

                new_user.name = attrs['name']

                new_user.lastname = attrs['lastname']

                self.sa.add(new_user)

                return new_user

            except (DatabaseError,):

                log.error(traceback.format_exc())

                self.sa.rollback()

                raise

        log.debug('this %s user exists skipping creation of ldap account',

                  username)

        return None

    def create_registration(self, form_data):

        from rhodecode.model.notification import NotificationModel

        try:

            new_user = User()

            for k, v in form_data.items():

                if k != 'admin':

                    setattr(new_user, k, v)

            self.sa.add(new_user)

            self.sa.flush()

            # notification to admins

            subject = _('new user registration')

            body = ('New user registration\n'

                    '---------------------\n'

                    '- Username: %s\n'

                    '- Full Name: %s\n'

                    '- Email: %s\n')

            body = body % (new_user.username, new_user.full_name,

                           new_user.email)

            edit_url = url('edit_user', id=new_user.user_id, qualified=True)

            kw = {'registered_user_url': edit_url}

            NotificationModel().create(created_by=new_user, subject=subject,

                                       body=body, recipients=None,

                                       type_=Notification.TYPE_REGISTRATION,

                                       email_kwargs=kw)

        except:

            log.error(traceback.format_exc())

            raise

    def update(self, user_id, form_data):

        try:

            user = self.get(user_id, cache=False)

            if user.username == 'default':

                raise DefaultUserException(

                                _("You can't Edit this user since it's"

                                  " crucial for entire application"))

            for k, v in form_data.items():

                if k == 'new_password' and v != '':

                    user.password = v

                    user.api_key = generate_api_key(user.username)

                else:

                    setattr(user, k, v)

            self.sa.add(user)

        except:

            log.error(traceback.format_exc())

            raise

    def update_my_account(self, user_id, form_data):

        try:

            user = self.get(user_id, cache=False)

            if user.username == 'default':

                raise DefaultUserException(

                                _("You can't Edit this user since it's"

                                  " crucial for entire application"))

            for k, v in form_data.items():

                if k == 'new_password' and v != '':

                    user.password = v

                    user.api_key = generate_api_key(user.username)

                else:

                    if k not in ['admin', 'active']:

                        setattr(user, k, v)

            self.sa.add(user)

        except:

            log.error(traceback.format_exc())

            raise

    def delete(self, user):

        user = self.__get_user(user)

        try:

            if user.username == 'default':

                raise DefaultUserException(

                    _("You can't remove this user since it's"

                      " crucial for entire application")

                )

            if user.repositories:

                raise UserOwnsReposException(

                    _('user "%s" still owns %s repositories and cannot be '

                      'removed. Switch owners or remove those repositories')

                    % (user.username, user.repositories)

                )

            self.sa.delete(user)

        except:

            log.error(traceback.format_exc())

            raise

    def reset_password_link(self, data):

        from rhodecode.lib.celerylib import tasks, run_task

        run_task(tasks.send_password_link, data['email'])

    def reset_password(self, data):

        from rhodecode.lib.celerylib import tasks, run_task

        run_task(tasks.reset_user_password, data['email'])

    def fill_data(self, auth_user, user_id=None, api_key=None):

        """

        Fetches auth_user by user_id,or api_key if present.

        Fills auth_user attributes with those taken from database.

        Additionally set's is_authenitated if lookup fails

        present in database

        :param auth_user: instance of user to set attributes

        :param user_id: user id to fetch by

        :param api_key: api key to fetch by

        """

        if user_id is None and api_key is None:

            raise Exception('You need to pass user_id or api_key')

        try:

            if api_key:

                dbuser = self.get_by_api_key(api_key)

            else:

                dbuser = self.get(user_id)

            if dbuser is not None and dbuser.active:

                log.debug('filling %s data' % dbuser)

                for k, v in dbuser.get_dict().items():

                    setattr(auth_user, k, v)

            else:

                return False

        except:

            log.error(traceback.format_exc())

            auth_user.is_authenticated = False

            return False

        return True

    def fill_perms(self, user):

        """

        Fills user permission attribute with permissions taken from database

        works for permissions given for repositories, and for permissions that

        are granted to groups

        :param user: user instance to fill his perms

        """

        RK = 'repositories'

        GK = 'repositories_groups'

        GLOBAL = 'global'

        user.permissions[RK] = {}

        user.permissions[GK] = {}

        user.permissions[GLOBAL] = set()

        #======================================================================

        # fetch default permissions

        #======================================================================

        default_user = User.get_by_username('default', cache=True)

        default_user_id = default_user.user_id

        default_repo_perms = Permission.get_default_perms(default_user_id)

        default_repo_groups_perms = Permission.get_default_group_perms(default_user_id)

        if user.is_admin:

            #==================================================================

            # admin user have all default rights for repositories

            # and groups set to admin

            #==================================================================

            user.permissions[GLOBAL].add('hg.admin')

            # repositories

            for perm in default_repo_perms:

                r_k = perm.UserRepoToPerm.repository.repo_name

                p = 'repository.admin'

                user.permissions[RK][r_k] = p

            # repositories groups

            for perm in default_repo_groups_perms:

                rg_k = perm.UserRepoGroupToPerm.group.group_name

                p = 'group.admin'

                user.permissions[GK][rg_k] = p

        else:

            #==================================================================

            # set default permissions first for repositories and groups

            #==================================================================

            uid = user.user_id

            # default global permissions

            default_global_perms = self.sa.query(UserToPerm)\

                .filter(UserToPerm.user_id == default_user_id)

            for perm in default_global_perms:

                user.permissions[GLOBAL].add(perm.permission.permission_name)

            for perm in default_repo_perms:

                r_k = perm.UserRepoToPerm.repository.repo_name

                if perm.Repository.private and not (perm.Repository.user_id == uid):

                    # disable defaults for private repos,

                    p = 'repository.none'

                elif perm.Repository.user_id == uid:

                    # set admin if owner

                    p = 'repository.admin'

                else:

                    p = perm.Permission.permission_name

                user.permissions[RK][r_k] = p

            for perm in default_repo_groups_perms:

                rg_k = perm.UserRepoGroupToPerm.group.group_name

                p = perm.Permission.permission_name

                user.permissions[GK][rg_k] = p

            #==================================================================

            #==================================================================

            user_perms = self.sa.query(UserToPerm)\

                    .options(joinedload(UserToPerm.permission))\

                    .filter(UserToPerm.user_id == uid).all()

            for perm in user_perms:

                user.permissions[GLOBAL].add(perm.permission.permission_name)

            user_repo_perms = \

             self.sa.query(UserRepoToPerm, Permission, Repository)\

             .join((Repository, UserRepoToPerm.repository_id == Repository.repo_id))\

             .join((Permission, UserRepoToPerm.permission_id == Permission.permission_id))\

             .filter(UserRepoToPerm.user_id == uid)\

             .all()