.. _changelog:

Changelog

=========

1.2.4 (**2012-01-19**)

======================

news

----

- RhodeCode is bundled with mercurial series 2.0.X by default, with

  full support to largefiles extension. Enabled by default in new installations

- #329 Ability to Add/Remove Groups to/from a Repository via AP

- added requires.txt file with requirements

fixes

-----

- fixes db session issues with celery when emailing admins

- #331 RhodeCode mangles repository names if the a repository group 

  contains the "full path" to the repositories

- #298 Conflicting e-mail addresses for LDAP and RhodeCode users

- DB session cleanup after hg protocol operations, fixes issues with

  `mysql has gone away` errors

- #333 doc fixes for get_repo api function

- #271 rare JSON serialization problem with statistics enabled

- #337 Fixes issues with validation of repository name conflicting with 

  a group name. A proper message is now displayed.

- #292 made ldap_dn in user edit readonly, to get rid of confusion that field

  doesn't work   

1.2.3 (**2011-11-02**)

======================

news

----

- added option to manage repos group for non admin users

- added following API methods for get_users, create_user, get_users_groups, 

  get_users_group, create_users_group, add_user_to_users_groups, get_repos, 

  get_repo, create_repo, add_user_to_repo

- implements #237 added password confirmation for my account 

  and admin edit user.

- implements #291 email notification for global events are now sent to all

  administrator users, and global config email.

fixes

-----

- added option for passing auth method for smtp mailer

- #276 issue with adding a single user with id>10 to usergroups

- #277 fixes windows LDAP settings in which missing values breaks the ldap auth 

- #288 fixes managing of repos in a group for non admin user

1.2.2 (**2011-10-17**)

======================

news

----

- #226 repo groups are available by path instead of numerical id

fixes

-----

- #259 Groups with the same name but with different parent group

- #260 Put repo in group, then move group to another group -> repo becomes unavailable

- #258 RhodeCode 1.2 assumes egg folder is writable (lockfiles problems)

- #265 ldap save fails sometimes on converting attributes to booleans, 

  added getter and setter into model that will prevent from this on db model level

- fixed problems with timestamps issues #251 and #213

- fixes #266 RhodeCode allows to create repo with the same name and in 

  the same parent as group

- fixes #245 Rescan of the repositories on Windows

- fixes #248 cannot edit repos inside a group on windows

- fixes #219 forking problems on windows

1.2.1 (**2011-10-08**)

======================

news

----

fixes

-----

- fixed problems with basic auth and push problems 

- gui fixes

- fixed logger

1.2.0 (**2011-10-07**)

======================

news

----

- implemented #47 repository groups

- implemented #89 Can setup google analytics code from settings menu

- implemented #91 added nicer looking archive urls with more download options

  like tags, branches

- implemented #44 into file browsing, and added follow branch option

- implemented #84 downloads can be enabled/disabled for each repository

- anonymous repository can be cloned without having to pass default:default

  into clone url

- fixed #90 whoosh indexer can index chooses repositories passed in command 

  line

- extended journal with day aggregates and paging

- implemented #107 source code lines highlight ranges

- implemented #93 customizable changelog on combined revision ranges - 

  equivalent of githubs compare view 

- implemented #108 extended and more powerful LDAP configuration

- implemented #56 users groups

- major code rewrites optimized codes for speed and memory usage

- raw and diff downloads are now in git format

- setup command checks for write access to given path

- fixed many issues with international characters and unicode. It uses utf8

  decode with replace to provide less errors even with non utf8 encoded strings

- #125 added API KEY access to feeds

- #109 Repository can be created from external Mercurial link (aka. remote 

  repository, and manually updated (via pull) from admin panel

- beta git support - push/pull server + basic view for git repos

- added followers page and forks page

- server side file creation (with binary file upload interface) 

  and edition with commits powered by codemirror 

- #111 file browser file finder, quick lookup files on whole file tree 

- added quick login sliding menu into main page

- changelog uses lazy loading of affected files details, in some scenarios 

  this can improve speed of changelog page dramatically especially for 

  larger repositories.

- implements #214 added support for downloading subrepos in download menu.

- Added basic API for direct operations on rhodecode via JSON

- Implemented advanced hook management

fixes

-----

- fixed file browser bug, when switching into given form revision the url was 

  not changing

- fixed propagation to error controller on simplehg and simplegit middlewares

- fixed error when trying to make a download on empty repository

- fixed problem with '[' chars in commit messages in journal

- fixed #99 Unicode errors, on file node paths with non utf-8 characters

- journal fork fixes

- removed issue with space inside renamed repository after deletion

- fixed strange issue on formencode imports

- fixed #126 Deleting repository on Windows, rename used incompatible chars. 

- #150 fixes for errors on repositories mapped in db but corrupted in 

  filesystem

- fixed problem with ascendant characters in realm #181

- fixed problem with sqlite file based database connection pool

- whoosh indexer and code stats share the same dynamic extensions map

- fixes #188 - relationship delete of repo_to_perm entry on user removal

- fixes issue #189 Trending source files shows "show more" when no more exist

- fixes issue #197 Relative paths for pidlocks

- fixes issue #198 password will require only 3 chars now for login form

- fixes issue #199 wrong redirection for non admin users after creating a repository

- fixes issues #202, bad db constraint made impossible to attach same group 

  more than one time. Affects only mysql/postgres

- fixes #218 os.kill patch for windows was missing sig param

- improved rendering of dag (they are not trimmed anymore when number of 

  heads exceeds 5)

1.1.8 (**2011-04-12**)

======================

news

----

- improved windows support

fixes

-----

- fixed #140 freeze of python dateutil library, since new version is python2.x

  incompatible

- setup-app will check for write permission in given path

- cleaned up license info issue #149

- fixes for issues #137,#116 and problems with unicode and accented characters.

- fixes crashes on gravatar, when passed in email as unicode

- fixed tooltip flickering problems

- fixed came_from redirection on windows

- fixed logging modules, and sql formatters

- windows fixes for os.kill issue #133

- fixes path splitting for windows issues #148

- fixed issue #143 wrong import on migration to 1.1.X

- fixed problems with displaying binary files, thanks to Thomas Waldmann

- removed name from archive files since it's breaking ui for long repo names

- fixed issue with archive headers sent to browser, thanks to Thomas Waldmann

- fixed compatibility for 1024px displays, and larger dpi settings, thanks to 

  Thomas Waldmann

- fixed issue #166 summary pager was skipping 10 revisions on second page

1.1.7 (**2011-03-23**)

======================

news

----

fixes

-----

- fixed (again) #136 installation support for FreeBSD

1.1.6 (**2011-03-21**)

======================

news

----

fixes

-----

- fixed #136 installation support for FreeBSD

- RhodeCode will check for python version during installation

1.1.5 (**2011-03-17**)

======================

news

----

- basic windows support, by exchanging pybcrypt into sha256 for windows only

  highly inspired by idea of mantis406

fixes

-----

- fixed sorting by author in main page

- fixed crashes with diffs on binary files

- fixed #131 problem with boolean values for LDAP

- fixed #122 mysql problems thanks to striker69 

- fixed problem with errors on calling raw/raw_files/annotate functions 

  with unknown revisions

- fixed returned rawfiles attachment names with international character

- cleaned out docs, big thanks to Jason Harris

1.1.4 (**2011-02-19**)

======================

news

----

fixes

-----

- fixed formencode import problem on settings page, that caused server crash

  when that page was accessed as first after server start

- journal fixes

- fixed option to access repository just by entering http://server/<repo_name> 

1.1.3 (**2011-02-16**)

======================

news

----

- implemented #102 allowing the '.' character in username

- added option to access repository just by entering http://server/<repo_name>

- celery task ignores result for better performance

fixes

-----

- fixed ehlo command and non auth mail servers on smtp_lib. Thanks to 

  apollo13 and Johan Walles

- small fixes in journal

- fixed problems with getting setting for celery from .ini files

- registration, password reset and login boxes share the same title as main 

  application now

- fixed #113: to high permissions to fork repository

- fixed problem with '[' chars in commit messages in journal

- removed issue with space inside renamed repository after deletion

- db transaction fixes when filesystem repository creation failed

- fixed #106 relation issues on databases different than sqlite

- fixed static files paths links to use of url() method

1.1.2 (**2011-01-12**)

======================

news

----

fixes

-----

- fixes #98 protection against float division of percentage stats

- fixed graph bug

- forced webhelpers version since it was making troubles during installation 

1.1.1 (**2011-01-06**)

======================

news

----

- added force https option into ini files for easier https usage (no need to

  set server headers with this options)

- small css updates

fixes

-----

- fixed #96 redirect loop on files view on repositories without changesets

- fixed #97 unicode string passed into server header in special cases (mod_wsgi)

  and server crashed with errors

- fixed large tooltips problems on main page

- fixed #92 whoosh indexer is more error proof

1.1.0 (**2010-12-18**)

======================

news

----

- rewrite of internals for vcs >=0.1.10

- uses mercurial 1.7 with dotencode disabled for maintaining compatibility 

  with older clients

- anonymous access, authentication via ldap

- performance upgrade for cached repos list - each repository has it's own 

  cache that's invalidated when needed.

- performance upgrades on repositories with large amount of commits (20K+)

- main page quick filter for filtering repositories

- user dashboards with ability to follow chosen repositories actions

- sends email to admin on new user registration

- added cache/statistics reset options into repository settings

- more detailed action logger (based on hooks) with pushed changesets lists

  and options to disable those hooks from admin panel

- introduced new enhanced changelog for merges that shows more accurate results

- new improved and faster code stats (based on pygments lexers mapping tables, 

  showing up to 10 trending sources for each repository. Additionally stats

  can be disabled in repository settings.

- gui optimizations, fixed application width to 1024px

- added cut off (for large files/changesets) limit into config files

- whoosh, celeryd, upgrade moved to paster command

- other than sqlite database backends can be used

fixes

-----

- fixes #61 forked repo was showing only after cache expired

- fixes #76 no confirmation on user deletes

- fixes #66 Name field misspelled

- fixes #72 block user removal when he owns repositories

- fixes #69 added password confirmation fields

- fixes #87 RhodeCode crashes occasionally on updating repository owner

- fixes #82 broken annotations on files with more than 1 blank line at the end

- a lot of fixes and tweaks for file browser

- fixed detached session issues

- fixed when user had no repos he would see all repos listed in my account

- fixed ui() instance bug when global hgrc settings was loaded for server 

  instance and all hgrc options were merged with our db ui() object

- numerous small bugfixes

(special thanks for TkSoh for detailed feedback)

1.0.2 (**2010-11-12**)

======================

news

----

- tested under python2.7

- bumped sqlalchemy and celery versions

fixes

-----

- fixed #59 missing graph.js

- fixed repo_size crash when repository had broken symlinks

- fixed python2.5 crashes.

1.0.1 (**2010-11-10**)

======================

news

----

- small css updated

fixes

-----

- fixed #53 python2.5 incompatible enumerate calls

- fixed #52 disable mercurial extension for web

- fixed #51 deleting repositories don't delete it's dependent objects

1.0.0 (**2010-11-02**)

======================

- security bugfix simplehg wasn't checking for permissions on commands

  other than pull or push.

- fixed doubled messages after push or pull in admin journal

- templating and css corrections, fixed repo switcher on chrome, updated titles

- admin menu accessible from options menu on repository view

- permissions cached queries

1.0.0rc4  (**2010-10-12**)

==========================

- fixed python2.5 missing simplejson imports (thanks to Jens Bäckman)

- removed cache_manager settings from sqlalchemy meta

- added sqlalchemy cache settings to ini files

- validated password length and added second try of failure on paster setup-app

# -*- coding: utf-8 -*-

"""

    rhodecode.model.repo

    ~~~~~~~~~~~~~~~~~~~~

    Repository model for rhodecode

    :created_on: Jun 5, 2010

    :author: marcink

    :copyright: (C) 2009-2011 Marcin Kuzminski <marcin@python-works.com>

    :license: GPLv3, see COPYING for more details.

"""

# This program is free software: you can redistribute it and/or modify

# it under the terms of the GNU General Public License as published by

# the Free Software Foundation, either version 3 of the License, or

# (at your option) any later version.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program.  If not, see <http://www.gnu.org/licenses/>.

import os

import shutil

import logging

import traceback

from datetime import datetime

from vcs.utils.lazy import LazyProperty

from vcs.backends import get_backend

from rhodecode.model import BaseModel

from rhodecode.model.caching_query import FromCache

from rhodecode.model.db import Repository, RepoToPerm, User, Permission, \

    Statistics, UsersGroup, UsersGroupRepoToPerm, RhodeCodeUi, Group

log = logging.getLogger(__name__)

class RepoModel(BaseModel):

    @LazyProperty

    def repos_path(self):

        """Get's the repositories root path from database

        """

        q = self.sa.query(RhodeCodeUi).filter(RhodeCodeUi.ui_key == '/').one()

        return q.ui_value

    def get(self, repo_id, cache=False):

        repo = self.sa.query(Repository)\

            .filter(Repository.repo_id == repo_id)

        if cache:

            repo = repo.options(FromCache("sql_cache_short",

                                          "get_repo_%s" % repo_id))

        return repo.scalar()

    def get_by_repo_name(self, repo_name, cache=False):

        repo = self.sa.query(Repository)\

            .filter(Repository.repo_name == repo_name)

        if cache:

            repo = repo.options(FromCache("sql_cache_short",

                                          "get_repo_%s" % repo_name))

        return repo.scalar()

    def get_users_js(self):

        users = self.sa.query(User).filter(User.active == True).all()

        u_tmpl = '''{id:%s, fname:"%s", lname:"%s", nname:"%s"},'''

        users_array = '[%s]' % '\n'.join([u_tmpl % (u.user_id, u.name,

                                                    u.lastname, u.username)

                                        for u in users])

        return users_array

    def get_users_groups_js(self):

        users_groups = self.sa.query(UsersGroup)\

            .filter(UsersGroup.users_group_active == True).all()

        g_tmpl = '''{id:%s, grname:"%s",grmembers:"%s"},'''

        users_groups_array = '[%s]' % '\n'.join([g_tmpl % \

                                    (gr.users_group_id, gr.users_group_name,

                                     len(gr.members))

                                        for gr in users_groups])

        return users_groups_array

    def _get_defaults(self, repo_name):

        """

        Get's information about repository, and returns a dict for 

        usage in forms

        :param repo_name:

        """

        repo_info = Repository.get_by_repo_name(repo_name)

        if repo_info is None:

            return None

        defaults = repo_info.get_dict()

        group, repo_name = repo_info.groups_and_repo

        defaults['repo_name'] = repo_name

        defaults['repo_group'] = getattr(group[-1] if group else None,

                                         'group_id', None)

        # fill owner

        if repo_info.user:

            defaults.update({'user': repo_info.user.username})

        else:

            replacement_user = User.query().filter(User.admin ==

                                                   True).first().username

            defaults.update({'user': replacement_user})

        # fill repository users

        for p in repo_info.repo_to_perm:

            defaults.update({'u_perm_%s' % p.user.username:

                             p.permission.permission_name})

        # fill repository groups

        for p in repo_info.users_group_to_perm:

            defaults.update({'g_perm_%s' % p.users_group.users_group_name:

                             p.permission.permission_name})

        return defaults

    def update(self, repo_name, form_data):

        try:

            cur_repo = self.get_by_repo_name(repo_name, cache=False)

            # update permissions

            for member, perm, member_type in form_data['perms_updates']:

                if member_type == 'user':

                    r2p = self.sa.query(RepoToPerm)\

                            .filter(RepoToPerm.user == User.get_by_username(member))\

                            .filter(RepoToPerm.repository == cur_repo)\

                            .one()

                    r2p.permission = self.sa.query(Permission)\

                                        .filter(Permission.permission_name ==

                                                perm).scalar()

                    self.sa.add(r2p)

                else:

                    g2p = self.sa.query(UsersGroupRepoToPerm)\

                            .filter(UsersGroupRepoToPerm.users_group ==

                                    UsersGroup.get_by_group_name(member))\

                            .filter(UsersGroupRepoToPerm.repository ==

                                    cur_repo).one()

                    g2p.permission = self.sa.query(Permission)\

                                        .filter(Permission.permission_name ==

                                                perm).scalar()

                    self.sa.add(g2p)

            # set new permissions

            for member, perm, member_type in form_data['perms_new']:

                if member_type == 'user':

                    r2p = RepoToPerm()

                    r2p.repository = cur_repo

                    r2p.user = User.get_by_username(member)

                    r2p.permission = self.sa.query(Permission)\

                                        .filter(Permission.

                                                permission_name == perm)\

                                                .scalar()

                    self.sa.add(r2p)

                else:

                    g2p = UsersGroupRepoToPerm()

                    g2p.repository = cur_repo

                    g2p.users_group = UsersGroup.get_by_group_name(member)

                    g2p.permission = self.sa.query(Permission)\

                                        .filter(Permission.

                                                permission_name == perm)\

                                                .scalar()

                    self.sa.add(g2p)

            # update current repo

            for k, v in form_data.items():

                if k == 'user':

                    cur_repo.user = User.get_by_username(v)

                elif k == 'repo_name':

                    pass

                elif k == 'repo_group':

                    cur_repo.group = Group.get(v)

                else:

                    setattr(cur_repo, k, v)

            new_name = cur_repo.get_new_name(form_data['repo_name'])

            cur_repo.repo_name = new_name

            self.sa.add(cur_repo)

            if repo_name != new_name:

                # rename repository

                self.__rename_repo(old=repo_name, new=new_name)

            self.sa.commit()

            return cur_repo

        except:

            log.error(traceback.format_exc())

            self.sa.rollback()

            raise

    def create(self, form_data, cur_user, just_db=False, fork=False):

        try:

            if fork:

                repo_name = form_data['fork_name']

                org_name = form_data['repo_name']

                org_full_name = org_name

            else:

                org_name = repo_name = form_data['repo_name']

                repo_name_full = form_data['repo_name_full']

            new_repo = Repository()

            new_repo.enable_statistics = False

            for k, v in form_data.items():

                if k == 'repo_name':

                    if fork:

                        v = repo_name

                    else:

                        v = repo_name_full

                if k == 'repo_group':

                    k = 'group_id'

                if k == 'description':

                setattr(new_repo, k, v)

            if fork:

                parent_repo = self.sa.query(Repository)\

                        .filter(Repository.repo_name == org_full_name).one()

                new_repo.fork = parent_repo

            new_repo.user_id = cur_user.user_id

            self.sa.add(new_repo)

            #create default permission

            repo_to_perm = RepoToPerm()

            default = 'repository.read'

            for p in User.get_by_username('default').user_perms:

                if p.permission.permission_name.startswith('repository.'):

                    default = p.permission.permission_name

                    break

            default_perm = 'repository.none' if form_data['private'] else default

            repo_to_perm.permission_id = self.sa.query(Permission)\

                    .filter(Permission.permission_name == default_perm)\

                    .one().permission_id

            repo_to_perm.repository = new_repo

            repo_to_perm.user_id = User.get_by_username('default').user_id

            self.sa.add(repo_to_perm)

            if not just_db:

                self.__create_repo(repo_name, form_data['repo_type'],

                                   form_data['repo_group'],

                                   form_data['clone_uri'])

            self.sa.commit()

            #now automatically start following this repository as owner

            from rhodecode.model.scm import ScmModel

            ScmModel(self.sa).toggle_following_repo(new_repo.repo_id,

                                             cur_user.user_id)

            return new_repo

        except:

            log.error(traceback.format_exc())

            self.sa.rollback()

            raise

    def create_fork(self, form_data, cur_user):

        from rhodecode.lib.celerylib import tasks, run_task

        run_task(tasks.create_repo_fork, form_data, cur_user)

    def delete(self, repo):

        try:

            self.sa.delete(repo)

            self.__delete_repo(repo)

            self.sa.commit()

        except:

            log.error(traceback.format_exc())

            self.sa.rollback()

            raise

    def delete_perm_user(self, form_data, repo_name):

        try:

            self.sa.query(RepoToPerm)\

                .filter(RepoToPerm.repository \

                        == self.get_by_repo_name(repo_name))\

                .filter(RepoToPerm.user_id == form_data['user_id']).delete()

            self.sa.commit()

        except:

            log.error(traceback.format_exc())

            self.sa.rollback()

            raise

    def delete_perm_users_group(self, form_data, repo_name):

        try:

            self.sa.query(UsersGroupRepoToPerm)\

                .filter(UsersGroupRepoToPerm.repository \

                        == self.get_by_repo_name(repo_name))\

                .filter(UsersGroupRepoToPerm.users_group_id \

                        == form_data['users_group_id']).delete()

            self.sa.commit()

        except:

            log.error(traceback.format_exc())

            self.sa.rollback()

            raise

    def delete_stats(self, repo_name):

        try:

            self.sa.query(Statistics)\

                .filter(Statistics.repository == \

                        self.get_by_repo_name(repo_name)).delete()

            self.sa.commit()

        except:

            log.error(traceback.format_exc())

            self.sa.rollback()

            raise

    def __create_repo(self, repo_name, alias, new_parent_id, clone_uri=False):

        """

        makes repository on filesystem. It's group aware means it'll create

        a repository within a group, and alter the paths accordingly of

        group location

        :param repo_name:

        :param alias:

        :param parent_id:

        :param clone_uri:

        """

        from rhodecode.lib.utils import is_valid_repo, is_valid_repos_group

        if new_parent_id:

            paths = Group.get(new_parent_id).full_path.split(Group.url_sep())

            new_parent_path = os.sep.join(paths)

        else:

            new_parent_path = ''

        repo_path = os.path.join(*map(lambda x:safe_str(x),

                                [self.repos_path, new_parent_path, repo_name]))

        # check if this path is not a repository

        if is_valid_repo(repo_path, self.repos_path):

            raise Exception('This path %s is a valid repository' % repo_path)

        # check if this path is a group

        if is_valid_repos_group(repo_path, self.repos_path):

            raise Exception('This path %s is a valid group' % repo_path)

        log.info('creating repo %s in %s @ %s', repo_name, repo_path,

                 clone_uri)

        backend = get_backend(alias)

        backend(repo_path, create=True, src_url=clone_uri)

    def __rename_repo(self, old, new):

        """

        renames repository on filesystem

        :param old: old name

        :param new: new name

        """

        log.info('renaming repo from %s to %s', old, new)

        old_path = os.path.join(self.repos_path, old)

        new_path = os.path.join(self.repos_path, new)

        if os.path.isdir(new_path):

            raise Exception('Was trying to rename to already existing dir %s' \

            		     % new_path)

        shutil.move(old_path, new_path)

    def __delete_repo(self, repo):

        """

        removes repo from filesystem, the removal is acctually made by

        added rm__ prefix into dir, and rename internat .hg/.git dirs so this

        repository is no longer valid for rhodecode, can be undeleted later on

        by reverting the renames on this repository

        :param repo: repo object

        """

        rm_path = os.path.join(self.repos_path, repo.repo_name)

        log.info("Removing %s", rm_path)

        #disable hg/git

        alias = repo.repo_type

        shutil.move(os.path.join(rm_path, '.%s' % alias),

                    os.path.join(rm_path, 'rm__.%s' % alias))

        #disable repo

        shutil.move(rm_path, os.path.join(self.repos_path, 'rm__%s__%s' \

                                          % (datetime.today()\

                                             .strftime('%Y%m%d_%H%M%S_%f'),

                                            repo.repo_name)))