.. _changelog:

=========

Changelog

=========

----------------------

:status: in-progress

:branch: beta

news

++++

- new codereview system

- email map, allowing users to have multiple email addresses mapped into

  their accounts

- improved git-hook system. Now all actions for git are logged into journal

  including pushed revisions, user and IP address

- changed setup-app into setup-rhodecode and added default options to it.

- new git repos are created as bare now by default

- #464 added links to groups in permission box

- #465 mentions autocomplete inside comments boxes

- #469 added --update-only option to whoosh to re-index only given list

  of repos in index 

- rhodecode-api CLI client

- new git http protocol replaced buggy dulwich implementation.

  Now based on pygrack & gitweb

- Improved RSS/ATOM feeds. Discoverable by browsers using proper headers, and 

  reformated based on user suggestions. Additional rss/atom feeds for user

  journal

- various i18n improvements

- #478 permissions overview for admin in user edit view

- File view now displays small gravatars off all authors of given file

- Implemented landing revisions. Each repository will get landing_rev attribute

  that defines 'default' revision/branch for generating readme files

- Implemented #509, RhodeCode enforces SSL for push/pulling if requested at 

  earliest possible call.

- Import remote svn repositories to mercurial using hgsubversion.

- Fixed #508 RhodeCode now has a option to explicitly set forking permissions

- RhodeCode can use alternative server for generating avatar icons

- implemented repositories locking. Pull locks, push unlocks. Also can be done

  via API calls

- #538 form for permissions can handle multiple users at once 

fixes

+++++

- improved translations

- fixes issue #455 Creating an archive generates an exception on Windows

- fixes #448 Download ZIP archive keeps file in /tmp open and results 

  in out of disk space

- fixes issue #454 Search results under Windows include proceeding

  backslash

- fixed issue #450. Rhodecode no longer will crash when bad revision is

  present in journal data.

- fix for issue #417, git execution was broken on windows for certain

  commands.

- fixed #413. Don't disable .git directory for bare repos on deleting

- fixed issue #459. Changed the way of obtaining logger in reindex task.

- fixed #453 added ID field in whoosh SCHEMA that solves the issue of

  reindexing modified files

- fixed #481 rhodecode emails are sent without Date header 

- fixed #458 wrong count when no repos are present

- fixed issue #492 missing `\ No newline at end of file` test at the end of 

  new chunk in html diff

- full text search now works also for commit messages

1.3.6 (**2012-05-17**)

----------------------

news

++++

- chinese traditional translation

- changed setup-app into setup-rhodecode and added arguments for auto-setup 

  mode that doesn't need user interaction 

fixes

+++++

- fixed no scm found warning

- fixed __future__ import error on rcextensions

- made simplejson required lib for speedup on JSON encoding

- fixes #449 bad regex could get more than revisions from parsing history

- don't clear DB session when CELERY_EAGER is turned ON

1.3.5 (**2012-05-10**)

----------------------

news

++++

- use ext_json for json module

- unified annotation view with file source view

- notification improvements, better inbox + css

- #419 don't strip passwords for login forms, make rhodecode 

  more compatible with LDAP servers

- Added HTTP_X_FORWARDED_FOR as another method of extracting 

  IP for pull/push logs. - moved all to base controller  

- #415: Adding comment to changeset causes reload. 

  Comments are now added via ajax and doesn't reload the page

- #374 LDAP config is discarded when LDAP can't be activated

- limited push/pull operations are now logged for git in the journal

- bumped mercurial to 2.2.X series

- added support for displaying submodules in file-browser

- #421 added bookmarks in changelog view

fixes

+++++

- fixed dev-version marker for stable when served from source codes

- fixed missing permission checks on show forks page

- #418 cast to unicode fixes in notification objects

- #426 fixed mention extracting regex

- fixed remote-pulling for git remotes remopositories

- fixed #434: Error when accessing files or changesets of a git repository 

  with submodules

- fixed issue with empty APIKEYS for users after registration ref. #438

- fixed issue with getting README files from git repositories

1.3.4 (**2012-03-28**)

----------------------

news

++++

- Whoosh logging is now controlled by the .ini files logging setup

- added clone-url into edit form on /settings page

- added help text into repo add/edit forms

- created rcextensions module with additional mappings (ref #322) and

  post push/pull/create repo hooks callbacks

- implemented #377 Users view for his own permissions on account page

- #399 added inheritance of permissions for users group on repos groups

- #401 repository group is automatically pre-selected when adding repos 

  inside a repository group

- added alternative HTTP 403 response when client failed to authenticate. Helps 

  solving issues with Mercurial and LDAP

- #402 removed group prefix from repository name when listing repositories 

  inside a group

- added gravatars into permission view and permissions autocomplete

- #347 when running multiple RhodeCode instances, properly invalidates cache 

  for all registered servers

fixes

+++++

- fixed #390 cache invalidation problems on repos inside group

- fixed #385 clone by ID url was loosing proxy prefix in URL

- fixed some unicode problems with waitress

- fixed issue with escaping < and > in changeset commits

- fixed error occurring during recursive group creation in API 

  create_repo function

- fixed #393 py2.5 fixes for routes url generator

- fixed #397 Private repository groups shows up before login

- fixed #396 fixed problems with revoking users in nested groups

- fixed mysql unicode issues + specified InnoDB as default engine with 

  utf8 charset

- #406 trim long branch/tag names in changelog to not break UI

1.3.3 (**2012-03-02**)

----------------------

news

++++

fixes

+++++

- fixed some python2.5 compatibility issues 

- fixed issues with removed repos was accidentally added as groups, after

  full rescan of paths

- fixes #376 Cannot edit user (using container auth)

- fixes #378 Invalid image urls on changeset screen with proxy-prefix 

  configuration

- fixed initial sorting of repos inside repo group

- fixes issue when user tried to resubmit same permission into user/user_groups

- bumped beaker version that fixes #375 leap error bug

- fixed raw_changeset for git. It was generated with hg patch headers

- fixed vcs issue with last_changeset for filenodes

- fixed missing commit after hook delete

- fixed #372 issues with git operation detection that caused a security issue 

  for git repos

1.3.2 (**2012-02-28**)

----------------------

news

++++

fixes

+++++

- fixed git protocol issues with repos-groups

- fixed git remote repos validator that prevented from cloning remote git repos

- fixes #370 ending slashes fixes for repo and groups

- fixes #368 improved git-protocol detection to handle other clients

- fixes #366 When Setting Repository Group To Blank Repo Group Wont Be 

  Moved To Root

- fixes #371 fixed issues with beaker/sqlalchemy and non-ascii cache keys 

- fixed #373 missing cascade drop on user_group_to_perm table

1.3.1 (**2012-02-27**)

----------------------

                enable_comments = False

                rev_start = rev_range[0]

                rev_end = rev_range[1]

                rev_ranges = c.rhodecode_repo.get_changesets(start=rev_start,

                                                            end=rev_end)

            else:

                rev_ranges = [c.rhodecode_repo.get_changeset(revision)]

            c.cs_ranges = list(rev_ranges)

            if not c.cs_ranges:

                raise RepositoryError('Changeset range returned empty result')

        except (RepositoryError, ChangesetDoesNotExistError, Exception), e:

            log.error(traceback.format_exc())

            h.flash(str(e), category='warning')

            return redirect(url('home'))

        c.changes = OrderedDict()

        c.lines_added = 0  # count of lines added

        c.lines_deleted = 0  # count of lines removes

        cumulative_diff = 0

        c.cut_off = False  # defines if cut off limit is reached

        c.changeset_statuses = ChangesetStatus.STATUSES

        c.comments = []

        c.statuses = []

        c.inline_comments = []

        c.inline_cnt = 0

        # Iterate over ranges (default changeset view is always one changeset)

        for changeset in c.cs_ranges:

            c.statuses.extend([ChangesetStatusModel()\

                              .get_status(c.rhodecode_db_repo.repo_id,

                                          changeset.raw_id)])

            c.comments.extend(ChangesetCommentsModel()\

                              .get_comments(c.rhodecode_db_repo.repo_id,

                                            revision=changeset.raw_id))

            inlines = ChangesetCommentsModel()\

                        .get_inline_comments(c.rhodecode_db_repo.repo_id,

                                             revision=changeset.raw_id)

            c.inline_comments.extend(inlines)

            c.changes[changeset.raw_id] = []

            try:

                changeset_parent = changeset.parents[0]

            except IndexError:

                changeset_parent = None

            #==================================================================

            # ADDED FILES

            #==================================================================

            for node in changeset.added:

                fid = h.FID(revision, node.path)

                line_context_lcl = get_line_ctx(fid, request.GET)

                ign_whitespace_lcl = get_ignore_ws(fid, request.GET)

                lim = self.cut_off_limit

                if cumulative_diff > self.cut_off_limit:

                    lim = -1 if limit_off is None else None

                size, cs1, cs2, diff, st = wrapped_diff(

                    filenode_old=None,

                    filenode_new=node,

                    cut_off_limit=lim,

                    ignore_whitespace=ign_whitespace_lcl,

                    line_context=line_context_lcl,

                    enable_comments=enable_comments

                )

                cumulative_diff += size

                c.lines_added += st[0]

                c.lines_deleted += st[1]

                c.changes[changeset.raw_id].append(

                    ('added', node, diff, cs1, cs2, st)

                )

            #==================================================================

            # CHANGED FILES

            #==================================================================

            for node in changeset.changed:

                try:

                    filenode_old = changeset_parent.get_node(node.path)

                except ChangesetError:

                    log.warning('Unable to fetch parent node for diff')

                    filenode_old = FileNode(node.path, '', EmptyChangeset())

                fid = h.FID(revision, node.path)

                line_context_lcl = get_line_ctx(fid, request.GET)

                ign_whitespace_lcl = get_ignore_ws(fid, request.GET)

                lim = self.cut_off_limit

                if cumulative_diff > self.cut_off_limit:

                    lim = -1 if limit_off is None else None

                size, cs1, cs2, diff, st = wrapped_diff(

                    filenode_old=filenode_old,

                    filenode_new=node,

                    cut_off_limit=lim,

                    ignore_whitespace=ign_whitespace_lcl,

                    line_context=line_context_lcl,

                    enable_comments=enable_comments

                )

                cumulative_diff += size

                c.lines_added += st[0]

                c.lines_deleted += st[1]

                c.changes[changeset.raw_id].append(

                    ('changed', node, diff, cs1, cs2, st)

                )

            #==================================================================

            # REMOVED FILES

            #==================================================================

            for node in changeset.removed:

                c.changes[changeset.raw_id].append(

                    ('removed', node, None, None, None, (0, 0))

                )

        # count inline comments

        for __, lines in c.inline_comments:

            for comments in lines.values():

                c.inline_cnt += len(comments)

        if len(c.cs_ranges) == 1:

            c.changeset = c.cs_ranges[0]

            c.changes = c.changes[c.changeset.raw_id]

            return render('changeset/changeset.html')

        else:

            return render('changeset/changeset_range.html')

    def raw_changeset(self, revision):

        method = request.GET.get('diff', 'show')

        ignore_whitespace = request.GET.get('ignorews') == '1'

        line_context = request.GET.get('context', 3)

        try:

            c.scm_type = c.rhodecode_repo.alias

            c.changeset = c.rhodecode_repo.get_changeset(revision)

        except RepositoryError:

            log.error(traceback.format_exc())

            return redirect(url('home'))

        else:

            try:

                c.changeset_parent = c.changeset.parents[0]

            except IndexError:

                c.changeset_parent = None

            c.changes = []

            for node in c.changeset.added:

                filenode_old = FileNode(node.path, '')

                if filenode_old.is_binary or node.is_binary:

                    diff = _('binary file') + '\n'

                else:

                    f_gitdiff = diffs.get_gitdiff(filenode_old, node,

                                           ignore_whitespace=ignore_whitespace,

                                           context=line_context)

                    diff = diffs.DiffProcessor(f_gitdiff,

                                                format='gitdiff').raw_diff()

                cs1 = None

                cs2 = node.changeset.raw_id

                c.changes.append(('added', node, diff, cs1, cs2))

            for node in c.changeset.changed:

                filenode_old = c.changeset_parent.get_node(node.path)

                if filenode_old.is_binary or node.is_binary:

                    diff = _('binary file')

                else:

                    f_gitdiff = diffs.get_gitdiff(filenode_old, node,

                                           ignore_whitespace=ignore_whitespace,

                                           context=line_context)

                    diff = diffs.DiffProcessor(f_gitdiff,

                                                format='gitdiff').raw_diff()

                cs1 = filenode_old.changeset.raw_id

                cs2 = node.changeset.raw_id

                c.changes.append(('changed', node, diff, cs1, cs2))

        response.content_type = 'text/plain'

        if method == 'download':

            response.content_disposition = 'attachment; filename=%s.patch' \

                                            % revision

        c.parent_tmpl = ''.join(['# Parent  %s\n' % x.raw_id

                                 for x in c.changeset.parents])

        c.diffs = ''

        for x in c.changes:

            c.diffs += x[2]

        return render('changeset/raw_changeset.html')

    @jsonify

    def comment(self, repo_name, revision):

        status = request.POST.get('changeset_status')

        change_status = request.POST.get('change_changeset_status')

        comm = ChangesetCommentsModel().create(

            repo=c.rhodecode_db_repo.repo_id,

            user=c.rhodecode_user.user_id,

            revision=revision,

            f_path=request.POST.get('f_path'),

            line_no=request.POST.get('line'),

            status_change=(ChangesetStatus.get_status_lbl(status)

                           if status and change_status else None)

        )

        # get status if set !

        if status and change_status:

            # if latest status was from pull request and it's closed

            # disallow changing status ! 

            # dont_allow_on_closed_pull_request = True !

            try:

                ChangesetStatusModel().set_status(

                    c.rhodecode_db_repo.repo_id,

                    status,

                    c.rhodecode_user.user_id,

                    comm,

                    revision=revision,

                    dont_allow_on_closed_pull_request=True

                )

            except StatusChangeOnClosedPullRequestError:

                log.error(traceback.format_exc())

                msg = _('Changing status on a changeset associated with'

                        'a closed pull request is not allowed')

                h.flash(msg, category='warning')

                return redirect(h.url('changeset_home', repo_name=repo_name,

                                      revision=revision))

        action_logger(self.rhodecode_user,

                      'user_commented_revision:%s' % revision,

                      c.rhodecode_db_repo, self.ip_addr, self.sa)

        Session().commit()

        if not request.environ.get('HTTP_X_PARTIAL_XHR'):

            return redirect(h.url('changeset_home', repo_name=repo_name,

                                  revision=revision))

        data = {

           'target_id': h.safeid(h.safe_unicode(request.POST.get('f_path'))),

        }

        if comm:

            c.co = comm

            data.update(comm.get_dict())

            data.update({'rendered_text':

                         render('changeset/changeset_comment_block.html')})

        return data

    @jsonify

    def delete_comment(self, repo_name, comment_id):

        co = ChangesetComment.get(comment_id)

        owner = lambda: co.author.user_id == c.rhodecode_user.user_id

        if h.HasPermissionAny('hg.admin', 'repository.admin')() or owner:

            ChangesetCommentsModel().delete(comment=co)

            Session().commit()

            return True

        else:

            raise HTTPForbidden()

    def create(self, repo_name):

        try:

            _form = PullRequestForm()().to_python(request.POST)

        except formencode.Invalid, errors:

            log.error(traceback.format_exc())

            if errors.error_dict.get('revisions'):

                msg = 'Revisions: %s' % errors.error_dict['revisions']

            elif errors.error_dict.get('pullrequest_title'):

                msg = _('Pull request requires a title with min. 3 chars')

            else:

                msg = _('error during creation of pull request')

            h.flash(msg, 'error')

            return redirect(url('pullrequest_home', repo_name=repo_name))

        org_repo = _form['org_repo']

        org_ref = _form['org_ref']

        other_repo = _form['other_repo']

        other_ref = _form['other_ref']

        revisions = _form['revisions']

        reviewers = _form['review_members']

        title = _form['pullrequest_title']

        description = _form['pullrequest_desc']

        try:

            pull_request = PullRequestModel().create(

                self.rhodecode_user.user_id, org_repo, org_ref, other_repo,

                other_ref, revisions, reviewers, title, description

            )

            Session().commit()

            h.flash(_('Successfully opened new pull request'),

                    category='success')

        except Exception:

            h.flash(_('Error occurred during sending pull request'),

                    category='error')

            log.error(traceback.format_exc())

            return redirect(url('pullrequest_home', repo_name=repo_name))

        return redirect(url('pullrequest_show', repo_name=other_repo,

                            pull_request_id=pull_request.pull_request_id))

    @NotAnonymous()

    @jsonify

    def update(self, repo_name, pull_request_id):

        pull_request = PullRequest.get_or_404(pull_request_id)

        if pull_request.is_closed():

            raise HTTPForbidden()

        #only owner or admin can update it

        owner = pull_request.author.user_id == c.rhodecode_user.user_id

        if h.HasPermissionAny('hg.admin', 'repository.admin')() or owner:

            reviewers_ids = map(int, filter(lambda v: v not in [None, ''],

                       request.POST.get('reviewers_ids', '').split(',')))

            PullRequestModel().update_reviewers(pull_request_id, reviewers_ids)

            Session.commit()

            return True

        raise HTTPForbidden()

    @NotAnonymous()

    @jsonify

    def delete(self, repo_name, pull_request_id):

        pull_request = PullRequest.get_or_404(pull_request_id)

        #only owner can delete it !

        if pull_request.author.user_id == c.rhodecode_user.user_id:

            PullRequestModel().delete(pull_request)

            Session().commit()

            h.flash(_('Successfully deleted pull request'),

                    category='success')

            return redirect(url('admin_settings_my_account'))

        raise HTTPForbidden()

    def _load_compare_data(self, pull_request, enable_comments=True):

        """

        Load context data needed for generating compare diff

        :param pull_request:

        :type pull_request:

        """

        org_repo = pull_request.org_repo

        (org_ref_type,

         org_ref_name,

         org_ref_rev) = pull_request.org_ref.split(':')

        other_repo = pull_request.other_repo

        (other_ref_type,

         other_ref_name,

         other_ref_rev) = pull_request.other_ref.split(':')

        # despite opening revisions for bookmarks/branches/tags, we always

        # convert this to rev to prevent changes after book or branch change

        org_ref = ('rev', org_ref_rev)

        other_ref = ('rev', other_ref_rev)

        c.org_repo = org_repo

        c.other_repo = other_repo

        c.cs_ranges, discovery_data = PullRequestModel().get_compare_data(

                                       org_repo, org_ref, other_repo, other_ref

                                      )

        c.statuses = c.rhodecode_db_repo.statuses([x.raw_id for x in

                                                   c.cs_ranges])

        # defines that we need hidden inputs with changesets

        c.as_form = request.GET.get('as_form', False)

        c.org_ref = org_ref[1]

        c.other_ref = other_ref[1]

        # diff needs to have swapped org with other to generate proper diff

        _diff = diffs.differ(other_repo, other_ref, org_repo, org_ref,

                             discovery_data)

        diff_processor = diffs.DiffProcessor(_diff, format='gitdiff')

        _parsed = diff_processor.prepare()

        c.files = []

        c.changes = {}

        for f in _parsed:

            fid = h.FID('', f['filename'])

            c.files.append([fid, f['operation'], f['filename'], f['stats']])

            diff = diff_processor.as_html(enable_comments=enable_comments,

                                          diff_lines=[f])

            c.changes[fid] = [f['operation'], f['filename'], diff]

    def show(self, repo_name, pull_request_id):

        repo_model = RepoModel()

        c.users_array = repo_model.get_users_js()

        c.users_groups_array = repo_model.get_users_groups_js()

        c.pull_request = PullRequest.get_or_404(pull_request_id)

        cc_model = ChangesetCommentsModel()

        cs_model = ChangesetStatusModel()

        _cs_statuses = cs_model.get_statuses(c.pull_request.org_repo,

                                            pull_request=c.pull_request,

                                            with_revisions=True)

        cs_statuses = defaultdict(list)

        for st in _cs_statuses:

            cs_statuses[st.author.username] += [st]

        c.pull_request_reviewers = []

        c.pull_request_pending_reviewers = []

        for o in c.pull_request.reviewers:

            st = cs_statuses.get(o.user.username, None)

            if st:

                sorter = lambda k: k.version

                st = [(x, list(y)[0])

                      for x, y in (groupby(sorted(st, key=sorter), sorter))]

            else:

                c.pull_request_pending_reviewers.append(o.user)

            c.pull_request_reviewers.append([o.user, st])

        # pull_requests repo_name we opened it against

        # ie. other_repo must match

        if repo_name != c.pull_request.other_repo.repo_name:

            raise HTTPNotFound

        # load compare data into template context

        enable_comments = not c.pull_request.is_closed()

        self._load_compare_data(c.pull_request, enable_comments=enable_comments)

        # inline comments

        c.inline_cnt = 0

        c.inline_comments = cc_model.get_inline_comments(

                                c.rhodecode_db_repo.repo_id,

                                pull_request=pull_request_id)

        # count inline comments

        for __, lines in c.inline_comments:

            for comments in lines.values():

                c.inline_cnt += len(comments)

        # comments

        c.comments = cc_model.get_comments(c.rhodecode_db_repo.repo_id,

                                           pull_request=pull_request_id)

        # changeset(pull-request) status

        c.current_changeset_status = cs_model.calculate_status(

                                        c.pull_request_reviewers

                                     )

        c.changeset_statuses = ChangesetStatus.STATUSES

        c.target_repo = c.pull_request.org_repo.repo_name

        return render('/pullrequests/pullrequest_show.html')

    @NotAnonymous()

    @jsonify

    def comment(self, repo_name, pull_request_id):

        pull_request = PullRequest.get_or_404(pull_request_id)

        if pull_request.is_closed():

            raise HTTPForbidden()

        status = request.POST.get('changeset_status')

        change_status = request.POST.get('change_changeset_status')

        comm = ChangesetCommentsModel().create(

            repo=c.rhodecode_db_repo.repo_id,

            user=c.rhodecode_user.user_id,

            pull_request=pull_request_id,

            f_path=request.POST.get('f_path'),

            line_no=request.POST.get('line'),

            status_change=(ChangesetStatus.get_status_lbl(status)

                           if status and change_status else None)

        )

        # get status if set !

        if status and change_status:

            ChangesetStatusModel().set_status(

                c.rhodecode_db_repo.repo_id,

                status,

                c.rhodecode_user.user_id,

                comm,

                pull_request=pull_request_id

            )

        action_logger(self.rhodecode_user,

                      'user_commented_pull_request:%s' % pull_request_id,

                      c.rhodecode_db_repo, self.ip_addr, self.sa)

        if request.POST.get('save_close'):

            PullRequestModel().close_pull_request(pull_request_id)

            action_logger(self.rhodecode_user,

                      'user_closed_pull_request:%s' % pull_request_id,

                      c.rhodecode_db_repo, self.ip_addr, self.sa)

        Session().commit()

        if not request.environ.get('HTTP_X_PARTIAL_XHR'):

            return redirect(h.url('pullrequest_show', repo_name=repo_name,

                                  pull_request_id=pull_request_id))

        data = {

           'target_id': h.safeid(h.safe_unicode(request.POST.get('f_path'))),

        }

        if comm:

            c.co = comm

            data.update(comm.get_dict())

            data.update({'rendered_text':

                         render('changeset/changeset_comment_block.html')})

        return data

    @NotAnonymous()

    @jsonify

    def delete_comment(self, repo_name, comment_id):

        co = ChangesetComment.get(comment_id)

        if co.pull_request.is_closed():

            #don't allow deleting comments on closed pull request

            raise HTTPForbidden()

        owner = lambda: co.author.user_id == c.rhodecode_user.user_id

        if h.HasPermissionAny('hg.admin', 'repository.admin')() or owner:

            ChangesetCommentsModel().delete(comment=co)

            Session().commit()

            return True

        else:

            raise HTTPForbidden()