.. _changelog:

=========

Changelog

=========

1.3.5 (**2012-XX-XX**)

----------------------

:status: in-progress

:branch: beta

news

++++

- use ext_json for json module

- unified annotation view with file source view

- notification improvements, better inbox + css

- #419 don't strip passwords for login forms, make rhodecode 

  more compatible with LDAP servers

- Added HTTP_X_FORWARDED_FOR as another method of extracting 

  IP for pull/push logs. - moved all to base controller  

- #415: Adding comment to changeset causes reload. 

  Comments are now added via ajax and doesn't reload the page

- #374 LDAP config is discarded when LDAP can't be activated

- limited push/pull operations are now logged for git in the journal

- bumped mercurial to 2.2.X series

fixes

+++++

- fixed dev-version marker for stable when served from source codes

- fixed missing permission checks on show forks page

- #418 cast to unicode fixes in notification objects

- #426 fixed mention extracting regex

- fixed remote-pulling for git remotes remopositories

1.3.4 (**2012-03-28**)

----------------------

news

++++

- Whoosh logging is now controlled by the .ini files logging setup

- added clone-url into edit form on /settings page

- added help text into repo add/edit forms

- created rcextensions module with additional mappings (ref #322) and

  post push/pull/create repo hooks callbacks

- implemented #377 Users view for his own permissions on account page

- #399 added inheritance of permissions for users group on repos groups

- #401 repository group is automatically pre-selected when adding repos 

  inside a repository group

- added alternative HTTP 403 response when client failed to authenticate. Helps 

  solving issues with Mercurial and LDAP

- #402 removed group prefix from repository name when listing repositories 

  inside a group

- added gravatars into permission view and permissions autocomplete

- #347 when running multiple RhodeCode instances, properly invalidates cache 

  for all registered servers

fixes

+++++

- fixed #390 cache invalidation problems on repos inside group

- fixed #385 clone by ID url was loosing proxy prefix in URL

- fixed some unicode problems with waitress

- fixed issue with escaping < and > in changeset commits

- fixed error occurring during recursive group creation in API 

  create_repo function

- fixed #393 py2.5 fixes for routes url generator

- fixed #397 Private repository groups shows up before login

- fixed #396 fixed problems with revoking users in nested groups

- fixed mysql unicode issues + specified InnoDB as default engine with 

  utf8 charset

- #406 trim long branch/tag names in changelog to not break UI

1.3.3 (**2012-03-02**)

----------------------

news

++++

fixes

+++++

- fixed some python2.5 compatibility issues 

- fixed issues with removed repos was accidentally added as groups, after

  full rescan of paths

- fixes #376 Cannot edit user (using container auth)

- fixes #378 Invalid image urls on changeset screen with proxy-prefix 

  configuration

- fixed initial sorting of repos inside repo group

- fixes issue when user tried to resubmit same permission into user/user_groups

- bumped beaker version that fixes #375 leap error bug

- fixed raw_changeset for git. It was generated with hg patch headers

- fixed vcs issue with last_changeset for filenodes

- fixed missing commit after hook delete

- fixed #372 issues with git operation detection that caused a security issue 

  for git repos

1.3.2 (**2012-02-28**)

----------------------

news

++++

fixes

+++++

- fixed git protocol issues with repos-groups

- fixed git remote repos validator that prevented from cloning remote git repos

- fixes #370 ending slashes fixes for repo and groups

- fixes #368 improved git-protocol detection to handle other clients

- fixes #366 When Setting Repository Group To Blank Repo Group Wont Be 

  Moved To Root

- fixes #371 fixed issues with beaker/sqlalchemy and non-ascii cache keys 

- fixed #373 missing cascade drop on user_group_to_perm table

1.3.1 (**2012-02-27**)

----------------------

news

++++

fixes

+++++

- redirection loop occurs when remember-me wasn't checked during login

- fixes issues with git blob history generation 

- don't fetch branch for git in file history dropdown. Causes unneeded slowness

1.3.0 (**2012-02-26**)

----------------------

news

++++

- code review, inspired by github code-comments 

- #215 rst and markdown README files support

- #252 Container-based and proxy pass-through authentication support

- #44 branch browser. Filtering of changelog by branches

- mercurial bookmarks support

- new hover top menu, optimized to add maximum size for important views

- configurable clone url template with possibility to specify  protocol like 

  ssh:// or http:// and also manually alter other parts of clone_url.

- enabled largefiles extension by default

- optimized summary file pages and saved a lot of unused space in them

- #239 option to manually mark repository as fork

- #320 mapping of commit authors to RhodeCode users

- #304 hashes are displayed using monospace font    

- diff configuration, toggle white lines and context lines

- #307 configurable diffs, whitespace toggle, increasing context lines

- sorting on branches, tags and bookmarks using YUI datatable

- improved file filter on files page

- implements #330 api method for listing nodes ar particular revision

- #73 added linking issues in commit messages to chosen issue tracker url

  based on user defined regular expression

- added linking of changesets in commit messages  

- new compact changelog with expandable commit messages

- firstname and lastname are optional in user creation

- #348 added post-create repository hook

- #212 global encoding settings is now configurable from .ini files 

- #227 added repository groups permissions

- markdown gets codehilite extensions

- new API methods, delete_repositories, grante/revoke permissions for groups 

  and repos

fixes

+++++

- rewrote dbsession management for atomic operations, and better error handling

- fixed sorting of repo tables

- #326 escape of special html entities in diffs

- normalized user_name => username in api attributes

- fixes #298 ldap created users with mixed case emails created conflicts 

  on saving a form

- fixes issue when owner of a repo couldn't revoke permissions for users 

  and groups

- fixes #271 rare JSON serialization problem with statistics

- fixes #337 missing validation check for conflicting names of a group with a

  repositories group

- #340 fixed session problem for mysql and celery tasks

- fixed #331 RhodeCode mangles repository names if the a repository group 

  contains the "full path" to the repositories

- #355 RhodeCode doesn't store encrypted LDAP passwords

1.2.5 (**2012-01-28**)

----------------------

news

++++

fixes

+++++

- #340 Celery complains about MySQL server gone away, added session cleanup

  for celery tasks

- #341 "scanning for repositories in None" log message during Rescan was missing

  a parameter

- fixed creating archives with subrepos. Some hooks were triggered during that

  operation leading to crash.

- fixed missing email in account page.

- Reverted Mercurial to 2.0.1 for windows due to bug in Mercurial that makes

  forking on windows impossible 

1.2.4 (**2012-01-19**)

----------------------

news

++++

- RhodeCode is bundled with mercurial series 2.0.X by default, with

  full support to largefiles extension. Enabled by default in new installations

- #329 Ability to Add/Remove Groups to/from a Repository via AP

- added requires.txt file with requirements

fixes

+++++

- fixes db session issues with celery when emailing admins

- #331 RhodeCode mangles repository names if the a repository group 

  contains the "full path" to the repositories

- #298 Conflicting e-mail addresses for LDAP and RhodeCode users

- DB session cleanup after hg protocol operations, fixes issues with

Pylons==1.0.0

Beaker==1.6.3

WebHelpers==1.3

formencode==1.2.4

SQLAlchemy==0.7.6

Mako==0.7.0

pygments>=1.4

celery>=2.2.5,<2.3

babel

python-dateutil>=1.5.0,<2.0.0

dulwich>=0.8.5,<0.9.0

webob==1.0.8

markdown==2.1.1

docutils==0.8.1

py-bcrypt

# -*- coding: utf-8 -*-

"""

    rhodecode.__init__

    ~~~~~~~~~~~~~~~~~~

    RhodeCode, a web based repository management based on pylons

    versioning implementation: http://www.python.org/dev/peps/pep-0386/

    :created_on: Apr 9, 2010

    :author: marcink

    :copyright: (C) 2010-2012 Marcin Kuzminski <marcin@python-works.com>

    :license: GPLv3, see COPYING for more details.

"""

# This program is free software: you can redistribute it and/or modify

# it under the terms of the GNU General Public License as published by

# the Free Software Foundation, either version 3 of the License, or

# (at your option) any later version.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program.  If not, see <http://www.gnu.org/licenses/>.

import sys

import platform

VERSION = (1, 4, 0, 'b')

try:

    from rhodecode.lib import get_current_revision

    _rev = get_current_revision()

    if _rev and len(VERSION) > 3:

        VERSION += ('dev%s' % _rev[0],)

except ImportError:

    pass

__version__ = ('.'.join((str(each) for each in VERSION[:3])) +

               '.'.join(VERSION[3:]))

__dbversion__ = 6  # defines current db version for migrations

__platform__ = platform.system()

__license__ = 'GPLv3'

__py_version__ = sys.version_info

PLATFORM_WIN = ('Windows')

PLATFORM_OTHERS = ('Linux', 'Darwin', 'FreeBSD', 'OpenBSD', 'SunOS')

requirements = [

    "Pylons==1.0.0",

    "Beaker==1.6.3",

    "WebHelpers==1.3",

    "formencode==1.2.4",

    "SQLAlchemy==0.7.6",

    "Mako==0.7.0",

    "pygments>=1.4",

    "celery>=2.2.5,<2.3",

    "babel",

    "python-dateutil>=1.5.0,<2.0.0",

    "dulwich>=0.8.5,<0.9.0",

    "webob==1.0.8",

    "markdown==2.1.1",

    "docutils==0.8.1",

]

if __py_version__ < (2, 6):

    requirements.append("simplejson")

    requirements.append("pysqlite")

if __platform__ in PLATFORM_WIN:

else:

    requirements.append("py-bcrypt")

def get_version():

    """Returns shorter version (digit parts only) as string."""

    return '.'.join((str(each) for each in VERSION[:3]))

BACKENDS = {

    'hg': 'Mercurial repository',

    'git': 'Git repository',

}

CELERY_ON = False

# link to config for pylons

CONFIG = {}

# Linked module for extensions

EXTENSIONS = {}

# -*- coding: utf-8 -*-

"""

    rhodecode.controllers.changelog

    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    changelog controller for rhodecode

    :created_on: Apr 21, 2010

    :author: marcink

    :copyright: (C) 2010-2012 Marcin Kuzminski <marcin@python-works.com>

    :license: GPLv3, see COPYING for more details.

"""

# This program is free software: you can redistribute it and/or modify

# it under the terms of the GNU General Public License as published by

# the Free Software Foundation, either version 3 of the License, or

# (at your option) any later version.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program.  If not, see <http://www.gnu.org/licenses/>.

import logging

import traceback

from mercurial import graphmod

from pylons import request, url, session, tmpl_context as c

from pylons.controllers.util import redirect

from pylons.i18n.translation import _

import rhodecode.lib.helpers as h

from rhodecode.lib.auth import LoginRequired, HasRepoPermissionAnyDecorator

from rhodecode.lib.base import BaseRepoController, render

from rhodecode.lib.helpers import RepoPage

from rhodecode.lib.compat import json

from rhodecode.lib.vcs.exceptions import RepositoryError, ChangesetDoesNotExistError

from rhodecode.model.db import Repository

log = logging.getLogger(__name__)

class ChangelogController(BaseRepoController):

    @LoginRequired()

    @HasRepoPermissionAnyDecorator('repository.read', 'repository.write',

                                   'repository.admin')

    def __before__(self):

        super(ChangelogController, self).__before__()

        c.affected_files_cut_off = 60

    def index(self):

        limit = 100

        default = 20

        if request.params.get('size'):

            try:

                int_size = int(request.params.get('size'))

            except ValueError:

                int_size = default

            int_size = int_size if int_size <= limit else limit

            c.size = int_size

            session['changelog_size'] = c.size

            session.save()

        else:

            c.size = int(session.get('changelog_size', default))

        p = int(request.params.get('page', 1))

        branch_name = request.params.get('branch', None)

        try:

            if branch_name:

                collection = [z for z in

                              c.rhodecode_repo.get_changesets(start=0,

                                                    branch_name=branch_name)]

                c.total_cs = len(collection)

            else:

                collection = c.rhodecode_repo

                c.total_cs = len(c.rhodecode_repo)

            c.pagination = RepoPage(collection, page=p, item_count=c.total_cs,

                                    items_per_page=c.size, branch=branch_name)

            collection = list(c.pagination)

            page_revisions = [x.raw_id for x in collection]

            c.comments = c.rhodecode_db_repo.comments(page_revisions)

            c.statuses = c.rhodecode_db_repo.statuses(page_revisions)

        except (RepositoryError, ChangesetDoesNotExistError, Exception), e:

            log.error(traceback.format_exc())

            h.flash(str(e), category='warning')

            return redirect(url('home'))

        self._graph(c.rhodecode_repo, collection, c.total_cs, c.size, p)

        c.branch_name = branch_name

        c.branch_filters = [('', _('All Branches'))] + \

            [(k, k) for k in c.rhodecode_repo.branches.keys()]

        return render('changelog/changelog.html')

    def changelog_details(self, cs):

        if request.environ.get('HTTP_X_PARTIAL_XHR'):

            c.cs = c.rhodecode_repo.get_changeset(cs)

            return render('changelog/changelog_details.html')

    def _graph(self, repo, collection, repo_size, size, p):

        """

        Generates a DAG graph for mercurial

        :param repo: repo instance

        :param size: number of commits to show

        :param p: page number

        """

        if not collection:

            c.jsdata = json.dumps([])

            return

        data = []

        revs = [x.revision for x in collection]

        if repo.alias == 'git':

            for _ in revs:

                vtx = [0, 1]

                edges = [[0, 0, 1]]

                data.append(['', vtx, edges])

        elif repo.alias == 'hg':

            dag = graphmod.dagwalker(repo._repo, revs)

                c.dag = graphmod.colored(dag, repo._repo)

            for (id, type, ctx, vtx, edges) in c.dag:

                if type != graphmod.CHANGESET:

                    continue

                data.append(['', vtx, edges])

        c.jsdata = json.dumps(data)

# -*- coding: utf-8 -*-

"""

    rhodecode.lib.diffs

    ~~~~~~~~~~~~~~~~~~~

    Set of diffing helpers, previously part of vcs

    :created_on: Dec 4, 2011

    :author: marcink

    :copyright: (C) 2010-2012 Marcin Kuzminski <marcin@python-works.com>

    :original copyright: 2007-2008 by Armin Ronacher

    :license: GPLv3, see COPYING for more details.

"""

# This program is free software: you can redistribute it and/or modify

# it under the terms of the GNU General Public License as published by

# the Free Software Foundation, either version 3 of the License, or

# (at your option) any later version.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program.  If not, see <http://www.gnu.org/licenses/>.

import re

import difflib

import markupsafe

from itertools import tee, imap

from pylons.i18n.translation import _

from rhodecode.lib.vcs.exceptions import VCSError

from rhodecode.lib.utils import EmptyChangeset

def wrap_to_table(str_):

    return '''<table class="code-difftable">

                <tr class="line no-comment">

                <td class="lineno new"></td>

                <td class="code no-comment"><pre>%s</pre></td>

                </tr>

              </table>''' % str_

def wrapped_diff(filenode_old, filenode_new, cut_off_limit=None,

                ignore_whitespace=True, line_context=3,

                enable_comments=False):

    """

    returns a wrapped diff into a table, checks for cut_off_limit and presents

    proper message

    """

    if filenode_old is None:

        filenode_old = FileNode(filenode_new.path, '', EmptyChangeset())

    if filenode_old.is_binary or filenode_new.is_binary:

        diff = wrap_to_table(_('binary file'))

        stats = (0, 0)

        size = 0

    elif cut_off_limit != -1 and (cut_off_limit is None or

    (filenode_old.size < cut_off_limit and filenode_new.size < cut_off_limit)):

        f_gitdiff = get_gitdiff(filenode_old, filenode_new,

                                ignore_whitespace=ignore_whitespace,

                                context=line_context)

        diff_processor = DiffProcessor(f_gitdiff, format='gitdiff')

        diff = diff_processor.as_html(enable_comments=enable_comments)

        stats = diff_processor.stat()

        size = len(diff or '')

    else:

        diff = wrap_to_table(_('Changeset was to big and was cut off, use '

                               'diff menu to display this diff'))

        stats = (0, 0)

        size = 0

    if not diff:

        diff = wrap_to_table(_('No changes detected'))

    cs1 = filenode_old.changeset.raw_id

    cs2 = filenode_new.changeset.raw_id

    return size, cs1, cs2, diff, stats

def get_gitdiff(filenode_old, filenode_new, ignore_whitespace=True, context=3):

    """

    Returns git style diff between given ``filenode_old`` and ``filenode_new``.

    :param ignore_whitespace: ignore whitespaces in diff

    """

    # make sure we pass in default context

    context = context or 3

    for filenode in (filenode_old, filenode_new):

        if not isinstance(filenode, FileNode):

            raise VCSError("Given object should be FileNode object, not %s"

                % filenode.__class__)

    repo = filenode_new.changeset.repository

    old_raw_id = getattr(filenode_old.changeset, 'raw_id', repo.EMPTY_CHANGESET)

    new_raw_id = getattr(filenode_new.changeset, 'raw_id', repo.EMPTY_CHANGESET)

    vcs_gitdiff = repo.get_diff(old_raw_id, new_raw_id, filenode_new.path,

                                 ignore_whitespace, context)

    return vcs_gitdiff

class DiffProcessor(object):

    """

    Give it a unified diff and it returns a list of the files that were

    mentioned in the diff together with a dict of meta information that

    can be used to render it in a HTML template.

    """

    _chunk_re = re.compile(r'@@ -(\d+)(?:,(\d+))? \+(\d+)(?:,(\d+))? @@(.*)')

    def __init__(self, diff, differ='diff', format='udiff'):

        """

        :param diff:   a text in diff format or generator

        :param format: format of diff passed, `udiff` or `gitdiff`

        """

        if isinstance(diff, basestring):

            diff = [diff]

        self.__udiff = diff

        self.__format = format

        self.adds = 0

        self.removes = 0

        if isinstance(self.__udiff, basestring):

            self.lines = iter(self.__udiff.splitlines(1))

        elif self.__format == 'gitdiff':

            udiff_copy = self.copy_iterator()

            self.lines = imap(self.escaper, self._parse_gitdiff(udiff_copy))

        else:

            udiff_copy = self.copy_iterator()

            self.lines = imap(self.escaper, udiff_copy)

        # Select a differ.

        if differ == 'difflib':

            self.differ = self._highlight_line_difflib

        else:

            self.differ = self._highlight_line_udiff

    def escaper(self, string):

        return markupsafe.escape(string)

    def copy_iterator(self):

        """

        make a fresh copy of generator, we should not iterate thru

        an original as it's needed for repeating operations on

        this instance of DiffProcessor

        """

        self.__udiff, iterator_copy = tee(self.__udiff)

        return iterator_copy

    def _extract_rev(self, line1, line2):

        """

        Extract the filename and revision hint from a line.

        """

        try:

            if line1.startswith('--- ') and line2.startswith('+++ '):

                l1 = line1[4:].split(None, 1)

                old_filename = (l1[0].replace('a/', '', 1)

                                if len(l1) >= 1 else None)

                old_rev = l1[1] if len(l1) == 2 else 'old'

                l2 = line2[4:].split(None, 1)

                new_filename = (l2[0].replace('b/', '', 1)

                                if len(l1) >= 1 else None)

                new_rev = l2[1] if len(l2) == 2 else 'new'

                filename = (old_filename

                            if old_filename != '/dev/null' else new_filename)

                return filename, new_rev, old_rev

        except (ValueError, IndexError):

            pass

        return None, None, None

    def _parse_gitdiff(self, diffiterator):

        def line_decoder(l):

            if l.startswith('+') and not l.startswith('+++'):

                self.adds += 1

            elif l.startswith('-') and not l.startswith('---'):

                self.removes += 1

            return l.decode('utf8', 'replace')

        output = list(diffiterator)

        size = len(output)

        if size == 2:

            l = []

            l.extend([output[0]])

            l.extend(output[1].splitlines(1))

            return map(line_decoder, l)

        elif size == 1:

            return  map(line_decoder, output[0].splitlines(1))

        elif size == 0:

            return []

        raise Exception('wrong size of diff %s' % size)

    def _highlight_line_difflib(self, line, next_):

        """

        Highlight inline changes in both lines.

        """

        if line['action'] == 'del':

            old, new = line, next_

        else:

            old, new = next_, line

        oldwords = re.split(r'(\W)', old['line'])

        newwords = re.split(r'(\W)', new['line'])

        sequence = difflib.SequenceMatcher(None, oldwords, newwords)

        oldfragments, newfragments = [], []

        for tag, i1, i2, j1, j2 in sequence.get_opcodes():

            oldfrag = ''.join(oldwords[i1:i2])

            newfrag = ''.join(newwords[j1:j2])

            if tag != 'equal':

                if oldfrag:

                    oldfrag = '<del>%s</del>' % oldfrag

                if newfrag:

                    newfrag = '<ins>%s</ins>' % newfrag

            oldfragments.append(oldfrag)

            newfragments.append(newfrag)

        old['line'] = "".join(oldfragments)

        new['line'] = "".join(newfragments)

    def _highlight_line_udiff(self, line, next_):

        """

        Highlight inline changes in both lines.

        """

        start = 0

        limit = min(len(line['line']), len(next_['line']))

        while start < limit and line['line'][start] == next_['line'][start]:

            start += 1

        end = -1

        limit -= start

        while -end <= limit and line['line'][end] == next_['line'][end]:

            end -= 1

        end += 1

        if start or end:

            def do(l):

                last = end + len(l['line'])

                if l['action'] == 'add':

                    tag = 'ins'

                else:

                    tag = 'del'

                l['line'] = '%s<%s>%s</%s>%s' % (

                    l['line'][:start],

                    tag,

                    l['line'][start:last],

                    tag,

                    l['line'][last:]

                )

            do(line)

            do(next_)

    def _parse_udiff(self):

        """

        Parse the diff an return data for the template.

        """

        lineiter = self.lines

        files = []

        try:

            line = lineiter.next()

            # skip first context

            skipfirst = True

            while 1:

                # continue until we found the old file

                if not line.startswith('--- '):

                    line = lineiter.next()

                    continue

                chunks = []

                filename, old_rev, new_rev = \

                    self._extract_rev(line, lineiter.next())

                files.append({

                    'filename':         filename,

                    'old_revision':     old_rev,

                    'new_revision':     new_rev,

                    'chunks':           chunks

                })

                line = lineiter.next()

                while line:

                    match = self._chunk_re.match(line)

                    if not match:

                        break

# -*- coding: utf-8 -*-

"""

    rhodecode.lib.hooks

    ~~~~~~~~~~~~~~~~~~~

    Hooks runned by rhodecode

    :created_on: Aug 6, 2010

    :author: marcink

    :copyright: (C) 2010-2012 Marcin Kuzminski <marcin@python-works.com>

    :license: GPLv3, see COPYING for more details.

"""

# This program is free software: you can redistribute it and/or modify

# it under the terms of the GNU General Public License as published by

# the Free Software Foundation, either version 3 of the License, or

# (at your option) any later version.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program.  If not, see <http://www.gnu.org/licenses/>.

import os

import sys

from mercurial.scmutil import revrange

from mercurial.node import nullrev

from rhodecode import EXTENSIONS

from rhodecode.lib import helpers as h

from rhodecode.lib.utils import action_logger

from inspect import isfunction

def _get_scm_size(alias, root_path):

    if not alias.startswith('.'):

        alias += '.'

    size_scm, size_root = 0, 0