.. _changelog:

=========

Changelog

=========

1.4.0 (**2012-XX-XX**)

----------------------

:status: in-progress

:branch: beta

news

++++

- new codereview system

- email map, allowing users to have multiple email addresses mapped into

  their accounts

- changed setup-app into setup-rhodecode and added default options to it.

- new git repos are created as bare now by default

- #464 added links to groups in permission box

- #465 mentions autocomplete inside comments boxes

- #469 added --update-only option to whoosh to re-index only given list

  of repos in index 

- rhodecode-api CLI client

- new git http protocol replaced buggy dulwich implementation.

  Now based on pygrack & gitweb

fixes

+++++

- improved translations

- fixes issue #455 Creating an archive generates an exception on Windows

- fixes #448 Download ZIP archive keeps file in /tmp open and results 

  in out of disk space

- fixes issue #454 Search results under Windows include proceeding

  backslash

- fixed issue #450. Rhodecode no longer will crash when bad revision is

  present in journal data.

- fix for issue #417, git execution was broken on windows for certain

  commands.

- fixed #413. Don't disable .git directory for bare repos on deleting

- fixed issue #459. Changed the way of obtaining logger in reindex task.

1.3.6 (**2012-05-17**)

----------------------

news

++++

- chinese traditional translation

- changed setup-app into setup-rhodecode and added arguments for auto-setup 

  mode that doesn't need user interaction 

fixes

+++++

- fixed no scm found warning

- fixed __future__ import error on rcextensions

- made simplejson required lib for speedup on JSON encoding

- fixes #449 bad regex could get more than revisions from parsing history

- don't clear DB session when CELERY_EAGER is turned ON

1.3.5 (**2012-05-10**)

----------------------

news

++++

- use ext_json for json module

- unified annotation view with file source view

- notification improvements, better inbox + css

- #419 don't strip passwords for login forms, make rhodecode 

  more compatible with LDAP servers

- Added HTTP_X_FORWARDED_FOR as another method of extracting 

  IP for pull/push logs. - moved all to base controller  

- #415: Adding comment to changeset causes reload. 

  Comments are now added via ajax and doesn't reload the page

- #374 LDAP config is discarded when LDAP can't be activated

- limited push/pull operations are now logged for git in the journal

- bumped mercurial to 2.2.X series

- added support for displaying submodules in file-browser

- #421 added bookmarks in changelog view

fixes

+++++

- fixed dev-version marker for stable when served from source codes

- fixed missing permission checks on show forks page

- #418 cast to unicode fixes in notification objects

- #426 fixed mention extracting regex

# Additional mappings that are not present in the pygments lexers

# used for building stats

# NOTE: that this will overide any mappings in LANGUAGES_EXTENSIONS_MAP

# build by pygments

EXTRA_MAPPINGS = {}

#==============================================================================

# WHOOSH INDEX EXTENSIONS

#==============================================================================

# if INDEX_EXTENSIONS is [] it'll use pygments lexers extensions by default.

# To set your own just add to this list extensions to index with content

INDEX_EXTENSIONS = []

# additional extensions for indexing besides the default from pygments

# those get's added to INDEX_EXTENSIONS

EXTRA_INDEX_EXTENSIONS = []

#==============================================================================

# POST CREATE REPOSITORY HOOK

#==============================================================================

# this function will be executed after each repository is created

def _crhook(*args, **kwargs):

    """

    Post create repository HOOK

    kwargs available:

     :param repo_name:

     :param repo_type:

     :param description:

     :param private:

     :param created_on:

     :param enable_downloads:

     :param repo_id:

     :param user_id:

     :param enable_statistics:

     :param clone_uri:

     :param fork_id:

     :param group_id:

     :param created_by:

    """

    return 0

CREATE_REPO_HOOK = _crhook

#==============================================================================

# POST PUSH HOOK

#==============================================================================

# this function will be executed after each push it's runned after the build-in

# hook that rhodecode uses for logging pushes

def _pushhook(*args, **kwargs):

    """

    Post push hook

    kwargs available:

      :param username: name of user who pushed

      :param ip: ip of who pushed

      :param action: pull

      :param repository: repository name

      :param pushed_revs: generator of pushed revisions

    """

    return 0

PUSH_HOOK = _pushhook

#==============================================================================

# POST PULL HOOK

#==============================================================================

# this function will be executed after each push it's runned after the build-in

# hook that rhodecode uses for logging pushes

def _pullhook(*args, **kwargs):

    """

    Post pull hook

    kwargs available::

      :param username: name of user who pulled

      :param ip: ip of who pushed

      :param action: pull

      :param repository: repository name

    """

    return 0

PULL_HOOK = _pullhook

    :author: marcink

    :copyright: (C) 2010-2012 Marcin Kuzminski <marcin@python-works.com>

    :license: GPLv3, see COPYING for more details.

"""

# This program is free software: you can redistribute it and/or modify

# it under the terms of the GNU General Public License as published by

# the Free Software Foundation, either version 3 of the License, or

# (at your option) any later version.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program.  If not, see <http://www.gnu.org/licenses/>.

from __future__ import with_statement

import os

import sys

import pkg_resources

import traceback

import logging

from os.path import dirname as dn, join as jn

#to get the rhodecode import

sys.path.append(dn(dn(dn(os.path.realpath(__file__)))))

from rhodecode.lib.utils import BasePasterCommand, Command, ask_ok

log = logging.getLogger(__name__)

class MakeRcExt(BasePasterCommand):

    max_args = 1

    min_args = 1

    usage = "CONFIG_FILE"

    summary = "Creates additional extensions for rhodecode"

    group_name = "RhodeCode"

    takes_config_file = -1

    parser = Command.standard_parser(verbose=True)

    def command(self):

        logging.config.fileConfig(self.path_to_ini_file)

        from pylons import config

            bdir = os.path.split(ext_file)[0]

            if not os.path.isdir(bdir):

                os.makedirs(bdir)

            with open(ext_file, 'wb') as f:

                f.write(tmpl)

                log.info('Writen new extensions file to %s' % ext_file)

        here = config['here']

        tmpl = pkg_resources.resource_string(

            'rhodecode', jn('config', 'rcextensions', '__init__.py')

        )

        ext_file = jn(here, 'rcextensions', '__init__.py')

        if os.path.exists(ext_file):

            msg = ('Extension file already exists, do you want '

                   'to overwrite it ? [y/n]')

            if ask_ok(msg):

            else:

                log.info('nothing done...')

        else:

    def update_parser(self):

        pass

                  action="my_account_update", conditions=dict(method=["PUT"]))

        m.connect("admin_settings_create_repository", "/create_repository",

                  action="create_repository", conditions=dict(method=["GET"]))

    #NOTIFICATION REST ROUTES

    with rmap.submapper(path_prefix=ADMIN_PREFIX,

                        controller='admin/notifications') as m:

        m.connect("notifications", "/notifications",

                  action="create", conditions=dict(method=["POST"]))

        m.connect("notifications", "/notifications",

                  action="index", conditions=dict(method=["GET"]))

        m.connect("notifications_mark_all_read", "/notifications/mark_all_read",

                  action="mark_all_read", conditions=dict(method=["GET"]))

        m.connect("formatted_notifications", "/notifications.{format}",

                  action="index", conditions=dict(method=["GET"]))

        m.connect("new_notification", "/notifications/new",

                  action="new", conditions=dict(method=["GET"]))

        m.connect("formatted_new_notification", "/notifications/new.{format}",

                  action="new", conditions=dict(method=["GET"]))

        m.connect("/notification/{notification_id}",

                  action="update", conditions=dict(method=["PUT"]))

        m.connect("/notification/{notification_id}",

                  action="delete", conditions=dict(method=["DELETE"]))

        m.connect("edit_notification", "/notification/{notification_id}/edit",

                  action="edit", conditions=dict(method=["GET"]))

        m.connect("formatted_edit_notification",

                  "/notification/{notification_id}.{format}/edit",

                  action="edit", conditions=dict(method=["GET"]))

        m.connect("notification", "/notification/{notification_id}",

                  action="show", conditions=dict(method=["GET"]))

        m.connect("formatted_notification", "/notifications/{notification_id}.{format}",

                  action="show", conditions=dict(method=["GET"]))

    #ADMIN MAIN PAGES

    with rmap.submapper(path_prefix=ADMIN_PREFIX,

                        controller='admin/admin') as m:

        m.connect('admin_home', '', action='index')

        m.connect('admin_add_repo', '/add_repo/{new_repo:[a-z0-9\. _-]*}',

                  action='add_repo')

    #==========================================================================

    # API V2

    #==========================================================================

    with rmap.submapper(path_prefix=ADMIN_PREFIX,

                        controller='api/api') as m:

        m.connect('api', '/api')

    #USER JOURNAL

    rmap.connect('public_journal', '%s/public_journal' % ADMIN_PREFIX,

                 controller='journal', action="public_journal")

    rmap.connect('public_journal_rss', '%s/public_journal/rss' % ADMIN_PREFIX,

                 controller='journal', action="public_journal_rss")

    rmap.connect('public_journal_rss_old', '%s/public_journal_rss' % ADMIN_PREFIX,

                 controller='journal', action="public_journal_rss")

    rmap.connect('public_journal_atom',

                 '%s/public_journal/atom' % ADMIN_PREFIX, controller='journal',

                 action="public_journal_atom")

    rmap.connect('public_journal_atom_old',

                 '%s/public_journal_atom' % ADMIN_PREFIX, controller='journal',

                 action="public_journal_atom")

    rmap.connect('toggle_following', '%s/toggle_following' % ADMIN_PREFIX,

                 controller='journal', action='toggle_following',

                 conditions=dict(method=["POST"]))

    #SEARCH

    rmap.connect('search', '%s/search' % ADMIN_PREFIX, controller='search',)

    rmap.connect('search_repo', '%s/search/{search_repo:.*}' % ADMIN_PREFIX,

                  controller='search')

    #LOGIN/LOGOUT/REGISTER/SIGN IN

    rmap.connect('login_home', '%s/login' % ADMIN_PREFIX, controller='login')

    rmap.connect('logout_home', '%s/logout' % ADMIN_PREFIX, controller='login',

                 action='logout')

    rmap.connect('register', '%s/register' % ADMIN_PREFIX, controller='login',

                 action='register')

    rmap.connect('reset_password', '%s/password_reset' % ADMIN_PREFIX,

                 controller='login', action='password_reset')

    rmap.connect('reset_password_confirmation',

                 '%s/password_reset_confirmation' % ADMIN_PREFIX,

                 controller='login', action='password_reset_confirmation')

    #FEEDS

    rmap.connect('rss_feed_home', '/{repo_name:.*}/feed/rss',

                controller='feed', action='rss',

                conditions=dict(function=check_repo))

    rmap.connect('atom_feed_home', '/{repo_name:.*}/feed/atom',

            h.flash(_('Whoosh reindex task scheduled'), category='success')

        if setting_id == 'global':

            application_form = ApplicationSettingsForm()()

            try:

                form_result = application_form.to_python(dict(request.POST))

                try:

                    hgsettings1 = RhodeCodeSetting.get_by_name('title')

                    hgsettings1.app_settings_value = \

                        form_result['rhodecode_title']

                    hgsettings2 = RhodeCodeSetting.get_by_name('realm')

                    hgsettings2.app_settings_value = \

                        form_result['rhodecode_realm']

                    hgsettings3 = RhodeCodeSetting.get_by_name('ga_code')

                    hgsettings3.app_settings_value = \

                        form_result['rhodecode_ga_code']

                    self.sa.add(hgsettings1)

                    self.sa.add(hgsettings2)

                    self.sa.add(hgsettings3)

                    self.sa.commit()

                    set_rhodecode_config(config)

                    h.flash(_('Updated application settings'),

                            category='success')

                except Exception:

                    log.error(traceback.format_exc())

                    h.flash(_('error occurred during updating '

                              'application settings'),

                            category='error')

                    self.sa.rollback()

            except formencode.Invalid, errors:

                return htmlfill.render(

                     render('admin/settings/settings.html'),

                     defaults=errors.value,

                     errors=errors.error_dict or {},

                     prefix_error=False,

                     encoding="UTF-8")

        if setting_id == 'mercurial':

            application_form = ApplicationUiSettingsForm()()

            try:

                form_result = application_form.to_python(dict(request.POST))

                try:

                    hgsettings1 = self.sa.query(RhodeCodeUi)\

                    .filter(RhodeCodeUi.ui_key == 'push_ssl').one()

                    hgsettings1.ui_value = form_result['web_push_ssl']

                    hgsettings2 = self.sa.query(RhodeCodeUi)\

                    .filter(RhodeCodeUi.ui_key == '/').one()

                    hgsettings2.ui_value = form_result['paths_root_path']

                    #HOOKS

                    hgsettings3 = self.sa.query(RhodeCodeUi)\

                    hgsettings4 = self.sa.query(RhodeCodeUi)\

                    hgsettings5 = self.sa.query(RhodeCodeUi)\

                    hgsettings6 = self.sa.query(RhodeCodeUi)\

                    self.sa.add(hgsettings1)

                    self.sa.add(hgsettings2)

                    self.sa.add(hgsettings3)

                    self.sa.add(hgsettings4)

                    self.sa.add(hgsettings5)

                    self.sa.add(hgsettings6)

                    self.sa.commit()

                    h.flash(_('Updated mercurial settings'),

                            category='success')

                except:

                    log.error(traceback.format_exc())

                    h.flash(_('error occurred during updating '

                              'application settings'), category='error')

                    self.sa.rollback()

            except formencode.Invalid, errors:

                return htmlfill.render(

                     render('admin/settings/settings.html'),

                     defaults=errors.value,

                     errors=errors.error_dict or {},

                     prefix_error=False,

                     encoding="UTF-8")

        if setting_id == 'hooks':

            ui_key = request.POST.get('new_hook_ui_key')

            ui_value = request.POST.get('new_hook_ui_value')

            try:

                if ui_value and ui_key:

                    RhodeCodeUi.create_or_update_hook(ui_key, ui_value)

                    h.flash(_('Added new hook'),

                            category='success')

                # check for edits

                update = False

                _d = request.POST.dict_of_lists()

                for k, v in zip(_d.get('hook_ui_key', []),

                                _d.get('hook_ui_value_new', [])):

                    RhodeCodeUi.create_or_update_hook(k, v)

                    update = True

                if update:

                    h.flash(_('Updated hooks'), category='success')

                self.sa.commit()

from pylons import url, response, tmpl_context as c

from pylons.i18n.translation import _

from webhelpers.feedgenerator import Atom1Feed, Rss201rev2Feed

from rhodecode.lib import helpers as h

from rhodecode.lib.auth import LoginRequired, HasRepoPermissionAnyDecorator

from rhodecode.lib.base import BaseRepoController

from rhodecode.lib.diffs import DiffProcessor

log = logging.getLogger(__name__)

class FeedController(BaseRepoController):

    @LoginRequired(api_access=True)

    @HasRepoPermissionAnyDecorator('repository.read', 'repository.write',

                                   'repository.admin')

    def __before__(self):

        super(FeedController, self).__before__()

        #common values for feeds

        self.description = _('Changes on %s repository')

        self.title = self.title = _('%s %s feed') % (c.rhodecode_name, '%s')

        self.language = 'en-us'

        self.ttl = "5"

        self.feed_nr = 20

    def _get_title(self, cs):

        return "%s" % (

            h.shorter(cs.message, 160)

        )

    def __changes(self, cs):

        changes = []

        diffprocessor = DiffProcessor(cs.diff())

        stats = diffprocessor.prepare(inline_diff=False)

        for st in stats:

            st.update({'added': st['stats'][0],

                       'removed': st['stats'][1]})

            changes.append('\n %(operation)s %(filename)s '

                           '(%(added)s lines added, %(removed)s lines removed)'

                            % st)

        return changes

    def __get_desc(self, cs):

        desc_msg = []

        desc_msg.append('%s %s %s:<br/>' % (cs.author, _('commited on'),

        desc_msg.append('<pre>')

        desc_msg.append(cs.message)

        desc_msg.append('\n')

        desc_msg.extend(self.__changes(cs))

        desc_msg.append('</pre>')

        return desc_msg

    def atom(self, repo_name):

        """Produce an atom-1.0 feed via feedgenerator module"""

        feed = Atom1Feed(

             title=self.title % repo_name,

             link=url('summary_home', repo_name=repo_name,

                      qualified=True),

             description=self.description % repo_name,

             language=self.language,

             ttl=self.ttl

        )

        for cs in reversed(list(c.rhodecode_repo[-self.feed_nr:])):

            feed.add_item(title=self._get_title(cs),

                          link=url('changeset_home', repo_name=repo_name,

                                   revision=cs.raw_id, qualified=True),

                          author_name=cs.author,

                          description=''.join(self.__get_desc(cs)),

                          pubdate=cs.date,

                          )

        response.content_type = feed.mime_type

        return feed.writeString('utf-8')

    def rss(self, repo_name):

        """Produce an rss2 feed via feedgenerator module"""

        feed = Rss201rev2Feed(

            title=self.title % repo_name,

            link=url('summary_home', repo_name=repo_name,

                     qualified=True),

            description=self.description % repo_name,

            language=self.language,

            ttl=self.ttl

        )

        for cs in reversed(list(c.rhodecode_repo[-self.feed_nr:])):

            feed.add_item(title=self._get_title(cs),

                          link=url('changeset_home', repo_name=repo_name,

                                   revision=cs.raw_id, qualified=True),

                          author_name=cs.author,

                          description=''.join(self.__get_desc(cs)),

                          pubdate=cs.date,

            line_context_lcl = get_line_ctx(fid, request.GET)

            ign_whitespace_lcl = get_ignore_ws(fid, request.GET)

            lim = request.GET.get('fulldiff') or self.cut_off_limit

            _, cs1, cs2, diff, st = diffs.wrapped_diff(filenode_old=node1,

                                         filenode_new=node2,

                                         cut_off_limit=lim,

                                         ignore_whitespace=ign_whitespace_lcl,

                                         line_context=line_context_lcl,

                                         enable_comments=False)

            c.changes = [('', node2, diff, cs1, cs2, st,)]

        return render('files/file_diff.html')

    def _get_node_history(self, cs, f_path):

        changesets = cs.get_file_history(f_path)

        hist_l = []

        changesets_group = ([], _("Changesets"))

        branches_group = ([], _("Branches"))

        tags_group = ([], _("Tags"))

        _hg = cs.repository.alias == 'hg'

        for chs in changesets:

            _branch = '(%s)' % chs.branch if _hg else ''

            n_desc = 'r%s:%s %s' % (chs.revision, chs.short_id, _branch)

            changesets_group[0].append((chs.raw_id, n_desc,))

        hist_l.append(changesets_group)

        for name, chs in c.rhodecode_repo.branches.items():

            branches_group[0].append((chs, name),)

        hist_l.append(branches_group)

        for name, chs in c.rhodecode_repo.tags.items():

            tags_group[0].append((chs, name),)

        hist_l.append(tags_group)

        return hist_l

    @jsonify

    @HasRepoPermissionAnyDecorator('repository.read', 'repository.write',

                                   'repository.admin')

    def nodelist(self, repo_name, revision, f_path):

        if request.environ.get('HTTP_X_PARTIAL_XHR'):

            cs = self.__get_cs_or_redirect(revision, repo_name)

            _d, _f = ScmModel().get_nodes(repo_name, cs.raw_id, f_path,

                                          flat=False)

    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    Journal controller for pylons

    :created_on: Nov 21, 2010

    :author: marcink

    :copyright: (C) 2010-2012 Marcin Kuzminski <marcin@python-works.com>

    :license: GPLv3, see COPYING for more details.

"""

# This program is free software: you can redistribute it and/or modify

# it under the terms of the GNU General Public License as published by

# the Free Software Foundation, either version 3 of the License, or

# (at your option) any later version.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program.  If not, see <http://www.gnu.org/licenses/>.

import logging

from itertools import groupby

from sqlalchemy import or_

from sqlalchemy.orm import joinedload

from webhelpers.paginate import Page

from webhelpers.feedgenerator import Atom1Feed, Rss201rev2Feed

from webob.exc import HTTPBadRequest

from pylons import request, tmpl_context as c, response, url

from pylons.i18n.translation import _

import rhodecode.lib.helpers as h

from rhodecode.lib.auth import LoginRequired, NotAnonymous

from rhodecode.lib.base import BaseController, render

from rhodecode.model.db import UserLog, UserFollowing, Repository, User

from rhodecode.model.meta import Session

from sqlalchemy.sql.expression import func

from rhodecode.model.scm import ScmModel

log = logging.getLogger(__name__)

class JournalController(BaseController):

    def __before__(self):

        super(JournalController, self).__before__()

        self.language = 'en-us'

        self.ttl = "5"

        self.feed_nr = 20

    @LoginRequired()

    @NotAnonymous()

    def index(self):

        # Return a rendered template

        p = int(request.params.get('page', 1))

        c.user = User.get(self.rhodecode_user.user_id)

        all_repos = self.sa.query(Repository)\

                     .filter(Repository.user_id == c.user.user_id)\

                     .order_by(func.lower(Repository.repo_name)).all()

        c.user_repos = ScmModel().get_repos(all_repos)

        c.following = self.sa.query(UserFollowing)\

            .filter(UserFollowing.user_id == self.rhodecode_user.user_id)\

            .options(joinedload(UserFollowing.follows_repository))\

            .all()

        journal = self._get_journal_data(c.following)

        c.journal_pager = Page(journal, page=p, items_per_page=20)

        c.journal_day_aggreagate = self._get_daily_aggregate(c.journal_pager)

        c.journal_data = render('journal/journal_data.html')

        if request.environ.get('HTTP_X_PARTIAL_XHR'):

            return c.journal_data

        return render('journal/journal.html')

    def _get_daily_aggregate(self, journal):

        groups = []

        for k, g in groupby(journal, lambda x: x.action_as_day):

            user_group = []

            for k2, g2 in groupby(list(g), lambda x: x.user.email):

                l = list(g2)

                user_group.append((l[0].user, l))

            groups.append((k, user_group,))

        return groups

    def _get_journal_data(self, following_repos):

        repo_ids = [x.follows_repository.repo_id for x in following_repos

                    if x.follows_repository is not None]

        user_ids = [x.follows_user.user_id for x in following_repos

                    if x.follows_user is not None]

        filtering_criterion = None

        if repo_ids and user_ids:

            filtering_criterion = or_(UserLog.repository_id.in_(repo_ids),

                        UserLog.user_id.in_(user_ids))

        if repo_ids and not user_ids:

            filtering_criterion = UserLog.repository_id.in_(repo_ids)

        if not repo_ids and user_ids: