.. _changelog:

Changelog

=========

1.2.0 (**2011-XX-XX**)

======================

:status: in-progress

:branch: beta

news

----

- implemented #89 Can setup google analytics code from settings menu

- implemented #91 added nicer looking archive urls with more download options

  like tags, branches

- implemented #44 into file browsing, and added follow branch option

- implemented #84 downloads can be enabled/disabled for each repository

- anonymous repository can be cloned without having to pass default:default

  into clone url

- fixed #90 whoosh indexer can index chooses repositories passed in command 

  line

- extended journal with day aggregates and paging

- implemented #107 source code lines highlight ranges

- implemented #93 customizable changelog on combined revision ranges - 

  equivalent of githubs compare view 

- implemented #108 extended and more powerful LDAP configuration

- implemented #56 users groups

- major code rewrites optimized codes for speed and memory usage

- raw and diff downloads are now in git format

- setup command checks for write access to given path

- fixed many issues with international characters and unicode. It uses utf8

  decode with replace to provide less errors even with non utf8 encoded strings

- #125 added API KEY access to feeds

- #109 Repository can be created from external Mercurial link (aka. remote 

  repository, and manually updated (via pull) from admin panel

- beta git support - push/pull server + basic view for git repos

- added followers page

fixes

-----

- fixed file browser bug, when switching into given form revision the url was 

  not changing

- fixed propagation to error controller on simplehg and simplegit middlewares

- fixed error when trying to make a download on empty repository

- fixed problem with '[' chars in commit messages in journal

- fixed #99 Unicode errors, on file node paths with non utf-8 characters

- journal fork fixes

- removed issue with space inside renamed repository after deletion

- fixed strange issue on formencode imports

- fixed #126 Deleting repository on Windows, rename used incompatible chars. 

- #150 fixes for errors on repositories mapped in db but corrupted in 

  filesystem

1.1.8 (**2011-04-12**)

======================

news

----

- improved windows support

fixes

-----

- fixed #140 freeze of python dateutil library, since new version is python2.x

  incompatible

- setup-app will check for write permission in given path

- cleaned up license info issue #149

- fixes for issues #137,#116 and problems with unicode and accented characters.

- fixes crashes on gravatar, when passed in email as unicode

- fixed tooltip flickering problems

- fixed came_from redirection on windows

- fixed logging modules, and sql formatters

- windows fixes for os.kill issue #133

- fixes path splitting for windows issues #148

- fixed issue #143 wrong import on migration to 1.1.X

- fixed problems with displaying binary files, thanks to Thomas Waldmann

- removed name from archive files since it's breaking ui for long repo names

- fixed issue with archive headers sent to browser, thanks to Thomas Waldmann

- fixed compatibility for 1024px displays, and larger dpi settings, thanks to 

  Thomas Waldmann

- fixed issue #166 summary pager was skipping 10 revisions on second page

1.1.7 (**2011-03-23**)

======================

news

----

fixes

-----

- fixed (again) #136 installation support for FreeBSD

1.1.6 (**2011-03-21**)

======================

news

----

fixes

-----

- fixed #136 installation support for FreeBSD

- RhodeCode will check for python version during installation

1.1.5 (**2011-03-17**)

======================

news

----

- basic windows support, by exchanging pybcrypt into sha256 for windows only

  highly inspired by idea of mantis406

fixes

-----

- fixed sorting by author in main page

- fixed crashes with diffs on binary files

- fixed #131 problem with boolean values for LDAP

- fixed #122 mysql problems thanks to striker69 

- fixed problem with errors on calling raw/raw_files/annotate functions 

  with unknown revisions

- fixed returned rawfiles attachment names with international character

- cleaned out docs, big thanks to Jason Harris

1.1.4 (**2011-02-19**)

======================

news

----

fixes

-----

- fixed formencode import problem on settings page, that caused server crash

  when that page was accessed as first after server start

- journal fixes

- fixed option to access repository just by entering http://server/<repo_name> 

1.1.3 (**2011-02-16**)

======================

news

----

- implemented #102 allowing the '.' character in username

- added option to access repository just by entering http://server/<repo_name>

- celery task ignores result for better performance

fixes

-----

- fixed ehlo command and non auth mail servers on smtp_lib. Thanks to 

  apollo13 and Johan Walles

- small fixes in journal

- fixed problems with getting setting for celery from .ini files

- registration, password reset and login boxes share the same title as main 

  application now

- fixed #113: to high permissions to fork repository

- fixed problem with '[' chars in commit messages in journal

- removed issue with space inside renamed repository after deletion

- db transaction fixes when filesystem repository creation failed

- fixed #106 relation issues on databases different than sqlite

- fixed static files paths links to use of url() method

1.1.2 (**2011-01-12**)

======================

news

----

fixes

-----

- fixes #98 protection against float division of percentage stats

- fixed graph bug

- forced webhelpers version since it was making troubles during installation 

1.1.1 (**2011-01-06**)

======================

news

----

- added force https option into ini files for easier https usage (no need to

  set server headers with this options)

- small css updates

fixes

-----

- fixed #96 redirect loop on files view on repositories without changesets

- fixed #97 unicode string passed into server header in special cases (mod_wsgi)

  and server crashed with errors

- fixed large tooltips problems on main page

- fixed #92 whoosh indexer is more error proof

1.1.0 (**2010-12-18**)

======================

news

----

- rewrite of internals for vcs >=0.1.10

- uses mercurial 1.7 with dotencode disabled for maintaining compatibility 

  with older clients

- anonymous access, authentication via ldap

- performance upgrade for cached repos list - each repository has it's own 

  cache that's invalidated when needed.

- performance upgrades on repositories with large amount of commits (20K+)

- main page quick filter for filtering repositories

- user dashboards with ability to follow chosen repositories actions

- sends email to admin on new user registration

- added cache/statistics reset options into repository settings

- more detailed action logger (based on hooks) with pushed changesets lists

  and options to disable those hooks from admin panel

- introduced new enhanced changelog for merges that shows more accurate results

- new improved and faster code stats (based on pygments lexers mapping tables, 

  showing up to 10 trending sources for each repository. Additionally stats

  can be disabled in repository settings.

- gui optimizations, fixed application width to 1024px

- added cut off (for large files/changesets) limit into config files

- whoosh, celeryd, upgrade moved to paster command

- other than sqlite database backends can be used

fixes

-----

- fixes #61 forked repo was showing only after cache expired

- fixes #76 no confirmation on user deletes

- fixes #66 Name field misspelled

- fixes #72 block user removal when he owns repositories

- fixes #69 added password confirmation fields

- fixes #87 RhodeCode crashes occasionally on updating repository owner

- fixes #82 broken annotations on files with more than 1 blank line at the end

- a lot of fixes and tweaks for file browser

- fixed detached session issues

- fixed when user had no repos he would see all repos listed in my account

- fixed ui() instance bug when global hgrc settings was loaded for server 

  instance and all hgrc options were merged with our db ui() object

- numerous small bugfixes

(special thanks for TkSoh for detailed feedback)

1.0.2 (**2010-11-12**)

======================

news

----

- tested under python2.7

- bumped sqlalchemy and celery versions

fixes

-----

- fixed #59 missing graph.js

- fixed repo_size crash when repository had broken symlinks

- fixed python2.5 crashes.

1.0.1 (**2010-11-10**)

======================

news

----

- small css updated

fixes

-----

- fixed #53 python2.5 incompatible enumerate calls

- fixed #52 disable mercurial extension for web

- fixed #51 deleting repositories don't delete it's dependent objects

1.0.0 (**2010-11-02**)

======================

- security bugfix simplehg wasn't checking for permissions on commands

  other than pull or push.

- fixed doubled messages after push or pull in admin journal

- templating and css corrections, fixed repo switcher on chrome, updated titles

- admin menu accessible from options menu on repository view

- permissions cached queries

1.0.0rc4  (**2010-10-12**)

==========================

- fixed python2.5 missing simplejson imports (thanks to Jens Bäckman)

- removed cache_manager settings from sqlalchemy meta

- added sqlalchemy cache settings to ini files

- validated password length and added second try of failure on paster setup-app

- fixed setup database destroy prompt even when there was no db

1.0.0rc3 (**2010-10-11**)

=========================

- fixed i18n during installation.

1.0.0rc2 (**2010-10-11**)

=========================

- Disabled dirsize in file browser, it's causing nasty bug when dir renames 

  occure. After vcs is fixed it'll be put back again.

- templating/css rewrites, optimized css.

# -*- coding: utf-8 -*-

"""

    rhodecode.lib.middleware.simplegit

    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    SimpleGit middleware for handling git protocol request (push/clone etc.)

    It's implemented with basic auth function

    :created_on: Apr 28, 2010

    :author: marcink

    :copyright: (C) 2009-2010 Marcin Kuzminski <marcin@python-works.com>

    :license: GPLv3, see COPYING for more details.

"""

# This program is free software: you can redistribute it and/or modify

# it under the terms of the GNU General Public License as published by

# the Free Software Foundation, either version 3 of the License, or

# (at your option) any later version.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program.  If not, see <http://www.gnu.org/licenses/>.

import os

import logging

import traceback

from dulwich import server as dulserver

class SimpleGitUploadPackHandler(dulserver.UploadPackHandler):

    def handle(self):

        write = lambda x: self.proto.write_sideband(1, x)

        graph_walker = dulserver.ProtocolGraphWalker(self,

                                                     self.repo.object_store,

                                                     self.repo.get_peeled)

        objects_iter = self.repo.fetch_objects(

          graph_walker.determine_wants, graph_walker, self.progress,

          get_tagged=self.get_tagged)

        # Do they want any objects?

        if len(objects_iter) == 0:

            return

        self.progress("counting objects: %d, done.\n" % len(objects_iter))

        dulserver.write_pack_data(dulserver.ProtocolFile(None, write),

                                  objects_iter, len(objects_iter))

        messages = []

        messages.append('thank you for using rhodecode')

        for msg in messages:

            self.progress(msg + "\n")

        # we are done

        self.proto.write("0000")

dulserver.DEFAULT_HANDLERS = {

  'git-upload-pack': SimpleGitUploadPackHandler,

  'git-receive-pack': dulserver.ReceivePackHandler,

}

from dulwich.repo import Repo

from dulwich.web import HTTPGitApplication

from paste.auth.basic import AuthBasicAuthenticator

from paste.httpheaders import REMOTE_USER, AUTH_TYPE

from rhodecode.lib.auth import authfunc, HasPermissionAnyMiddleware

from rhodecode.lib.utils import invalidate_cache, check_repo_fast

from rhodecode.model.user import UserModel

from webob.exc import HTTPNotFound, HTTPForbidden, HTTPInternalServerError

log = logging.getLogger(__name__)

def is_git(environ):

    """Returns True if request's target is git server.

    ``HTTP_USER_AGENT`` would then have git client version given.

    :param environ:

    """

    http_user_agent = environ.get('HTTP_USER_AGENT')

    if http_user_agent and http_user_agent.startswith('git'):

        return True

    return False

class SimpleGit(object):

    def __init__(self, application, config):

        self.application = application

        self.config = config

        #authenticate this git request using

        self.authenticate = AuthBasicAuthenticator('', authfunc)

        self.ipaddr = '0.0.0.0'

        self.username = None

        self.action = None

    def __call__(self, environ, start_response):

        if not is_git(environ):

            return self.application(environ, start_response)

        proxy_key = 'HTTP_X_REAL_IP'

        def_key = 'REMOTE_ADDR'

        self.ipaddr = environ.get(proxy_key, environ.get(def_key, '0.0.0.0'))

        # skip passing error to error controller

        environ['pylons.status_code_redirect'] = True

        #======================================================================

        # GET ACTION PULL or PUSH

        #======================================================================

        self.action = self.__get_action(environ)

        try:

            #==================================================================

            # GET REPOSITORY NAME

            #==================================================================

            self.repo_name = self.__get_repository(environ)

        except:

            return HTTPInternalServerError()(environ, start_response)

        #======================================================================

        # CHECK ANONYMOUS PERMISSION

        #======================================================================

        if self.action in ['pull', 'push']:

            anonymous_user = self.__get_user('default')

            self.username = anonymous_user.username

            anonymous_perm = self.__check_permission(self.action,

                                                     anonymous_user,

                                                     self.repo_name)

            if anonymous_perm is not True or anonymous_user.active is False:

                if anonymous_perm is not True:

                    log.debug('Not enough credentials to access this '

                              'repository as anonymous user')

                if anonymous_user.active is False:

                    log.debug('Anonymous access is disabled, running '

                              'authentication')

                #==============================================================

                # DEFAULT PERM FAILED OR ANONYMOUS ACCESS IS DISABLED SO WE

                # NEED TO AUTHENTICATE AND ASK FOR AUTH USER PERMISSIONS

                #==============================================================

                if not REMOTE_USER(environ):

                    result = self.authenticate(environ)

                    if isinstance(result, str):

                        AUTH_TYPE.update(environ, 'basic')

                        REMOTE_USER.update(environ, result)

                    else:

                        return result.wsgi_application(environ, start_response)

                #==============================================================

                # CHECK PERMISSIONS FOR THIS REQUEST USING GIVEN USERNAME FROM

                # BASIC AUTH

                #==============================================================

                if self.action in ['pull', 'push']:

                    username = REMOTE_USER(environ)

                    try:

                        user = self.__get_user(username)

                        self.username = user.username

                    except:

                        log.error(traceback.format_exc())

                        return HTTPInternalServerError()(environ,

                                                         start_response)

                    #check permissions for this repository

                    perm = self.__check_permission(self.action, user,

                                                   self.repo_name)

                    if perm is not True:

                        return HTTPForbidden()(environ, start_response)

        self.extras = {'ip': self.ipaddr,

                       'username': self.username,

                       'action': self.action,

                       'repository': self.repo_name}

        #===================================================================

        # GIT REQUEST HANDLING

        #===================================================================

        self.basepath = self.config['base_path']

        self.repo_path = os.path.join(self.basepath, self.repo_name)

        #quick check if that dir exists...

        if check_repo_fast(self.repo_name, self.basepath):

            return HTTPNotFound()(environ, start_response)

        try:

            app = self.__make_app()

        except:

            log.error(traceback.format_exc())

            return HTTPInternalServerError()(environ, start_response)

        #invalidate cache on push

        if self.action == 'push':

            self.__invalidate_cache(self.repo_name)

        return app(environ, start_response)

    def __make_app(self):

        _d = {'/' + self.repo_name: Repo(self.repo_path)}

        backend = dulserver.DictBackend(_d)

        gitserve = HTTPGitApplication(backend)

        return gitserve

    def __check_permission(self, action, user, repo_name):

        """Checks permissions using action (push/pull) user and repository

        name

        :param action: push or pull action

        :param user: user instance

        :param repo_name: repository name

        """

        if action == 'push':

            if not HasPermissionAnyMiddleware('repository.write',

                                              'repository.admin')(user,

                                                                  repo_name):

                return False

        else:

            #any other action need at least read permission

            if not HasPermissionAnyMiddleware('repository.read',

                                              'repository.write',

                                              'repository.admin')(user,

                                                                  repo_name):

                return False

        return True

    def __get_repository(self, environ):

        """Get's repository name out of PATH_INFO header

        :param environ: environ where PATH_INFO is stored

        """

        try:

            repo_name = '/'.join(environ['PATH_INFO'].split('/')[1:])

            if repo_name.endswith('/'):

                repo_name = repo_name.rstrip('/')

        except:

            log.error(traceback.format_exc())

            raise

        repo_name = repo_name.split('/')[0]

        return repo_name

    def __get_user(self, username):

        return UserModel().get_by_username(username, cache=True)

    def __get_action(self, environ):

        """Maps git request commands into a pull or push command.

        :param environ:

        """

        service = environ['QUERY_STRING'].split('=')

        if len(service) > 1:

            service_cmd = service[1]

            mapping = {'git-receive-pack': 'push',

                       'git-upload-pack': 'pull',

                       }

            return mapping.get(service_cmd,

                               service_cmd if service_cmd else 'other')

        else:

            return 'other'

    def __invalidate_cache(self, repo_name):

        """we know that some change was made to repositories and we should

        invalidate the cache to see the changes right away but only for

        push requests"""

        invalidate_cache('get_repo_cached_%s' % repo_name)

# -*- coding: utf-8 -*-

"""

    rhodecode.lib.middleware.simplehg

    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    SimpleHG middleware for handling mercurial protocol request

    (push/clone etc.). It's implemented with basic auth function

    :created_on: Apr 28, 2010

    :author: marcink

    :copyright: (C) 2009-2010 Marcin Kuzminski <marcin@python-works.com>

    :license: GPLv3, see COPYING for more details.

"""

# This program is free software: you can redistribute it and/or modify

# it under the terms of the GNU General Public License as published by

# the Free Software Foundation, either version 3 of the License, or

# (at your option) any later version.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program.  If not, see <http://www.gnu.org/licenses/>.

import os

import logging

import traceback

from mercurial.error import RepoError

from mercurial.hgweb import hgweb

from mercurial.hgweb.request import wsgiapplication

from paste.auth.basic import AuthBasicAuthenticator

from paste.httpheaders import REMOTE_USER, AUTH_TYPE

from rhodecode.lib.auth import authfunc, HasPermissionAnyMiddleware

from rhodecode.lib.utils import make_ui, invalidate_cache, \

    check_repo_fast, ui_sections

from rhodecode.model.user import UserModel

from webob.exc import HTTPNotFound, HTTPForbidden, HTTPInternalServerError

log = logging.getLogger(__name__)

def is_mercurial(environ):

    """Returns True if request's target is mercurial server - header

    ``HTTP_ACCEPT`` of such request would start with ``application/mercurial``.

    """

    http_accept = environ.get('HTTP_ACCEPT')

    if http_accept and http_accept.startswith('application/mercurial'):

        return True

    return False

class SimpleHg(object):

    def __init__(self, application, config):

        self.application = application

        self.config = config

        #authenticate this mercurial request using authfunc

        self.authenticate = AuthBasicAuthenticator('', authfunc)

        self.ipaddr = '0.0.0.0'

        self.repo_name = None

        self.username = None

        self.action = None

    def __call__(self, environ, start_response):

        if not is_mercurial(environ):

            return self.application(environ, start_response)

        proxy_key = 'HTTP_X_REAL_IP'

        def_key = 'REMOTE_ADDR'

        self.ipaddr = environ.get(proxy_key, environ.get(def_key, '0.0.0.0'))

        # skip passing error to error controller

        environ['pylons.status_code_redirect'] = True

        #======================================================================

        # GET ACTION PULL or PUSH

        #======================================================================

        self.action = self.__get_action(environ)

        try:

            #==================================================================

            # GET REPOSITORY NAME

            #==================================================================

            self.repo_name = self.__get_repository(environ)

        except:

            return HTTPInternalServerError()(environ, start_response)

        #======================================================================

        # CHECK ANONYMOUS PERMISSION

        #======================================================================

        if self.action in ['pull', 'push']:

            anonymous_user = self.__get_user('default')

            self.username = anonymous_user.username

            anonymous_perm = self.__check_permission(self.action,

                                                     anonymous_user,

                                                     self.repo_name)

            if anonymous_perm is not True or anonymous_user.active is False:

                if anonymous_perm is not True:

                    log.debug('Not enough credentials to access this '

                              'repository as anonymous user')

                if anonymous_user.active is False:

                    log.debug('Anonymous access is disabled, running '

                              'authentication')

                #==============================================================

                # DEFAULT PERM FAILED OR ANONYMOUS ACCESS IS DISABLED SO WE

                # NEED TO AUTHENTICATE AND ASK FOR AUTH USER PERMISSIONS

                #==============================================================

                if not REMOTE_USER(environ):

                    result = self.authenticate(environ)

                    if isinstance(result, str):

                        AUTH_TYPE.update(environ, 'basic')

                        REMOTE_USER.update(environ, result)

                    else:

                        return result.wsgi_application(environ, start_response)

                #==============================================================

                # CHECK PERMISSIONS FOR THIS REQUEST USING GIVEN USERNAME FROM

                # BASIC AUTH

                #==============================================================

                if self.action in ['pull', 'push']:

                    username = REMOTE_USER(environ)

                    try:

                        user = self.__get_user(username)

                        self.username = user.username

                    except:

                        log.error(traceback.format_exc())

                        return HTTPInternalServerError()(environ,

                                                         start_response)

                    #check permissions for this repository

                    perm = self.__check_permission(self.action, user,

                                                   self.repo_name)

                    if perm is not True:

                        return HTTPForbidden()(environ, start_response)

        self.extras = {'ip': self.ipaddr,

                       'username': self.username,

                       'action': self.action,

                       'repository': self.repo_name}

        #======================================================================

        # MERCURIAL REQUEST HANDLING

        #======================================================================

        environ['PATH_INFO'] = '/'  # since we wrap into hgweb, reset the path

        self.baseui = make_ui('db')

        self.basepath = self.config['base_path']

        self.repo_path = os.path.join(self.basepath, self.repo_name)

        #quick check if that dir exists...

        if check_repo_fast(self.repo_name, self.basepath):

            return HTTPNotFound()(environ, start_response)

        try:

            app = wsgiapplication(self.__make_app)

        except RepoError, e:

            if str(e).find('not found') != -1:

                return HTTPNotFound()(environ, start_response)

        except Exception:

            log.error(traceback.format_exc())

            return HTTPInternalServerError()(environ, start_response)

        #invalidate cache on push

        if self.action == 'push':

            self.__invalidate_cache(self.repo_name)

        return app(environ, start_response)

    def __make_app(self):

        """

        Make an wsgi application using hgweb, and inject generated baseui

        instance, additionally inject some extras into ui object

        """

        self.__inject_extras(self.baseui, self.extras)

        return hgweb(str(self.repo_path), baseui=self.baseui)

    def __check_permission(self, action, user, repo_name):

        """

        Checks permissions using action (push/pull) user and repository

        name

        :param action: push or pull action

        :param user: user instance

        :param repo_name: repository name

        """

        if action == 'push':

            if not HasPermissionAnyMiddleware('repository.write',

                                              'repository.admin')(user,

                                                                  repo_name):

                return False

        else:

            #any other action need at least read permission

            if not HasPermissionAnyMiddleware('repository.read',

                                              'repository.write',

                                              'repository.admin')(user,

                                                                  repo_name):

                return False

        return True

    def __get_repository(self, environ):

        """

        Get's repository name out of PATH_INFO header

        :param environ: environ where PATH_INFO is stored

        """

        try:

            repo_name = '/'.join(environ['PATH_INFO'].split('/')[1:])

            if repo_name.endswith('/'):

                repo_name = repo_name.rstrip('/')

        except:

            log.error(traceback.format_exc())

            raise

        return repo_name

    def __get_user(self, username):

        return UserModel().get_by_username(username, cache=True)

    def __get_action(self, environ):

        """

        Maps mercurial request commands into a clone,pull or push command.

        This should always return a valid command string

        :param environ:

        """

        mapping = {'changegroup': 'pull',

                   'changegroupsubset': 'pull',

                   'stream_out': 'pull',

                   'listkeys': 'pull',

                   'unbundle': 'push',

                   'pushkey': 'push', }

        for qry in environ['QUERY_STRING'].split('&'):

            if qry.startswith('cmd'):

                cmd = qry.split('=')[-1]

                if cmd in mapping:

                    return mapping[cmd]

                else:

                    return 'pull'

    def __invalidate_cache(self, repo_name):

        """we know that some change was made to repositories and we should

        invalidate the cache to see the changes right away but only for

        push requests"""

        invalidate_cache('get_repo_cached_%s' % repo_name)

    def __inject_extras(self, baseui, extras={}):

        hgrc = os.path.join(self.repo_path, '.hg', 'hgrc')

        #inject some additional parameters that will be available in ui

        #for hooks

        for k, v in extras.items():

            baseui.setconfig('rhodecode_extras', k, v)

        repoui = make_ui('file', hgrc, False)

        if repoui: