.. _changelog:

Changelog

=========

1.0.0 (**2010-11-02**)

----------------------

- security bugfix simplehg wasn't checking for permissions on commands

  other than pull or push.

- fixed doubled messages after push or pull in admin journal

- templating and css corrections, fixed repo switcher on chrome, updated titles

- admin menu accessible from options menu on repository view

- permissions cached queries

1.0.0rc4  (**2010-10-12**)

--------------------------

- fixed python2.5 missing simplejson imports (thanks to Jens Bäckman)

- removed cache_manager settings from sqlalchemy meta

- added sqlalchemy cache settings to ini files

- validated password length and added second try of failure on paster setup-app

- fixed setup database destroy prompt even when there was no db

1.0.0rc3 (**2010-10-11**)

-------------------------

- fixed i18n during installation.

1.0.0rc2 (**2010-10-11**)

-------------------------

- Disabled dirsize in file browser, it's causing nasty bug when dir renames 

  occure. After vcs is fixed it'll be put back again.

- templating/css rewrites, optimized css.

.. _installation:

Installation

============

``RhodeCode`` is written entirely in Python, but in order to use it's full

potential there are some third-party requirements. When RhodeCode is used 

recommended one is rabbitmq_ to make the async tasks work.

Of course RhodeCode works in sync mode also, then You don't have to install

any third party apps. Celery_ will give You large speed improvement when using

will work just fine without celery running.

After You decide to Run it with celery make sure You run celeryd and

message broker together with the application.   

Requirements for Celery

-----------------------

**Message Broker** 

- preferred is `RabbitMq <http://www.rabbitmq.com/>`_

- possible other is `Redis <http://code.google.com/p/redis/>`_

For installation instructions You can visit: 

http://ask.github.com/celery/getting-started/index.html

It's very nice tutorial how to start celery_ with rabbitmq_

Install from Cheese Shop

------------------------

Easiest way to install ``rhodecode`` is to run::

Or::

 pip install rhodecode

If you prefer to install manually simply grab latest release from

http://pypi.python.org/pypi/rhodecode, decompres archive and run::

Step by step installation example

---------------------------------

::

 virtualenv --no-site-packages /var/www/rhodecode-venv

- this will install new virtualenv_ into `/var/www/rhodecode-venv`. 

- Activate the virtualenv_ by running 

::

  source activate /var/www/rhodecode-venv/bin/activate

- Make a folder for rhodecode somewhere on the filesystem for example 

::

  mkdir /var/www/rhodecode

- Run this command to install rhodecode

::

  easy_install rhodecode 

- this will install rhodecode together with pylons

  and all other required python libraries

You can now proceed to :ref:`setup`

.. _virtualenv: http://pypi.python.org/pypi/virtualenv  

.. _python: http://www.python.org/

.. _mercurial: http://mercurial.selenic.com/

.. _celery: http://celeryproject.org/

.. _rabbitmq: http://www.rabbitmq.com/

.. _setup:

Setup

=====

Setting up the application

--------------------------

::

 paster make-config RhodeCode production.ini

- This will create `production.ini` config inside the directory

  this config contain various settings for rhodecode, e.g port, email settings

  static files, cache and logging.

::

 paster setup-app production.ini

- This command will create all needed tables and an admin account. 

  When asked for a path You can either use a new location of one with already 

  existing ones. RhodeCode will simply add all new found repositories to 

  it's database. Also make sure You specify correct path to repositories.

- Remember that the given path for mercurial_ repositories must be write 

  accessible for the application. It's very important since RhodeCode web interface

  will work even without such an access but, when trying to do a push it'll 

  eventually fail with permission denied errors. 

- Run 

::

 paster serve production.ini

- This command runs the rhodecode server the app should be available at the 

  127.0.0.1:5000. This ip and port is configurable via the production.ini 

  file  created in previous step

- Use admin account you created to login.

- Default permissions on each repository is read, and owner is admin. So 

  remember to update these if needed.

Setting up Whoosh

-----------------

- For full text search You can either put crontab entry for

::

 python /var/www/rhodecode/<rhodecode_installation_path>/lib/indexers/daemon.py incremental <put_here_path_to_repos>

When using incremental mode whoosh will check last modification date of each file

and add it to reindex if newer file is available. Also indexing daemon checks

for removed files and removes them from index. Sometime You might want to rebuild

and in standalone daemon You can pass `full` instead on incremental to build

Nginx virtual host example

--------------------------

Sample config for nginx::

 server {

    listen          80;

    server_name     hg.myserver.com;

    access_log      /var/log/nginx/rhodecode.access.log;

    error_log       /var/log/nginx/rhodecode.error.log;

    location / {

            root /var/www/rhodecode/rhodecode/public/;

            if (!-f $request_filename){

                proxy_pass      http://127.0.0.1:5000;

            }

            #this is important for https !!!

            proxy_set_header X-Url-Scheme $scheme;

            include         /etc/nginx/proxy.conf;  

    }

 }  

Here's the proxy.conf. It's tuned so it'll not timeout on long

pushes and also on large pushes::

    proxy_redirect              off;

    proxy_set_header            Host $host;

    proxy_set_header            X-Host $http_host;

    proxy_set_header            X-Real-IP $remote_addr;

    proxy_set_header            X-Forwarded-For $proxy_add_x_forwarded_for;

    proxy_set_header            Proxy-host $proxy_host;

    client_max_body_size        400m;

    client_body_buffer_size     128k;

    proxy_buffering             off;

    proxy_connect_timeout       3600;

    proxy_send_timeout          3600;

    proxy_read_timeout          3600;

    proxy_buffer_size           8k;

    proxy_buffers               8 32k;

    proxy_busy_buffers_size     64k;

    proxy_temp_file_write_size  64k;

Also when using root path with nginx You might set the static files to false

in production.ini file::

  [app:main]

    use = egg:rhodecode

    full_stack = true

    static_files = false

    lang=en

    cache_dir = %(here)s/data

To not have the statics served by the application. And improve speed.

Other configuration files

-------------------------

Some extra configuration files and examples can be found here:

http://hg.python-works.com/rhodecode/files/tip/init.d

and also an celeryconfig file can be use from here:

http://hg.python-works.com/rhodecode/files/tip/celeryconfig.py

.. _virtualenv: http://pypi.python.org/pypi/virtualenv

.. _python: http://www.python.org/

.. _mercurial: http://mercurial.selenic.com/

.. _celery: http://celeryproject.org/

.. _rabbitmq: http://www.rabbitmq.com/

#!/usr/bin/env python

# encoding: utf-8

# RhodeCode, a web based repository management based on pylons

# Copyright (C) 2009-2010 Marcin Kuzminski <marcin@python-works.com>

# 

# This program is free software; you can redistribute it and/or

# modify it under the terms of the GNU General Public License

# as published by the Free Software Foundation; version 2

# of the License or (at your opinion) any later version of the license.

# 

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

# 

# You should have received a copy of the GNU General Public License

# along with this program; if not, write to the Free Software

# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,

# MA  02110-1301, USA.

"""

Created on April 9, 2010

RhodeCode, a web based repository management based on pylons

versioning implementation: http://semver.org/

@author: marcink

"""

__version__ = '.'.join((str(each) for each in VERSION[:4]))

def get_version():

    """

    Returns shorter version (digit parts only) as string.

    """

    return '.'.join((str(each) for each in VERSION[:3]))

"""Helper functions

Consists of functions to typically be used within templates, but also

available to Controllers. This module is available to both as 'h'.

"""

from pygments.formatters import HtmlFormatter

from pygments import highlight as code_highlight

from pylons import url, app_globals as g

from pylons.i18n.translation import _, ungettext

from vcs.utils.annotate import annotate_highlight

from webhelpers.html import literal, HTML, escape

from webhelpers.html.tools import *

from webhelpers.html.builder import make_tag

from webhelpers.html.tags import auto_discovery_link, checkbox, css_classes, \

    end_form, file, form, hidden, image, javascript_link, link_to, link_to_if, \

    link_to_unless, ol, required_legend, select, stylesheet_link, submit, text, \

    password, textarea, title, ul, xml_declaration, radio

from webhelpers.html.tools import auto_link, button_to, highlight, js_obfuscate, \

    mail_to, strip_links, strip_tags, tag_re

from webhelpers.number import format_byte_size, format_bit_size

from webhelpers.pylonslib import Flash as _Flash

from webhelpers.pylonslib.secure_form import secure_form

from webhelpers.text import chop_at, collapse, convert_accented_entities, \

    convert_misc_entities, lchop, plural, rchop, remove_formatting, \

    replace_whitespace, urlify, truncate, wrap_paragraphs

#Custom helpers here :)

class _Link(object):

    '''

    Make a url based on label and url with help of url_for

    :param label:name of link    if not defined url is used

    :param url: the url for link

    '''

    def __call__(self, label='', *url_, **urlargs):

        if label is None or '':

            label = url

        link_fn = link_to(label, url(*url_, **urlargs))

        return link_fn

link = _Link()

class _GetError(object):

    def __call__(self, field_name, form_errors):

        tmpl = """<span class="error_msg">%s</span>"""

        if form_errors and form_errors.has_key(field_name):

            return literal(tmpl % form_errors.get(field_name))

get_error = _GetError()

def recursive_replace(str, replace=' '):

    """

    Recursive replace of given sign to just one instance

    :param str: given string

    :param replace:char to find and replace multiple instances

    Examples::

    >>> recursive_replace("Mighty---Mighty-Bo--sstones",'-')

    'Mighty-Mighty-Bo-sstones'

    """

    if str.find(replace * 2) == -1:

        return str

    else:

        str = str.replace(replace * 2, replace)

class _ToolTip(object):

    def __call__(self, tooltip_title, trim_at=50):

        """

        Special function just to wrap our text into nice formatted autowrapped

        text

        :param tooltip_title:

        """

        return wrap_paragraphs(escape(tooltip_title), trim_at)\

                       .replace('\n', '<br/>')

    def activate(self):

        """

        Adds tooltip mechanism to the given Html all tooltips have to have 

        set class tooltip and set attribute tooltip_title.

        Then a tooltip will be generated based on that

        All with yui js tooltip

        """

        js = '''

        YAHOO.util.Event.onDOMReady(function(){

            function toolTipsId(){

                var ids = [];

                var tts = YAHOO.util.Dom.getElementsByClassName('tooltip');

                for (var i = 0; i < tts.length; i++) {

                    //if element doesn not have and id autgenerate one for tooltip

                    if (!tts[i].id){

                        tts[i].id='tt'+i*100;

                    }

                    ids.push(tts[i].id);

                }

                return ids        

            };

            var myToolTips = new YAHOO.widget.Tooltip("tooltip", { 

                context: toolTipsId(),

                monitorresize:false,

                xyoffset :[0,0],

                autodismissdelay:300000,

                hidedelay:5,

                showdelay:20,

            });

            //Mouse Over event disabled for new repositories since they dont

            //have last commit message

            myToolTips.contextMouseOverEvent.subscribe(

                function(type, args) {

                    var context = args[0];

                    var txt = context.getAttribute('tooltip_title');

                    if(txt){                                       

                        return true;

                    }

                    else{

                        return false;

                    }

                });

            // Set the text for the tooltip just before we display it. Lazy method

            myToolTips.contextTriggerEvent.subscribe( 

                 function(type, args) { 

                        var context = args[0]; 

                        var txt = context.getAttribute('tooltip_title');

                        this.cfg.setProperty("text", txt);

                        // positioning of tooltip

                        var tt_w = this.element.clientWidth;

                        var tt_h = this.element.clientHeight;

                        var context_w = context.offsetWidth;

                        var context_h = context.offsetHeight;

                        var pos_x = YAHOO.util.Dom.getX(context);

                        var pos_y = YAHOO.util.Dom.getY(context);

                        var display_strategy = 'top';

                        var xy_pos = [0,0];

                        switch (display_strategy){

                            case 'top':

                                var cur_x = (pos_x+context_w/2)-(tt_w/2);

                                var cur_y = pos_y-tt_h-4;

                                xy_pos = [cur_x,cur_y];                                

                                break;

                            case 'bottom':

                                var cur_x = (pos_x+context_w/2)-(tt_w/2);

                                var cur_y = pos_y+context_h+4;

                                xy_pos = [cur_x,cur_y];                                

                                break;

                            case 'left':

                                var cur_x = (pos_x-tt_w-4);

                                var cur_y = pos_y-((tt_h/2)-context_h/2);

                                xy_pos = [cur_x,cur_y];                                

                                break;

                            case 'right':

                                var cur_x = (pos_x+context_w+4);

                                var cur_y = pos_y-((tt_h/2)-context_h/2);

                                xy_pos = [cur_x,cur_y];                                

                                break;

                             default:

                                var cur_x = (pos_x+context_w/2)-(tt_w/2);

                                var cur_y = pos_y-tt_h-4;

                                xy_pos = [cur_x,cur_y];                                

                                break;                             

                        }

                        this.cfg.setProperty("xy",xy_pos);

                  });

            //Mouse out 

            myToolTips.contextMouseOutEvent.subscribe(

                function(type, args) {

                    var context = args[0];

                });

        });

        return literal(js)

tooltip = _ToolTip()

class _FilesBreadCrumbs(object):

    def __call__(self, repo_name, rev, paths):

        url_l = [link_to(repo_name, url('files_home',

                                        repo_name=repo_name,

                                        revision=rev, f_path=''))]

        paths_l = paths.split('/')

            if p != '':

                url_l.append(link_to(p, url('files_home',

                                            repo_name=repo_name,

                                            revision=rev,

        return literal('/'.join(url_l))

files_breadcrumbs = _FilesBreadCrumbs()

class CodeHtmlFormatter(HtmlFormatter):

    def wrap(self, source, outfile):

        return self._wrap_div(self._wrap_pre(self._wrap_code(source)))

    def _wrap_code(self, source):

            i, t = it

            yield i, t

def pygmentize(filenode, **kwargs):

    """

    pygmentize function using pygments

    :param filenode:

    """

    return literal(code_highlight(filenode.content,

                                  filenode.lexer, CodeHtmlFormatter(**kwargs)))

def pygmentize_annotation(filenode, **kwargs):

    """

    pygmentize function for annotation

    :param filenode:

    """

    color_dict = {}

    def gen_color():

        """generator for getting 10k of evenly distibuted colors using hsv color

        and golden ratio.

        import colorsys

        n = 10000

        golden_ratio = 0.618033988749895

        h = 0.22717784590367374

        #generate 10k nice web friendly colors in the same order

        for c in xrange(n):

            h += golden_ratio

            h %= 1

            HSV_tuple = [h, 0.95, 0.95]

            RGB_tuple = colorsys.hsv_to_rgb(*HSV_tuple)

    cgenerator = gen_color()

    def get_color_string(cs):

        if color_dict.has_key(cs):

            col = color_dict[cs]

        else:

            col = color_dict[cs] = cgenerator.next()

        return "color: rgb(%s)! important;" % (', '.join(col))

    def url_func(changeset):

        tooltip_html = "<div style='font-size:0.8em'><b>Author:</b>" + \

        tooltip_html = tooltip_html % (changeset.author,

                                               changeset.date,

                                               tooltip(changeset.message))

        lnk_format = 'r%-5s:%s' % (changeset.revision,

                                 changeset.short_id)

        uri = link_to(

                lnk_format,

                url('changeset_home', repo_name=changeset.repository.name,

                    revision=changeset.short_id),

                style=get_color_string(changeset.short_id),

                class_='tooltip',

                tooltip_title=tooltip_html

              )

        uri += '\n'

    return literal(annotate_highlight(filenode, url_func, **kwargs))

def repo_name_slug(value):

    """Return slug of name of repository

    This function is called on each creation/modification

    of repository to prevent bad names in repo

    """

    slug = remove_formatting(value)

    slug = strip_tags(slug)

    for c in """=[]\;'"<>,/~!@#$%^&*()+{}|: """:

        slug = slug.replace(c, '-')

    slug = recursive_replace(slug, '-')

    slug = collapse(slug, '-')

    return slug

def get_changeset_safe(repo, rev):

    from vcs.backends.base import BaseRepository

    from vcs.exceptions import RepositoryError

    if not isinstance(repo, BaseRepository):

        raise Exception('You must pass an Repository '

                        'object as first argument got %s', type(repo))

    try:

        cs = repo.get_changeset(rev)

    except RepositoryError:

        from rhodecode.lib.utils import EmptyChangeset

        cs = EmptyChangeset()

    return cs

flash = _Flash()

#===============================================================================

# MERCURIAL FILTERS available via h.

#===============================================================================

from mercurial import util

from mercurial.templatefilters import age as _age, person as _person

age = lambda  x:_age(x)

capitalize = lambda x: x.capitalize()

date = lambda x: util.datestr(x)

email = util.email

email_or_none = lambda x: util.email(x) if util.email(x) != x else None

person = lambda x: _person(x)

hgdate = lambda  x: "%d %d" % x

isodate = lambda  x: util.datestr(x, '%Y-%m-%d %H:%M %1%2')

isodatesec = lambda  x: util.datestr(x, '%Y-%m-%d %H:%M:%S %1%2')

localdate = lambda  x: (x[0], util.makedate()[1])

rfc822date = lambda  x: util.datestr(x, "%a, %d %b %Y %H:%M:%S %1%2")

rfc822date_notz = lambda  x: util.datestr(x, "%a, %d %b %Y %H:%M:%S")

rfc3339date = lambda  x: util.datestr(x, "%Y-%m-%dT%H:%M:%S%1:%2")

time_ago = lambda x: util.datestr(_age(x), "%a, %d %b %Y %H:%M:%S %1%2")

#===============================================================================

# PERMS

#===============================================================================

from rhodecode.lib.auth import HasPermissionAny, HasPermissionAll, \

HasRepoPermissionAny, HasRepoPermissionAll

#===============================================================================

# GRAVATAR URL

#===============================================================================

import hashlib

import urllib

from pylons import request

def gravatar_url(email_address, size=30):

    ssl_enabled = 'https' == request.environ.get('HTTP_X_URL_SCHEME')

    default = 'identicon'

    baseurl_nossl = "http://www.gravatar.com/avatar/"

    baseurl_ssl = "https://secure.gravatar.com/avatar/"

    baseurl = baseurl_ssl if ssl_enabled else baseurl_nossl

    # construct the url

    gravatar_url = baseurl + hashlib.md5(email_address.lower()).hexdigest() + "?"

    gravatar_url += urllib.urlencode({'d':default, 's':str(size)})

    return gravatar_url

def safe_unicode(str):

    """safe unicode function. In case of UnicodeDecode error we try to return

    unicode with errors replace, if this failes we return unicode with 

    string_escape decoding """

    try:

        u_str = unicode(str)

    except UnicodeDecodeError:

        try:

            u_str = unicode(str, 'utf-8', 'replace')

        except UnicodeDecodeError:

            #incase we have a decode error just represent as byte string

            u_str = unicode(str(str).encode('string_escape'))

    return u_str

def ask_ok(prompt, retries=4, complaint='Yes or no, please!'):

    while True:

        ok = raw_input(prompt)

        if ok in ('y', 'ye', 'yes'): return True

        if ok in ('n', 'no', 'nop', 'nope'): return False

        retries = retries - 1

        if retries < 0: raise IOError

        print complaint

@cache_region('super_short_term', 'cached_hg_ui')

def get_hg_ui_cached():

    try:

        sa = meta.Session

        ret = sa.query(RhodeCodeUi).all()

    finally:

        meta.Session.remove()

    return ret

def get_hg_settings():

    try:

        sa = meta.Session

        ret = sa.query(RhodeCodeSettings).all()

    finally:

        meta.Session.remove()

    if not ret:

        raise Exception('Could not get application settings !')

    settings = {}

    for each in ret:

        settings['rhodecode_' + each.app_settings_name] = each.app_settings_value

    return settings

def get_hg_ui_settings():

    try:

        sa = meta.Session

        ret = sa.query(RhodeCodeUi).all()

    finally:

        meta.Session.remove()

    if not ret:

        raise Exception('Could not get application ui settings !')

    settings = {}

    for each in ret:

        k = each.ui_key

        v = each.ui_value

        if k == '/':

            k = 'root_path'

        if k.find('.') != -1:

            k = k.replace('.', '_')

        if each.ui_section == 'hooks':

            v = each.ui_active

        settings[each.ui_section + '_' + k] = v

    return settings

#propagated from mercurial documentation

ui_sections = ['alias', 'auth',

                'decode/encode', 'defaults',

                'diff', 'email',

                'extensions', 'format',

                'merge-patterns', 'merge-tools',

                'hooks', 'http_proxy',

                'smtp', 'patch',

                'paths', 'profiling',

                'server', 'trusted',

                'ui', 'web', ]

def make_ui(read_from='file', path=None, checkpaths=True):

    """

    A function that will read python rc files or database

    and make an mercurial ui object from read options

    :param path: path to mercurial config file

    :param checkpaths: check the path

    :param read_from: read from 'file' or 'db'

    """

    baseui = ui.ui()

    if read_from == 'file':

        if not os.path.isfile(path):

            log.warning('Unable to read config file %s' % path)

            return False

        log.debug('reading hgrc from %s', path)

        cfg = config.config()

        cfg.read(path)

        for section in ui_sections:

            for k, v in cfg.items(section):

                baseui.setconfig(section, k, v)

                log.debug('settings ui from file[%s]%s:%s', section, k, v)

        if checkpaths:check_repo_dir(cfg.items('paths'))

    elif read_from == 'db':