.. _changelog:

Changelog

=========

1.2.0 (**2011-XX-XX**)

======================

:status: in-progress

:branch: beta

news

----

- implemented #89 Can setup google analytics code from settings menu

- implemented #91 added nicer looking archive urls with more download options

  like tags, branches

- implemented #44 into file browsing, and added follow branch option

- implemented #102 allowing '.' in username

- implemented #84 downloads can be enabled/disabled for each repository 

- anonymous repository can be cloned without having to pass default:default

  into clone url

- fixed #90 whoosh indexer can index chooses repositories passed in command 

  line

- extended journal with day aggregates and paging

- implemented #107 customizable code highlights on code sources

- implemented #93 customizable changelog on combined revision ranges - 

  equivalent of githubs compare view 

- implemented #108 extended and more powerfull LDAP configuration

fixes

-----

- fixed file browser bug, when switching into given form revision the url was 

  not changing

- fixed propagation to error controller on simplehg and simplegit middlewares

- fixed error when trying to make a download on empty repository

- fixed ehlo command and non auth mail servers on smtp_lib. Thanks to 

  apollo13 and Johan Walles

- fixed problem with '[' chars in commit messages in journal

- fixed #99 Unicode errors, on file node paths with non utf-8 characters

- fixed #106 relation issues on databases different than sqlite

- registration,password reset and login boxes share the same title as main 

  application now

- fixed problems with getting setting for celery from .ini files

1.1.2 (**2011-01-12**)

======================

news

----

fixes

-----

- fixes #98 protection against float division of percentage stats

- fixed graph bug

- forced webhelpers version since it was making troubles during installation 

1.1.1 (**2011-01-06**)

======================

news

----

- added force https option into ini files for easier https usage (no need to

  set server headers with this options)

- small css updates

fixes

-----

- fixed #96 redirect loop on files view on repositories without changesets

- fixed #97 unicode string passed into server header in special cases (mod_wsgi)

  and server crashed with errors

- fixed large tooltips problems on main page

- fixed #92 whoosh indexer is more error proof

1.1.0 (**2010-12-18**)

======================

news

----

- rewrite of internals for vcs >=0.1.10

- uses mercurial 1.7 with dotencode disabled for maintaining compatibility 

  with older clients

- anonymous access, authentication via ldap

- performance upgrade for cached repos list - each repository has it's own 

  cache that's invalidated when needed.

- performance upgrades on repositories with large amount of commits (20K+)

- main page quick filter for filtering repositories

- user dashboards with ability to follow chosen repositories actions

- sends email to admin on new user registration

- added cache/statistics reset options into repository settings

- more detailed action logger (based on hooks) with pushed changesets lists

  and options to disable those hooks from admin panel

- introduced new enhanced changelog for merges that shows more accurate results

- new improved and faster code stats (based on pygments lexers mapping tables, 

  showing up to 10 trending sources for each repository. Additionally stats

  can be disabled in repository settings.

- gui optimizations, fixed application width to 1024px

- added cut off (for large files/changesets) limit into config files

- whoosh, celeryd, upgrade moved to paster command

- other than sqlite database backends can be used

fixes

-----

- fixes #61 forked repo was showing only after cache expired

- fixes #76 no confirmation on user deletes

- fixes #66 Name field misspelled

- fixes #72 block user removal when he owns repositories

- fixes #69 added password confirmation fields

- fixes #87 RhodeCode crashes occasionally on updating repository owner

- fixes #82 broken annotations on files with more than 1 blank line at the end

- a lot of fixes and tweaks for file browser

- fixed detached session issues

- fixed when user had no repos he would see all repos listed in my account

- fixed ui() instance bug when global hgrc settings was loaded for server 

  instance and all hgrc options were merged with our db ui() object

- numerous small bugfixes

(special thanks for TkSoh for detailed feedback)

1.0.2 (**2010-11-12**)

======================

news

----

- tested under python2.7

- bumped sqlalchemy and celery versions

fixes

-----

- fixed #59 missing graph.js

- fixed repo_size crash when repository had broken symlinks

- fixed python2.5 crashes.

1.0.1 (**2010-11-10**)

======================

news

----

- small css updated

fixes

-----

- fixed #53 python2.5 incompatible enumerate calls

- fixed #52 disable mercurial extension for web

- fixed #51 deleting repositories don't delete it's dependent objects

1.0.0 (**2010-11-02**)

======================

- security bugfix simplehg wasn't checking for permissions on commands

  other than pull or push.

- fixed doubled messages after push or pull in admin journal

- templating and css corrections, fixed repo switcher on chrome, updated titles

- admin menu accessible from options menu on repository view

- permissions cached queries

1.0.0rc4  (**2010-10-12**)

==========================

- fixed python2.5 missing simplejson imports (thanks to Jens Bäckman)

- removed cache_manager settings from sqlalchemy meta

- added sqlalchemy cache settings to ini files

- validated password length and added second try of failure on paster setup-app

- fixed setup database destroy prompt even when there was no db

1.0.0rc3 (**2010-10-11**)

=========================

- fixed i18n during installation.

1.0.0rc2 (**2010-10-11**)

=========================

- Disabled dirsize in file browser, it's causing nasty bug when dir renames 

  occure. After vcs is fixed it'll be put back again.

- templating/css rewrites, optimized css.

"""Helper functions

Consists of functions to typically be used within templates, but also

available to Controllers. This module is available to both as 'h'.

"""

import random

import hashlib

import StringIO

from pygments.formatters import HtmlFormatter

from pygments import highlight as code_highlight

from pylons import url

from pylons.i18n.translation import _, ungettext

from vcs.utils.annotate import annotate_highlight

from rhodecode.lib.utils import repo_name_slug

from webhelpers.html import literal, HTML, escape

from webhelpers.html.tools import *

from webhelpers.html.builder import make_tag

from webhelpers.html.tags import auto_discovery_link, checkbox, css_classes, \

    end_form, file, form, hidden, image, javascript_link, link_to, link_to_if, \

    link_to_unless, ol, required_legend, select, stylesheet_link, submit, text, \

    password, textarea, title, ul, xml_declaration, radio

from webhelpers.html.tools import auto_link, button_to, highlight, js_obfuscate, \

    mail_to, strip_links, strip_tags, tag_re

from webhelpers.number import format_byte_size, format_bit_size

from webhelpers.pylonslib import Flash as _Flash

from webhelpers.pylonslib.secure_form import secure_form

from webhelpers.text import chop_at, collapse, convert_accented_entities, \

    convert_misc_entities, lchop, plural, rchop, remove_formatting, \

    replace_whitespace, urlify, truncate, wrap_paragraphs

from webhelpers.date import time_ago_in_words

from webhelpers.html.tags import _set_input_attrs, _set_id_attr, \

    convert_boolean_attrs, NotGiven

def _reset(name, value=None, id=NotGiven, type="reset", **attrs):

    """Reset button

    """

    _set_input_attrs(attrs, type, name, value)

    _set_id_attr(attrs, id, name)

    convert_boolean_attrs(attrs, ["disabled"])

    return HTML.input(**attrs)

reset = _reset

def get_token():

    """Return the current authentication token, creating one if one doesn't

    already exist.

    """

    token_key = "_authentication_token"

    from pylons import session

    if not token_key in session:

        try:

            token = hashlib.sha1(str(random.getrandbits(128))).hexdigest()

        except AttributeError: # Python < 2.4

            token = hashlib.sha1(str(random.randrange(2 ** 128))).hexdigest()

        session[token_key] = token

        if hasattr(session, 'save'):

            session.save()

    return session[token_key]

class _GetError(object):

    """Get error from form_errors, and represent it as span wrapped error

    message

    :param field_name: field to fetch errors for

    :param form_errors: form errors dict

    """

    def __call__(self, field_name, form_errors):

        tmpl = """<span class="error_msg">%s</span>"""

        if form_errors and form_errors.has_key(field_name):

            return literal(tmpl % form_errors.get(field_name))

get_error = _GetError()

class _ToolTip(object):

    def __call__(self, tooltip_title, trim_at=50):

        """Special function just to wrap our text into nice formatted 

        autowrapped text

        :param tooltip_title:

        """

        return wrap_paragraphs(escape(tooltip_title), trim_at)\

                       .replace('\n', '<br/>')

    def activate(self):

        """Adds tooltip mechanism to the given Html all tooltips have to have 

        set class `tooltip` and set attribute `tooltip_title`.

        Then a tooltip will be generated based on that. All with yui js tooltip

        """

        js = '''

        YAHOO.util.Event.onDOMReady(function(){

            function toolTipsId(){

                var ids = [];

                var tts = YAHOO.util.Dom.getElementsByClassName('tooltip');

                for (var i = 0; i < tts.length; i++) {

                    //if element doesn't not have and id autogenerate one for tooltip

                    if (!tts[i].id){

                        tts[i].id='tt'+i*100;

                    }

                    ids.push(tts[i].id);

                }

                return ids        

            };

            var myToolTips = new YAHOO.widget.Tooltip("tooltip", { 

                context: toolTipsId(),

                monitorresize:false,

                xyoffset :[0,0],

                autodismissdelay:300000,

                hidedelay:5,

                showdelay:20,

            });

            // Set the text for the tooltip just before we display it. Lazy method

            myToolTips.contextTriggerEvent.subscribe( 

                 function(type, args) { 

                        var context = args[0]; 

                        //positioning of tooltip

                        var tt_w = this.element.clientWidth;//tooltip width

                        var tt_h = this.element.clientHeight;//tooltip height

                        var context_w = context.offsetWidth;

                        var context_h = context.offsetHeight;

                        var pos_x = YAHOO.util.Dom.getX(context);

                        var pos_y = YAHOO.util.Dom.getY(context);

                        var display_strategy = 'right';

                        var xy_pos = [0,0];

                        switch (display_strategy){

                            case 'top':

                                var cur_x = (pos_x+context_w/2)-(tt_w/2);

                                var cur_y = (pos_y-tt_h-4);

                                xy_pos = [cur_x,cur_y];                                

                                break;

                            case 'bottom':

                                var cur_x = (pos_x+context_w/2)-(tt_w/2);

                                var cur_y = pos_y+context_h+4;

                                xy_pos = [cur_x,cur_y];                                

                                break;

                            case 'left':

                                var cur_x = (pos_x-tt_w-4);

                                var cur_y = pos_y-((tt_h/2)-context_h/2);

                                xy_pos = [cur_x,cur_y];                                

                                break;

                            case 'right':

                                var cur_x = (pos_x+context_w+4);

                                var cur_y = pos_y-((tt_h/2)-context_h/2);

                                xy_pos = [cur_x,cur_y];                                

                                break;

                             default:

                                var cur_x = (pos_x+context_w/2)-(tt_w/2);

                                var cur_y = pos_y-tt_h-4;

                                xy_pos = [cur_x,cur_y];                                

                                break;                             

                        }

                        this.cfg.setProperty("xy",xy_pos);

                  });

            //Mouse out 

            myToolTips.contextMouseOutEvent.subscribe(

                function(type, args) {

                    var context = args[0];

                });

        });

        '''

        return literal(js)

tooltip = _ToolTip()

class _FilesBreadCrumbs(object):

    def __call__(self, repo_name, rev, paths):

        if isinstance(paths, str):

            paths = paths.decode('utf-8')

        url_l = [link_to(repo_name, url('files_home',

                                        repo_name=repo_name,

                                        revision=rev, f_path=''))]

        paths_l = paths.split('/')

        for cnt, p in enumerate(paths_l):

            if p != '':

                url_l.append(link_to(p, url('files_home',

                                            repo_name=repo_name,

                                            revision=rev,

                                            f_path='/'.join(paths_l[:cnt + 1]))))

        return literal('/'.join(url_l))

files_breadcrumbs = _FilesBreadCrumbs()

class CodeHtmlFormatter(HtmlFormatter):

    """My code Html Formatter for source codes

    """

    def wrap(self, source, outfile):

        return self._wrap_div(self._wrap_pre(self._wrap_code(source)))

    def _wrap_code(self, source):

        for cnt, it in enumerate(source):

            i, t = it

            t = '<div id="L%s">%s</div>' % (cnt + 1, t)

            yield i, t

    def _wrap_tablelinenos(self, inner):

        dummyoutfile = StringIO.StringIO()

        lncount = 0

        for t, line in inner:

            if t:

                lncount += 1

            dummyoutfile.write(line)

        fl = self.linenostart

        mw = len(str(lncount + fl - 1))

        sp = self.linenospecial

        st = self.linenostep

        la = self.lineanchors

        aln = self.anchorlinenos

        nocls = self.noclasses

        if sp:

            lines = []

            for i in range(fl, fl + lncount):

                if i % st == 0:

                    if i % sp == 0:

                        if aln:

                            lines.append('<a href="#%s%d" class="special">%*d</a>' %

                                         (la, i, mw, i))

                        else:

                            lines.append('<span class="special">%*d</span>' % (mw, i))

                    else:

                        if aln:

                            lines.append('<a href="#%s%d">%*d</a>' % (la, i, mw, i))

                        else:

                            lines.append('%*d' % (mw, i))

                else:

                    lines.append('')

            ls = '\n'.join(lines)

        else:

            lines = []

            for i in range(fl, fl + lncount):

                if i % st == 0:

                    if aln:

                        lines.append('<a href="#%s%d">%*d</a>' % (la, i, mw, i))

                    else:

                        lines.append('%*d' % (mw, i))

                else:

                    lines.append('')

            ls = '\n'.join(lines)

        # in case you wonder about the seemingly redundant <div> here: since the

        # content in the other cell also is wrapped in a div, some browsers in

        # some configurations seem to mess up the formatting...

        if nocls:

            yield 0, ('<table class="%stable">' % self.cssclass +

                      '<tr><td><div class="linenodiv" '

                      'style="background-color: #f0f0f0; padding-right: 10px">'

                      '<pre style="line-height: 125%">' +

                      ls + '</pre></div></td><td class="code">')

        else:

            yield 0, ('<table class="%stable">' % self.cssclass +

                      '<tr><td class="linenos"><div class="linenodiv"><pre>' +

                      ls + '</pre></div></td><td class="code">')

        yield 0, dummyoutfile.getvalue()

        yield 0, '</td></tr></table>'

def pygmentize(filenode, **kwargs):

    """pygmentize function using pygments

    :param filenode:

    """

    return literal(code_highlight(filenode.content,

                                  filenode.lexer, CodeHtmlFormatter(**kwargs)))

def pygmentize_annotation(filenode, **kwargs):

    """pygmentize function for annotation

    :param filenode:

    """

    color_dict = {}

    def gen_color(n=10000):

        """generator for getting n of evenly distributed colors using 

        hsv color and golden ratio. It always return same order of colors

        :returns: RGB tuple

        """

        import colorsys

        golden_ratio = 0.618033988749895

        h = 0.22717784590367374

        for c in xrange(n):

            h += golden_ratio

            h %= 1

            HSV_tuple = [h, 0.95, 0.95]

            RGB_tuple = colorsys.hsv_to_rgb(*HSV_tuple)

            yield map(lambda x:str(int(x * 256)), RGB_tuple)

    cgenerator = gen_color()

    def get_color_string(cs):

        if color_dict.has_key(cs):

            col = color_dict[cs]

        else:

            col = color_dict[cs] = cgenerator.next()

        return "color: rgb(%s)! important;" % (', '.join(col))

    def url_func(changeset):

        tooltip_html = "<div style='font-size:0.8em'><b>Author:</b>" + \

        " %s<br/><b>Date:</b> %s</b><br/><b>Message:</b> %s<br/></div>"

        tooltip_html = tooltip_html % (changeset.author,

                                               changeset.date,

                                               tooltip(changeset.message))

        lnk_format = '%5s:%s' % ('r%s' % changeset.revision,

                                 short_id(changeset.raw_id))

        uri = link_to(

                lnk_format,

                url('changeset_home', repo_name=changeset.repository.name,

                    revision=changeset.raw_id),

                style=get_color_string(changeset.raw_id),

                class_='tooltip',

                title=tooltip_html

              )

        uri += '\n'

        return uri

    return literal(annotate_highlight(filenode, url_func, **kwargs))

def get_changeset_safe(repo, rev):

    from vcs.backends.base import BaseRepository

    from vcs.exceptions import RepositoryError

    if not isinstance(repo, BaseRepository):

        raise Exception('You must pass an Repository '

                        'object as first argument got %s', type(repo))

    try:

        cs = repo.get_changeset(rev)

    except RepositoryError:

        from rhodecode.lib.utils import EmptyChangeset

        cs = EmptyChangeset()

    return cs

def is_following_repo(repo_name, user_id):

    from rhodecode.model.scm import ScmModel

    return ScmModel().is_following_repo(repo_name, user_id)

flash = _Flash()

#==============================================================================

# MERCURIAL FILTERS available via h.

#==============================================================================

from mercurial import util

from mercurial.templatefilters import person as _person

def _age(curdate):

    """turns a datetime into an age string."""

    if not curdate:

        return ''

    from datetime import timedelta, datetime

    agescales = [("year", 3600 * 24 * 365),

                 ("month", 3600 * 24 * 30),

                 ("day", 3600 * 24),

                 ("hour", 3600),

                 ("minute", 60),

                 ("second", 1), ]

    age = datetime.now() - curdate

    age_seconds = (age.days * agescales[2][1]) + age.seconds

    pos = 1

    for scale in agescales:

        if scale[1] <= age_seconds:

            if pos == 6:pos = 5

            return time_ago_in_words(curdate, agescales[pos][0]) + ' ' + _('ago')

        pos += 1

    return _('just now')

age = lambda  x:_age(x)

capitalize = lambda x: x.capitalize()

email = util.email

email_or_none = lambda x: util.email(x) if util.email(x) != x else None

person = lambda x: _person(x)

short_id = lambda x: x[:12]

def bool2icon(value):

    """Returns True/False values represented as small html image of true/false

    icons

    :param value: bool value

    """

    if value is True:

        return HTML.tag('img', src=url("/images/icons/accept.png"),

                        alt=_('True'))

    if value is False:

        return HTML.tag('img', src=url("/images/icons/cancel.png"),

                        alt=_('False'))

    return value

def action_parser(user_log):

    """This helper will map the specified string action into translated

    fancy names with icons and links

    :param user_log: user log instance

    """

    action = user_log.action

    action_params = ' '

    x = action.split(':')

    if len(x) > 1:

        action, action_params = x

    def get_cs_links():

        revs_limit = 5 #display this amount always

        revs_top_limit = 50 #show upto this amount of changesets hidden

        revs = action_params.split(',')

        repo_name = user_log.repository.repo_name

        from rhodecode.model.scm import ScmModel

        repo, dbrepo = ScmModel().get(repo_name, retval='repo',

                                      invalidation_list=[])

        message = lambda rev: get_changeset_safe(repo, rev).message

        cs_links = " " + ', '.join ([link_to(rev,

                url('changeset_home',

                repo_name=repo_name,

                revision=rev), title=tooltip(message(rev)),

                class_='tooltip') for rev in revs[:revs_limit] ])

        compare_view = (' <div class="compare_view tooltip" title="%s">'

                        '<a href="%s">%s</a> '

                        '</div>' % (_('Show all combined changesets %s->%s' \

                                      % (revs[0], revs[-1])),

                                    url('changeset_home', repo_name=repo_name,

                                        revision='%s...%s' % (revs[0], revs[-1])

                                    ),

                                    _('compare view'))

                        )

        if len(revs) > revs_limit:

            uniq_id = revs[0]

            html_tmpl = ('<span> %s '

            '<a class="show_more" id="_%s" href="#more">%s</a> '

            '%s</span>')

            cs_links += html_tmpl % (_('and'), uniq_id, _('%s more') \

                                        % (len(revs) - revs_limit),

                                        _('revisions'))

            html_tmpl = '<span id="%s" style="display:none"> %s </span>'

            cs_links += html_tmpl % (uniq_id, ', '.join([link_to(rev,

                url('changeset_home',

                repo_name=repo_name, revision=rev),

                title=message(rev), class_='tooltip')

                for rev in revs[revs_limit:revs_top_limit]]))

        if len(revs) > 1:

            cs_links += compare_view

        return cs_links

    def get_fork_name():

        repo_name = action_params

                                          repo_name=repo_name,)))

    map = {'user_deleted_repo':(_('[deleted] repository'), None),

           'user_created_repo':(_('[created] repository'), None),

           'user_updated_repo':(_('[updated] repository'), None),

           'admin_deleted_repo':(_('[delete] repository'), None),

           'admin_created_repo':(_('[created] repository'), None),

           'admin_forked_repo':(_('[forked] repository'), None),

           'admin_updated_repo':(_('[updated] repository'), None),

           'push':(_('[pushed] into'), get_cs_links),

           'pull':(_('[pulled] from'), None),

           'started_following_repo':(_('[started following] repository'), None),

           'stopped_following_repo':(_('[stopped following] repository'), None),

            }

    action_str = map.get(action, action)

    action = action_str[0].replace('[', '<span class="journal_highlight">')\

                   .replace(']', '</span>')

    action_params_func = lambda :""

    if action_str[1] is not None:

        action_params_func = action_str[1]

    return [literal(action), action_params_func]

def action_parser_icon(user_log):

    action = user_log.action

    action_params = None

    x = action.split(':')

    if len(x) > 1:

        action, action_params = x

    tmpl = """<img src="%s/%s" alt="%s"/>"""

    map = {'user_deleted_repo':'database_delete.png',

           'user_created_repo':'database_add.png',

           'user_forked_repo':'arrow_divide.png',

           'user_updated_repo':'database_edit.png',

           'admin_deleted_repo':'database_delete.png',

           'admin_created_repo':'database_add.png',

           'admin_forked_repo':'arrow_divide.png',

           'admin_updated_repo':'database_edit.png',

           'push':'script_add.png',

           'pull':'down_16.png',

           'started_following_repo':'heart_add.png',

           'stopped_following_repo':'heart_delete.png',

            }

    return literal(tmpl % ((url('/images/icons/')),

                           map.get(action, action), action))

#==============================================================================

# PERMS

#==============================================================================

from rhodecode.lib.auth import HasPermissionAny, HasPermissionAll, \

HasRepoPermissionAny, HasRepoPermissionAll

#==============================================================================

# GRAVATAR URL

#==============================================================================

import hashlib

import urllib

from pylons import request

def gravatar_url(email_address, size=30):

    ssl_enabled = 'https' == request.environ.get('wsgi.url_scheme')

    default = 'identicon'

    baseurl_nossl = "http://www.gravatar.com/avatar/"

    baseurl_ssl = "https://secure.gravatar.com/avatar/"

    baseurl = baseurl_ssl if ssl_enabled else baseurl_nossl

    # construct the url

    gravatar_url = baseurl + hashlib.md5(email_address.lower()).hexdigest() + "?"

    gravatar_url += urllib.urlencode({'d':default, 's':str(size)})

    return gravatar_url

def safe_unicode(str):

    """safe unicode function. In case of UnicodeDecode error we try to return

    unicode with errors replace, if this failes we return unicode with 

    string_escape decoding """

    try:

        u_str = unicode(str)

    except UnicodeDecodeError:

        try:

            u_str = unicode(str, 'utf-8', 'replace')

        except UnicodeDecodeError:

            #incase we have a decode error just represent as byte string

            u_str = unicode(str(str).encode('string_escape'))

    return u_str

def changed_tooltip(nodes):

    if nodes:

        pref = ': <br/> '

        suf = ''

        if len(nodes) > 30:

            suf = '<br/>' + _(' and %s more') % (len(nodes) - 30)

        return literal(pref + '<br/> '.join([x.path for x in nodes[:30]]) + suf)

    else:

        return ': ' + _('No Files')