1.2.0 (**2011-XX-XX**)

======================

:status: in-progress

:branch: beta

news

----

- implemented #89 Can setup google analytics code from settings menu

- implemented #91 added nicer looking archive urls with more download options

  like tags, branches

- implemented #44 into file browsing, and added follow branch option

- implemented #84 downloads can be enabled/disabled for each repository 

- anonymous repository can be cloned without having to pass default:default

  into clone url

- fixed #90 whoosh indexer can index chooses repositories passed in command 

  line

- extended journal with day aggregates and paging

- implemented #107 source code lines highlight ranges

- implemented #93 customizable changelog on combined revision ranges - 

  equivalent of githubs compare view 

- implemented #108 extended and more powerful LDAP configuration

- implemented #56 users groups

- major code rewrites optimized codes for speed and memory usage

- raw and diff downloads are now in git format

- setup command checks for write access to given path

- fixed many issues with international characters and unicode. It uses utf8

  decode with replace to provide less errors even with non utf8 encoded strings

- #125 added API KEY access to feeds

- #109 Repository can be created from external Mercurial link (aka. remote 

  repository, and manually updated (via pull) from admin panel

fixes

-----

- fixed file browser bug, when switching into given form revision the url was 

  not changing

- fixed propagation to error controller on simplehg and simplegit middlewares

- fixed error when trying to make a download on empty repository

- fixed problem with '[' chars in commit messages in journal

- fixed #99 Unicode errors, on file node paths with non utf-8 characters

- journal fork fixes

- removed issue with space inside renamed repository after deletion

- fixed strange issue on formencode imports

- fixed #126 Deleting repository on Windows, rename used incompatible chars. 

- windows fixes for os.kill and path spliting, issues #148 and #133

1.1.7 (**2011-03-23**)

======================

news

----

fixes

-----

- fixed (again) #136 installation support for FreeBSD

1.1.6 (**2011-03-21**)

======================

news

----

fixes

-----

- fixed #136 installation support for FreeBSD

- RhodeCode will check for python version during installation

1.1.5 (**2011-03-17**)

======================

news

----

- basic windows support, by exchanging pybcrypt into sha256 for windows only

  highly inspired by idea of mantis406

fixes

-----

- fixed sorting by author in main page

- fixed crashes with diffs on binary files

- fixed #131 problem with boolean values for LDAP

- fixed #122 mysql problems thanks to striker69 

- fixed problem with errors on calling raw/raw_files/annotate functions 

  with unknown revisions

- fixed returned rawfiles attachment names with international character

- cleaned out docs, big thanks to Jason Harris

1.1.4 (**2011-02-19**)

======================

# (at your option) any later version.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program.  If not, see <http://www.gnu.org/licenses/>.

import os

import time

import traceback

import logging

from mercurial import ui

from sqlalchemy.exc import DatabaseError

from sqlalchemy.orm import make_transient

from beaker.cache import cache_region, region_invalidate

from vcs import get_backend

from vcs.utils.helpers import get_scm

from vcs.exceptions import RepositoryError, VCSError

from vcs.utils.lazy import LazyProperty

from rhodecode import BACKENDS

from rhodecode.lib import helpers as h

from rhodecode.lib.auth import HasRepoPermissionAny

from rhodecode.lib.utils import get_repos as get_filesystem_repos, make_ui, \

    action_logger

from rhodecode.model import BaseModel

from rhodecode.model.user import UserModel

from rhodecode.model.repo import RepoModel

from rhodecode.model.db import Repository, RhodeCodeUi, CacheInvalidation, \

    UserFollowing, UserLog

from rhodecode.model.caching_query import FromCache

log = logging.getLogger(__name__)

class UserTemp(object):

    def __init__(self, user_id):

        self.user_id = user_id

    def __repr__(self):

        return "<%s('id:%s')>" % (self.__class__.__name__, self.user_id)

class RepoTemp(object):

    def __init__(self, repo_id):

        self.repo_id = repo_id

    def __repr__(self):

        return "<%s('id:%s')>" % (self.__class__.__name__, self.repo_id)

class ScmModel(BaseModel):

    """Generic Scm Model

    """

    @LazyProperty

    def repos_path(self):

        """Get's the repositories root path from database

        """

        q = self.sa.query(RhodeCodeUi).filter(RhodeCodeUi.ui_key == '/').one()

        return q.ui_value

    def repo_scan(self, repos_path=None):

        """Listing of repositories in given path. This path should not be a

        repository itself. Return a dictionary of repository objects

        :param repos_path: path to directory containing repositories

        """

        log.info('scanning for repositories in %s', repos_path)

        if repos_path is None:

            repos_path = self.repos_path

        baseui = make_ui('db')

        repos_list = {}

        for name, path in get_filesystem_repos(repos_path, recursive=True):

            try:

                    raise RepositoryError('Duplicate repository name %s '

                                    'found in %s' % (name, path))

                else:

                    klass = get_backend(path[0])

                    if path[0] == 'hg' and path[0] in BACKENDS.keys():

                        repos_list[name] = klass(path[1], baseui=baseui)

                    if path[0] == 'git' and path[0] in BACKENDS.keys():

                        repos_list[name] = klass(path[1])

            except OSError:

                continue

        return repos_list

    def get_repos(self, all_repos=None):

        :param all_repos: give specific repositories list, good for filtering

            this have to be a list of  just the repository names

        """

        if all_repos is None:

            repos = self.sa.query(Repository)\

                        .order_by(Repository.repo_name).all()

            all_repos = [r.repo_name for r in repos]

        #get the repositories that should be invalidated

        invalidation_list = [str(x.cache_key) for x in \

                             self.sa.query(CacheInvalidation.cache_key)\

                             .filter(CacheInvalidation.cache_active == False)\

                             .all()]

        for r_name in all_repos:

            r_dbr = self.get(r_name, invalidation_list)

            if r_dbr is not None:

                repo, dbrepo = r_dbr

                last_change = repo.last_change

                tip = h.get_changeset_safe(repo, 'tip')

                tmp_d = {}

                tmp_d['name'] = dbrepo.repo_name

                tmp_d['name_sort'] = tmp_d['name'].lower()

                tmp_d['description'] = dbrepo.description

                tmp_d['description_sort'] = tmp_d['description']

                tmp_d['last_change'] = last_change

                tmp_d['tip'] = tip.raw_id

                tmp_d['tip_sort'] = tip.revision

                tmp_d['rev'] = tip.revision

                tmp_d['contact'] = dbrepo.user.full_contact

                tmp_d['contact_sort'] = tmp_d['contact']

                tmp_d['owner_sort'] = tmp_d['contact']

                tmp_d['repo_archives'] = list(repo._get_archives())

                tmp_d['last_msg'] = tip.message

                tmp_d['repo'] = repo

                tmp_d['dbrepo'] = dbrepo.get_dict()

                yield tmp_d

    def get(self, repo_name, invalidation_list=None, retval='all'):

        """Returns a tuple of Repository,DbRepository,

        Get's repository from given name, creates BackendInstance and

        propagates it's data from database with all additional information

        :param repo_name:

        :param invalidation_list: if a invalidation list is given the get

            method should not manually check if this repository needs

            invalidation and just invalidate the repositories in list

        :param retval: string specifing what to return one of 'repo','dbrepo',

            'all'if repo or dbrepo is given it'll just lazy load chosen type

            and return None as the second

        """

        if not HasRepoPermissionAny('repository.read', 'repository.write',

                            'repository.admin')(repo_name, 'get repo check'):

            return

        #======================================================================

        # CACHE FUNCTION

        #======================================================================

        @cache_region('long_term')

        def _get_repo(repo_name):

            repo_path = os.path.join(self.repos_path, repo_name)

            try:

                alias = get_scm(repo_path)[0]

                log.debug('Creating instance of %s repository', alias)

                backend = get_backend(alias)

            except VCSError:

                log.error(traceback.format_exc())

                log.error('Perhaps this repository is in db and not in '

                          'filesystem run rescan repositories with '

                          '"destroy old data " option from admin panel')

                return

            if alias == 'hg':

                repo = backend(repo_path, create=False, baseui=make_ui('db'))

                #skip hidden web repository

                if repo._get_hidden():

                    return

            else:

                repo = backend(repo_path, create=False)

            return repo

        pre_invalidate = True

        dbinvalidate = False

        if invalidation_list is not None:

            pre_invalidate = repo_name in invalidation_list

        if pre_invalidate:

            #this returns object to invalidate

            invalidate = self._should_invalidate(repo_name)

            if invalidate:

                log.info('invalidating cache for repository %s', repo_name)

                region_invalidate(_get_repo, None, repo_name)

                self._mark_invalidated(invalidate)

                dbinvalidate = True

        r, dbr = None, None

        if retval == 'repo' or 'all':

            r = _get_repo(repo_name)

        if retval == 'dbrepo' or 'all':

            dbr = RepoModel().get_full(repo_name, cache=True,

                                          invalidate=dbinvalidate)

        return r, dbr

    def mark_for_invalidation(self, repo_name):

        """Puts cache invalidation task into db for

        further global cache invalidation

        :param repo_name: this repo that should invalidation take place

        """

        log.debug('marking %s for invalidation', repo_name)

        cache = self.sa.query(CacheInvalidation)\

            .filter(CacheInvalidation.cache_key == repo_name).scalar()

        if cache:

            #mark this cache as inactive

            cache.cache_active = False

        else:

            log.debug('cache key not found in invalidation db -> creating one')

            cache = CacheInvalidation(repo_name)

        try:

            self.sa.add(cache)

            self.sa.commit()

        except (DatabaseError,):

            log.error(traceback.format_exc())

            self.sa.rollback()

    def toggle_following_repo(self, follow_repo_id, user_id):

        f = self.sa.query(UserFollowing)\

            .filter(UserFollowing.follows_repo_id == follow_repo_id)\

            .filter(UserFollowing.user_id == user_id).scalar()

        if f is not None:

            try:

                self.sa.delete(f)

                self.sa.commit()

                action_logger(UserTemp(user_id),

                              'stopped_following_repo',

                              RepoTemp(follow_repo_id))

                return

            except:

                log.error(traceback.format_exc())

                self.sa.rollback()

                raise

        try:

            f = UserFollowing()

            f.user_id = user_id

            f.follows_repo_id = follow_repo_id

            self.sa.add(f)

            self.sa.commit()

            action_logger(UserTemp(user_id),

                          'started_following_repo',

                          RepoTemp(follow_repo_id))

        except:

            log.error(traceback.format_exc())

            self.sa.rollback()

            raise

    def toggle_following_user(self, follow_user_id , user_id):

        f = self.sa.query(UserFollowing)\

            .filter(UserFollowing.follows_user_id == follow_user_id)\

            .filter(UserFollowing.user_id == user_id).scalar()

        if f is not None:

            try:

                self.sa.delete(f)

                self.sa.commit()

                return

            except:

                log.error(traceback.format_exc())

                self.sa.rollback()

                raise

        try:

            f = UserFollowing()

            f.user_id = user_id

            f.follows_user_id = follow_user_id

            self.sa.add(f)

            self.sa.commit()

        except:

            log.error(traceback.format_exc())

            self.sa.rollback()

            raise

    def is_following_repo(self, repo_name, user_id, cache=False):

        r = self.sa.query(Repository)\

            .filter(Repository.repo_name == repo_name).scalar()

        f = self.sa.query(UserFollowing)\

            .filter(UserFollowing.follows_repository == r)\

            .filter(UserFollowing.user_id == user_id).scalar()

        return f is not None

    def is_following_user(self, username, user_id, cache=False):

        u = UserModel(self.sa).get_by_username(username)

        f = self.sa.query(UserFollowing)\

            .filter(UserFollowing.follows_user == u)\

            .filter(UserFollowing.user_id == user_id).scalar()

        return f is not None

    def get_followers(self, repo_id):

        if isinstance(repo_id, int):

            return self.sa.query(UserFollowing)\

                    .filter(UserFollowing.follows_repo_id == repo_id).count()

        else:

            return self.sa.query(UserFollowing)\

                    .filter(UserFollowing.follows_repository \

                            == RepoModel().get_by_repo_name(repo_id)).count()

    def get_forks(self, repo_id):

        if isinstance(repo_id, int):

            return self.sa.query(Repository)\

                .filter(Repository.fork_id == repo_id).count()

        else:

            return self.sa.query(Repository)\

                .filter(Repository.fork \

                        == RepoModel().get_by_repo_name(repo_id)).count()

    def pull_changes(self, repo_name, username):

        repo, dbrepo = self.get(repo_name, retval='all')

        try:

            extras = {'ip':'',

                      'username':username,

                      'action':'push_remote',

                      'repository':repo_name}

            #inject ui extra param to log this action via push logger

            for k, v in extras.items():

                repo._repo.ui.setconfig('rhodecode_extras', k, v)

            repo.pull(dbrepo.clone_uri)

            self.mark_for_invalidation(repo_name)

        except:

            log.error(traceback.format_exc())

            raise

    def get_unread_journal(self):

        return self.sa.query(UserLog).count()

    def _should_invalidate(self, repo_name):

        """Looks up database for invalidation signals for this repo_name

        :param repo_name:

        """

        ret = self.sa.query(CacheInvalidation)\

            .filter(CacheInvalidation.cache_key == repo_name)\

            .filter(CacheInvalidation.cache_active == False)\

            .scalar()

        return ret

    def _mark_invalidated(self, cache_key):

        """ Marks all occurrences of cache to invalidation as already

        invalidated

        :param cache_key:

        """

        if cache_key:

            log.debug('marking %s as already invalidated', cache_key)

        try:

            cache_key.cache_active = True

            self.sa.add(cache_key)

            self.sa.commit()

        except (DatabaseError,):

            log.error(traceback.format_exc())

            self.sa.rollback()