kallithea Changeset - d5527cebf76a

Changeset - d5527cebf76a

Parent rev.

Child rev.

[Not reviewed]

beta

0 3 0

Marcin Kuzminski - 14 years ago 2012-03-11 16:51:23
marcin@python-works.com

Resolve error occurring during recursive group creation in API create-repo function

3 files changed with 17 insertions and 22 deletions:

docs/changelog.rst

rhodecode/controllers/api/api.py

rhodecode/lib/utils.py

0 comments (0 inline, 0 general)

docs/changelog.rst

➞

Show inline comments

 .. _changelog:
 =========
 Changelog
 =========
 .3.4 (**2012-XX-XX**)
 ----------------------
 :status: in-progress
 :branch: beta
 news
 ++++
 - Whoosh logging is now controlled by the .ini files logging setup
 - added clone-url into edit form on /settings page
 - added help text into repo add/edit forms
 - created rcextensions module with additional mappings (ref #322) and
   post push/pull/create repo hooks callbacks
 fixes
 +++++
 - fixed #390 cache invalidation problems on repos inside group
 - fixed #385 clone by ID url was loosing proxy prefix in URL
 - fixed some unicode problems with waitress
 - fixed issue with escaping < and > in changeset commits
 - fixed error occurring during recursive group creation in API
   create_repo function
 .3.3 (**2012-03-02**)
 ----------------------
 news
 ++++
 fixes
 +++++
 - fixed some python2.5 compatibility issues
 - fixed issues with removed repos was accidentally added as groups, after
   full rescan of paths
 - fixes #376 Cannot edit user (using container auth)
 - fixes #378 Invalid image urls on changeset screen with proxy-prefix
   configuration
 - fixed initial sorting of repos inside repo group
 - fixes issue when user tried to resubmit same permission into user/user_groups
 - bumped beaker version that fixes #375 leap error bug
 - fixed raw_changeset for git. It was generated with hg patch headers
 - fixed vcs issue with last_changeset for filenodes
 - fixed missing commit after hook delete
 - fixed #372 issues with git operation detection that caused a security issue
   for git repos
 .3.2 (**2012-02-28**)
 ----------------------
 news
 ++++
 fixes
 +++++
 - fixed git protocol issues with repos-groups
 - fixed git remote repos validator that prevented from cloning remote git repos
 - fixes #370 ending slashes fixes for repo and groups
 - fixes #368 improved git-protocol detection to handle other clients
 - fixes #366 When Setting Repository Group To Blank Repo Group Wont Be
   Moved To Root
 - fixes #371 fixed issues with beaker/sqlalchemy and non-ascii cache keys
 - fixed #373 missing cascade drop on user_group_to_perm table
 .3.1 (**2012-02-27**)
 ----------------------
 news
 ++++
 fixes
 +++++
 - redirection loop occurs when remember-me wasn't checked during login
 - fixes issues with git blob history generation
 - don't fetch branch for git in file history dropdown. Causes unneeded slowness
 .3.0 (**2012-02-26**)
 ----------------------
 news
 ++++
 - code review, inspired by github code-comments
 - #215 rst and markdown README files support
 - #252 Container-based and proxy pass-through authentication support
 - #44 branch browser. Filtering of changelog by branches
 - mercurial bookmarks support
 - new hover top menu, optimized to add maximum size for important views
 - configurable clone url template with possibility to specify  protocol like
   ssh:// or http:// and also manually alter other parts of clone_url.
 - enabled largefiles extension by default
 - optimized summary file pages and saved a lot of unused space in them
 - #239 option to manually mark repository as fork
 - #320 mapping of commit authors to RhodeCode users
 - #304 hashes are displayed using monospace font
 - diff configuration, toggle white lines and context lines
 - #307 configurable diffs, whitespace toggle, increasing context lines
 - sorting on branches, tags and bookmarks using YUI datatable
 - improved file filter on files page
 - implements #330 api method for listing nodes ar particular revision
 - #73 added linking issues in commit messages to chosen issue tracker url
   based on user defined regular expression
 - added linking of changesets in commit messages
 - new compact changelog with expandable commit messages
 - firstname and lastname are optional in user creation
 - #348 added post-create repository hook
 - #212 global encoding settings is now configurable from .ini files
 - #227 added repository groups permissions
 - markdown gets codehilite extensions
 - new API methods, delete_repositories, grante/revoke permissions for groups
   and repos
 fixes
 +++++
 - rewrote dbsession management for atomic operations, and better error handling
 - fixed sorting of repo tables
 - #326 escape of special html entities in diffs
 - normalized user_name => username in api attributes
 - fixes #298 ldap created users with mixed case emails created conflicts
   on saving a form
 - fixes issue when owner of a repo couldn't revoke permissions for users
   and groups
 - fixes #271 rare JSON serialization problem with statistics
 - fixes #337 missing validation check for conflicting names of a group with a
   repositories group
 - #340 fixed session problem for mysql and celery tasks
 - fixed #331 RhodeCode mangles repository names if the a repository group
   contains the "full path" to the repositories
 - #355 RhodeCode doesn't store encrypted LDAP passwords
 .2.5 (**2012-01-28**)
 ----------------------
 news
 ++++
 fixes
 +++++
 - #340 Celery complains about MySQL server gone away, added session cleanup
   for celery tasks
 - #341 "scanning for repositories in None" log message during Rescan was missing
   a parameter
 - fixed creating archives with subrepos. Some hooks were triggered during that
   operation leading to crash.
 - fixed missing email in account page.
 - Reverted Mercurial to 2.0.1 for windows due to bug in Mercurial that makes
   forking on windows impossible
 .2.4 (**2012-01-19**)
 ----------------------
 news
 ++++
 - RhodeCode is bundled with mercurial series 2.0.X by default, with
   full support to largefiles extension. Enabled by default in new installations
 - #329 Ability to Add/Remove Groups to/from a Repository via AP
 - added requires.txt file with requirements
 fixes
 +++++
 - fixes db session issues with celery when emailing admins
 - #331 RhodeCode mangles repository names if the a repository group
   contains the "full path" to the repositories
 - #298 Conflicting e-mail addresses for LDAP and RhodeCode users
 - DB session cleanup after hg protocol operations, fixes issues with
   `mysql has gone away` errors
 - #333 doc fixes for get_repo api function
 - #271 rare JSON serialization problem with statistics enabled
 - #337 Fixes issues with validation of repository name conflicting with
   a group name. A proper message is now displayed.
 - #292 made ldap_dn in user edit readonly, to get rid of confusion that field
   doesn't work
 - #316 fixes issues with web description in hgrc files
 .2.3 (**2011-11-02**)
 ----------------------
 news
 ++++
 - added option to manage repos group for non admin users
 - added following API methods for get_users, create_user, get_users_groups,
   get_users_group, create_users_group, add_user_to_users_groups, get_repos,
   get_repo, create_repo, add_user_to_repo
 - implements #237 added password confirmation for my account
   and admin edit user.
 - implements #291 email notification for global events are now sent to all
   administrator users, and global config email.
 fixes
 +++++
 - added option for passing auth method for smtp mailer
 - #276 issue with adding a single user with id>10 to usergroups
 - #277 fixes windows LDAP settings in which missing values breaks the ldap auth
 - #288 fixes managing of repos in a group for non admin user
 .2.2 (**2011-10-17**)
 ----------------------
 news
 ++++
 - #226 repo groups are available by path instead of numerical id

rhodecode/controllers/api/api.py

➞

Show inline comments

 # -*- coding: utf-8 -*-
 """
     rhodecode.controllers.api
     ~~~~~~~~~~~~~~~~~~~~~~~~~
     API controller for RhodeCode
     :created_on: Aug 20, 2011
     :author: marcink
     :copyright: (C) 2011-2012 Marcin Kuzminski <marcin@python-works.com>
     :license: GPLv3, see COPYING for more details.
 """
 # This program is free software; you can redistribute it and/or
 # modify it under the terms of the GNU General Public License
 # as published by the Free Software Foundation; version 2
 # of the License or (at your opinion) any later version of the license.
+#
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
+#
 # You should have received a copy of the GNU General Public License
 # along with this program; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
 # MA  02110-1301, USA.
 import traceback
 import logging
 from rhodecode.controllers.api import JSONRPCController, JSONRPCError
 from rhodecode.lib.auth import HasPermissionAllDecorator, \
     HasPermissionAnyDecorator, PasswordGenerator
 from rhodecode.model.meta import Session
 from rhodecode.model.scm import ScmModel
 from rhodecode.model.db import User, UsersGroup, RepoGroup, Repository
 from rhodecode.model.repo import RepoModel
 from rhodecode.model.user import UserModel
 from rhodecode.model.users_group import UsersGroupModel
 from rhodecode.model.repos_group import ReposGroupModel
 from rhodecode.lib.utils import map_groups
 log = logging.getLogger(__name__)
 class ApiController(JSONRPCController):
     """
     API Controller
     Each method needs to have USER as argument this is then based on given
     API_KEY propagated as instance of user object
     Preferably this should be first argument also
     Each function should also **raise** JSONRPCError for any
     errors that happens
     """
     @HasPermissionAllDecorator('hg.admin')
     def pull(self, apiuser, repo_name):
         """
         Dispatch pull action on given repo
         :param user:
         :param repo_name:
         """
         if Repository.is_valid(repo_name) is False:
             raise JSONRPCError('Unknown repo "%s"' % repo_name)
         try:
             ScmModel().pull_changes(repo_name, self.rhodecode_user.username)
             return 'Pulled from %s' % repo_name
         except Exception:
             raise JSONRPCError('Unable to pull changes from "%s"' % repo_name)
     @HasPermissionAllDecorator('hg.admin')
     def get_user(self, apiuser, userid):
         """"
         Get a user by username
         :param apiuser:
         :param username:
         """
         user = UserModel().get_user(userid)
         if user is None:
             return user
         return dict(
             id=user.user_id,
             username=user.username,
             firstname=user.name,
             lastname=user.lastname,
             email=user.email,
             active=user.active,
             admin=user.admin,
             ldap_dn=user.ldap_dn
+        )
     @HasPermissionAllDecorator('hg.admin')
     def get_users(self, apiuser):
         """"
         Get all users
         :param apiuser:
         """
         result = []
         for user in User.getAll():
             result.append(
                 dict(
                     id=user.user_id,
                     username=user.username,
                     firstname=user.name,
                     lastname=user.lastname,
                     email=user.email,
                     active=user.active,
                     admin=user.admin,
                     ldap_dn=user.ldap_dn
+                )
+            )
         return result
     @HasPermissionAllDecorator('hg.admin')
     def create_user(self, apiuser, username, email, password, firstname=None,
                     lastname=None, active=True, admin=False, ldap_dn=None):
         """
         Create new user
         :param apiuser:
         :param username:
         :param password:
         :param email:
         :param name:
         :param lastname:
         :param active:
         :param admin:
         :param ldap_dn:
         """
         if User.get_by_username(username):
             raise JSONRPCError("user %s already exist" % username)
         if User.get_by_email(email, case_insensitive=True):
             raise JSONRPCError("email %s already exist" % email)
         if ldap_dn:
             # generate temporary password if ldap_dn
             password = PasswordGenerator().gen_password(length=8)
         try:
             usr = UserModel().create_or_update(
                 username, password, email, firstname,
                 lastname, active, admin, ldap_dn
+            )
             Session.commit()
             return dict(
                 id=usr.user_id,
                 msg='created new user %s' % username
+            )
         except Exception:
             log.error(traceback.format_exc())
             raise JSONRPCError('failed to create user %s' % username)
     @HasPermissionAllDecorator('hg.admin')
     def update_user(self, apiuser, userid, username, password, email,
                     firstname, lastname, active, admin, ldap_dn):
         """
         Updates given user
         :param apiuser:
         :param username:
         :param password:
         :param email:
         :param name:
         :param lastname:
         :param active:
         :param admin:
         :param ldap_dn:
         """
         if not UserModel().get_user(userid):
             raise JSONRPCError("user %s does not exist" % username)
         try:
             usr = UserModel().create_or_update(
                 username, password, email, firstname,
                 lastname, active, admin, ldap_dn
+            )
             Session.commit()
             return dict(
                 id=usr.user_id,
                 msg='updated user %s' % username
+            )
         except Exception:
             log.error(traceback.format_exc())
             raise JSONRPCError('failed to update user %s' % username)
     @HasPermissionAllDecorator('hg.admin')
     def get_users_group(self, apiuser, group_name):
         """"
         Get users group by name
         :param apiuser:
         :param group_name:
         """
         users_group = UsersGroup.get_by_group_name(group_name)
         if not users_group:
             return None
         members = []
         for user in users_group.members:
             user = user.user
             members.append(dict(id=user.user_id,
                             username=user.username,
                             firstname=user.name,
                             lastname=user.lastname,
                             email=user.email,
                             active=user.active,
                             admin=user.admin,
                             ldap=user.ldap_dn))
         return dict(id=users_group.users_group_id,
                     group_name=users_group.users_group_name,
                     active=users_group.users_group_active,
                     members=members)
     @HasPermissionAllDecorator('hg.admin')
@@ @@ -275,386 +276,381 @@ class ApiController(JSONRPCController): @@
             Session.commit()
             return dict(id=ug.users_group_id,
                         msg='created new users group %s' % group_name)
         except Exception:
             log.error(traceback.format_exc())
             raise JSONRPCError('failed to create group %s' % group_name)
     @HasPermissionAllDecorator('hg.admin')
     def add_user_to_users_group(self, apiuser, group_name, username):
         """"
         Add a user to a group
         :param apiuser:
         :param group_name:
         :param username:
         """
         try:
             users_group = UsersGroup.get_by_group_name(group_name)
             if not users_group:
                 raise JSONRPCError('unknown users group %s' % group_name)
             user = User.get_by_username(username)
             if user is None:
                 raise JSONRPCError('unknown user %s' % username)
             ugm = UsersGroupModel().add_user_to_group(users_group, user)
             success = True if ugm != True else False
             msg = 'added member %s to users group %s' % (username, group_name)
             msg = msg if success else 'User is already in that group'
             Session.commit()
             return dict(
                 id=ugm.users_group_member_id if ugm != True else None,
                 success=success,
                 msg=msg
+            )
         except Exception:
             log.error(traceback.format_exc())
             raise JSONRPCError('failed to add users group member')
     @HasPermissionAllDecorator('hg.admin')
     def remove_user_from_users_group(self, apiuser, group_name, username):
         """
         Remove user from a group
         :param apiuser
         :param group_name
         :param username
         """
         try:
             users_group = UsersGroup.get_by_group_name(group_name)
             if not users_group:
                 raise JSONRPCError('unknown users group %s' % group_name)
             user = User.get_by_username(username)
             if user is None:
                 raise JSONRPCError('unknown user %s' % username)
             success = UsersGroupModel().remove_user_from_group(users_group, user)
             msg = 'removed member %s from users group %s' % (username, group_name)
             msg = msg if success else "User wasn't in group"
             Session.commit()
             return dict(success=success, msg=msg)
         except Exception:
             log.error(traceback.format_exc())
             raise JSONRPCError('failed to remove user from group')
     @HasPermissionAnyDecorator('hg.admin')
     def get_repo(self, apiuser, repoid):
         """"
         Get repository by name
         :param apiuser:
         :param repo_name:
         """
         repo = RepoModel().get_repo(repoid)
         if repo is None:
             raise JSONRPCError('unknown repository %s' % repo)
         members = []
         for user in repo.repo_to_perm:
             perm = user.permission.permission_name
             user = user.user
             members.append(
                 dict(
                     type_="user",
                     id=user.user_id,
                     username=user.username,
                     firstname=user.name,
                     lastname=user.lastname,
                     email=user.email,
                     active=user.active,
                     admin=user.admin,
                     ldap=user.ldap_dn,
                     permission=perm
+                )
+            )
         for users_group in repo.users_group_to_perm:
             perm = users_group.permission.permission_name
             users_group = users_group.users_group
             members.append(
                 dict(
                     type_="users_group",
                     id=users_group.users_group_id,
                     name=users_group.users_group_name,
                     active=users_group.users_group_active,
                     permission=perm
+                )
+            )
         return dict(
             id=repo.repo_id,
             repo_name=repo.repo_name,
             type=repo.repo_type,
             description=repo.description,
             members=members
+        )
     @HasPermissionAnyDecorator('hg.admin')
     def get_repos(self, apiuser):
         """"
         Get all repositories
         :param apiuser:
         """
         result = []
         for repository in Repository.getAll():
             result.append(
                 dict(
                     id=repository.repo_id,
                     repo_name=repository.repo_name,
                     type=repository.repo_type,
                     description=repository.description
+                )
+            )
         return result
     @HasPermissionAnyDecorator('hg.admin')
     def get_repo_nodes(self, apiuser, repo_name, revision, root_path,
                        ret_type='all'):
         """
         returns a list of nodes and it's children
         for a given path at given revision. It's possible to specify ret_type
         to show only files or dirs
         :param apiuser:
         :param repo_name: name of repository
         :param revision: revision for which listing should be done
         :param root_path: path from which start displaying
         :param ret_type: return type 'all|files|dirs' nodes
         """
         try:
             _d, _f = ScmModel().get_nodes(repo_name, revision, root_path,
                                           flat=False)
             _map = {
                 'all': _d + _f,
                 'files': _f,
                 'dirs': _d,
+            }
             return _map[ret_type]
         except KeyError:
             raise JSONRPCError('ret_type must be one of %s' % _map.keys())
         except Exception, e:
             raise JSONRPCError(e)
     @HasPermissionAnyDecorator('hg.admin', 'hg.create.repository')
     def create_repo(self, apiuser, repo_name, owner_name, description='',
                     repo_type='hg', private=False, clone_uri=None):
         """
         Create repository, if clone_url is given it makes a remote clone
         :param apiuser:
         :param repo_name:
         :param owner_name:
         :param description:
         :param repo_type:
         :param private:
         :param clone_uri:
         """
         try:
             owner = User.get_by_username(owner_name)
             if owner is None:
                 raise JSONRPCError('unknown user %s' % owner_name)
             if Repository.get_by_repo_name(repo_name):
                 raise JSONRPCError("repo %s already exist" % repo_name)
-            groups = repo_name.split('/')
+            groups = repo_name.split(Repository.url_sep())
             real_name = groups[-1]
             groups = groups[:-1]
             parent_id = None
             for g in groups:
                 group = RepoGroup.get_by_group_name(g)
                 if not group:
                     group = ReposGroupModel().create(g, '', parent_id)
                 parent_id = group.group_id
             # create structure of groups
             group = map_groups(repo_name)
             repo = RepoModel().create(
                 dict(
                     repo_name=real_name,
                     repo_name_full=repo_name,
                     description=description,
                     private=private,
                     repo_type=repo_type,
-                    repo_group=parent_id,
+                    repo_group=group.group_id if group else None,
                     clone_uri=clone_uri
                 ),
                 owner
+            )
             Session.commit()
             return dict(
                 id=repo.repo_id,
                 msg="Created new repository %s" % repo.repo_name
+            )
         except Exception:
             log.error(traceback.format_exc())
             raise JSONRPCError('failed to create repository %s' % repo_name)
     @HasPermissionAnyDecorator('hg.admin')
     def delete_repo(self, apiuser, repo_name):
         """
         Deletes a given repository
         :param repo_name:
         """
         if not Repository.get_by_repo_name(repo_name):
             raise JSONRPCError("repo %s does not exist" % repo_name)
         try:
             RepoModel().delete(repo_name)
             Session.commit()
             return dict(
                 msg='Deleted repository %s' % repo_name
+            )
         except Exception:
             log.error(traceback.format_exc())
             raise JSONRPCError('failed to delete repository %s' % repo_name)
     @HasPermissionAnyDecorator('hg.admin')
     def grant_user_permission(self, apiuser, repo_name, username, perm):
         """
         Grant permission for user on given repository, or update existing one
         if found
         :param repo_name:
         :param username:
         :param perm:
         """
         try:
             repo = Repository.get_by_repo_name(repo_name)
             if repo is None:
                 raise JSONRPCError('unknown repository %s' % repo)
             user = User.get_by_username(username)
             if user is None:
                 raise JSONRPCError('unknown user %s' % username)
             RepoModel().grant_user_permission(repo=repo, user=user, perm=perm)
             Session.commit()
             return dict(
                 msg='Granted perm: %s for user: %s in repo: %s' % (
                     perm, username, repo_name
+                )
+            )
         except Exception:
             log.error(traceback.format_exc())
             raise JSONRPCError(
                 'failed to edit permission %(repo)s for %(user)s' % dict(
                     user=username, repo=repo_name
+                )
+            )
     @HasPermissionAnyDecorator('hg.admin')
     def revoke_user_permission(self, apiuser, repo_name, username):
         """
         Revoke permission for user on given repository
         :param repo_name:
         :param username:
         """
         try:
             repo = Repository.get_by_repo_name(repo_name)
             if repo is None:
                 raise JSONRPCError('unknown repository %s' % repo)
             user = User.get_by_username(username)
             if user is None:
                 raise JSONRPCError('unknown user %s' % username)
             RepoModel().revoke_user_permission(repo=repo_name, user=username)
             Session.commit()
             return dict(
                 msg='Revoked perm for user: %s in repo: %s' % (
                     username, repo_name
+                )
+            )
         except Exception:
             log.error(traceback.format_exc())
             raise JSONRPCError(
                 'failed to edit permission %(repo)s for %(user)s' % dict(
                     user=username, repo=repo_name
+                )
+            )
     @HasPermissionAnyDecorator('hg.admin')
     def grant_users_group_permission(self, apiuser, repo_name, group_name, perm):
         """
         Grant permission for users group on given repository, or update
         existing one if found
         :param repo_name:
         :param group_name:
         :param perm:
         """
         try:
             repo = Repository.get_by_repo_name(repo_name)
             if repo is None:
                 raise JSONRPCError('unknown repository %s' % repo)
             user_group = UsersGroup.get_by_group_name(group_name)
             if user_group is None:
                 raise JSONRPCError('unknown users group %s' % user_group)
             RepoModel().grant_users_group_permission(repo=repo_name,
                                                      group_name=group_name,
                                                      perm=perm)
             Session.commit()
             return dict(
                 msg='Granted perm: %s for group: %s in repo: %s' % (
                     perm, group_name, repo_name
+                )
+            )
         except Exception:
             log.error(traceback.format_exc())
             raise JSONRPCError(
                 'failed to edit permission %(repo)s for %(usersgr)s' % dict(
                     usersgr=group_name, repo=repo_name
+                )
+            )
     @HasPermissionAnyDecorator('hg.admin')
     def revoke_users_group_permission(self, apiuser, repo_name, group_name):
         """
         Revoke permission for users group on given repository
         :param repo_name:
         :param group_name:
         """
         try:
             repo = Repository.get_by_repo_name(repo_name)
             if repo is None:
                 raise JSONRPCError('unknown repository %s' % repo)
             user_group = UsersGroup.get_by_group_name(group_name)
             if user_group is None:
                 raise JSONRPCError('unknown users group %s' % user_group)
             RepoModel().revoke_users_group_permission(repo=repo_name,
                                                       group_name=group_name)
             Session.commit()
             return dict(
                 msg='Revoked perm for group: %s in repo: %s' % (
                     group_name, repo_name
+                )
+            )
         except Exception:
             log.error(traceback.format_exc())
             raise JSONRPCError(
                 'failed to edit permission %(repo)s for %(usersgr)s' % dict(
                     usersgr=group_name, repo=repo_name
+                )
+            )

rhodecode/lib/utils.py

➞

Show inline comments

@@ @@ -191,443 +191,440 @@ def get_repos(path, recursive=False): @@
                 scm_info = get_scm(cur_path)
                 yield scm_info[1].split(path, 1)[-1].lstrip(os.sep), scm_info
             except VCSError:
                 if not recursive:
                     continue
                 #check if this dir containts other repos for recursive scan
                 rec_path = os.path.join(p, dirpath)
                 if os.path.isdir(rec_path):
                     for inner_scm in _get_repos(rec_path):
                         yield inner_scm
     return _get_repos(path)
 def is_valid_repo(repo_name, base_path):
     """
     Returns True if given path is a valid repository False otherwise
     :param repo_name:
     :param base_path:
     :return True: if given path is a valid repository
     """
     full_path = os.path.join(safe_str(base_path), safe_str(repo_name))
     try:
         get_scm(full_path)
         return True
     except VCSError:
         return False
 def is_valid_repos_group(repos_group_name, base_path):
     """
     Returns True if given path is a repos group False otherwise
     :param repo_name:
     :param base_path:
     """
     full_path = os.path.join(safe_str(base_path), safe_str(repos_group_name))
     # check if it's not a repo
     if is_valid_repo(repos_group_name, base_path):
         return False
     # check if it's a valid path
     if os.path.isdir(full_path):
         return True
     return False
 def ask_ok(prompt, retries=4, complaint='Yes or no, please!'):
     while True:
         ok = raw_input(prompt)
         if ok in ('y', 'ye', 'yes'):
             return True
         if ok in ('n', 'no', 'nop', 'nope'):
             return False
         retries = retries - 1
         if retries < 0:
             raise IOError
         print complaint
 #propagated from mercurial documentation
 ui_sections = ['alias', 'auth',
                 'decode/encode', 'defaults',
                 'diff', 'email',
                 'extensions', 'format',
                 'merge-patterns', 'merge-tools',
                 'hooks', 'http_proxy',
                 'smtp', 'patch',
                 'paths', 'profiling',
                 'server', 'trusted',
                 'ui', 'web', ]
 def make_ui(read_from='file', path=None, checkpaths=True):
     """
     A function that will read python rc files or database
     and make an mercurial ui object from read options
     :param path: path to mercurial config file
     :param checkpaths: check the path
     :param read_from: read from 'file' or 'db'
     """
     baseui = ui.ui()
     # clean the baseui object
     baseui._ocfg = config.config()
     baseui._ucfg = config.config()
     baseui._tcfg = config.config()
     if read_from == 'file':
         if not os.path.isfile(path):
             log.debug('hgrc file is not present at %s skipping...' % path)
             return False
         log.debug('reading hgrc from %s' % path)
         cfg = config.config()
         cfg.read(path)
         for section in ui_sections:
             for k, v in cfg.items(section):
                 log.debug('settings ui from file[%s]%s:%s' % (section, k, v))
                 baseui.setconfig(section, k, v)
     elif read_from == 'db':
         sa = meta.Session
         ret = sa.query(RhodeCodeUi)\
             .options(FromCache("sql_cache_short", "get_hg_ui_settings"))\
             .all()
         hg_ui = ret
         for ui_ in hg_ui:
             if ui_.ui_active:
                 log.debug('settings ui from db[%s]%s:%s', ui_.ui_section,
                           ui_.ui_key, ui_.ui_value)
                 baseui.setconfig(ui_.ui_section, ui_.ui_key, ui_.ui_value)
         meta.Session.remove()
     return baseui
 def set_rhodecode_config(config):
     """
     Updates pylons config with new settings from database
     :param config:
     """
     hgsettings = RhodeCodeSetting.get_app_settings()
     for k, v in hgsettings.items():
         config[k] = v
 def invalidate_cache(cache_key, *args):
     """
     Puts cache invalidation task into db for
     further global cache invalidation
     """
     from rhodecode.model.scm import ScmModel
     if cache_key.startswith('get_repo_cached_'):
         name = cache_key.split('get_repo_cached_')[-1]
         ScmModel().mark_for_invalidation(name)
 class EmptyChangeset(BaseChangeset):
     """
     An dummy empty changeset. It's possible to pass hash when creating
     an EmptyChangeset
     """
     def __init__(self, cs='0' * 40, repo=None, requested_revision=None,
                  alias=None):
         self._empty_cs = cs
         self.revision = -1
         self.message = ''
         self.author = ''
         self.date = ''
         self.repository = repo
         self.requested_revision = requested_revision
         self.alias = alias
     @LazyProperty
     def raw_id(self):
         """
         Returns raw string identifying this changeset, useful for web
         representation.
         """
         return self._empty_cs
     @LazyProperty
     def branch(self):
         return get_backend(self.alias).DEFAULT_BRANCH_NAME
     @LazyProperty
     def short_id(self):
         return self.raw_id[:12]
     def get_file_changeset(self, path):
         return self
     def get_file_content(self, path):
         return u''
     def get_file_size(self, path):
         return 0
-def map_groups(groups):
+def map_groups(path):
     """
     Checks for groups existence, and creates groups structures.
     It returns last group in structure
     Given a full path to a repository, create all nested groups that this
     repo is inside. This function creates parent-child relationships between
     groups and creates default perms for all new groups.
-    :param groups: list of groups structure
+    :param paths: full path to repository
     """
     sa = meta.Session
+    groups = path.split(Repository.url_sep())
     parent = None
     group = None
     # last element is repo in nested groups structure
     groups = groups[:-1]
     rgm = ReposGroupModel(sa)
     for lvl, group_name in enumerate(groups):
         group_name = '/'.join(groups[:lvl] + [group_name])
         group = RepoGroup.get_by_group_name(group_name)
         desc = '%s group' % group_name
 #        # WTF that doesn't work !?
 #        if group is None:
 #            group = rgm.create(group_name, desc, parent, just_db=True)
 #            sa.commit()
         # skip folders that are now removed repos
         if REMOVED_REPO_PAT.match(group_name):
             break
         if group is None:
             log.debug('creating group level: %s group_name: %s' % (lvl, group_name))
             log.debug('creating group level: %s group_name: %s' % (lvl,
                                                                    group_name))
             group = RepoGroup(group_name, parent)
             group.group_description = desc
             sa.add(group)
             rgm._create_default_perms(group)
-            sa.commit()
+            sa.flush()
         parent = group
     return group
 def repo2db_mapper(initial_repo_list, remove_obsolete=False):
     """
     maps all repos given in initial_repo_list, non existing repositories
     are created, if remove_obsolete is True it also check for db entries
     that are not in initial_repo_list and removes them.
     :param initial_repo_list: list of repositories found by scanning methods
     :param remove_obsolete: check for obsolete entries in database
     """
     from rhodecode.model.repo import RepoModel
     sa = meta.Session
     rm = RepoModel()
     user = sa.query(User).filter(User.admin == True).first()
     if user is None:
         raise Exception('Missing administrative account !')
     added = []
     for name, repo in initial_repo_list.items():
-        group = map_groups(name.split(Repository.url_sep()))
         group = map_groups(name)
         if not rm.get_by_repo_name(name, cache=False):
             log.info('repository %s not found creating default' % name)
             added.append(name)
             form_data = {
              'repo_name': name,
              'repo_name_full': name,
              'repo_type': repo.alias,
              'description': repo.description \
                 if repo.description != 'unknown' else '%s repository' % name,
              'private': False,
              'group_id': getattr(group, 'group_id', None)
+            }
             rm.create(form_data, user, just_db=True)
     sa.commit()
     removed = []
     if remove_obsolete:
         #remove from database those repositories that are not in the filesystem
         for repo in sa.query(Repository).all():
             if repo.repo_name not in initial_repo_list.keys():
                 removed.append(repo.repo_name)
                 sa.delete(repo)
                 sa.commit()
     return added, removed
 # set cache regions for beaker so celery can utilise it
 def add_cache(settings):
     cache_settings = {'regions': None}
     for key in settings.keys():
         for prefix in ['beaker.cache.', 'cache.']:
             if key.startswith(prefix):
                 name = key.split(prefix)[1].strip()
                 cache_settings[name] = settings[key].strip()
     if cache_settings['regions']:
         for region in cache_settings['regions'].split(','):
             region = region.strip()
             region_settings = {}
             for key, value in cache_settings.items():
                 if key.startswith(region):
                     region_settings[key.split('.')[1]] = value
             region_settings['expire'] = int(region_settings.get('expire',
 ))
             region_settings.setdefault('lock_dir',
                                        cache_settings.get('lock_dir'))
             region_settings.setdefault('data_dir',
                                        cache_settings.get('data_dir'))
             if 'type' not in region_settings:
                 region_settings['type'] = cache_settings.get('type',
                                                              'memory')
             beaker.cache.cache_regions[region] = region_settings
 def load_rcextensions(root_path):
     import rhodecode
     from rhodecode.config import conf
     path = os.path.join(root_path, 'rcextensions', '__init__.py')
     if os.path.isfile(path):
         rcext = create_module('rc', path)
         EXT = rhodecode.EXTENSIONS = rcext
         log.debug('Found rcextensions now loading %s...' % rcext)
         # Additional mappings that are not present in the pygments lexers
         conf.LANGUAGES_EXTENSIONS_MAP.update(getattr(EXT, 'EXTRA_MAPPINGS', {}))
         #OVERRIDE OUR EXTENSIONS FROM RC-EXTENSIONS (if present)
         if getattr(EXT, 'INDEX_EXTENSIONS', []) != []:
             log.debug('settings custom INDEX_EXTENSIONS')
             conf.INDEX_EXTENSIONS = getattr(EXT, 'INDEX_EXTENSIONS', [])
         #ADDITIONAL MAPPINGS
         log.debug('adding extra into INDEX_EXTENSIONS')
         conf.INDEX_EXTENSIONS.extend(getattr(EXT, 'EXTRA_INDEX_EXTENSIONS', []))
 #==============================================================================
 # TEST FUNCTIONS AND CREATORS
 #==============================================================================
 def create_test_index(repo_location, config, full_index):
     """
     Makes default test index
     :param config: test config
     :param full_index:
     """
     from rhodecode.lib.indexers.daemon import WhooshIndexingDaemon
     from rhodecode.lib.pidlock import DaemonLock, LockHeld
     repo_location = repo_location
     index_location = os.path.join(config['app_conf']['index_dir'])
     if not os.path.exists(index_location):
         os.makedirs(index_location)
     try:
         l = DaemonLock(file_=jn(dn(index_location), 'make_index.lock'))
         WhooshIndexingDaemon(index_location=index_location,
                              repo_location=repo_location)\
             .run(full_index=full_index)
         l.release()
     except LockHeld:
         pass
 def create_test_env(repos_test_path, config):
     """
     Makes a fresh database and
     install test repository into tmp dir
     """
     from rhodecode.lib.db_manage import DbManage
     from rhodecode.tests import HG_REPO, TESTS_TMP_PATH
     # PART ONE create db
     dbconf = config['sqlalchemy.db1.url']
     log.debug('making test db %s' % dbconf)
     # create test dir if it doesn't exist
     if not os.path.isdir(repos_test_path):
         log.debug('Creating testdir %s' % repos_test_path)
         os.makedirs(repos_test_path)
     dbmanage = DbManage(log_sql=True, dbconf=dbconf, root=config['here'],
                         tests=True)
     dbmanage.create_tables(override=True)
     dbmanage.create_settings(dbmanage.config_prompt(repos_test_path))
     dbmanage.create_default_user()
     dbmanage.admin_prompt()
     dbmanage.create_permissions()
     dbmanage.populate_default_permissions()
     Session.commit()
     # PART TWO make test repo
     log.debug('making test vcs repositories')
     idx_path = config['app_conf']['index_dir']
     data_path = config['app_conf']['cache_dir']
     #clean index and data
     if idx_path and os.path.exists(idx_path):
         log.debug('remove %s' % idx_path)
         shutil.rmtree(idx_path)
     if data_path and os.path.exists(data_path):
         log.debug('remove %s' % data_path)
         shutil.rmtree(data_path)
     #CREATE DEFAULT HG REPOSITORY
     cur_dir = dn(dn(abspath(__file__)))
     tar = tarfile.open(jn(cur_dir, 'tests', "vcs_test_hg.tar.gz"))
     tar.extractall(jn(TESTS_TMP_PATH, HG_REPO))
     tar.close()
 #==============================================================================
 # PASTER COMMANDS
 #==============================================================================
 class BasePasterCommand(Command):
     """
     Abstract Base Class for paster commands.
     The celery commands are somewhat aggressive about loading
     celery.conf, and since our module sets the `CELERY_LOADER`
     environment variable to our loader, we have to bootstrap a bit and
     make sure we've had a chance to load the pylons config off of the
     command line, otherwise everything fails.
     """
     min_args = 1
     min_args_error = "Please provide a paster config file as an argument."
     takes_config_file = 1
     requires_config_file = True
     def notify_msg(self, msg, log=False):
         """Make a notification to user, additionally if logger is passed
         it logs this action using given logger
         :param msg: message that will be printed to user
         :param log: logging instance, to use to additionally log this message
         """
         if log and isinstance(log, logging):
             log(msg)
     def run(self, args):
         """
         Overrides Command.run
         Checks for a config file argument and loads it.
         """
         if len(args) < self.min_args:

0 comments (0 inline, 0 general)