kallithea Changeset - ebd76deee70d

Changeset - ebd76deee70d

Parent rev.

Child rev.

[Not reviewed]

beta

0 5 0

Marcin Kuzminski - 13 years ago 2012-12-17 22:46:50
marcin@python-works.com

Default parameters are now also used for creating repos using API calls, and initial repo scanner
- added new parameters to API call to override defaults
- docs update
- repo_type is now optional as it's a part of defaults

5 files changed with 54 insertions and 10 deletions:

docs/api/api.rst

rhodecode/controllers/api/api.py

rhodecode/lib/utils.py

rhodecode/model/db.py

rhodecode/model/repo.py

0 comments (0 inline, 0 general)

docs/api/api.rst

➞

Show inline comments

 .. _api:
 ===
 API
 ===
 Starting from RhodeCode version 1.2 a simple API was implemented.
 There's a single schema for calling all api methods. API is implemented
 with JSON protocol both ways. An url to send API request to RhodeCode is
 <your_server>/_admin/api
 API ACCESS FOR WEB VIEWS
 ++++++++++++++++++++++++
 API access can also be turned on for each web view in RhodeCode that is
 decorated with `@LoginRequired` decorator. To enable API access simple change
 the standard login decorator to `@LoginRequired(api_access=True)`.
 After this change, a rhodecode view can be accessed without login by adding a
 GET parameter `?api_key=<api_key>` to url. By default this is only
 enabled on RSS/ATOM feed views.
 API ACCESS
 ++++++++++
 All clients are required to send JSON-RPC spec JSON data::
+    {
         "id:"<id>",
         "api_key":"<api_key>",
         "method":"<method_name>",
         "args":{"<arg_key>":"<arg_val>"}
+    }
 Example call for autopulling remotes repos using curl::
     curl https://server.com/_admin/api -X POST -H 'content-type:text/plain' --data-binary '{"id":1,"api_key":"xe7cdb2v278e4evbdf5vs04v832v0efvcbcve4a3","method":"pull","args":{"repo":"CPython"}}'
 Simply provide
  - *id* A value of any type, which is used to match the response with the request that it is replying to.
  - *api_key* for access and permission validation.
  - *method* is name of method to call
  - *args* is an key:value list of arguments to pass to method
 .. note::
     api_key can be found in your user account page
 RhodeCode API will return always a JSON-RPC response::
+    {
         "id":<id>, # matching id sent by request
         "result": "<result>"|null, # JSON formatted result, null if any errors
         "error": "null"|<error_message> # JSON formatted error (if any)
+    }
 All responses from API will be `HTTP/1.0 200 OK`, if there's an error while
 calling api *error* key from response will contain failure description
 and result will be null.
 API CLIENT
 ++++++++++
 From version 1.4 RhodeCode adds a script that allows to easily
 communicate with API. After installing RhodeCode a `rhodecode-api` script
 will be available.
 To get started quickly simply run::
   rhodecode-api _create_config --apikey=<youapikey> --apihost=<rhodecode host>
 This will create a file named .config in the directory you executed it storing
 json config file with credentials. You can skip this step and always provide
 both of the arguments to be able to communicate with server
 after that simply run any api command for example get_repo::
  rhodecode-api get_repo
  calling {"api_key": "<apikey>", "id": 75, "args": {}, "method": "get_repo"} to http://127.0.0.1:5000
  rhodecode said:
  {'error': 'Missing non optional `repoid` arg in JSON DATA',
   'id': 75,
   'result': None}
 Ups looks like we forgot to add an argument
 Let's try again now giving the repoid as parameters::
     rhodecode-api get_repo repoid:rhodecode
     calling {"api_key": "<apikey>", "id": 39, "args": {"repoid": "rhodecode"}, "method": "get_repo"} to http://127.0.0.1:5000
     rhodecode said:
     {'error': None,
      'id': 39,
      'result': <json data...>}
 API METHODS
 +++++++++++
 pull
 ----
 Pulls given repo from remote location. Can be used to automatically keep
 remote repos up to date. This command can be executed only using api_key
 belonging to user with admin rights
 INPUT::
     id : <id_for_response>
     api_key : "<api_key>"
     method :  "pull"
     args :    {
                 "repoid" : "<reponame or repo_id>"
+              }
 OUTPUT::
     id : <id_given_in_input>
     result : "Pulled from `<reponame>`"
     error :  null
 rescan_repos
 ------------
 Dispatch rescan repositories action. If remove_obsolete is set
 RhodeCode will delete repos that are in database but not in the filesystem.
 This command can be executed only using api_key belonging to user with admin
 rights.
 INPUT::
     id : <id_for_response>
     api_key : "<api_key>"
     method :  "rescan_repos"
     args :    {
                 "remove_obsolete" : "<boolean = Optional(False)>"
+              }
 OUTPUT::
     id : <id_given_in_input>
     result : "{'added': [<list of names of added repos>],
                'removed': [<list of names of removed repos>]}"
     error :  null
 lock
 ----
 Set locking state on given repository by given user.
 This command can be executed only using api_key belonging to user with admin
 rights.
 INPUT::
     id : <id_for_response>
     api_key : "<api_key>"
     method :  "lock"
     args :    {
                 "repoid" : "<reponame or repo_id>"
                 "userid" : "<user_id or username>",
                 "locked" : "<bool true|false>"
+              }
 OUTPUT::
     id : <id_given_in_input>
     result : "User `<username>` set lock state for repo `<reponame>` to `true|false`"
     error :  null
 get_user
 --------
 Get's an user by username or user_id, Returns empty result if user is not found.
 This command can be executed only using api_key belonging to user with admin
 rights.
 INPUT::
     id : <id_for_response>
     api_key : "<api_key>"
     method :  "get_user"
     args :    {
                 "userid" : "<username or user_id>"
+              }
 OUTPUT::
     id : <id_given_in_input>
     result: None if user does not exist or
+            {
                 "user_id" :  "<user_id>",
                 "username" : "<username>",
                 "firstname": "<firstname>",
                 "lastname" : "<lastname>",
                 "email" :    "<email>",
                 "emails":    "<list_of_all_additional_emails>",
                 "active" :   "<bool>",
                 "admin" :    "<bool>",
                 "ldap_dn" :  "<ldap_dn>",
                 "last_login": "<last_login>",
                 "permissions": {
                     "global": ["hg.create.repository",
                                "repository.read",
                                "hg.register.manual_activate"],
                     "repositories": {"repo1": "repository.none"},
                     "repositories_groups": {"Group1": "group.read"}
                  },
+            }
     error:  null
 get_users
 ---------
 Lists all existing users. This command can be executed only using api_key
 belonging to user with admin rights.
 INPUT::
     id : <id_for_response>
     api_key : "<api_key>"
     method :  "get_users"
     args :    { }
 OUTPUT::
     id : <id_given_in_input>
     result: [
+              {
                 "user_id" :  "<user_id>",
                 "username" : "<username>",
                 "firstname": "<firstname>",
                 "lastname" : "<lastname>",
                 "email" :    "<email>",
                 "emails":    "<list_of_all_additional_emails>",
                 "active" :   "<bool>",
                 "admin" :    "<bool>",
                 "ldap_dn" :  "<ldap_dn>",
                 "last_login": "<last_login>",
               },
     	      …
+            ]
     error:  null
 create_user
 -----------
 Creates new user. This command can
 be executed only using api_key belonging to user with admin rights.
 INPUT::
     id : <id_for_response>
     api_key : "<api_key>"
     method :  "create_user"
     args :    {
                 "username" :  "<username>",
                 "email" :     "<useremail>",
                 "password" :  "<password>",
                 "firstname" : "<firstname> = Optional(None)",
                 "lastname" :  "<lastname> = Optional(None)",
                 "active" :    "<bool> = Optional(True)",
                 "admin" :     "<bool> = Optional(False)",
                 "ldap_dn" :   "<ldap_dn> = Optional(None)"
+              }
 OUTPUT::
     id : <id_given_in_input>
     result: {
               "msg" : "created new user `<username>`",
               "user": {
                 "user_id" :  "<user_id>",
                 "username" : "<username>",
                 "firstname": "<firstname>",
                 "lastname" : "<lastname>",
                 "email" :    "<email>",
                 "emails":    "<list_of_all_additional_emails>",
                 "active" :   "<bool>",
                 "admin" :    "<bool>",
                 "ldap_dn" :  "<ldap_dn>",
                 "last_login": "<last_login>",
               },
+            }
     error:  null
 update_user
 -----------
 updates given user if such user exists. This command can
 be executed only using api_key belonging to user with admin rights.
 INPUT::
     id : <id_for_response>
     api_key : "<api_key>"
     method :  "update_user"
     args :    {
                 "userid" : "<user_id or username>",
                 "username" :  "<username> = Optional",
                 "email" :     "<useremail> = Optional",
                 "password" :  "<password> = Optional",
                 "firstname" : "<firstname> = Optional",
                 "lastname" :  "<lastname> = Optional",
                 "active" :    "<bool> = Optional",
                 "admin" :     "<bool> = Optional",
                 "ldap_dn" :   "<ldap_dn> = Optional"
+              }
 OUTPUT::
     id : <id_given_in_input>
     result: {
               "msg" : "updated user ID:<userid> <username>",
               "user": {
                 "user_id" :  "<user_id>",
                 "username" : "<username>",
                 "firstname": "<firstname>",
                 "lastname" : "<lastname>",
                 "email" :    "<email>",
                 "emails":    "<list_of_all_additional_emails>",
                 "active" :   "<bool>",
                 "admin" :    "<bool>",
                 "ldap_dn" :  "<ldap_dn>",
                 "last_login": "<last_login>",
               },
+            }
     error:  null
 delete_user
 -----------
 deletes givenuser if such user exists. This command can
 be executed only using api_key belonging to user with admin rights.
 INPUT::
     id : <id_for_response>
     api_key : "<api_key>"
     method :  "delete_user"
     args :    {
                 "userid" : "<user_id or username>",
+              }
 OUTPUT::
     id : <id_given_in_input>
     result: {
               "msg" : "deleted user ID:<userid> <username>",
               "user": null
+            }
     error:  null
 get_users_group
 ---------------
 Gets an existing users group. This command can be executed only using api_key
 belonging to user with admin rights.
 INPUT::
     id : <id_for_response>
     api_key : "<api_key>"
     method :  "get_users_group"
     args :    {
                 "usersgroupid" : "<users group id or name>"
+              }
 OUTPUT::
     id : <id_given_in_input>
     result : None if group not exist
+             {
                "users_group_id" : "<id>",
                "group_name" :     "<groupname>",
                "active":          "<bool>",
                "members" :  [
+                              {
                                 "user_id" :  "<user_id>",
                                 "username" : "<username>",
                                 "firstname": "<firstname>",
                                 "lastname" : "<lastname>",
                                 "email" :    "<email>",
                                 "emails":    "<list_of_all_additional_emails>",
                                 "active" :   "<bool>",
                                 "admin" :    "<bool>",
                                 "ldap_dn" :  "<ldap_dn>",
                                 "last_login": "<last_login>",
                               },
                               …
+                            ]
+             }
     error : null
 get_users_groups
 ----------------
 Lists all existing users groups. This command can be executed only using
 api_key belonging to user with admin rights.
 INPUT::
     id : <id_for_response>
     api_key : "<api_key>"
     method :  "get_users_groups"
     args :    { }
 OUTPUT::
     id : <id_given_in_input>
     result : [
+               {
                "users_group_id" : "<id>",
                "group_name" :     "<groupname>",
                "active":          "<bool>",
                },
                …
+              ]
     error : null
 create_users_group
 ------------------
 Creates new users group. This command can be executed only using api_key
 belonging to user with admin rights
 INPUT::
     id : <id_for_response>
     api_key : "<api_key>"
     method :  "create_users_group"
     args:     {
                 "group_name":  "<groupname>",
                 "active":"<bool> = Optional(True)"
+              }
 OUTPUT::
     id : <id_given_in_input>
     result: {
               "msg": "created new users group `<groupname>`",
               "users_group": {
                      "users_group_id" : "<id>",
                      "group_name" :     "<groupname>",
                      "active":          "<bool>",
                },
+            }
     error:  null
 add_user_to_users_group
 -----------------------
 Adds a user to a users group. If user exists in that group success will be
 `false`. This command can be executed only using api_key
 belonging to user with admin rights
 INPUT::
     id : <id_for_response>
     api_key : "<api_key>"
     method :  "add_user_users_group"
     args:     {
                 "usersgroupid" : "<users group id or name>",
                 "userid" : "<user_id or username>",
+              }
 OUTPUT::
     id : <id_given_in_input>
     result: {
               "success": True|False # depends on if member is in group
               "msg": "added member `<username>` to users group `<groupname>` |
                       User is already in that group"
+            }
     error:  null
 remove_user_from_users_group
 ----------------------------
 Removes a user from a users group. If user is not in given group success will
 be `false`. This command can be executed only
 using api_key belonging to user with admin rights
 INPUT::
     id : <id_for_response>
     api_key : "<api_key>"
     method :  "remove_user_from_users_group"
     args:     {
                 "usersgroupid" : "<users group id or name>",
                 "userid" : "<user_id or username>",
+              }
 OUTPUT::
     id : <id_given_in_input>
     result: {
               "success":  True|False,  # depends on if member is in group
               "msg": "removed member <username> from users group <groupname> |
                       User wasn't in group"
+            }
     error:  null
 get_repo
 --------
 Gets an existing repository by it's name or repository_id. Members will return
 either users_group or user associated to that repository. This command can
 be executed only using api_key belonging to user with admin rights.
 INPUT::
     id : <id_for_response>
     api_key : "<api_key>"
     method :  "get_repo"
     args:     {
                 "repoid" : "<reponame or repo_id>"
+              }
 OUTPUT::
     id : <id_given_in_input>
     result: None if repository does not exist or
+            {
                 "repo_id" :     "<repo_id>",
                 "repo_name" :   "<reponame>"
                 "repo_type" :   "<repo_type>",
                 "clone_uri" :   "<clone_uri>",
                 "private": :    "<bool>",
                 "enable_downloads":  "<bool>",
                 "enable_locking":    "<bool>",
                 "enable_statistics": "<bool>",
                 "private":           "<bool>",
                 "created_on" :  "<datetimecreated>",
                 "description" : "<description>",
                 "landing_rev":  "<landing_rev>",
                 "owner":        "<repo_owner>",
                 "fork_of":  "<name_of_fork_parent>",
                 "members" :     [
+                                  {
                                     "type": "user",
                                     "user_id" :  "<user_id>",
                                     "username" : "<username>",
                                     "firstname": "<firstname>",
                                     "lastname" : "<lastname>",
                                     "email" :    "<email>",
                                     "emails":    "<list_of_all_additional_emails>",
                                     "active" :   "<bool>",
                                     "admin" :    "<bool>",
                                     "ldap_dn" :  "<ldap_dn>",
                                     "last_login": "<last_login>",
                                     "permission" : "repository.(read|write|admin)"
                                   },
                                   …
+                                  {
                                     "type": "users_group",
                                     "id" :       "<usersgroupid>",
                                     "name" :     "<usersgroupname>",
                                     "active":    "<bool>",
                                     "permission" : "repository.(read|write|admin)"
                                   },
                                   …
+                                ]
+            }
     error:  null
 get_repos
 ---------
 Lists all existing repositories. This command can be executed only using api_key
 belonging to user with admin rights
 INPUT::
     id : <id_for_response>
     api_key : "<api_key>"
     method :  "get_repos"
     args:     { }
 OUTPUT::
     id : <id_given_in_input>
     result: [
+              {
                 "repo_id" :     "<repo_id>",
                 "repo_name" :   "<reponame>"
                 "repo_type" :   "<repo_type>",
                 "clone_uri" :   "<clone_uri>",
                 "private": :    "<bool>",
                 "created_on" :  "<datetimecreated>",
                 "description" : "<description>",
                 "landing_rev":  "<landing_rev>",
                 "owner":        "<repo_owner>",
                 "fork_of":  "<name_of_fork_parent>",
                 "enable_downloads":  "<bool>",
                 "enable_locking":    "<bool>",
                 "enable_statistics": "<bool>",
               },
               …
+            ]
     error:  null
 get_repo_nodes
 --------------
 returns a list of nodes and it's children in a flat list for a given path
 at given revision. It's possible to specify ret_type to show only `files` or
 `dirs`. This command can be executed only using api_key belonging to user
 with admin rights
 INPUT::
     id : <id_for_response>
     api_key : "<api_key>"
     method :  "get_repo_nodes"
     args:     {
                 "repoid" : "<reponame or repo_id>"
                 "revision"  : "<revision>",
                 "root_path" : "<root_path>",
                 "ret_type"  : "<ret_type> = Optional('all')"
+              }
 OUTPUT::
     id : <id_given_in_input>
     result: [
+              {
                 "name" :        "<name>"
                 "type" :        "<type>",
               },
               …
+            ]
     error:  null
 create_repo
 -----------
 Creates a repository. This command can be executed only using api_key
 belonging to user with admin rights.
 If repository name contains "/", all needed repository groups will be created.
 For example "foo/bar/baz" will create groups "foo", "bar" (with "foo" as parent),
 and create "baz" repository with "bar" as group.
 INPUT::
     id : <id_for_response>
     api_key : "<api_key>"
     method :  "create_repo"
     args:     {
                 "repo_name" :   "<reponame>",
                 "owner" :       "<onwer_name_or_id>",
                 "repo_type" :   "<repo_type>",
+                "repo_type" :        "<repo_type> = Optional('hg')",
                 "description" : "<description> = Optional('')",
                 "private" :     "<bool> = Optional(False)",
                 "clone_uri" :   "<clone_uri> = Optional(None)",
                 "landing_rev" : "<landing_rev> = Optional('tip')",
                 "enable_downloads":  "<bool> = Optional(False)",
                 "enable_locking":    "<bool> = Optional(False)",
                 "enable_statistics": "<bool> = Optional(False)",
+              }
 OUTPUT::
     id : <id_given_in_input>
     result: {
               "msg": "Created new repository `<reponame>`",
               "repo": {
                 "repo_id" :     "<repo_id>",
                 "repo_name" :   "<reponame>"
                 "repo_type" :   "<repo_type>",
                 "clone_uri" :   "<clone_uri>",
                 "private": :    "<bool>",
                 "created_on" :  "<datetimecreated>",
                 "description" : "<description>",
                 "landing_rev":  "<landing_rev>",
                 "owner":        "<repo_owner>",
                 "fork_of":  "<name_of_fork_parent>",
                 "enable_downloads":  "<bool>",
                 "enable_locking":    "<bool>",
                 "enable_statistics": "<bool>",
               },
+            }
     error:  null
 delete_repo
 -----------
 Deletes a repository. This command can be executed only using api_key
 belonging to user with admin rights.
 INPUT::
     id : <id_for_response>
     api_key : "<api_key>"
     method :  "delete_repo"
     args:     {
                 "repoid" : "<reponame or repo_id>"
+              }
 OUTPUT::
     id : <id_given_in_input>
     result: {
               "msg": "Deleted repository `<reponame>`",
               "success": true
+            }
     error:  null
 grant_user_permission
 ---------------------
 Grant permission for user on given repository, or update existing one
 if found. This command can be executed only using api_key belonging to user
 with admin rights.
 INPUT::
     id : <id_for_response>
     api_key : "<api_key>"
     method :  "grant_user_permission"
     args:     {
                 "repoid" : "<reponame or repo_id>"
                 "userid" : "<username or user_id>"
                 "perm" :       "(repository.(none|read|write|admin))",
+              }
 OUTPUT::
     id : <id_given_in_input>
     result: {
               "msg" : "Granted perm: `<perm>` for user: `<username>` in repo: `<reponame>`",
               "success": true
+            }
     error:  null
 revoke_user_permission
 ----------------------
 Revoke permission for user on given repository. This command can be executed
 only using api_key belonging to user with admin rights.
 INPUT::
     id : <id_for_response>
     api_key : "<api_key>"
     method  : "revoke_user_permission"
     args:     {
                 "repoid" : "<reponame or repo_id>"
                 "userid" : "<username or user_id>"
+              }
 OUTPUT::
     id : <id_given_in_input>
     result: {
               "msg" : "Revoked perm for user: `<username>` in repo: `<reponame>`",
               "success": true
+            }
     error:  null
 grant_users_group_permission
 ----------------------------
 Grant permission for users group on given repository, or update
 existing one if found. This command can be executed only using
 api_key belonging to user with admin rights.
 INPUT::
     id : <id_for_response>
     api_key : "<api_key>"
     method :  "grant_users_group_permission"
     args:     {
                 "repoid" : "<reponame or repo_id>"
                 "usersgroupid" : "<users group id or name>"
                 "perm" : "(repository.(none|read|write|admin))",
+              }
 OUTPUT::
     id : <id_given_in_input>
     result: {
               "msg" : "Granted perm: `<perm>` for group: `<usersgroupname>` in repo: `<reponame>`",
               "success": true
+            }
     error:  null
 revoke_users_group_permission
 -----------------------------
 Revoke permission for users group on given repository.This command can be
 executed only using api_key belonging to user with admin rights.
 INPUT::
     id : <id_for_response>
     api_key : "<api_key>"
     method  : "revoke_users_group_permission"
     args:     {
                 "repoid" : "<reponame or repo_id>"
                 "usersgroupid" : "<users group id or name>"
+              }
 OUTPUT::
     id : <id_given_in_input>
     result: {
               "msg" : "Revoked perm for group: `<usersgroupname>` in repo: `<reponame>`",
               "success": true
+            }
     error:  null
@@ \ No newline at end of file @@

rhodecode/controllers/api/api.py

➞

Show inline comments

 # -*- coding: utf-8 -*-
 """
     rhodecode.controllers.api
     ~~~~~~~~~~~~~~~~~~~~~~~~~
     API controller for RhodeCode
     :created_on: Aug 20, 2011
     :author: marcink
     :copyright: (C) 2011-2012 Marcin Kuzminski <marcin@python-works.com>
     :license: GPLv3, see COPYING for more details.
 """
 # This program is free software; you can redistribute it and/or
 # modify it under the terms of the GNU General Public License
 # as published by the Free Software Foundation; version 2
 # of the License or (at your opinion) any later version of the license.
+#
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
+#
 # You should have received a copy of the GNU General Public License
 # along with this program; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
 # MA  02110-1301, USA.
 import traceback
 import logging
 from rhodecode.controllers.api import JSONRPCController, JSONRPCError
 from rhodecode.lib.auth import HasPermissionAllDecorator, \
     HasPermissionAnyDecorator, PasswordGenerator, AuthUser
 from rhodecode.lib.utils import map_groups, repo2db_mapper
 from rhodecode.model.meta import Session
 from rhodecode.model.scm import ScmModel
 from rhodecode.model.repo import RepoModel
 from rhodecode.model.user import UserModel
 from rhodecode.model.users_group import UsersGroupModel
 from rhodecode.model.permission import PermissionModel
 from rhodecode.model.db import Repository
+from rhodecode.model.db import Repository, RhodeCodeSetting
 log = logging.getLogger(__name__)
 class Optional(object):
     """
     Defines an optional parameter::
         param = param.getval() if isinstance(param, Optional) else param
         param = param() if isinstance(param, Optional) else param
     is equivalent of::
         param = Optional.extract(param)
     """
     def __init__(self, type_):
         self.type_ = type_
     def __repr__(self):
         return '<Optional:%s>' % self.type_.__repr__()
     def __call__(self):
         return self.getval()
     def getval(self):
         """
         returns value from this Optional instance
         """
         return self.type_
     @classmethod
     def extract(cls, val):
         if isinstance(val, cls):
             return val.getval()
         return val
 def get_user_or_error(userid):
     """
     Get user by id or name or return JsonRPCError if not found
     :param userid:
     """
     user = UserModel().get_user(userid)
     if user is None:
         raise JSONRPCError("user `%s` does not exist" % userid)
     return user
 def get_repo_or_error(repoid):
     """
     Get repo by id or name or return JsonRPCError if not found
     :param userid:
     """
     repo = RepoModel().get_repo(repoid)
     if repo is None:
         raise JSONRPCError('repository `%s` does not exist' % (repoid))
     return repo
 def get_users_group_or_error(usersgroupid):
     """
     Get users group by id or name or return JsonRPCError if not found
     :param userid:
     """
     users_group = UsersGroupModel().get_group(usersgroupid)
     if users_group is None:
         raise JSONRPCError('users group `%s` does not exist' % usersgroupid)
     return users_group
 def get_perm_or_error(permid):
     """
     Get permission by id or name or return JsonRPCError if not found
     :param userid:
     """
     perm = PermissionModel().get_permission_by_name(permid)
     if perm is None:
         raise JSONRPCError('permission `%s` does not exist' % (permid))
     return perm
 class ApiController(JSONRPCController):
     """
     API Controller
     Each method needs to have USER as argument this is then based on given
     API_KEY propagated as instance of user object
     Preferably this should be first argument also
     Each function should also **raise** JSONRPCError for any
     errors that happens
     """
     @HasPermissionAllDecorator('hg.admin')
     def pull(self, apiuser, repoid):
         """
         Dispatch pull action on given repo
         :param apiuser:
         :param repoid:
         """
         repo = get_repo_or_error(repoid)
         try:
             ScmModel().pull_changes(repo.repo_name,
                                     self.rhodecode_user.username)
             return 'Pulled from `%s`' % repo.repo_name
         except Exception:
             log.error(traceback.format_exc())
             raise JSONRPCError(
                 'Unable to pull changes from `%s`' % repo.repo_name
+            )
     @HasPermissionAllDecorator('hg.admin')
     def rescan_repos(self, apiuser, remove_obsolete=Optional(False)):
         """
         Dispatch rescan repositories action. If remove_obsolete is set
         than also delete repos that are in database but not in the filesystem.
         aka "clean zombies"
         :param apiuser:
         :param remove_obsolete:
         """
         try:
             rm_obsolete = Optional.extract(remove_obsolete)
             added, removed = repo2db_mapper(ScmModel().repo_scan(),
                                             remove_obsolete=rm_obsolete)
             return {'added': added, 'removed': removed}
         except Exception:
             log.error(traceback.format_exc())
             raise JSONRPCError(
                 'Error occurred during rescan repositories action'
+            )
     @HasPermissionAllDecorator('hg.admin')
     def lock(self, apiuser, repoid, userid, locked):
         """
         Set locking state on particular repository by given user
         :param apiuser:
         :param repoid:
         :param userid:
         :param locked:
         """
         repo = get_repo_or_error(repoid)
         user = get_user_or_error(userid)
         locked = bool(locked)
         try:
             if locked:
                 Repository.lock(repo, user.user_id)
             else:
                 Repository.unlock(repo)
             return ('User `%s` set lock state for repo `%s` to `%s`'
                     % (user.username, repo.repo_name, locked))
         except Exception:
             log.error(traceback.format_exc())
             raise JSONRPCError(
                 'Error occurred locking repository `%s`' % repo.repo_name
+            )
     @HasPermissionAllDecorator('hg.admin')
     def get_user(self, apiuser, userid):
         """"
         Get a user by username
         :param apiuser:
         :param userid:
         """
         user = get_user_or_error(userid)
         data = user.get_api_data()
         data['permissions'] = AuthUser(user_id=user.user_id).permissions
         return data
     @HasPermissionAllDecorator('hg.admin')
     def get_users(self, apiuser):
         """"
         Get all users
         :param apiuser:
         """
         result = []
         for user in UserModel().get_all():
             result.append(user.get_api_data())
         return result
     @HasPermissionAllDecorator('hg.admin')
     def create_user(self, apiuser, username, email, password,
                     firstname=Optional(None), lastname=Optional(None),
                     active=Optional(True), admin=Optional(False),
                     ldap_dn=Optional(None)):
         """
         Create new user
         :param apiuser:
         :param username:
         :param email:
         :param password:
         :param firstname:
         :param lastname:
         :param active:
         :param admin:
         :param ldap_dn:
         """
         if UserModel().get_by_username(username):
             raise JSONRPCError("user `%s` already exist" % username)
         if UserModel().get_by_email(email, case_insensitive=True):
             raise JSONRPCError("email `%s` already exist" % email)
         if Optional.extract(ldap_dn):
             # generate temporary password if ldap_dn
             password = PasswordGenerator().gen_password(length=8)
         try:
             user = UserModel().create_or_update(
                 username=Optional.extract(username),
                 password=Optional.extract(password),
                 email=Optional.extract(email),
                 firstname=Optional.extract(firstname),
                 lastname=Optional.extract(lastname),
                 active=Optional.extract(active),
                 admin=Optional.extract(admin),
                 ldap_dn=Optional.extract(ldap_dn)
+            )
             Session().commit()
             return dict(
                 msg='created new user `%s`' % username,
                 user=user.get_api_data()
+            )
         except Exception:
             log.error(traceback.format_exc())
             raise JSONRPCError('failed to create user `%s`' % username)
     @HasPermissionAllDecorator('hg.admin')
     def update_user(self, apiuser, userid, username=Optional(None),
                     email=Optional(None), firstname=Optional(None),
                     lastname=Optional(None), active=Optional(None),
                     admin=Optional(None), ldap_dn=Optional(None),
                     password=Optional(None)):
         """
         Updates given user
         :param apiuser:
         :param userid:
         :param username:
         :param email:
         :param firstname:
         :param lastname:
         :param active:
         :param admin:
         :param ldap_dn:
         :param password:
         """
         user = get_user_or_error(userid)
         # call function and store only updated arguments
         updates = {}
         def store_update(attr, name):
             if not isinstance(attr, Optional):
                 updates[name] = attr
         try:
             store_update(username, 'username')
             store_update(password, 'password')
             store_update(email, 'email')
             store_update(firstname, 'name')
             store_update(lastname, 'lastname')
             store_update(active, 'active')
             store_update(admin, 'admin')
             store_update(ldap_dn, 'ldap_dn')
             user = UserModel().update_user(user, **updates)
             Session().commit()
             return dict(
                 msg='updated user ID:%s %s' % (user.user_id, user.username),
                 user=user.get_api_data()
+            )
         except Exception:
             log.error(traceback.format_exc())
             raise JSONRPCError('failed to update user `%s`' % userid)
     @HasPermissionAllDecorator('hg.admin')
     def delete_user(self, apiuser, userid):
         """"
         Deletes an user
         :param apiuser:
         :param userid:
         """
         user = get_user_or_error(userid)
         try:
             UserModel().delete(userid)
             Session().commit()
             return dict(
                 msg='deleted user ID:%s %s' % (user.user_id, user.username),
                 user=None
+            )
         except Exception:
             log.error(traceback.format_exc())
             raise JSONRPCError('failed to delete ID:%s %s' % (user.user_id,
                                                               user.username))
     @HasPermissionAllDecorator('hg.admin')
     def get_users_group(self, apiuser, usersgroupid):
         """"
         Get users group by name or id
         :param apiuser:
         :param usersgroupid:
         """
         users_group = get_users_group_or_error(usersgroupid)
         data = users_group.get_api_data()
         members = []
         for user in users_group.members:
             user = user.user
             members.append(user.get_api_data())
         data['members'] = members
         return data
     @HasPermissionAllDecorator('hg.admin')
     def get_users_groups(self, apiuser):
         """"
         Get all users groups
         :param apiuser:
         """
         result = []
         for users_group in UsersGroupModel().get_all():
             result.append(users_group.get_api_data())
         return result
     @HasPermissionAllDecorator('hg.admin')
     def create_users_group(self, apiuser, group_name, active=Optional(True)):
         """
         Creates an new usergroup
         :param apiuser:
         :param group_name:
         :param active:
         """
         if UsersGroupModel().get_by_name(group_name):
             raise JSONRPCError("users group `%s` already exist" % group_name)
         try:
             active = Optional.extract(active)
             ug = UsersGroupModel().create(name=group_name, active=active)
             Session().commit()
             return dict(
                 msg='created new users group `%s`' % group_name,
                 users_group=ug.get_api_data()
+            )
         except Exception:
             log.error(traceback.format_exc())
             raise JSONRPCError('failed to create group `%s`' % group_name)
     @HasPermissionAllDecorator('hg.admin')
     def add_user_to_users_group(self, apiuser, usersgroupid, userid):
         """"
         Add a user to a users group
         :param apiuser:
         :param usersgroupid:
         :param userid:
         """
         user = get_user_or_error(userid)
         users_group = get_users_group_or_error(usersgroupid)
         try:
             ugm = UsersGroupModel().add_user_to_group(users_group, user)
             success = True if ugm != True else False
             msg = 'added member `%s` to users group `%s`' % (
                         user.username, users_group.users_group_name
+                    )
             msg = msg if success else 'User is already in that group'
             Session().commit()
             return dict(
                 success=success,
                 msg=msg
+            )
         except Exception:
             log.error(traceback.format_exc())
             raise JSONRPCError(
                 'failed to add member to users group `%s`' % (
                     users_group.users_group_name
+                )
+            )
     @HasPermissionAllDecorator('hg.admin')
     def remove_user_from_users_group(self, apiuser, usersgroupid, userid):
         """
         Remove user from a group
         :param apiuser:
         :param usersgroupid:
         :param userid:
         """
         user = get_user_or_error(userid)
         users_group = get_users_group_or_error(usersgroupid)
         try:
             success = UsersGroupModel().remove_user_from_group(users_group,
                                                                user)
             msg = 'removed member `%s` from users group `%s`' % (
                         user.username, users_group.users_group_name
+                    )
             msg = msg if success else "User wasn't in group"
             Session().commit()
             return dict(success=success, msg=msg)
         except Exception:
             log.error(traceback.format_exc())
             raise JSONRPCError(
                 'failed to remove member from users group `%s`' % (
                         users_group.users_group_name
+                    )
+            )
     @HasPermissionAnyDecorator('hg.admin')
     def get_repo(self, apiuser, repoid):
         """"
         Get repository by name
         :param apiuser:
         :param repoid:
         """
         repo = get_repo_or_error(repoid)
         members = []
         for user in repo.repo_to_perm:
             perm = user.permission.permission_name
             user = user.user
             user_data = user.get_api_data()
             user_data['type'] = "user"
             user_data['permission'] = perm
             members.append(user_data)
         for users_group in repo.users_group_to_perm:
             perm = users_group.permission.permission_name
             users_group = users_group.users_group
             users_group_data = users_group.get_api_data()
             users_group_data['type'] = "users_group"
             users_group_data['permission'] = perm
             members.append(users_group_data)
         data = repo.get_api_data()
         data['members'] = members
         return data
     @HasPermissionAnyDecorator('hg.admin')
     def get_repos(self, apiuser):
         """"
         Get all repositories
         :param apiuser:
         """
         result = []
         for repo in RepoModel().get_all():
             result.append(repo.get_api_data())
         return result
     @HasPermissionAnyDecorator('hg.admin')
     def get_repo_nodes(self, apiuser, repoid, revision, root_path,
                        ret_type='all'):
         """
         returns a list of nodes and it's children
         for a given path at given revision. It's possible to specify ret_type
         to show only files or dirs
         :param apiuser:
         :param repoid: name or id of repository
         :param revision: revision for which listing should be done
         :param root_path: path from which start displaying
         :param ret_type: return type 'all|files|dirs' nodes
         """
         repo = get_repo_or_error(repoid)
         try:
             _d, _f = ScmModel().get_nodes(repo, revision, root_path,
                                           flat=False)
             _map = {
                 'all': _d + _f,
                 'files': _f,
                 'dirs': _d,
+            }
             return _map[ret_type]
         except KeyError:
             raise JSONRPCError('ret_type must be one of %s' % _map.keys())
         except Exception:
             log.error(traceback.format_exc())
             raise JSONRPCError(
                 'failed to get repo: `%s` nodes' % repo.repo_name
+            )
     @HasPermissionAnyDecorator('hg.admin', 'hg.create.repository')
     def create_repo(self, apiuser, repo_name, owner, repo_type,
+    def create_repo(self, apiuser, repo_name, owner, repo_type=Optional('hg'),
                     description=Optional(''), private=Optional(False),
                     clone_uri=Optional(None), landing_rev=Optional('tip')):
                     clone_uri=Optional(None), landing_rev=Optional('tip'),
                     enable_statistics=Optional(False),
                     enable_locking=Optional(False),
                     enable_downloads=Optional(False)):
         """
         Create repository, if clone_url is given it makes a remote clone
         if repo_name is withina  group name the groups will be created
         automatically if they aren't present
         :param apiuser:
         :param repo_name:
         :param onwer:
         :param repo_type:
         :param description:
         :param private:
         :param clone_uri:
         :param landing_rev:
         """
         owner = get_user_or_error(owner)
         if RepoModel().get_by_repo_name(repo_name):
             raise JSONRPCError("repo `%s` already exist" % repo_name)
         private = Optional.extract(private)
         defs = RhodeCodeSetting.get_default_repo_settings(strip_prefix=True)
         if isinstance(private, Optional):
             private = defs.get('repo_private') or Optional.extract(private)
         if isinstance(repo_type, Optional):
             repo_type = defs.get('repo_type')
         if isinstance(enable_statistics, Optional):
             enable_statistics = defs.get('repo_enable_statistics')
         if isinstance(enable_locking, Optional):
             enable_locking = defs.get('repo_enable_locking')
         if isinstance(enable_downloads, Optional):
             enable_downloads = defs.get('repo_enable_downloads')
         clone_uri = Optional.extract(clone_uri)
         description = Optional.extract(description)
         landing_rev = Optional.extract(landing_rev)
         try:
             # create structure of groups and return the last group
             group = map_groups(repo_name)
             repo = RepoModel().create_repo(
                 repo_name=repo_name,
                 repo_type=repo_type,
                 description=description,
                 owner=owner,
                 private=private,
                 clone_uri=clone_uri,
                 repos_group=group,
                 landing_rev=landing_rev,
                 enable_statistics=enable_statistics,
                 enable_downloads=enable_downloads,
                 enable_locking=enable_locking
+            )
             Session().commit()
             return dict(
                 msg="Created new repository `%s`" % (repo.repo_name),
                 repo=repo.get_api_data()
+            )
         except Exception:
             log.error(traceback.format_exc())
             raise JSONRPCError('failed to create repository `%s`' % repo_name)
     @HasPermissionAnyDecorator('hg.admin')
     def fork_repo(self, apiuser, repoid, fork_name, owner,
                   description=Optional(''), copy_permissions=Optional(False),
                   private=Optional(False), landing_rev=Optional('tip')):
         repo = get_repo_or_error(repoid)
         repo_name = repo.repo_name
         owner = get_user_or_error(owner)
         _repo = RepoModel().get_by_repo_name(fork_name)
         if _repo:
             type_ = 'fork' if _repo.fork else 'repo'
             raise JSONRPCError("%s `%s` already exist" % (type_, fork_name))
         try:
             # create structure of groups and return the last group
             group = map_groups(fork_name)
             form_data = dict(
                 repo_name=fork_name,
                 repo_name_full=fork_name,
                 repo_group=group,
                 repo_type=repo.repo_type,
                 description=Optional.extract(description),
                 private=Optional.extract(private),
                 copy_permissions=Optional.extract(copy_permissions),
                 landing_rev=Optional.extract(landing_rev),
                 update_after_clone=False,
                 fork_parent_id=repo.repo_id,
+            )
             RepoModel().create_fork(form_data, cur_user=owner)
             return dict(
                 msg='Created fork of `%s` as `%s`' % (repo.repo_name,
                                                       fork_name),
                 success=True  # cannot return the repo data here since fork
                               # cann be done async
+            )
         except Exception:
             log.error(traceback.format_exc())
             raise JSONRPCError(
                 'failed to fork repository `%s` as `%s`' % (repo_name,
                                                             fork_name)
+            )
     @HasPermissionAnyDecorator('hg.admin')
     def delete_repo(self, apiuser, repoid):
         """
         Deletes a given repository
         :param apiuser:
         :param repoid:
         """
         repo = get_repo_or_error(repoid)
         try:
             RepoModel().delete(repo)
             Session().commit()
             return dict(
                 msg='Deleted repository `%s`' % repo.repo_name,
                 success=True
+            )
         except Exception:
             log.error(traceback.format_exc())
             raise JSONRPCError(
                 'failed to delete repository `%s`' % repo.repo_name
+            )
     @HasPermissionAnyDecorator('hg.admin')
     def grant_user_permission(self, apiuser, repoid, userid, perm):
         """
         Grant permission for user on given repository, or update existing one
         if found
         :param repoid:
         :param userid:
         :param perm:
         """
         repo = get_repo_or_error(repoid)
         user = get_user_or_error(userid)
         perm = get_perm_or_error(perm)
         try:
             RepoModel().grant_user_permission(repo=repo, user=user, perm=perm)
             Session().commit()
             return dict(
                 msg='Granted perm: `%s` for user: `%s` in repo: `%s`' % (
                     perm.permission_name, user.username, repo.repo_name
                 ),
                 success=True
+            )
         except Exception:
             log.error(traceback.format_exc())
             raise JSONRPCError(
                 'failed to edit permission for user: `%s` in repo: `%s`' % (
                     userid, repoid
+                )
+            )
     @HasPermissionAnyDecorator('hg.admin')
     def revoke_user_permission(self, apiuser, repoid, userid):
         """
         Revoke permission for user on given repository
         :param apiuser:
         :param repoid:
         :param userid:
         """
         repo = get_repo_or_error(repoid)
         user = get_user_or_error(userid)
         try:
             RepoModel().revoke_user_permission(repo=repo, user=user)
             Session().commit()
             return dict(
                 msg='Revoked perm for user: `%s` in repo: `%s`' % (
                     user.username, repo.repo_name
                 ),
                 success=True
+            )
         except Exception:
             log.error(traceback.format_exc())
             raise JSONRPCError(
                 'failed to edit permission for user: `%s` in repo: `%s`' % (
                     userid, repoid
+                )
+            )
     @HasPermissionAnyDecorator('hg.admin')
     def grant_users_group_permission(self, apiuser, repoid, usersgroupid,
                                      perm):
         """
         Grant permission for users group on given repository, or update
         existing one if found
         :param apiuser:
         :param repoid:
         :param usersgroupid:
         :param perm:
         """
         repo = get_repo_or_error(repoid)
         perm = get_perm_or_error(perm)
         users_group = get_users_group_or_error(usersgroupid)
         try:
             RepoModel().grant_users_group_permission(repo=repo,
                                                      group_name=users_group,
                                                      perm=perm)
             Session().commit()
             return dict(
                 msg='Granted perm: `%s` for users group: `%s` in '
                     'repo: `%s`' % (
                     perm.permission_name, users_group.users_group_name,
                     repo.repo_name
                 ),
                 success=True
+            )
         except Exception:
             log.error(traceback.format_exc())
             raise JSONRPCError(
                 'failed to edit permission for users group: `%s` in '
                 'repo: `%s`' % (
                     usersgroupid, repo.repo_name
+                )
+            )
     @HasPermissionAnyDecorator('hg.admin')
     def revoke_users_group_permission(self, apiuser, repoid, usersgroupid):
         """
         Revoke permission for users group on given repository
         :param apiuser:
         :param repoid:
         :param usersgroupid:
         """
         repo = get_repo_or_error(repoid)
         users_group = get_users_group_or_error(usersgroupid)
         try:
             RepoModel().revoke_users_group_permission(repo=repo,
                                                       group_name=users_group)
             Session().commit()
             return dict(
                 msg='Revoked perm for users group: `%s` in repo: `%s`' % (
                     users_group.users_group_name, repo.repo_name
                 ),
                 success=True
+            )
         except Exception:
             log.error(traceback.format_exc())
             raise JSONRPCError(
                 'failed to edit permission for users group: `%s` in '
                 'repo: `%s`' % (
                     users_group.users_group_name, repo.repo_name
+                )
+            )

rhodecode/lib/utils.py

➞

Show inline comments

 # -*- coding: utf-8 -*-
 """
     rhodecode.lib.utils
     ~~~~~~~~~~~~~~~~~~~
     Utilities library for RhodeCode
     :created_on: Apr 18, 2010
     :author: marcink
     :copyright: (C) 2010-2012 Marcin Kuzminski <marcin@python-works.com>
     :license: GPLv3, see COPYING for more details.
 """
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation, either version 3 of the License, or
 # (at your option) any later version.
+#
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
+#
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 import os
 import re
 import logging
 import datetime
 import traceback
 import paste
 import beaker
 import tarfile
 import shutil
 import decorator
 import warnings
 from os.path import abspath
 from os.path import dirname as dn, join as jn
 from paste.script.command import Command, BadCommand
 from mercurial import ui, config
 from webhelpers.text import collapse, remove_formatting, strip_tags
 from rhodecode.lib.vcs import get_backend
 from rhodecode.lib.vcs.backends.base import BaseChangeset
 from rhodecode.lib.vcs.utils.lazy import LazyProperty
 from rhodecode.lib.vcs.utils.helpers import get_scm
 from rhodecode.lib.vcs.exceptions import VCSError
 from rhodecode.lib.caching_query import FromCache
 from rhodecode.model import meta
 from rhodecode.model.db import Repository, User, RhodeCodeUi, \
     UserLog, RepoGroup, RhodeCodeSetting, CacheInvalidation
 from rhodecode.model.meta import Session
 from rhodecode.model.repos_group import ReposGroupModel
 from rhodecode.lib.utils2 import safe_str, safe_unicode
 from rhodecode.lib.vcs.utils.fakemod import create_module
 log = logging.getLogger(__name__)
 REMOVED_REPO_PAT = re.compile(r'rm__\d{8}_\d{6}_\d{6}__.*')
 def recursive_replace(str_, replace=' '):
     """
     Recursive replace of given sign to just one instance
     :param str_: given string
     :param replace: char to find and replace multiple instances
     Examples::
     >>> recursive_replace("Mighty---Mighty-Bo--sstones",'-')
     'Mighty-Mighty-Bo-sstones'
     """
     if str_.find(replace * 2) == -1:
         return str_
     else:
         str_ = str_.replace(replace * 2, replace)
         return recursive_replace(str_, replace)
 def repo_name_slug(value):
     """
     Return slug of name of repository
     This function is called on each creation/modification
     of repository to prevent bad names in repo
     """
     slug = remove_formatting(value)
     slug = strip_tags(slug)
     for c in """`?=[]\;'"<>,/~!@#$%^&*()+{}|: """:
         slug = slug.replace(c, '-')
     slug = recursive_replace(slug, '-')
     slug = collapse(slug, '-')
     return slug
 def get_repo_slug(request):
     _repo = request.environ['pylons.routes_dict'].get('repo_name')
     if _repo:
         _repo = _repo.rstrip('/')
     return _repo
 def get_repos_group_slug(request):
     _group = request.environ['pylons.routes_dict'].get('group_name')
     if _group:
         _group = _group.rstrip('/')
     return _group
 def action_logger(user, action, repo, ipaddr='', sa=None, commit=False):
     """
     Action logger for various actions made by users
     :param user: user that made this action, can be a unique username string or
         object containing user_id attribute
     :param action: action to log, should be on of predefined unique actions for
         easy translations
     :param repo: string name of repository or object containing repo_id,
         that action was made on
     :param ipaddr: optional ip address from what the action was made
     :param sa: optional sqlalchemy session
     """
     if not sa:
         sa = meta.Session()
     try:
         if hasattr(user, 'user_id'):
             user_obj = User.get(user.user_id)
         elif isinstance(user, basestring):
             user_obj = User.get_by_username(user)
         else:
             raise Exception('You have to provide a user object or a username')
         if hasattr(repo, 'repo_id'):
             repo_obj = Repository.get(repo.repo_id)
             repo_name = repo_obj.repo_name
         elif  isinstance(repo, basestring):
             repo_name = repo.lstrip('/')
             repo_obj = Repository.get_by_repo_name(repo_name)
         else:
             repo_obj = None
             repo_name = ''
         user_log = UserLog()
         user_log.user_id = user_obj.user_id
         user_log.username = user_obj.username
         user_log.action = safe_unicode(action)
         user_log.repository = repo_obj
         user_log.repository_name = repo_name
         user_log.action_date = datetime.datetime.now()
         user_log.user_ip = ipaddr
         sa.add(user_log)
         log.info(
             'Adding user %s, action %s on %s' % (user_obj, action,
                                                  safe_unicode(repo))
+        )
         if commit:
             sa.commit()
     except:
         log.error(traceback.format_exc())
         raise
 def get_repos(path, recursive=False):
     """
     Scans given path for repos and return (name,(type,path)) tuple
     :param path: path to scan for repositories
     :param recursive: recursive search and return names with subdirs in front
     """
     # remove ending slash for better results
     path = path.rstrip(os.sep)
     def _get_repos(p):
         if not os.access(p, os.W_OK):
             return
         for dirpath in os.listdir(p):
             if os.path.isfile(os.path.join(p, dirpath)):
                 continue
             cur_path = os.path.join(p, dirpath)
             try:
                 scm_info = get_scm(cur_path)
                 yield scm_info[1].split(path, 1)[-1].lstrip(os.sep), scm_info
             except VCSError:
                 if not recursive:
                     continue
                 #check if this dir containts other repos for recursive scan
                 rec_path = os.path.join(p, dirpath)
                 if os.path.isdir(rec_path):
                     for inner_scm in _get_repos(rec_path):
                         yield inner_scm
     return _get_repos(path)
 def is_valid_repo(repo_name, base_path, scm=None):
     """
     Returns True if given path is a valid repository False otherwise.
     If scm param is given also compare if given scm is the same as expected
     from scm parameter
     :param repo_name:
     :param base_path:
     :param scm:
     :return True: if given path is a valid repository
     """
     full_path = os.path.join(safe_str(base_path), safe_str(repo_name))
     try:
         scm_ = get_scm(full_path)
         if scm:
             return scm_[0] == scm
         return True
     except VCSError:
         return False
 def is_valid_repos_group(repos_group_name, base_path):
     """
     Returns True if given path is a repos group False otherwise
     :param repo_name:
     :param base_path:
     """
     full_path = os.path.join(safe_str(base_path), safe_str(repos_group_name))
     # check if it's not a repo
     if is_valid_repo(repos_group_name, base_path):
         return False
     try:
         # we need to check bare git repos at higher level
         # since we might match branches/hooks/info/objects or possible
         # other things inside bare git repo
         get_scm(os.path.dirname(full_path))
         return False
     except VCSError:
         pass
     # check if it's a valid path
     if os.path.isdir(full_path):
         return True
     return False
 def ask_ok(prompt, retries=4, complaint='Yes or no please!'):
     while True:
         ok = raw_input(prompt)
         if ok in ('y', 'ye', 'yes'):
             return True
         if ok in ('n', 'no', 'nop', 'nope'):
             return False
         retries = retries - 1
         if retries < 0:
             raise IOError
         print complaint
 #propagated from mercurial documentation
 ui_sections = ['alias', 'auth',
                 'decode/encode', 'defaults',
                 'diff', 'email',
                 'extensions', 'format',
                 'merge-patterns', 'merge-tools',
                 'hooks', 'http_proxy',
                 'smtp', 'patch',
                 'paths', 'profiling',
                 'server', 'trusted',
                 'ui', 'web', ]
 def make_ui(read_from='file', path=None, checkpaths=True, clear_session=True):
     """
     A function that will read python rc files or database
     and make an mercurial ui object from read options
     :param path: path to mercurial config file
     :param checkpaths: check the path
     :param read_from: read from 'file' or 'db'
     """
     baseui = ui.ui()
     # clean the baseui object
     baseui._ocfg = config.config()
     baseui._ucfg = config.config()
     baseui._tcfg = config.config()
     if read_from == 'file':
         if not os.path.isfile(path):
             log.debug('hgrc file is not present at %s, skipping...' % path)
             return False
         log.debug('reading hgrc from %s' % path)
         cfg = config.config()
         cfg.read(path)
         for section in ui_sections:
             for k, v in cfg.items(section):
                 log.debug('settings ui from file[%s]%s:%s' % (section, k, v))
                 baseui.setconfig(safe_str(section), safe_str(k), safe_str(v))
     elif read_from == 'db':
         sa = meta.Session()
         ret = sa.query(RhodeCodeUi)\
             .options(FromCache("sql_cache_short", "get_hg_ui_settings"))\
             .all()
         hg_ui = ret
         for ui_ in hg_ui:
             if ui_.ui_active:
                 log.debug('settings ui from db[%s]%s:%s', ui_.ui_section,
                           ui_.ui_key, ui_.ui_value)
                 baseui.setconfig(safe_str(ui_.ui_section), safe_str(ui_.ui_key),
                                  safe_str(ui_.ui_value))
             if ui_.ui_key == 'push_ssl':
                 # force set push_ssl requirement to False, rhodecode
                 # handles that
                 baseui.setconfig(safe_str(ui_.ui_section), safe_str(ui_.ui_key),
                                  False)
         if clear_session:
             meta.Session.remove()
     return baseui
 def set_rhodecode_config(config):
     """
     Updates pylons config with new settings from database
     :param config:
     """
     hgsettings = RhodeCodeSetting.get_app_settings()
     for k, v in hgsettings.items():
         config[k] = v
 def invalidate_cache(cache_key, *args):
     """
     Puts cache invalidation task into db for
     further global cache invalidation
     """
     from rhodecode.model.scm import ScmModel
     if cache_key.startswith('get_repo_cached_'):
         name = cache_key.split('get_repo_cached_')[-1]
         ScmModel().mark_for_invalidation(name)
 def map_groups(path):
     """
     Given a full path to a repository, create all nested groups that this
     repo is inside. This function creates parent-child relationships between
     groups and creates default perms for all new groups.
     :param paths: full path to repository
     """
     sa = meta.Session()
     groups = path.split(Repository.url_sep())
     parent = None
     group = None
     # last element is repo in nested groups structure
     groups = groups[:-1]
     rgm = ReposGroupModel(sa)
     for lvl, group_name in enumerate(groups):
         group_name = '/'.join(groups[:lvl] + [group_name])
         group = RepoGroup.get_by_group_name(group_name)
         desc = '%s group' % group_name
         # skip folders that are now removed repos
         if REMOVED_REPO_PAT.match(group_name):
             break
         if group is None:
             log.debug('creating group level: %s group_name: %s' % (lvl,
                                                                    group_name))
             group = RepoGroup(group_name, parent)
             group.group_description = desc
             sa.add(group)
             rgm._create_default_perms(group)
             sa.flush()
         parent = group
     return group
 def repo2db_mapper(initial_repo_list, remove_obsolete=False,
                    install_git_hook=False):
     """
     maps all repos given in initial_repo_list, non existing repositories
     are created, if remove_obsolete is True it also check for db entries
     that are not in initial_repo_list and removes them.
     :param initial_repo_list: list of repositories found by scanning methods
     :param remove_obsolete: check for obsolete entries in database
     :param install_git_hook: if this is True, also check and install githook
         for a repo if missing
     """
     from rhodecode.model.repo import RepoModel
     from rhodecode.model.scm import ScmModel
     sa = meta.Session()
     rm = RepoModel()
     user = sa.query(User).filter(User.admin == True).first()
     if user is None:
         raise Exception('Missing administrative account!')
     added = []
 #    # clear cache keys
 #    log.debug("Clearing cache keys now...")
 #    CacheInvalidation.clear_cache()
 #    sa.commit()
     ##creation defaults
     defs = RhodeCodeSetting.get_default_repo_settings(strip_prefix=True)
     enable_statistics = defs.get('repo_enable_statistics')
     enable_locking = defs.get('repo_enable_locking')
     enable_downloads = defs.get('repo_enable_downloads')
     private = defs.get('repo_private')
     for name, repo in initial_repo_list.items():
         group = map_groups(name)
         db_repo = rm.get_by_repo_name(name)
         # found repo that is on filesystem not in RhodeCode database
         if not db_repo:
             log.info('repository %s not found, creating now' % name)
             added.append(name)
             desc = (repo.description
                     if repo.description != 'unknown'
                     else '%s repository' % name)
             new_repo = rm.create_repo(
                 repo_name=name,
                 repo_type=repo.alias,
                 description=desc,
                 repos_group=getattr(group, 'group_id', None),
                 owner=user,
                 just_db=True
                 just_db=True,
                 enable_locking=enable_locking,
                 enable_downloads=enable_downloads,
                 enable_statistics=enable_statistics,
                 private=private
+            )
             # we added that repo just now, and make sure it has githook
             # installed
             if new_repo.repo_type == 'git':
                 ScmModel().install_git_hook(new_repo.scm_instance)
         elif install_git_hook:
             if db_repo.repo_type == 'git':
                 ScmModel().install_git_hook(db_repo.scm_instance)
         # during starting install all cache keys for all repositories in the
         # system, this will register all repos and multiple instances
         key, _prefix, _org_key = CacheInvalidation._get_key(name)
         CacheInvalidation.invalidate(name)
         log.debug("Creating a cache key for %s instance_id=>`%s`"
                   % (name, _prefix or '-'))
     sa.commit()
     removed = []
     if remove_obsolete:
         # remove from database those repositories that are not in the filesystem
         for repo in sa.query(Repository).all():
             if repo.repo_name not in initial_repo_list.keys():
                 log.debug("Removing non-existing repository found in db `%s`" %
                           repo.repo_name)
                 try:
                     sa.delete(repo)
                     sa.commit()
                     removed.append(repo.repo_name)
                 except:
                     #don't hold further removals on error
                     log.error(traceback.format_exc())
                     sa.rollback()
     return added, removed
 # set cache regions for beaker so celery can utilise it
 def add_cache(settings):
     cache_settings = {'regions': None}
     for key in settings.keys():
         for prefix in ['beaker.cache.', 'cache.']:
             if key.startswith(prefix):
                 name = key.split(prefix)[1].strip()
                 cache_settings[name] = settings[key].strip()
     if cache_settings['regions']:
         for region in cache_settings['regions'].split(','):
             region = region.strip()
             region_settings = {}
             for key, value in cache_settings.items():
                 if key.startswith(region):
                     region_settings[key.split('.')[1]] = value
             region_settings['expire'] = int(region_settings.get('expire',
 ))
             region_settings.setdefault('lock_dir',
                                        cache_settings.get('lock_dir'))
             region_settings.setdefault('data_dir',
                                        cache_settings.get('data_dir'))
             if 'type' not in region_settings:
                 region_settings['type'] = cache_settings.get('type',
                                                              'memory')
             beaker.cache.cache_regions[region] = region_settings
 def load_rcextensions(root_path):
     import rhodecode
     from rhodecode.config import conf
     path = os.path.join(root_path, 'rcextensions', '__init__.py')
     if os.path.isfile(path):
         rcext = create_module('rc', path)
         EXT = rhodecode.EXTENSIONS = rcext
         log.debug('Found rcextensions now loading %s...' % rcext)
         # Additional mappings that are not present in the pygments lexers
         conf.LANGUAGES_EXTENSIONS_MAP.update(getattr(EXT, 'EXTRA_MAPPINGS', {}))
         #OVERRIDE OUR EXTENSIONS FROM RC-EXTENSIONS (if present)
         if getattr(EXT, 'INDEX_EXTENSIONS', []) != []:
             log.debug('settings custom INDEX_EXTENSIONS')
             conf.INDEX_EXTENSIONS = getattr(EXT, 'INDEX_EXTENSIONS', [])
         #ADDITIONAL MAPPINGS
         log.debug('adding extra into INDEX_EXTENSIONS')
         conf.INDEX_EXTENSIONS.extend(getattr(EXT, 'EXTRA_INDEX_EXTENSIONS', []))
 #==============================================================================
 # TEST FUNCTIONS AND CREATORS
 #==============================================================================
 def create_test_index(repo_location, config, full_index):
     """
     Makes default test index
     :param config: test config
     :param full_index:
     """
     from rhodecode.lib.indexers.daemon import WhooshIndexingDaemon
     from rhodecode.lib.pidlock import DaemonLock, LockHeld
     repo_location = repo_location
     index_location = os.path.join(config['app_conf']['index_dir'])
     if not os.path.exists(index_location):
         os.makedirs(index_location)
     try:
         l = DaemonLock(file_=jn(dn(index_location), 'make_index.lock'))
         WhooshIndexingDaemon(index_location=index_location,
                              repo_location=repo_location)\
             .run(full_index=full_index)
         l.release()
     except LockHeld:
         pass
 def create_test_env(repos_test_path, config):
     """
     Makes a fresh database and
     install test repository into tmp dir
     """
     from rhodecode.lib.db_manage import DbManage
     from rhodecode.tests import HG_REPO, GIT_REPO, TESTS_TMP_PATH
     # PART ONE create db
     dbconf = config['sqlalchemy.db1.url']
     log.debug('making test db %s' % dbconf)
     # create test dir if it doesn't exist
     if not os.path.isdir(repos_test_path):
         log.debug('Creating testdir %s' % repos_test_path)
         os.makedirs(repos_test_path)
     dbmanage = DbManage(log_sql=True, dbconf=dbconf, root=config['here'],
                         tests=True)
     dbmanage.create_tables(override=True)
     dbmanage.create_settings(dbmanage.config_prompt(repos_test_path))
     dbmanage.create_default_user()
     dbmanage.admin_prompt()
     dbmanage.create_permissions()
     dbmanage.populate_default_permissions()
     Session().commit()
     # PART TWO make test repo
     log.debug('making test vcs repositories')
     idx_path = config['app_conf']['index_dir']
     data_path = config['app_conf']['cache_dir']
     #clean index and data
     if idx_path and os.path.exists(idx_path):
         log.debug('remove %s' % idx_path)
         shutil.rmtree(idx_path)
     if data_path and os.path.exists(data_path):
         log.debug('remove %s' % data_path)
         shutil.rmtree(data_path)
     #CREATE DEFAULT TEST REPOS
     cur_dir = dn(dn(abspath(__file__)))
     tar = tarfile.open(jn(cur_dir, 'tests', "vcs_test_hg.tar.gz"))
     tar.extractall(jn(TESTS_TMP_PATH, HG_REPO))
     tar.close()
     cur_dir = dn(dn(abspath(__file__)))
     tar = tarfile.open(jn(cur_dir, 'tests', "vcs_test_git.tar.gz"))
     tar.extractall(jn(TESTS_TMP_PATH, GIT_REPO))
     tar.close()
     #LOAD VCS test stuff
     from rhodecode.tests.vcs import setup_package
     setup_package()
 #==============================================================================
 # PASTER COMMANDS
 #==============================================================================
 class BasePasterCommand(Command):
     """
     Abstract Base Class for paster commands.
     The celery commands are somewhat aggressive about loading
     celery.conf, and since our module sets the `CELERY_LOADER`
     environment variable to our loader, we have to bootstrap a bit and
     make sure we've had a chance to load the pylons config off of the
     command line, otherwise everything fails.
     """
     min_args = 1
     min_args_error = "Please provide a paster config file as an argument."
     takes_config_file = 1
     requires_config_file = True
     def notify_msg(self, msg, log=False):
         """Make a notification to user, additionally if logger is passed
         it logs this action using given logger
         :param msg: message that will be printed to user
         :param log: logging instance, to use to additionally log this message
         """
         if log and isinstance(log, logging):
             log(msg)
     def run(self, args):
         """
         Overrides Command.run
         Checks for a config file argument and loads it.
         """
         if len(args) < self.min_args:
             raise BadCommand(
                 self.min_args_error % {'min_args': self.min_args,
                                        'actual_args': len(args)})
         # Decrement because we're going to lob off the first argument.
         # @@ This is hacky
         self.min_args -= 1
         self.bootstrap_config(args[0])
         self.update_parser()
         return super(BasePasterCommand, self).run(args[1:])
     def update_parser(self):
         """
         Abstract method.  Allows for the class's parser to be updated
         before the superclass's `run` method is called.  Necessary to
         allow options/arguments to be passed through to the underlying
         celery command.
         """
         raise NotImplementedError("Abstract Method.")
     def bootstrap_config(self, conf):
         """
         Loads the pylons configuration.
         """
         from pylons import config as pylonsconfig
         self.path_to_ini_file = os.path.realpath(conf)
         conf = paste.deploy.appconfig('config:' + self.path_to_ini_file)
         pylonsconfig.init_app(conf.global_conf, conf.local_conf)
 def check_git_version():
     """
     Checks what version of git is installed in system, and issues a warning
     if it's too old for RhodeCode to properly work.
     """
     import subprocess
     from distutils.version import StrictVersion
     from rhodecode import BACKENDS
     p = subprocess.Popen('git --version', shell=True,
                         stdout=subprocess.PIPE, stderr=subprocess.PIPE)
     stdout, stderr = p.communicate()
     ver = (stdout.split(' ')[-1] or '').strip() or '0.0.0'
     if len(ver.split('.')) > 3:
         #StrictVersion needs to be only 3 element type
         ver = '.'.join(ver.split('.')[:3])
     try:
         _ver = StrictVersion(ver)
     except:
         _ver = StrictVersion('0.0.0')
         stderr = traceback.format_exc()
     req_ver = '1.7.4'
     to_old_git = False
     if  _ver < StrictVersion(req_ver):
         to_old_git = True
     if 'git' in BACKENDS:
         log.debug('GIT version detected: %s' % stdout)
         if stderr:
             log.warning('Unable to detect git version org error was:%r' % stderr)
         elif to_old_git:
             log.warning('RhodeCode detected git version %s, which is too old '
                         'for the system to function properly. Make sure '
                         'its version is at least %s' % (ver, req_ver))
     return _ver
 @decorator.decorator
 def jsonify(func, *args, **kwargs):
     """Action decorator that formats output for JSON
     Given a function that will return content, this decorator will turn
     the result into JSON, with a content-type of 'application/json' and
     output it.
     """
     from pylons.decorators.util import get_pylons
     from rhodecode.lib.ext_json import json
     pylons = get_pylons(args)
     pylons.response.headers['Content-Type'] = 'application/json; charset=utf-8'
     data = func(*args, **kwargs)
     if isinstance(data, (list, tuple)):
         msg = "JSON responses with Array envelopes are susceptible to " \
               "cross-site data leak attacks, see " \
               "http://wiki.pylonshq.com/display/pylonsfaq/Warnings"
         warnings.warn(msg, Warning, 2)
         log.warning(msg)
     log.debug("Returning JSON wrapped action output")
     return json.dumps(data, encoding='utf-8')
@@ \ No newline at end of file @@

rhodecode/model/db.py

➞

Show inline comments

@@ @@ -76,1537 +76,1540 @@ class BaseModel(object): @@
         """
         return dict with keys and values corresponding
         to this model data """
         d = {}
         for k in self._get_keys():
             d[k] = getattr(self, k)
         # also use __json__() if present to get additional fields
         _json_attr = getattr(self, '__json__', None)
         if _json_attr:
             # update with attributes from __json__
             if callable(_json_attr):
                 _json_attr = _json_attr()
             for k, val in _json_attr.iteritems():
                 d[k] = val
         return d
     def get_appstruct(self):
         """return list with keys and values tupples corresponding
         to this model data """
         l = []
         for k in self._get_keys():
             l.append((k, getattr(self, k),))
         return l
     def populate_obj(self, populate_dict):
         """populate model with data from given populate_dict"""
         for k in self._get_keys():
             if k in populate_dict:
                 setattr(self, k, populate_dict[k])
     @classmethod
     def query(cls):
         return Session().query(cls)
     @classmethod
     def get(cls, id_):
         if id_:
             return cls.query().get(id_)
     @classmethod
     def get_or_404(cls, id_):
         try:
             id_ = int(id_)
         except (TypeError, ValueError):
             raise HTTPNotFound
         res = cls.query().get(id_)
         if not res:
             raise HTTPNotFound
         return res
     @classmethod
     def getAll(cls):
         return cls.query().all()
     @classmethod
     def delete(cls, id_):
         obj = cls.query().get(id_)
         Session().delete(obj)
     def __repr__(self):
         if hasattr(self, '__unicode__'):
             # python repr needs to return str
             return safe_str(self.__unicode__())
         return '<DB:%s>' % (self.__class__.__name__)
 class RhodeCodeSetting(Base, BaseModel):
     __tablename__ = 'rhodecode_settings'
     __table_args__ = (
         UniqueConstraint('app_settings_name'),
         {'extend_existing': True, 'mysql_engine': 'InnoDB',
          'mysql_charset': 'utf8'}
+    )
     app_settings_id = Column("app_settings_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     app_settings_name = Column("app_settings_name", String(255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     _app_settings_value = Column("app_settings_value", String(255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     def __init__(self, k='', v=''):
         self.app_settings_name = k
         self.app_settings_value = v
     @validates('_app_settings_value')
     def validate_settings_value(self, key, val):
         assert type(val) == unicode
         return val
     @hybrid_property
     def app_settings_value(self):
         v = self._app_settings_value
         if self.app_settings_name in ["ldap_active",
                                       "default_repo_enable_statistics",
                                       "default_repo_enable_locking",
                                       "default_repo_private",
                                       "default_repo_enable_downloads"]:
             v = str2bool(v)
         return v
     @app_settings_value.setter
     def app_settings_value(self, val):
         """
         Setter that will always make sure we use unicode in app_settings_value
         :param val:
         """
         self._app_settings_value = safe_unicode(val)
     def __unicode__(self):
         return u"<%s('%s:%s')>" % (
             self.__class__.__name__,
             self.app_settings_name, self.app_settings_value
+        )
     @classmethod
     def get_by_name(cls, key):
         return cls.query()\
             .filter(cls.app_settings_name == key).scalar()
     @classmethod
     def get_by_name_or_create(cls, key):
         res = cls.get_by_name(key)
         if not res:
             res = cls(key)
         return res
     @classmethod
     def get_app_settings(cls, cache=False):
         ret = cls.query()
         if cache:
             ret = ret.options(FromCache("sql_cache_short", "get_hg_settings"))
         if not ret:
             raise Exception('Could not get application settings !')
         settings = {}
         for each in ret:
             settings['rhodecode_' + each.app_settings_name] = \
                 each.app_settings_value
         return settings
     @classmethod
     def get_ldap_settings(cls, cache=False):
         ret = cls.query()\
                 .filter(cls.app_settings_name.startswith('ldap_')).all()
         fd = {}
         for row in ret:
             fd.update({row.app_settings_name: row.app_settings_value})
         return fd
     @classmethod
     def get_default_repo_settings(cls, cache=False, strip_prefix=False):
         ret = cls.query()\
                 .filter(cls.app_settings_name.startswith('default_')).all()
         fd = {}
         for row in ret:
             key = row.app_settings_name
             if strip_prefix:
                 key = remove_prefix(key, prefix='default_')
             fd.update({key: row.app_settings_value})
         return fd
 class RhodeCodeUi(Base, BaseModel):
     __tablename__ = 'rhodecode_ui'
     __table_args__ = (
         UniqueConstraint('ui_key'),
         {'extend_existing': True, 'mysql_engine': 'InnoDB',
          'mysql_charset': 'utf8'}
+    )
     HOOK_UPDATE = 'changegroup.update'
     HOOK_REPO_SIZE = 'changegroup.repo_size'
     HOOK_PUSH = 'changegroup.push_logger'
     HOOK_PRE_PUSH = 'prechangegroup.pre_push'
     HOOK_PULL = 'outgoing.pull_logger'
     HOOK_PRE_PULL = 'preoutgoing.pre_pull'
     ui_id = Column("ui_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     ui_section = Column("ui_section", String(255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     ui_key = Column("ui_key", String(255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     ui_value = Column("ui_value", String(255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     ui_active = Column("ui_active", Boolean(), nullable=True, unique=None, default=True)
     @classmethod
     def get_by_key(cls, key):
         return cls.query().filter(cls.ui_key == key).scalar()
     @classmethod
     def get_builtin_hooks(cls):
         q = cls.query()
         q = q.filter(cls.ui_key.in_([cls.HOOK_UPDATE, cls.HOOK_REPO_SIZE,
                                      cls.HOOK_PUSH, cls.HOOK_PRE_PUSH,
                                      cls.HOOK_PULL, cls.HOOK_PRE_PULL]))
         return q.all()
     @classmethod
     def get_custom_hooks(cls):
         q = cls.query()
         q = q.filter(~cls.ui_key.in_([cls.HOOK_UPDATE, cls.HOOK_REPO_SIZE,
                                       cls.HOOK_PUSH, cls.HOOK_PRE_PUSH,
                                       cls.HOOK_PULL, cls.HOOK_PRE_PULL]))
         q = q.filter(cls.ui_section == 'hooks')
         return q.all()
     @classmethod
     def get_repos_location(cls):
         return cls.get_by_key('/').ui_value
     @classmethod
     def create_or_update_hook(cls, key, val):
         new_ui = cls.get_by_key(key) or cls()
         new_ui.ui_section = 'hooks'
         new_ui.ui_active = True
         new_ui.ui_key = key
         new_ui.ui_value = val
         Session().add(new_ui)
     def __repr__(self):
         return '<DB:%s[%s:%s]>' % (self.__class__.__name__, self.ui_key,
                                    self.ui_value)
 class User(Base, BaseModel):
     __tablename__ = 'users'
     __table_args__ = (
         UniqueConstraint('username'), UniqueConstraint('email'),
         Index('u_username_idx', 'username'),
         Index('u_email_idx', 'email'),
         {'extend_existing': True, 'mysql_engine': 'InnoDB',
          'mysql_charset': 'utf8'}
+    )
     DEFAULT_USER = 'default'
     DEFAULT_PERMISSIONS = [
         'hg.register.manual_activate', 'hg.create.repository',
         'hg.fork.repository', 'repository.read', 'group.read'
+    ]
     user_id = Column("user_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     username = Column("username", String(255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     password = Column("password", String(255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     active = Column("active", Boolean(), nullable=True, unique=None, default=True)
     admin = Column("admin", Boolean(), nullable=True, unique=None, default=False)
     name = Column("firstname", String(255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     lastname = Column("lastname", String(255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     _email = Column("email", String(255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     last_login = Column("last_login", DateTime(timezone=False), nullable=True, unique=None, default=None)
     ldap_dn = Column("ldap_dn", String(255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     api_key = Column("api_key", String(255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     inherit_default_permissions = Column("inherit_default_permissions", Boolean(), nullable=False, unique=None, default=True)
     user_log = relationship('UserLog')
     user_perms = relationship('UserToPerm', primaryjoin="User.user_id==UserToPerm.user_id", cascade='all')
     repositories = relationship('Repository')
     user_followers = relationship('UserFollowing', primaryjoin='UserFollowing.follows_user_id==User.user_id', cascade='all')
     followings = relationship('UserFollowing', primaryjoin='UserFollowing.user_id==User.user_id', cascade='all')
     repo_to_perm = relationship('UserRepoToPerm', primaryjoin='UserRepoToPerm.user_id==User.user_id', cascade='all')
     repo_group_to_perm = relationship('UserRepoGroupToPerm', primaryjoin='UserRepoGroupToPerm.user_id==User.user_id', cascade='all')
     group_member = relationship('UsersGroupMember', cascade='all')
     notifications = relationship('UserNotification', cascade='all')
     # notifications assigned to this user
     user_created_notifications = relationship('Notification', cascade='all')
     # comments created by this user
     user_comments = relationship('ChangesetComment', cascade='all')
     #extra emails for this user
     user_emails = relationship('UserEmailMap', cascade='all')
     @hybrid_property
     def email(self):
         return self._email
     @email.setter
     def email(self, val):
         self._email = val.lower() if val else None
     @property
     def firstname(self):
         # alias for future
         return self.name
     @property
     def emails(self):
         other = UserEmailMap.query().filter(UserEmailMap.user==self).all()
         return [self.email] + [x.email for x in other]
     @property
     def username_and_name(self):
         return '%s (%s %s)' % (self.username, self.firstname, self.lastname)
     @property
     def full_name(self):
         return '%s %s' % (self.firstname, self.lastname)
     @property
     def full_name_or_username(self):
         return ('%s %s' % (self.firstname, self.lastname)
                 if (self.firstname and self.lastname) else self.username)
     @property
     def full_contact(self):
         return '%s %s <%s>' % (self.firstname, self.lastname, self.email)
     @property
     def short_contact(self):
         return '%s %s' % (self.firstname, self.lastname)
     @property
     def is_admin(self):
         return self.admin
     def __unicode__(self):
         return u"<%s('id:%s:%s')>" % (self.__class__.__name__,
                                      self.user_id, self.username)
     @classmethod
     def get_by_username(cls, username, case_insensitive=False, cache=False):
         if case_insensitive:
             q = cls.query().filter(cls.username.ilike(username))
         else:
             q = cls.query().filter(cls.username == username)
         if cache:
             q = q.options(FromCache(
                             "sql_cache_short",
                             "get_user_%s" % _hash_key(username)
+                          )
+            )
         return q.scalar()
     @classmethod
     def get_by_api_key(cls, api_key, cache=False):
         q = cls.query().filter(cls.api_key == api_key)
         if cache:
             q = q.options(FromCache("sql_cache_short",
                                     "get_api_key_%s" % api_key))
         return q.scalar()
     @classmethod
     def get_by_email(cls, email, case_insensitive=False, cache=False):
         if case_insensitive:
             q = cls.query().filter(cls.email.ilike(email))
         else:
             q = cls.query().filter(cls.email == email)
         if cache:
             q = q.options(FromCache("sql_cache_short",
                                     "get_email_key_%s" % email))
         ret = q.scalar()
         if ret is None:
             q = UserEmailMap.query()
             # try fetching in alternate email map
             if case_insensitive:
                 q = q.filter(UserEmailMap.email.ilike(email))
             else:
                 q = q.filter(UserEmailMap.email == email)
             q = q.options(joinedload(UserEmailMap.user))
             if cache:
                 q = q.options(FromCache("sql_cache_short",
                                         "get_email_map_key_%s" % email))
             ret = getattr(q.scalar(), 'user', None)
         return ret
     def update_lastlogin(self):
         """Update user lastlogin"""
         self.last_login = datetime.datetime.now()
         Session().add(self)
         log.debug('updated user %s lastlogin' % self.username)
     def get_api_data(self):
         """
         Common function for generating user related data for API
         """
         user = self
         data = dict(
             user_id=user.user_id,
             username=user.username,
             firstname=user.name,
             lastname=user.lastname,
             email=user.email,
             emails=user.emails,
             api_key=user.api_key,
             active=user.active,
             admin=user.admin,
             ldap_dn=user.ldap_dn,
             last_login=user.last_login,
+        )
         return data
     def __json__(self):
         data = dict(
             full_name=self.full_name,
             full_name_or_username=self.full_name_or_username,
             short_contact=self.short_contact,
             full_contact=self.full_contact
+        )
         data.update(self.get_api_data())
         return data
 class UserEmailMap(Base, BaseModel):
     __tablename__ = 'user_email_map'
     __table_args__ = (
         Index('uem_email_idx', 'email'),
         UniqueConstraint('email'),
         {'extend_existing': True, 'mysql_engine': 'InnoDB',
          'mysql_charset': 'utf8'}
+    )
     __mapper_args__ = {}
     email_id = Column("email_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     user_id = Column("user_id", Integer(), ForeignKey('users.user_id'), nullable=True, unique=None, default=None)
     _email = Column("email", String(255, convert_unicode=False, assert_unicode=None), nullable=True, unique=False, default=None)
     user = relationship('User', lazy='joined')
     @validates('_email')
     def validate_email(self, key, email):
         # check if this email is not main one
         main_email = Session().query(User).filter(User.email == email).scalar()
         if main_email is not None:
             raise AttributeError('email %s is present is user table' % email)
         return email
     @hybrid_property
     def email(self):
         return self._email
     @email.setter
     def email(self, val):
         self._email = val.lower() if val else None
 class UserLog(Base, BaseModel):
     __tablename__ = 'user_logs'
     __table_args__ = (
         {'extend_existing': True, 'mysql_engine': 'InnoDB',
          'mysql_charset': 'utf8'},
+    )
     user_log_id = Column("user_log_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     user_id = Column("user_id", Integer(), ForeignKey('users.user_id'), nullable=True, unique=None, default=None)
     username = Column("username", String(255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     repository_id = Column("repository_id", Integer(), ForeignKey('repositories.repo_id'), nullable=True)
     repository_name = Column("repository_name", String(255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     user_ip = Column("user_ip", String(255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     action = Column("action", UnicodeText(1200000, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     action_date = Column("action_date", DateTime(timezone=False), nullable=True, unique=None, default=None)
     @property
     def action_as_day(self):
         return datetime.date(*self.action_date.timetuple()[:3])
     user = relationship('User')
     repository = relationship('Repository', cascade='')
 class UsersGroup(Base, BaseModel):
     __tablename__ = 'users_groups'
     __table_args__ = (
         {'extend_existing': True, 'mysql_engine': 'InnoDB',
          'mysql_charset': 'utf8'},
+    )
     users_group_id = Column("users_group_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     users_group_name = Column("users_group_name", String(255, convert_unicode=False, assert_unicode=None), nullable=False, unique=True, default=None)
     users_group_active = Column("users_group_active", Boolean(), nullable=True, unique=None, default=None)
     inherit_default_permissions = Column("users_group_inherit_default_permissions", Boolean(), nullable=False, unique=None, default=True)
     members = relationship('UsersGroupMember', cascade="all, delete, delete-orphan", lazy="joined")
     users_group_to_perm = relationship('UsersGroupToPerm', cascade='all')
     users_group_repo_to_perm = relationship('UsersGroupRepoToPerm', cascade='all')
     def __unicode__(self):
         return u'<userGroup(%s)>' % (self.users_group_name)
     @classmethod
     def get_by_group_name(cls, group_name, cache=False,
                           case_insensitive=False):
         if case_insensitive:
             q = cls.query().filter(cls.users_group_name.ilike(group_name))
         else:
             q = cls.query().filter(cls.users_group_name == group_name)
         if cache:
             q = q.options(FromCache(
                             "sql_cache_short",
                             "get_user_%s" % _hash_key(group_name)
+                          )
+            )
         return q.scalar()
     @classmethod
     def get(cls, users_group_id, cache=False):
         users_group = cls.query()
         if cache:
             users_group = users_group.options(FromCache("sql_cache_short",
                                     "get_users_group_%s" % users_group_id))
         return users_group.get(users_group_id)
     def get_api_data(self):
         users_group = self
         data = dict(
             users_group_id=users_group.users_group_id,
             group_name=users_group.users_group_name,
             active=users_group.users_group_active,
+        )
         return data
 class UsersGroupMember(Base, BaseModel):
     __tablename__ = 'users_groups_members'
     __table_args__ = (
         {'extend_existing': True, 'mysql_engine': 'InnoDB',
          'mysql_charset': 'utf8'},
+    )
     users_group_member_id = Column("users_group_member_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     users_group_id = Column("users_group_id", Integer(), ForeignKey('users_groups.users_group_id'), nullable=False, unique=None, default=None)
     user_id = Column("user_id", Integer(), ForeignKey('users.user_id'), nullable=False, unique=None, default=None)
     user = relationship('User', lazy='joined')
     users_group = relationship('UsersGroup')
     def __init__(self, gr_id='', u_id=''):
         self.users_group_id = gr_id
         self.user_id = u_id
 class Repository(Base, BaseModel):
     __tablename__ = 'repositories'
     __table_args__ = (
         UniqueConstraint('repo_name'),
         Index('r_repo_name_idx', 'repo_name'),
         {'extend_existing': True, 'mysql_engine': 'InnoDB',
          'mysql_charset': 'utf8'},
+    )
     repo_id = Column("repo_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     repo_name = Column("repo_name", String(255, convert_unicode=False, assert_unicode=None), nullable=False, unique=True, default=None)
     clone_uri = Column("clone_uri", String(255, convert_unicode=False, assert_unicode=None), nullable=True, unique=False, default=None)
     repo_type = Column("repo_type", String(255, convert_unicode=False, assert_unicode=None), nullable=False, unique=False, default=None)
     user_id = Column("user_id", Integer(), ForeignKey('users.user_id'), nullable=False, unique=False, default=None)
     private = Column("private", Boolean(), nullable=True, unique=None, default=None)
     enable_statistics = Column("statistics", Boolean(), nullable=True, unique=None, default=True)
     enable_downloads = Column("downloads", Boolean(), nullable=True, unique=None, default=True)
     description = Column("description", String(10000, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     created_on = Column('created_on', DateTime(timezone=False), nullable=True, unique=None, default=datetime.datetime.now)
     updated_on = Column('updated_on', DateTime(timezone=False), nullable=True, unique=None, default=datetime.datetime.now)
     landing_rev = Column("landing_revision", String(255, convert_unicode=False, assert_unicode=None), nullable=False, unique=False, default=None)
     enable_locking = Column("enable_locking", Boolean(), nullable=False, unique=None, default=False)
     _locked = Column("locked", String(255, convert_unicode=False, assert_unicode=None), nullable=True, unique=False, default=None)
     fork_id = Column("fork_id", Integer(), ForeignKey('repositories.repo_id'), nullable=True, unique=False, default=None)
     group_id = Column("group_id", Integer(), ForeignKey('groups.group_id'), nullable=True, unique=False, default=None)
     user = relationship('User')
     fork = relationship('Repository', remote_side=repo_id)
     group = relationship('RepoGroup')
     repo_to_perm = relationship('UserRepoToPerm', cascade='all', order_by='UserRepoToPerm.repo_to_perm_id')
     users_group_to_perm = relationship('UsersGroupRepoToPerm', cascade='all')
     stats = relationship('Statistics', cascade='all', uselist=False)
     followers = relationship('UserFollowing',
                              primaryjoin='UserFollowing.follows_repo_id==Repository.repo_id',
                              cascade='all')
     logs = relationship('UserLog')
     comments = relationship('ChangesetComment', cascade="all, delete, delete-orphan")
     pull_requests_org = relationship('PullRequest',
                     primaryjoin='PullRequest.org_repo_id==Repository.repo_id',
                     cascade="all, delete, delete-orphan")
     pull_requests_other = relationship('PullRequest',
                     primaryjoin='PullRequest.other_repo_id==Repository.repo_id',
                     cascade="all, delete, delete-orphan")
     def __unicode__(self):
         return u"<%s('%s:%s')>" % (self.__class__.__name__, self.repo_id,
                                    self.repo_name)
     @hybrid_property
     def locked(self):
         # always should return [user_id, timelocked]
         if self._locked:
             _lock_info = self._locked.split(':')
             return int(_lock_info[0]), _lock_info[1]
         return [None, None]
     @locked.setter
     def locked(self, val):
         if val and isinstance(val, (list, tuple)):
             self._locked = ':'.join(map(str, val))
         else:
             self._locked = None
     @classmethod
     def url_sep(cls):
         return URL_SEP
     @classmethod
     def get_by_repo_name(cls, repo_name):
         q = Session().query(cls).filter(cls.repo_name == repo_name)
         q = q.options(joinedload(Repository.fork))\
                 .options(joinedload(Repository.user))\
                 .options(joinedload(Repository.group))
         return q.scalar()
     @classmethod
     def get_by_full_path(cls, repo_full_path):
         repo_name = repo_full_path.split(cls.base_path(), 1)[-1]
         return cls.get_by_repo_name(repo_name.strip(URL_SEP))
     @classmethod
     def get_repo_forks(cls, repo_id):
         return cls.query().filter(Repository.fork_id == repo_id)
     @classmethod
     def base_path(cls):
         """
         Returns base path when all repos are stored
         :param cls:
         """
         q = Session().query(RhodeCodeUi)\
             .filter(RhodeCodeUi.ui_key == cls.url_sep())
         q = q.options(FromCache("sql_cache_short", "repository_repo_path"))
         return q.one().ui_value
     @property
     def forks(self):
         """
         Return forks of this repo
         """
         return Repository.get_repo_forks(self.repo_id)
     @property
     def parent(self):
         """
         Returns fork parent
         """
         return self.fork
     @property
     def just_name(self):
         return self.repo_name.split(Repository.url_sep())[-1]
     @property
     def groups_with_parents(self):
         groups = []
         if self.group is None:
             return groups
         cur_gr = self.group
         groups.insert(0, cur_gr)
         while 1:
             gr = getattr(cur_gr, 'parent_group', None)
             cur_gr = cur_gr.parent_group
             if gr is None:
                 break
             groups.insert(0, gr)
         return groups
     @property
     def groups_and_repo(self):
         return self.groups_with_parents, self.just_name
     @LazyProperty
     def repo_path(self):
         """
         Returns base full path for that repository means where it actually
         exists on a filesystem
         """
         q = Session().query(RhodeCodeUi).filter(RhodeCodeUi.ui_key ==
                                               Repository.url_sep())
         q = q.options(FromCache("sql_cache_short", "repository_repo_path"))
         return q.one().ui_value
     @property
     def repo_full_path(self):
         p = [self.repo_path]
         # we need to split the name by / since this is how we store the
         # names in the database, but that eventually needs to be converted
         # into a valid system path
         p += self.repo_name.split(Repository.url_sep())
         return os.path.join(*p)
     @property
     def cache_keys(self):
         """
         Returns associated cache keys for that repo
         """
         return CacheInvalidation.query()\
             .filter(CacheInvalidation.cache_args == self.repo_name)\
             .order_by(CacheInvalidation.cache_key)\
             .all()
     def get_new_name(self, repo_name):
         """
         returns new full repository name based on assigned group and new new
         :param group_name:
         """
         path_prefix = self.group.full_path_splitted if self.group else []
         return Repository.url_sep().join(path_prefix + [repo_name])
     @property
     def _ui(self):
         """
         Creates an db based ui object for this repository
         """
         from rhodecode.lib.utils import make_ui
         return make_ui('db', clear_session=False)
     @classmethod
     def inject_ui(cls, repo, extras={}):
         from rhodecode.lib.vcs.backends.hg import MercurialRepository
         from rhodecode.lib.vcs.backends.git import GitRepository
         required = (MercurialRepository, GitRepository)
         if not isinstance(repo, required):
             raise Exception('repo must be instance of %s' % required)
         # inject ui extra param to log this action via push logger
         for k, v in extras.items():
             repo._repo.ui.setconfig('rhodecode_extras', k, v)
     @classmethod
     def is_valid(cls, repo_name):
         """
         returns True if given repo name is a valid filesystem repository
         :param cls:
         :param repo_name:
         """
         from rhodecode.lib.utils import is_valid_repo
         return is_valid_repo(repo_name, cls.base_path())
     def get_api_data(self):
         """
         Common function for generating repo api data
         """
         repo = self
         data = dict(
             repo_id=repo.repo_id,
             repo_name=repo.repo_name,
             repo_type=repo.repo_type,
             clone_uri=repo.clone_uri,
             private=repo.private,
             created_on=repo.created_on,
             description=repo.description,
             landing_rev=repo.landing_rev,
             owner=repo.user.username,
             fork_of=repo.fork.repo_name if repo.fork else None
             fork_of=repo.fork.repo_name if repo.fork else None,
             enable_statistics=repo.enable_statistics,
             enable_locking=repo.enable_locking,
             enable_downloads=repo.enable_downloads
+        )
         return data
     @classmethod
     def lock(cls, repo, user_id):
         repo.locked = [user_id, time.time()]
         Session().add(repo)
         Session().commit()
     @classmethod
     def unlock(cls, repo):
         repo.locked = None
         Session().add(repo)
         Session().commit()
     @property
     def last_db_change(self):
         return self.updated_on
     #==========================================================================
     # SCM PROPERTIES
     #==========================================================================
     def get_changeset(self, rev=None):
         return get_changeset_safe(self.scm_instance, rev)
     def get_landing_changeset(self):
         """
         Returns landing changeset, or if that doesn't exist returns the tip
         """
         cs = self.get_changeset(self.landing_rev) or self.get_changeset()
         return cs
     def update_last_change(self, last_change=None):
         if last_change is None:
             last_change = datetime.datetime.now()
         if self.updated_on is None or self.updated_on != last_change:
             log.debug('updated repo %s with new date %s' % (self, last_change))
             self.updated_on = last_change
             Session().add(self)
             Session().commit()
     @property
     def tip(self):
         return self.get_changeset('tip')
     @property
     def author(self):
         return self.tip.author
     @property
     def last_change(self):
         return self.scm_instance.last_change
     def get_comments(self, revisions=None):
         """
         Returns comments for this repository grouped by revisions
         :param revisions: filter query by revisions only
         """
         cmts = ChangesetComment.query()\
             .filter(ChangesetComment.repo == self)
         if revisions:
             cmts = cmts.filter(ChangesetComment.revision.in_(revisions))
         grouped = defaultdict(list)
         for cmt in cmts.all():
             grouped[cmt.revision].append(cmt)
         return grouped
     def statuses(self, revisions=None):
         """
         Returns statuses for this repository
         :param revisions: list of revisions to get statuses for
         :type revisions: list
         """
         statuses = ChangesetStatus.query()\
             .filter(ChangesetStatus.repo == self)\
             .filter(ChangesetStatus.version == 0)
         if revisions:
             statuses = statuses.filter(ChangesetStatus.revision.in_(revisions))
         grouped = {}
         #maybe we have open new pullrequest without a status ?
         stat = ChangesetStatus.STATUS_UNDER_REVIEW
         status_lbl = ChangesetStatus.get_status_lbl(stat)
         for pr in PullRequest.query().filter(PullRequest.org_repo == self).all():
             for rev in pr.revisions:
                 pr_id = pr.pull_request_id
                 pr_repo = pr.other_repo.repo_name
                 grouped[rev] = [stat, status_lbl, pr_id, pr_repo]
         for stat in statuses.all():
             pr_id = pr_repo = None
             if stat.pull_request:
                 pr_id = stat.pull_request.pull_request_id
                 pr_repo = stat.pull_request.other_repo.repo_name
             grouped[stat.revision] = [str(stat.status), stat.status_lbl,
                                       pr_id, pr_repo]
         return grouped
     #==========================================================================
     # SCM CACHE INSTANCE
     #==========================================================================
     @property
     def invalidate(self):
         return CacheInvalidation.invalidate(self.repo_name)
     def set_invalidate(self):
         """
         set a cache for invalidation for this instance
         """
         CacheInvalidation.set_invalidate(repo_name=self.repo_name)
     @LazyProperty
     def scm_instance(self):
         import rhodecode
         full_cache = str2bool(rhodecode.CONFIG.get('vcs_full_cache'))
         if full_cache:
             return self.scm_instance_cached()
         return self.__get_instance()
     def scm_instance_cached(self, cache_map=None):
         @cache_region('long_term')
         def _c(repo_name):
             return self.__get_instance()
         rn = self.repo_name
         log.debug('Getting cached instance of repo')
         if cache_map:
             # get using prefilled cache_map
             invalidate_repo = cache_map[self.repo_name]
             if invalidate_repo:
                 invalidate_repo = (None if invalidate_repo.cache_active
                                    else invalidate_repo)
         else:
             # get from invalidate
             invalidate_repo = self.invalidate
         if invalidate_repo is not None:
             region_invalidate(_c, None, rn)
             # update our cache
             CacheInvalidation.set_valid(invalidate_repo.cache_key)
         return _c(rn)
     def __get_instance(self):
         repo_full_path = self.repo_full_path
         try:
             alias = get_scm(repo_full_path)[0]
             log.debug('Creating instance of %s repository' % alias)
             backend = get_backend(alias)
         except VCSError:
             log.error(traceback.format_exc())
             log.error('Perhaps this repository is in db and not in '
                       'filesystem run rescan repositories with '
                       '"destroy old data " option from admin panel')
             return
         if alias == 'hg':
             repo = backend(safe_str(repo_full_path), create=False,
                            baseui=self._ui)
             # skip hidden web repository
             if repo._get_hidden():
                 return
         else:
             repo = backend(repo_full_path, create=False)
         return repo
 class RepoGroup(Base, BaseModel):
     __tablename__ = 'groups'
     __table_args__ = (
         UniqueConstraint('group_name', 'group_parent_id'),
         CheckConstraint('group_id != group_parent_id'),
         {'extend_existing': True, 'mysql_engine': 'InnoDB',
          'mysql_charset': 'utf8'},
+    )
     __mapper_args__ = {'order_by': 'group_name'}
     group_id = Column("group_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     group_name = Column("group_name", String(255, convert_unicode=False, assert_unicode=None), nullable=False, unique=True, default=None)
     group_parent_id = Column("group_parent_id", Integer(), ForeignKey('groups.group_id'), nullable=True, unique=None, default=None)
     group_description = Column("group_description", String(10000, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     enable_locking = Column("enable_locking", Boolean(), nullable=False, unique=None, default=False)
     repo_group_to_perm = relationship('UserRepoGroupToPerm', cascade='all', order_by='UserRepoGroupToPerm.group_to_perm_id')
     users_group_to_perm = relationship('UsersGroupRepoGroupToPerm', cascade='all')
     parent_group = relationship('RepoGroup', remote_side=group_id)
     def __init__(self, group_name='', parent_group=None):
         self.group_name = group_name
         self.parent_group = parent_group
     def __unicode__(self):
         return u"<%s('%s:%s')>" % (self.__class__.__name__, self.group_id,
                                   self.group_name)
     @classmethod
     def groups_choices(cls, check_perms=False):
         from webhelpers.html import literal as _literal
         from rhodecode.model.scm import ScmModel
         groups = cls.query().all()
         if check_perms:
             #filter group user have access to, it's done
             #magically inside ScmModel based on current user
             groups = ScmModel().get_repos_groups(groups)
         repo_groups = [('', '')]
         sep = ' &raquo; '
         _name = lambda k: _literal(sep.join(k))
         repo_groups.extend([(x.group_id, _name(x.full_path_splitted))
                               for x in groups])
         repo_groups = sorted(repo_groups, key=lambda t: t[1].split(sep)[0])
         return repo_groups
     @classmethod
     def url_sep(cls):
         return URL_SEP
     @classmethod
     def get_by_group_name(cls, group_name, cache=False, case_insensitive=False):
         if case_insensitive:
             gr = cls.query()\
                 .filter(cls.group_name.ilike(group_name))
         else:
             gr = cls.query()\
                 .filter(cls.group_name == group_name)
         if cache:
             gr = gr.options(FromCache(
                             "sql_cache_short",
                             "get_group_%s" % _hash_key(group_name)
+                            )
+            )
         return gr.scalar()
     @property
     def parents(self):
         parents_recursion_limit = 5
         groups = []
         if self.parent_group is None:
             return groups
         cur_gr = self.parent_group
         groups.insert(0, cur_gr)
         cnt = 0
         while 1:
             cnt += 1
             gr = getattr(cur_gr, 'parent_group', None)
             cur_gr = cur_gr.parent_group
             if gr is None:
                 break
             if cnt == parents_recursion_limit:
                 # this will prevent accidental infinit loops
                 log.error('group nested more than %s' %
                           parents_recursion_limit)
                 break
             groups.insert(0, gr)
         return groups
     @property
     def children(self):
         return RepoGroup.query().filter(RepoGroup.parent_group == self)
     @property
     def name(self):
         return self.group_name.split(RepoGroup.url_sep())[-1]
     @property
     def full_path(self):
         return self.group_name
     @property
     def full_path_splitted(self):
         return self.group_name.split(RepoGroup.url_sep())
     @property
     def repositories(self):
         return Repository.query()\
                 .filter(Repository.group == self)\
                 .order_by(Repository.repo_name)
     @property
     def repositories_recursive_count(self):
         cnt = self.repositories.count()
         def children_count(group):
             cnt = 0
             for child in group.children:
                 cnt += child.repositories.count()
                 cnt += children_count(child)
             return cnt
         return cnt + children_count(self)
     def recursive_groups_and_repos(self):
         """
         Recursive return all groups, with repositories in those groups
         """
         all_ = []
         def _get_members(root_gr):
             for r in root_gr.repositories:
                 all_.append(r)
             childs = root_gr.children.all()
             if childs:
                 for gr in childs:
                     all_.append(gr)
                     _get_members(gr)
         _get_members(self)
         return [self] + all_
     def get_new_name(self, group_name):
         """
         returns new full group name based on parent and new name
         :param group_name:
         """
         path_prefix = (self.parent_group.full_path_splitted if
                        self.parent_group else [])
         return RepoGroup.url_sep().join(path_prefix + [group_name])
 class Permission(Base, BaseModel):
     __tablename__ = 'permissions'
     __table_args__ = (
         Index('p_perm_name_idx', 'permission_name'),
         {'extend_existing': True, 'mysql_engine': 'InnoDB',
          'mysql_charset': 'utf8'},
+    )
     PERMS = [
         ('repository.none', _('Repository no access')),
         ('repository.read', _('Repository read access')),
         ('repository.write', _('Repository write access')),
         ('repository.admin', _('Repository admin access')),
         ('group.none', _('Repositories Group no access')),
         ('group.read', _('Repositories Group read access')),
         ('group.write', _('Repositories Group write access')),
         ('group.admin', _('Repositories Group admin access')),
         ('hg.admin', _('RhodeCode Administrator')),
         ('hg.create.none', _('Repository creation disabled')),
         ('hg.create.repository', _('Repository creation enabled')),
         ('hg.fork.none', _('Repository forking disabled')),
         ('hg.fork.repository', _('Repository forking enabled')),
         ('hg.register.none', _('Register disabled')),
         ('hg.register.manual_activate', _('Register new user with RhodeCode '
                                           'with manual activation')),
         ('hg.register.auto_activate', _('Register new user with RhodeCode '
                                         'with auto activation')),
+    ]
     # defines which permissions are more important higher the more important
     PERM_WEIGHTS = {
         'repository.none': 0,
         'repository.read': 1,
         'repository.write': 3,
         'repository.admin': 4,
         'group.none': 0,
         'group.read': 1,
         'group.write': 3,
         'group.admin': 4,
         'hg.fork.none': 0,
         'hg.fork.repository': 1,
         'hg.create.none': 0,
         'hg.create.repository':1
+    }
     permission_id = Column("permission_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     permission_name = Column("permission_name", String(255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     permission_longname = Column("permission_longname", String(255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     def __unicode__(self):
         return u"<%s('%s:%s')>" % (
             self.__class__.__name__, self.permission_id, self.permission_name
+        )
     @classmethod
     def get_by_key(cls, key):
         return cls.query().filter(cls.permission_name == key).scalar()
     @classmethod
     def get_default_perms(cls, default_user_id):
         q = Session().query(UserRepoToPerm, Repository, cls)\
          .join((Repository, UserRepoToPerm.repository_id == Repository.repo_id))\
          .join((cls, UserRepoToPerm.permission_id == cls.permission_id))\
          .filter(UserRepoToPerm.user_id == default_user_id)
         return q.all()
     @classmethod
     def get_default_group_perms(cls, default_user_id):
         q = Session().query(UserRepoGroupToPerm, RepoGroup, cls)\
          .join((RepoGroup, UserRepoGroupToPerm.group_id == RepoGroup.group_id))\
          .join((cls, UserRepoGroupToPerm.permission_id == cls.permission_id))\
          .filter(UserRepoGroupToPerm.user_id == default_user_id)
         return q.all()
 class UserRepoToPerm(Base, BaseModel):
     __tablename__ = 'repo_to_perm'
     __table_args__ = (
         UniqueConstraint('user_id', 'repository_id', 'permission_id'),
         {'extend_existing': True, 'mysql_engine': 'InnoDB',
          'mysql_charset': 'utf8'}
+    )
     repo_to_perm_id = Column("repo_to_perm_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     user_id = Column("user_id", Integer(), ForeignKey('users.user_id'), nullable=False, unique=None, default=None)
     permission_id = Column("permission_id", Integer(), ForeignKey('permissions.permission_id'), nullable=False, unique=None, default=None)
     repository_id = Column("repository_id", Integer(), ForeignKey('repositories.repo_id'), nullable=False, unique=None, default=None)
     user = relationship('User')
     repository = relationship('Repository')
     permission = relationship('Permission')
     @classmethod
     def create(cls, user, repository, permission):
         n = cls()
         n.user = user
         n.repository = repository
         n.permission = permission
         Session().add(n)
         return n
     def __unicode__(self):
         return u'<user:%s => %s >' % (self.user, self.repository)
 class UserToPerm(Base, BaseModel):
     __tablename__ = 'user_to_perm'
     __table_args__ = (
         UniqueConstraint('user_id', 'permission_id'),
         {'extend_existing': True, 'mysql_engine': 'InnoDB',
          'mysql_charset': 'utf8'}
+    )
     user_to_perm_id = Column("user_to_perm_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     user_id = Column("user_id", Integer(), ForeignKey('users.user_id'), nullable=False, unique=None, default=None)
     permission_id = Column("permission_id", Integer(), ForeignKey('permissions.permission_id'), nullable=False, unique=None, default=None)
     user = relationship('User')
     permission = relationship('Permission', lazy='joined')
 class UsersGroupRepoToPerm(Base, BaseModel):
     __tablename__ = 'users_group_repo_to_perm'
     __table_args__ = (
         UniqueConstraint('repository_id', 'users_group_id', 'permission_id'),
         {'extend_existing': True, 'mysql_engine': 'InnoDB',
          'mysql_charset': 'utf8'}
+    )
     users_group_to_perm_id = Column("users_group_to_perm_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     users_group_id = Column("users_group_id", Integer(), ForeignKey('users_groups.users_group_id'), nullable=False, unique=None, default=None)
     permission_id = Column("permission_id", Integer(), ForeignKey('permissions.permission_id'), nullable=False, unique=None, default=None)
     repository_id = Column("repository_id", Integer(), ForeignKey('repositories.repo_id'), nullable=False, unique=None, default=None)
     users_group = relationship('UsersGroup')
     permission = relationship('Permission')
     repository = relationship('Repository')
     @classmethod
     def create(cls, users_group, repository, permission):
         n = cls()
         n.users_group = users_group
         n.repository = repository
         n.permission = permission
         Session().add(n)
         return n
     def __unicode__(self):
         return u'<userGroup:%s => %s >' % (self.users_group, self.repository)
 class UsersGroupToPerm(Base, BaseModel):
     __tablename__ = 'users_group_to_perm'
     __table_args__ = (
         UniqueConstraint('users_group_id', 'permission_id',),
         {'extend_existing': True, 'mysql_engine': 'InnoDB',
          'mysql_charset': 'utf8'}
+    )
     users_group_to_perm_id = Column("users_group_to_perm_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     users_group_id = Column("users_group_id", Integer(), ForeignKey('users_groups.users_group_id'), nullable=False, unique=None, default=None)
     permission_id = Column("permission_id", Integer(), ForeignKey('permissions.permission_id'), nullable=False, unique=None, default=None)
     users_group = relationship('UsersGroup')
     permission = relationship('Permission')
 class UserRepoGroupToPerm(Base, BaseModel):
     __tablename__ = 'user_repo_group_to_perm'
     __table_args__ = (
         UniqueConstraint('user_id', 'group_id', 'permission_id'),
         {'extend_existing': True, 'mysql_engine': 'InnoDB',
          'mysql_charset': 'utf8'}
+    )
     group_to_perm_id = Column("group_to_perm_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     user_id = Column("user_id", Integer(), ForeignKey('users.user_id'), nullable=False, unique=None, default=None)
     group_id = Column("group_id", Integer(), ForeignKey('groups.group_id'), nullable=False, unique=None, default=None)
     permission_id = Column("permission_id", Integer(), ForeignKey('permissions.permission_id'), nullable=False, unique=None, default=None)
     user = relationship('User')
     group = relationship('RepoGroup')
     permission = relationship('Permission')
 class UsersGroupRepoGroupToPerm(Base, BaseModel):
     __tablename__ = 'users_group_repo_group_to_perm'
     __table_args__ = (
         UniqueConstraint('users_group_id', 'group_id'),
         {'extend_existing': True, 'mysql_engine': 'InnoDB',
          'mysql_charset': 'utf8'}
+    )
     users_group_repo_group_to_perm_id = Column("users_group_repo_group_to_perm_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     users_group_id = Column("users_group_id", Integer(), ForeignKey('users_groups.users_group_id'), nullable=False, unique=None, default=None)
     group_id = Column("group_id", Integer(), ForeignKey('groups.group_id'), nullable=False, unique=None, default=None)
     permission_id = Column("permission_id", Integer(), ForeignKey('permissions.permission_id'), nullable=False, unique=None, default=None)
     users_group = relationship('UsersGroup')
     permission = relationship('Permission')
     group = relationship('RepoGroup')
 class Statistics(Base, BaseModel):
     __tablename__ = 'statistics'
     __table_args__ = (
          UniqueConstraint('repository_id'),
          {'extend_existing': True, 'mysql_engine': 'InnoDB',
           'mysql_charset': 'utf8'}
+    )
     stat_id = Column("stat_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     repository_id = Column("repository_id", Integer(), ForeignKey('repositories.repo_id'), nullable=False, unique=True, default=None)
     stat_on_revision = Column("stat_on_revision", Integer(), nullable=False)
     commit_activity = Column("commit_activity", LargeBinary(1000000), nullable=False)#JSON data
     commit_activity_combined = Column("commit_activity_combined", LargeBinary(), nullable=False)#JSON data
     languages = Column("languages", LargeBinary(1000000), nullable=False)#JSON data
     repository = relationship('Repository', single_parent=True)
 class UserFollowing(Base, BaseModel):
     __tablename__ = 'user_followings'
     __table_args__ = (
         UniqueConstraint('user_id', 'follows_repository_id'),
         UniqueConstraint('user_id', 'follows_user_id'),
         {'extend_existing': True, 'mysql_engine': 'InnoDB',
          'mysql_charset': 'utf8'}
+    )
     user_following_id = Column("user_following_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     user_id = Column("user_id", Integer(), ForeignKey('users.user_id'), nullable=False, unique=None, default=None)
     follows_repo_id = Column("follows_repository_id", Integer(), ForeignKey('repositories.repo_id'), nullable=True, unique=None, default=None)
     follows_user_id = Column("follows_user_id", Integer(), ForeignKey('users.user_id'), nullable=True, unique=None, default=None)
     follows_from = Column('follows_from', DateTime(timezone=False), nullable=True, unique=None, default=datetime.datetime.now)
     user = relationship('User', primaryjoin='User.user_id==UserFollowing.user_id')
     follows_user = relationship('User', primaryjoin='User.user_id==UserFollowing.follows_user_id')
     follows_repository = relationship('Repository', order_by='Repository.repo_name')
     @classmethod
     def get_repo_followers(cls, repo_id):
         return cls.query().filter(cls.follows_repo_id == repo_id)
 class CacheInvalidation(Base, BaseModel):
     __tablename__ = 'cache_invalidation'
     __table_args__ = (
         UniqueConstraint('cache_key'),
         Index('key_idx', 'cache_key'),
         {'extend_existing': True, 'mysql_engine': 'InnoDB',
          'mysql_charset': 'utf8'},
+    )
     cache_id = Column("cache_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     cache_key = Column("cache_key", String(255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     cache_args = Column("cache_args", String(255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     cache_active = Column("cache_active", Boolean(), nullable=True, unique=None, default=False)
     def __init__(self, cache_key, cache_args=''):
         self.cache_key = cache_key
         self.cache_args = cache_args
         self.cache_active = False
     def __unicode__(self):
         return u"<%s('%s:%s')>" % (self.__class__.__name__,
                                   self.cache_id, self.cache_key)
     @property
     def prefix(self):
         _split = self.cache_key.split(self.cache_args, 1)
         if _split and len(_split) == 2:
             return _split[0]
         return ''
     @classmethod
     def clear_cache(cls):
         cls.query().delete()
     @classmethod
     def _get_key(cls, key):
         """
         Wrapper for generating a key, together with a prefix
         :param key:
         """
         import rhodecode
         prefix = ''
         org_key = key
         iid = rhodecode.CONFIG.get('instance_id')
         if iid:
             prefix = iid
         return "%s%s" % (prefix, key), prefix, org_key
     @classmethod
     def get_by_key(cls, key):
         return cls.query().filter(cls.cache_key == key).scalar()
     @classmethod
     def get_by_repo_name(cls, repo_name):
         return cls.query().filter(cls.cache_args == repo_name).all()
     @classmethod
     def _get_or_create_key(cls, key, repo_name, commit=True):
         inv_obj = Session().query(cls).filter(cls.cache_key == key).scalar()
         if not inv_obj:
             try:
                 inv_obj = CacheInvalidation(key, repo_name)
                 Session().add(inv_obj)
                 if commit:
                     Session().commit()
             except Exception:
                 log.error(traceback.format_exc())
                 Session().rollback()
         return inv_obj
     @classmethod
     def invalidate(cls, key):
         """
         Returns Invalidation object if this given key should be invalidated
         None otherwise. `cache_active = False` means that this cache
         state is not valid and needs to be invalidated
         :param key:
         """
         repo_name = key
         repo_name = remove_suffix(repo_name, '_README')
         repo_name = remove_suffix(repo_name, '_RSS')
         repo_name = remove_suffix(repo_name, '_ATOM')
         # adds instance prefix
         key, _prefix, _org_key = cls._get_key(key)
         inv = cls._get_or_create_key(key, repo_name)
         if inv and inv.cache_active is False:
             return inv
     @classmethod
     def set_invalidate(cls, key=None, repo_name=None):
         """
         Mark this Cache key for invalidation, either by key or whole
         cache sets based on repo_name
         :param key:
         """
         if key:
             key, _prefix, _org_key = cls._get_key(key)
             inv_objs = Session().query(cls).filter(cls.cache_key == key).all()
         elif repo_name:
             inv_objs = Session().query(cls).filter(cls.cache_args == repo_name).all()
         log.debug('marking %s key[s] for invalidation based on key=%s,repo_name=%s'
                   % (len(inv_objs), key, repo_name))
         try:
             for inv_obj in inv_objs:
                 inv_obj.cache_active = False
                 Session().add(inv_obj)
             Session().commit()
         except Exception:
             log.error(traceback.format_exc())
             Session().rollback()
     @classmethod
     def set_valid(cls, key):
         """
         Mark this cache key as active and currently cached
         :param key:
         """
         inv_obj = cls.get_by_key(key)
         inv_obj.cache_active = True
         Session().add(inv_obj)
         Session().commit()
     @classmethod
     def get_cache_map(cls):
         class cachemapdict(dict):
             def __init__(self, *args, **kwargs):
                 fixkey = kwargs.get('fixkey')
                 if fixkey:
                     del kwargs['fixkey']
                 self.fixkey = fixkey
                 super(cachemapdict, self).__init__(*args, **kwargs)
             def __getattr__(self, name):
                 key = name
                 if self.fixkey:
                     key, _prefix, _org_key = cls._get_key(key)
                 if key in self.__dict__:
                     return self.__dict__[key]
                 else:
                     return self[key]
             def __getitem__(self, key):
                 if self.fixkey:
                     key, _prefix, _org_key = cls._get_key(key)
                 try:
                     return super(cachemapdict, self).__getitem__(key)
                 except KeyError:
                     return
         cache_map = cachemapdict(fixkey=True)
         for obj in cls.query().all():
             cache_map[obj.cache_key] = cachemapdict(obj.get_dict())
         return cache_map
 class ChangesetComment(Base, BaseModel):
     __tablename__ = 'changeset_comments'
     __table_args__ = (
         Index('cc_revision_idx', 'revision'),
         {'extend_existing': True, 'mysql_engine': 'InnoDB',
          'mysql_charset': 'utf8'},
+    )
     comment_id = Column('comment_id', Integer(), nullable=False, primary_key=True)
     repo_id = Column('repo_id', Integer(), ForeignKey('repositories.repo_id'), nullable=False)
     revision = Column('revision', String(40), nullable=True)
     pull_request_id = Column("pull_request_id", Integer(), ForeignKey('pull_requests.pull_request_id'), nullable=True)
     line_no = Column('line_no', Unicode(10), nullable=True)
     hl_lines = Column('hl_lines', Unicode(512), nullable=True)
     f_path = Column('f_path', Unicode(1000), nullable=True)
     user_id = Column('user_id', Integer(), ForeignKey('users.user_id'), nullable=False)
     text = Column('text', UnicodeText(25000), nullable=False)
     created_on = Column('created_on', DateTime(timezone=False), nullable=False, default=datetime.datetime.now)
     modified_at = Column('modified_at', DateTime(timezone=False), nullable=False, default=datetime.datetime.now)
     author = relationship('User', lazy='joined')
     repo = relationship('Repository')
     status_change = relationship('ChangesetStatus', cascade="all, delete, delete-orphan")
     pull_request = relationship('PullRequest', lazy='joined')
     @classmethod
     def get_users(cls, revision=None, pull_request_id=None):
         """

rhodecode/model/repo.py

➞

Show inline comments

 # -*- coding: utf-8 -*-
 """
     rhodecode.model.repo
     ~~~~~~~~~~~~~~~~~~~~
     Repository model for rhodecode
     :created_on: Jun 5, 2010
     :author: marcink
     :copyright: (C) 2010-2012 Marcin Kuzminski <marcin@python-works.com>
     :license: GPLv3, see COPYING for more details.
 """
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation, either version 3 of the License, or
 # (at your option) any later version.
+#
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
+#
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 from __future__ import with_statement
 import os
 import shutil
 import logging
 import traceback
 from datetime import datetime
 from rhodecode.lib.vcs.backends import get_backend
 from rhodecode.lib.compat import json
 from rhodecode.lib.utils2 import LazyProperty, safe_str, safe_unicode,\
     remove_prefix
 from rhodecode.lib.caching_query import FromCache
 from rhodecode.lib.hooks import log_create_repository, log_delete_repository
 from rhodecode.model import BaseModel
 from rhodecode.model.db import Repository, UserRepoToPerm, User, Permission, \
     Statistics, UsersGroup, UsersGroupRepoToPerm, RhodeCodeUi, RepoGroup,\
     RhodeCodeSetting
 from rhodecode.lib import helpers as h
 log = logging.getLogger(__name__)
 class RepoModel(BaseModel):
     cls = Repository
     URL_SEPARATOR = Repository.url_sep()
     def __get_users_group(self, users_group):
         return self._get_instance(UsersGroup, users_group,
                                   callback=UsersGroup.get_by_group_name)
     def _get_repos_group(self, repos_group):
         return self._get_instance(RepoGroup, repos_group,
                                   callback=RepoGroup.get_by_group_name)
     @LazyProperty
     def repos_path(self):
         """
         Get's the repositories root path from database
         """
         q = self.sa.query(RhodeCodeUi).filter(RhodeCodeUi.ui_key == '/').one()
         return q.ui_value
     def get(self, repo_id, cache=False):
         repo = self.sa.query(Repository)\
             .filter(Repository.repo_id == repo_id)
         if cache:
             repo = repo.options(FromCache("sql_cache_short",
                                           "get_repo_%s" % repo_id))
         return repo.scalar()
     def get_repo(self, repository):
         return self._get_repo(repository)
     def get_by_repo_name(self, repo_name, cache=False):
         repo = self.sa.query(Repository)\
             .filter(Repository.repo_name == repo_name)
         if cache:
             repo = repo.options(FromCache("sql_cache_short",
                                           "get_repo_%s" % repo_name))
         return repo.scalar()
     def get_users_js(self):
         users = self.sa.query(User).filter(User.active == True).all()
         return json.dumps([
+            {
              'id': u.user_id,
              'fname': u.name,
              'lname': u.lastname,
              'nname': u.username,
              'gravatar_lnk': h.gravatar_url(u.email, 14)
             } for u in users]
+        )
     def get_users_groups_js(self):
         users_groups = self.sa.query(UsersGroup)\
             .filter(UsersGroup.users_group_active == True).all()
         return json.dumps([
+            {
              'id': gr.users_group_id,
              'grname': gr.users_group_name,
              'grmembers': len(gr.members),
             } for gr in users_groups]
+        )
     def _get_defaults(self, repo_name):
         """
         Get's information about repository, and returns a dict for
         usage in forms
         :param repo_name:
         """
         repo_info = Repository.get_by_repo_name(repo_name)
         if repo_info is None:
             return None
         defaults = repo_info.get_dict()
         group, repo_name = repo_info.groups_and_repo
         defaults['repo_name'] = repo_name
         defaults['repo_group'] = getattr(group[-1] if group else None,
                                          'group_id', None)
         for strip, k in [(0, 'repo_type'), (1, 'repo_enable_downloads'),
                   (1, 'repo_description'), (1, 'repo_enable_locking'),
                   (1, 'repo_landing_rev'), (0, 'clone_uri'),
                   (1, 'repo_private'), (1, 'repo_enable_statistics')]:
             attr = k
             if strip:
                 attr = remove_prefix(k, 'repo_')
             defaults[k] = defaults[attr]
         # fill owner
         if repo_info.user:
             defaults.update({'user': repo_info.user.username})
         else:
             replacement_user = User.query().filter(User.admin ==
                                                    True).first().username
             defaults.update({'user': replacement_user})
         # fill repository users
         for p in repo_info.repo_to_perm:
             defaults.update({'u_perm_%s' % p.user.username:
                              p.permission.permission_name})
         # fill repository groups
         for p in repo_info.users_group_to_perm:
             defaults.update({'g_perm_%s' % p.users_group.users_group_name:
                              p.permission.permission_name})
         return defaults
     def update(self, org_repo_name, **kwargs):
         try:
             cur_repo = self.get_by_repo_name(org_repo_name, cache=False)
             # update permissions
             for member, perm, member_type in kwargs['perms_updates']:
                 if member_type == 'user':
                     # this updates existing one
                     RepoModel().grant_user_permission(
                         repo=cur_repo, user=member, perm=perm
+                    )
                 else:
                     RepoModel().grant_users_group_permission(
                         repo=cur_repo, group_name=member, perm=perm
+                    )
             # set new permissions
             for member, perm, member_type in kwargs['perms_new']:
                 if member_type == 'user':
                     RepoModel().grant_user_permission(
                         repo=cur_repo, user=member, perm=perm
+                    )
                 else:
                     RepoModel().grant_users_group_permission(
                         repo=cur_repo, group_name=member, perm=perm
+                    )
             if 'user' in kwargs:
                 cur_repo.user = User.get_by_username(kwargs['user'])
             if 'repo_group' in kwargs:
                 cur_repo.group = RepoGroup.get(kwargs['repo_group'])
             for strip, k in [(0, 'repo_type'), (1, 'repo_enable_downloads'),
                       (1, 'repo_description'), (1, 'repo_enable_locking'),
                       (1, 'repo_landing_rev'), (0, 'clone_uri'),
                       (1, 'repo_private'), (1, 'repo_enable_statistics')]:
                 if k in kwargs:
                     val = kwargs[k]
                     if strip:
                         k = remove_prefix(k, 'repo_')
                     setattr(cur_repo, k, val)
             new_name = cur_repo.get_new_name(kwargs['repo_name'])
             cur_repo.repo_name = new_name
             self.sa.add(cur_repo)
             if org_repo_name != new_name:
                 # rename repository
                 self.__rename_repo(old=org_repo_name, new=new_name)
             return cur_repo
         except:
             log.error(traceback.format_exc())
             raise
     def create_repo(self, repo_name, repo_type, description, owner,
                     private=False, clone_uri=None, repos_group=None,
                     landing_rev='tip', just_db=False, fork_of=None,
                     copy_fork_permissions=False, enable_statistics=False,
                     enable_locking=False, enable_downloads=False):
         """
         Create repository
         """
         from rhodecode.model.scm import ScmModel
         owner = self._get_user(owner)
         fork_of = self._get_repo(fork_of)
         repos_group = self._get_repos_group(repos_group)
         try:
             # repo name is just a name of repository
             # while repo_name_full is a full qualified name that is combined
             # with name and path of group
             repo_name_full = repo_name
             repo_name = repo_name.split(self.URL_SEPARATOR)[-1]
             new_repo = Repository()
             new_repo.enable_statistics = False
             new_repo.repo_name = repo_name_full
             new_repo.repo_type = repo_type
             new_repo.user = owner
             new_repo.group = repos_group
             new_repo.description = description or repo_name
             new_repo.private = private
             new_repo.clone_uri = clone_uri
             new_repo.landing_rev = landing_rev
             new_repo.enable_statistics = enable_statistics
             new_repo.enable_locking = enable_locking
             new_repo.enable_downloads = enable_downloads
             if repos_group:
                 new_repo.enable_locking = repos_group.enable_locking
             if fork_of:
                 parent_repo = fork_of
                 new_repo.fork = parent_repo
             self.sa.add(new_repo)
             def _create_default_perms():
                 # create default permission
                 repo_to_perm = UserRepoToPerm()
                 default = 'repository.read'
                 for p in User.get_by_username('default').user_perms:
                     if p.permission.permission_name.startswith('repository.'):
                         default = p.permission.permission_name
                         break
                 default_perm = 'repository.none' if private else default
                 repo_to_perm.permission_id = self.sa.query(Permission)\
                         .filter(Permission.permission_name == default_perm)\
                         .one().permission_id
                 repo_to_perm.repository = new_repo
                 repo_to_perm.user_id = User.get_by_username('default').user_id
                 self.sa.add(repo_to_perm)
             if fork_of:
                 if copy_fork_permissions:
                     repo = fork_of
                     user_perms = UserRepoToPerm.query()\
                         .filter(UserRepoToPerm.repository == repo).all()
                     group_perms = UsersGroupRepoToPerm.query()\
                         .filter(UsersGroupRepoToPerm.repository == repo).all()
                     for perm in user_perms:
                         UserRepoToPerm.create(perm.user, new_repo,
                                               perm.permission)
                     for perm in group_perms:
                         UsersGroupRepoToPerm.create(perm.users_group, new_repo,
                                                     perm.permission)
                 else:
                     _create_default_perms()
             else:
                 _create_default_perms()
             if not just_db:
                 self.__create_repo(repo_name, repo_type,
                                    repos_group,
                                    clone_uri)
                 log_create_repository(new_repo.get_dict(),
                                       created_by=owner.username)
             # now automatically start following this repository as owner
             ScmModel(self.sa).toggle_following_repo(new_repo.repo_id,
                                                     owner.user_id)
             return new_repo
         except:
             log.error(traceback.format_exc())
             raise
     def create(self, form_data, cur_user, just_db=False, fork=None):
         """
         Backward compatibility function, just a wrapper on top of create_repo
         :param form_data:
         :param cur_user:
         :param just_db:
         :param fork:
         """
         owner = cur_user
         repo_name = form_data['repo_name_full']
         repo_type = form_data['repo_type']
         description = form_data['repo_description']
         private = form_data['repo_private']
         clone_uri = form_data.get('clone_uri')
         repos_group = form_data['repo_group']
         landing_rev = form_data['repo_landing_rev']
         copy_fork_permissions = form_data.get('copy_permissions')
         fork_of = form_data.get('fork_parent_id')
         ##defaults
+        ## repo creation defaults, private and repo_type are filled in form
         defs = RhodeCodeSetting.get_default_repo_settings(strip_prefix=True)
-        enable_statistics = defs.get('repo_enable_statistic')
+        enable_statistics = defs.get('repo_enable_statistics')
         enable_locking = defs.get('repo_enable_locking')
         enable_downloads = defs.get('repo_enable_downloads')
         return self.create_repo(
             repo_name, repo_type, description, owner, private, clone_uri,
             repos_group, landing_rev, just_db, fork_of, copy_fork_permissions,
             enable_statistics, enable_locking, enable_downloads
+        )
     def create_fork(self, form_data, cur_user):
         """
         Simple wrapper into executing celery task for fork creation
         :param form_data:
         :param cur_user:
         """
         from rhodecode.lib.celerylib import tasks, run_task
         run_task(tasks.create_repo_fork, form_data, cur_user)
     def delete(self, repo):
         repo = self._get_repo(repo)
         if repo:
             old_repo_dict = repo.get_dict()
             owner = repo.user
             try:
                 self.sa.delete(repo)
                 self.__delete_repo(repo)
                 log_delete_repository(old_repo_dict,
                                       deleted_by=owner.username)
             except:
                 log.error(traceback.format_exc())
                 raise
     def grant_user_permission(self, repo, user, perm):
         """
         Grant permission for user on given repository, or update existing one
         if found
         :param repo: Instance of Repository, repository_id, or repository name
         :param user: Instance of User, user_id or username
         :param perm: Instance of Permission, or permission_name
         """
         user = self._get_user(user)
         repo = self._get_repo(repo)
         permission = self._get_perm(perm)
         # check if we have that permission already
         obj = self.sa.query(UserRepoToPerm)\
             .filter(UserRepoToPerm.user == user)\
             .filter(UserRepoToPerm.repository == repo)\
             .scalar()
         if obj is None:
             # create new !
             obj = UserRepoToPerm()
         obj.repository = repo
         obj.user = user
         obj.permission = permission
         self.sa.add(obj)
         log.debug('Granted perm %s to %s on %s' % (perm, user, repo))
     def revoke_user_permission(self, repo, user):
         """
         Revoke permission for user on given repository
         :param repo: Instance of Repository, repository_id, or repository name
         :param user: Instance of User, user_id or username
         """
         user = self._get_user(user)
         repo = self._get_repo(repo)
         obj = self.sa.query(UserRepoToPerm)\
             .filter(UserRepoToPerm.repository == repo)\
             .filter(UserRepoToPerm.user == user)\
             .scalar()
         if obj:
             self.sa.delete(obj)
             log.debug('Revoked perm on %s on %s' % (repo, user))
     def grant_users_group_permission(self, repo, group_name, perm):
         """
         Grant permission for users group on given repository, or update
         existing one if found
         :param repo: Instance of Repository, repository_id, or repository name
         :param group_name: Instance of UserGroup, users_group_id,
             or users group name
         :param perm: Instance of Permission, or permission_name
         """
         repo = self._get_repo(repo)
         group_name = self.__get_users_group(group_name)
         permission = self._get_perm(perm)
         # check if we have that permission already
         obj = self.sa.query(UsersGroupRepoToPerm)\
             .filter(UsersGroupRepoToPerm.users_group == group_name)\
             .filter(UsersGroupRepoToPerm.repository == repo)\
             .scalar()
         if obj is None:
             # create new
             obj = UsersGroupRepoToPerm()
         obj.repository = repo
         obj.users_group = group_name
         obj.permission = permission
         self.sa.add(obj)
         log.debug('Granted perm %s to %s on %s' % (perm, group_name, repo))
     def revoke_users_group_permission(self, repo, group_name):
         """
         Revoke permission for users group on given repository
         :param repo: Instance of Repository, repository_id, or repository name
         :param group_name: Instance of UserGroup, users_group_id,
             or users group name
         """
         repo = self._get_repo(repo)
         group_name = self.__get_users_group(group_name)
         obj = self.sa.query(UsersGroupRepoToPerm)\
             .filter(UsersGroupRepoToPerm.repository == repo)\
             .filter(UsersGroupRepoToPerm.users_group == group_name)\
             .scalar()
         if obj:
             self.sa.delete(obj)
             log.debug('Revoked perm to %s on %s' % (repo, group_name))
     def delete_stats(self, repo_name):
         """
         removes stats for given repo
         :param repo_name:
         """
         try:
             obj = self.sa.query(Statistics)\
                     .filter(Statistics.repository ==
                             self.get_by_repo_name(repo_name))\
                     .one()
             self.sa.delete(obj)
         except:
             log.error(traceback.format_exc())
             raise
     def __create_repo(self, repo_name, alias, parent, clone_uri=False):
         """
         makes repository on filesystem. It's group aware means it'll create
         a repository within a group, and alter the paths accordingly of
         group location
         :param repo_name:
         :param alias:
         :param parent_id:
         :param clone_uri:
         """
         from rhodecode.lib.utils import is_valid_repo, is_valid_repos_group
         from rhodecode.model.scm import ScmModel
         if parent:
             new_parent_path = os.sep.join(parent.full_path_splitted)
         else:
             new_parent_path = ''
         # we need to make it str for mercurial
         repo_path = os.path.join(*map(lambda x: safe_str(x),
                                 [self.repos_path, new_parent_path, repo_name]))
         # check if this path is not a repository
         if is_valid_repo(repo_path, self.repos_path):
             raise Exception('This path %s is a valid repository' % repo_path)
         # check if this path is a group
         if is_valid_repos_group(repo_path, self.repos_path):
             raise Exception('This path %s is a valid group' % repo_path)
         log.info('creating repo %s in %s @ %s' % (
                      repo_name, safe_unicode(repo_path), clone_uri
+                )
+        )
         backend = get_backend(alias)
         if alias == 'hg':
             backend(repo_path, create=True, src_url=clone_uri)
         elif alias == 'git':
             r = backend(repo_path, create=True, src_url=clone_uri, bare=True)
             # add rhodecode hook into this repo
             ScmModel().install_git_hook(repo=r)
         else:
             raise Exception('Undefined alias %s' % alias)
     def __rename_repo(self, old, new):
         """
         renames repository on filesystem
         :param old: old name
         :param new: new name
         """
         log.info('renaming repo from %s to %s' % (old, new))
         old_path = os.path.join(self.repos_path, old)
         new_path = os.path.join(self.repos_path, new)
         if os.path.isdir(new_path):
             raise Exception(
                 'Was trying to rename to already existing dir %s' % new_path
+            )
         shutil.move(old_path, new_path)
     def __delete_repo(self, repo):
         """
         removes repo from filesystem, the removal is acctually made by
         added rm__ prefix into dir, and rename internat .hg/.git dirs so this
         repository is no longer valid for rhodecode, can be undeleted later on
         by reverting the renames on this repository
         :param repo: repo object
         """
         rm_path = os.path.join(self.repos_path, repo.repo_name)
         log.info("Removing %s" % (rm_path))
         # disable hg/git internal that it doesn't get detected as repo
         alias = repo.repo_type
         bare = getattr(repo.scm_instance, 'bare', False)
         if not bare:
             # skip this for bare git repos
             shutil.move(os.path.join(rm_path, '.%s' % alias),
                         os.path.join(rm_path, 'rm__.%s' % alias))
         # disable repo
         _now = datetime.now()
         _ms = str(_now.microsecond).rjust(6, '0')
         _d = 'rm__%s__%s' % (_now.strftime('%Y%m%d_%H%M%S_' + _ms),
                              repo.just_name)
         if repo.group:
             args = repo.group.full_path_splitted + [_d]
             _d = os.path.join(*args)
         shutil.move(rm_path, os.path.join(self.repos_path, _d))

0 comments (0 inline, 0 general)