kallithea Changeset - 7ac09514a178

Changeset - 7ac09514a178

Parent rev.

Child rev.

[Not reviewed]

beta

1 7 3

Marcin Kuzminski - 13 years ago 2012-06-03 20:35:13
marcin@python-works.com

created rhodecode-api binary script for working with api via cli
- created docs
- moved the backup script to bin folder

9 files changed with 293 insertions and 18 deletions:

.hgignore

docs/api/api.rst

docs/changelog.rst

rhodecode/bin/__init__.py

new file

rhodecode/bin/rhodecode_api.py

216

rhodecode/bin/rhodecode_backup.py

rhodecode/controllers/api/__init__.py

rhodecode/controllers/api/api.py

setup.py

0 comments (0 inline, 0 general)

.hgignore

➞

Show inline comments

@@ @@ -10,12 +10,13 @@ syntax: regexp @@
 ^docs/build/
 ^docs/_build/
 ^data$
 ^\.settings$
 ^\.project$
 ^\.pydevproject$
 ^\.coverage$
 ^rhodecode\.db$
 ^test\.db$
 ^RhodeCode\.egg-info$
 ^rc\.ini$
 ^fabfile.py
 ^\.rhodecode$

docs/api/api.rst

➞

Show inline comments

@@ @@ -50,24 +50,65 @@ Simply provide @@
 RhodeCode API will return always a JSON-RPC response::
+    {
         "id":<id>, # matching id sent by request
         "result": "<result>"|null, # JSON formatted result, null if any errors
         "error": "null"|<error_message> # JSON formatted error (if any)
+    }
 All responses from API will be `HTTP/1.0 200 OK`, if there's an error while
 calling api *error* key from response will contain failure description
 and result will be null.
 API CLIENT
 ++++++++++
 From version 1.4 RhodeCode adds a binary script that allows to easily
 communicate with API. After installing RhodeCode a `rhodecode-api` script
 will be available.
 To get started quickly simply run::
   rhodecode-api _create_config --apikey=<youapikey> --apihost=<rhodecode host>
 This will create a file named .config in the directory you executed it storing
 json config file with credentials. You can skip this step and always provide
 both of the arguments to be able to communicate with server
 after that simply run any api command for example get_repo::
  rhodecode-api get_repo
  calling {"api_key": "<apikey>", "id": 75, "args": {}, "method": "get_repo"} to http://127.0.0.1:5000
  rhodecode said:
  {'error': 'Missing non optional `repoid` arg in JSON DATA',
   'id': 75,
   'result': None}
 Ups looks like we forgot to add an argument
 Let's try again now giving the repoid as parameters::
     rhodecode-api get_repo repoid:rhodecode
     calling {"api_key": "<apikey>", "id": 39, "args": {"repoid": "rhodecode"}, "method": "get_repo"} to http://127.0.0.1:5000
     rhodecode said:
     {'error': None,
      'id': 39,
      'result': <json data...>}
 API METHODS
 +++++++++++
 pull
 ----
 Pulls given repo from remote location. Can be used to automatically keep
 remote repos up to date. This command can be executed only using api_key
 belonging to user with admin rights
 INPUT::

docs/changelog.rst

➞

Show inline comments

@@ @@ -13,24 +13,25 @@ Changelog @@
 news
 ++++
 - new codereview system
 - email map, allowing users to have multiple email addresses mapped into
   their accounts
 - changed setup-app into setup-rhodecode and added default options to it.
 - new git repos are created as bare now by default
 - #464 added links to groups in permission box
 - #465 mentions autocomplete inside comments boxes
 - #469 added --update-only option to whoosh to re-index only given list
   of repos in index
 - rhodecode-api CLI client
 fixes
 +++++
 - improved translations
 - fixes issue #455 Creating an archive generates an exception on Windows
 - fixes #448 Download ZIP archive keeps file in /tmp open and results
   in out of disk space
 - fixes issue #454 Search results under Windows include proceeding
   backslash
 - fixed issue #450. Rhodecode no longer will crash when bad revision is
   present in journal data.

rhodecode/bin/__init__.py

➞

Show inline comments

new file 100644

rhodecode/bin/rhodecode_api.py

➞

Show inline comments

@@ new file 100755 @@
 # -*- coding: utf-8 -*-
 """
     rhodecode.bin.backup_manager
     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     Api CLI client for RhodeCode
     :created_on: Jun 3, 2012
     :author: marcink
     :copyright: (C) 2010-2012 Marcin Kuzminski <marcin@python-works.com>
     :license: GPLv3, see COPYING for more details.
 """
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation, either version 3 of the License, or
 # (at your option) any later version.
+#
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
+#
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 from __future__ import with_statement
 import os
 import sys
 import random
 import urllib2
 import pprint
 import argparse
 try:
     from rhodecode.lib.ext_json import json
 except ImportError:
     try:
         import simplejson as json
     except ImportError:
         import json
 CONFIG_NAME = '.rhodecode'
 class RcConf(object):
     """
     RhodeCode config for API
     conf = RcConf()
     conf['key']
     """
     def __init__(self, autoload=True, autocreate=False, config=None):
         self._conf_name = CONFIG_NAME
         self._conf = {}
         if autocreate:
             self.make_config(config)
         if autoload:
             self._conf = self.load_config()
     def __getitem__(self, key):
         return self._conf[key]
     def __nonzero__(self):
         if self._conf:
             return True
         return False
     def __eq__(self):
         return self._conf.__eq__()
     def __repr__(self):
         return 'RcConf<%s>' % self._conf.__repr__()
     def make_config(self, config):
         """
         Saves given config as a JSON dump in the _conf_name location
         :param config:
         :type config:
         """
         with open(self._conf_name, 'wb') as f:
             json.dump(config, f, indent=4)
             sys.stdout.write('Updated conf\n')
     def update_config(self, new_config):
         """
         Reads the JSON config updates it's values with new_config and
         saves it back as JSON dump
         :param new_config:
         """
         config = {}
         try:
             with open(self._conf_name, 'rb') as conf:
                 config = json.load(conf)
         except IOError, e:
             sys.stderr.write(str(e) + '\n')
         config.update(new_config)
         self.make_config(config)
     def load_config(self):
         """
         Loads config from file and returns loaded JSON object
         """
         try:
             with open(self._conf_name, 'rb') as conf:
                 return  json.load(conf)
         except IOError, e:
             #sys.stderr.write(str(e) + '\n')
             pass
 def api_call(apikey, apihost, method=None, **kw):
     """
     Api_call wrapper for RhodeCode
     :param apikey:
     :param apihost:
     :param method:
     """
     def _build_data(random_id):
         """
         Builds API data with given random ID
         :param random_id:
         :type random_id:
         """
         return {
             "id": random_id,
             "api_key": apikey,
             "method": method,
             "args": kw
+        }
     if not method:
         raise Exception('please specify method name !')
     id_ = random.randrange(1, 200)
     req = urllib2.Request('%s/_admin/api' % apihost,
                       data=json.dumps(_build_data(id_)),
                       headers={'content-type': 'text/plain'})
     print 'calling %s to %s' % (req.get_data(), apihost)
     ret = urllib2.urlopen(req)
     json_data = json.loads(ret.read())
     id_ret = json_data['id']
     _formatted_json = pprint.pformat(json_data)
     if id_ret == id_:
         print 'rhodecode said:\n%s' % (_formatted_json)
     else:
         raise Exception('something went wrong. '
                         'ID mismatch got %s, expected %s | %s' % (
                                             id_ret, id_, _formatted_json))
 def argparser(argv):
     usage = ("rhodecode_api [-h] [--apikey APIKEY] [--apihost APIHOST] "
              "_create_config or METHOD <key:val> <key2:val> ...")
     parser = argparse.ArgumentParser(description='RhodeCode API cli',
                                      usage=usage)
     ## config
     group = parser.add_argument_group('config')
     group.add_argument('--apikey', help='api access key')
     group.add_argument('--apihost', help='api host')
     group = parser.add_argument_group('API')
     group.add_argument('method', metavar='METHOD', type=str,
             help='API method name to call followed by key:value attributes',
+    )
     args, other = parser.parse_known_args()
     return parser, args, other
 def main(argv=None):
     """
     Main execution function for cli
     :param argv:
     :type argv:
     """
     if argv is None:
         argv = sys.argv
     conf = None
     parser, args, other = argparser(argv)
     api_credentials_given = (args.apikey and args.apihost)
     if args.method == '_create_config':
         if not api_credentials_given:
             raise parser.error('_create_config requires --apikey and --apihost')
         conf = RcConf(autocreate=True, config={'apikey': args.apikey,
                                                'apihost': args.apihost})
         sys.stdout.write('Create new config in %s\n' % CONFIG_NAME)
     if not conf:
         conf = RcConf(autoload=True)
         if not conf:
             if not api_credentials_given:
                 parser.error('Could not find config file and missing '
                              '--apikey or --apihost in params')
     apikey = args.apikey or conf['apikey']
     host = args.apihost or conf['apihost']
     method = args.method
     margs = dict(map(lambda s: s.split(':', 1), other))
     api_call(apikey, host, method, **margs)
     return 0
 if __name__ == '__main__':
     sys.exit(main(sys.argv))

rhodecode/bin/rhodecode_backup.py

➞

Show inline comments

@@ modified file chmod 100644 => 100755 @@
 # -*- coding: utf-8 -*-
 """
-    rhodecode.lib.backup_manager
+    rhodecode.bin.backup_manager
     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-    Mercurial repositories backup manager, it allows to backups all
+    Repositories backup manager, it allows to backups all
     repositories and send it to backup server using RSA key via ssh.
     :created_on: Feb 28, 2010
     :author: marcink
     :copyright: (C) 2010-2012 Marcin Kuzminski <marcin@python-works.com>
     :license: GPLv3, see COPYING for more details.
 """
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation, either version 3 of the License, or
 # (at your option) any later version.
+#
@@ @@ -30,25 +30,25 @@ import sys @@
 import logging
 import tarfile
 import datetime
 import subprocess
 logging.basicConfig(level=logging.DEBUG,
                     format="%(asctime)s %(levelname)-5.5s %(message)s")
 class BackupManager(object):
     def __init__(self, repos_location, rsa_key, backup_server):
         today = datetime.datetime.now().weekday() + 1
-        self.backup_file_name = "mercurial_repos.%s.tar.gz" % today
+        self.backup_file_name = "rhodecode_repos.%s.tar.gz" % today
         self.id_rsa_path = self.get_id_rsa(rsa_key)
         self.repos_path = self.get_repos_path(repos_location)
         self.backup_server = backup_server
         self.backup_file_path = '/tmp'
         logging.info('starting backup for %s', self.repos_path)
         logging.info('backup target %s', self.backup_file_path)
     def get_id_rsa(self, rsa_key):
         if not os.path.isfile(rsa_key):

rhodecode/controllers/api/__init__.py

➞

Show inline comments

@@ @@ -48,33 +48,34 @@ log = logging.getLogger('JSONRPC') @@
 class JSONRPCError(BaseException):
     def __init__(self, message):
         self.message = message
         super(JSONRPCError, self).__init__()
     def __str__(self):
         return str(self.message)
 def jsonrpc_error(message, code=None):
+def jsonrpc_error(message, retid=None, code=None):
     """
     Generate a Response object with a JSON-RPC error body
     """
     from pylons.controllers.util import Response
     resp = Response(body=json.dumps(dict(id=None, result=None, error=message)),
     return Response(
             body=json.dumps(dict(id=retid, result=None, error=message)),
                     status=code,
                     content_type='application/json')
     return resp
             content_type='application/json'
+    )
 class JSONRPCController(WSGIController):
     """
      A WSGI-speaking JSON-RPC controller class
      See the specification:
      <http://json-rpc.org/wiki/specification>`.
      Valid controller return values should be json-serializable objects.
      Sub-classes should catch their exceptions and raise JSONRPCError
@@ @@ -85,109 +86,121 @@ class JSONRPCController(WSGIController): @@
     def _get_method_args(self):
         """
         Return `self._rpc_args` to dispatched controller method
         chosen by __call__
         """
         return self._rpc_args
     def __call__(self, environ, start_response):
         """
         Parse the request body as JSON, look up the method on the
         controller and if it exists, dispatch to it.
         """
         self._req_id = None
         if 'CONTENT_LENGTH' not in environ:
             log.debug("No Content-Length")
             return jsonrpc_error(message="No Content-Length in request")
             return jsonrpc_error(retid=self._req_id,
                                  message="No Content-Length in request")
         else:
             length = environ['CONTENT_LENGTH'] or 0
             length = int(environ['CONTENT_LENGTH'])
             log.debug('Content-Length: %s' % length)
         if length == 0:
             log.debug("Content-Length is 0")
             return jsonrpc_error(message="Content-Length is 0")
             return jsonrpc_error(retid=self._req_id,
                                  message="Content-Length is 0")
         raw_body = environ['wsgi.input'].read(length)
         try:
             json_body = json.loads(urllib.unquote_plus(raw_body))
         except ValueError, e:
             # catch JSON errors Here
             return jsonrpc_error(message="JSON parse error ERR:%s RAW:%r" \
             return jsonrpc_error(retid=self._req_id,
                                  message="JSON parse error ERR:%s RAW:%r" \
                                  % (e, urllib.unquote_plus(raw_body)))
         # check AUTH based on API KEY
         try:
             self._req_api_key = json_body['api_key']
             self._req_id = json_body['id']
             self._req_method = json_body['method']
             self._request_params = json_body['args']
             log.debug(
                 'method: %s, params: %s' % (self._req_method,
                                             self._request_params)
+            )
         except KeyError, e:
             return jsonrpc_error(message='Incorrect JSON query missing %s' % e)
             return jsonrpc_error(retid=self._req_id,
                                  message='Incorrect JSON query missing %s' % e)
         # check if we can find this session using api_key
         try:
             u = User.get_by_api_key(self._req_api_key)
             if u is None:
                 return jsonrpc_error(message='Invalid API KEY')
                 return jsonrpc_error(retid=self._req_id,
                                      message='Invalid API KEY')
             auth_u = AuthUser(u.user_id, self._req_api_key)
         except Exception, e:
             return jsonrpc_error(message='Invalid API KEY')
             return jsonrpc_error(retid=self._req_id,
                                  message='Invalid API KEY')
         self._error = None
         try:
             self._func = self._find_method()
         except AttributeError, e:
             return jsonrpc_error(message=str(e))
             return jsonrpc_error(retid=self._req_id,
                                  message=str(e))
         # now that we have a method, add self._req_params to
         # self.kargs and dispatch control to WGIController
         argspec = inspect.getargspec(self._func)
         arglist = argspec[0][1:]
         defaults = map(type, argspec[3] or [])
         default_empty = types.NotImplementedType
         # kw arguments required by this method
         func_kwargs = dict(izip_longest(reversed(arglist), reversed(defaults),
                                         fillvalue=default_empty))
         # this is little trick to inject logged in user for
         # perms decorators to work they expect the controller class to have
         # rhodecode_user attribute set
         self.rhodecode_user = auth_u
         # This attribute will need to be first param of a method that uses
         # api_key, which is translated to instance of user at that name
         USER_SESSION_ATTR = 'apiuser'
         if USER_SESSION_ATTR not in arglist:
             return jsonrpc_error(message='This method [%s] does not support '
                                  'authentication (missing %s param)' %
                                  (self._func.__name__, USER_SESSION_ATTR))
             return jsonrpc_error(
                 retid=self._req_id,
                 message='This method [%s] does not support '
                          'authentication (missing %s param)' % (
                                     self._func.__name__, USER_SESSION_ATTR)
+            )
         # get our arglist and check if we provided them as args
         for arg, default in func_kwargs.iteritems():
             if arg == USER_SESSION_ATTR:
                 # USER_SESSION_ATTR is something translated from api key and
                 # this is checked before so we don't need validate it
                 continue
             # skip the required param check if it's default value is
             # NotImplementedType (default_empty)
             if (default == default_empty and arg not in self._request_params):
                 return jsonrpc_error(
                     retid=self._req_id,
                     message=(
                         'Missing non optional `%s` arg in JSON DATA' % arg
+                    )
+                )
         self._rpc_args = {USER_SESSION_ATTR: u}
         self._rpc_args.update(self._request_params)
         self._rpc_args['action'] = self._req_method
         self._rpc_args['environ'] = environ
         self._rpc_args['start_response'] = start_response

rhodecode/controllers/api/api.py

➞

Show inline comments

@@ @@ -380,25 +380,25 @@ class ApiController(JSONRPCController): @@
     @HasPermissionAnyDecorator('hg.admin')
     def get_repo(self, apiuser, repoid):
         """"
         Get repository by name
         :param apiuser:
         :param repo_name:
         """
         repo = RepoModel().get_repo(repoid)
         if repo is None:
             raise JSONRPCError('unknown repository %s' % repo)
+            raise JSONRPCError('unknown repository "%s"' % (repo or repoid))
         members = []
         for user in repo.repo_to_perm:
             perm = user.permission.permission_name
             user = user.user
             members.append(
                 dict(
                     type="user",
                     id=user.user_id,
                     username=user.username,
                     firstname=user.name,
                     lastname=user.lastname,

setup.py

➞

Show inline comments

@@ @@ -78,24 +78,27 @@ setup( @@
     packages=packages,
     include_package_data=True,
     test_suite='nose.collector',
     package_data=package_data,
     message_extractors={'rhodecode': [
             ('**.py', 'python', None),
             ('templates/**.mako', 'mako', {'input_encoding': 'utf-8'}),
             ('templates/**.html', 'mako', {'input_encoding': 'utf-8'}),
             ('public/**', 'ignore', None)]},
     zip_safe=False,
     paster_plugins=['PasteScript', 'Pylons'],
     entry_points="""
     [console_scripts]
     rhodecode-api =  rhodecode.bin.rhodecode_api:main
     [paste.app_factory]
     main = rhodecode.config.middleware:make_app
     [paste.app_install]
     main = pylons.util:PylonsInstaller
     [paste.global_paster_command]
     setup-rhodecode=rhodecode.config.setup_rhodecode:SetupCommand
     make-index=rhodecode.lib.indexers:MakeIndex
     make-rcext=rhodecode.config.rcextensions.make_rcextensions:MakeRcExt
     upgrade-db=rhodecode.lib.dbmigrate:UpgradeDb
     celeryd=rhodecode.lib.celerypylons.commands:CeleryDaemonCommand

0 comments (0 inline, 0 general)