.. _api:

===

API

===

Starting from RhodeCode version 1.2 a simple API was implemented.

There's a single schema for calling all api methods. API is implemented

with JSON protocol both ways. An url to send API request to RhodeCode is

<your_server>/_admin/api

API ACCESS FOR WEB VIEWS

++++++++++++++++++++++++

API access can also be turned on for each web view in RhodeCode that is 

decorated with `@LoginRequired` decorator. To enable API access simple change 

the standard login decorator to `@LoginRequired(api_access=True)`. 

After this change, a rhodecode view can be accessed without login by adding a 

GET parameter `?api_key=<api_key>` to url. By default this is only

enabled on RSS/ATOM feed views.

API ACCESS

++++++++++

All clients are required to send JSON-RPC spec JSON data::

    {   

        "id:"<id>",

        "api_key":"<api_key>",

        "method":"<method_name>",

        "args":{"<arg_key>":"<arg_val>"}

    }

Example call for autopulling remotes repos using curl::

    curl https://server.com/_admin/api -X POST -H 'content-type:text/plain' --data-binary '{"id":1,"api_key":"xe7cdb2v278e4evbdf5vs04v832v0efvcbcve4a3","method":"pull","args":{"repo":"CPython"}}'

Simply provide

 - *id* A value of any type, which is used to match the response with the request that it is replying to.

 - *api_key* for access and permission validation.

 - *method* is name of method to call

 - *args* is an key:value list of arguments to pass to method

.. note::

    api_key can be found in your user account page

RhodeCode API will return always a JSON-RPC response::

    {   

        "id":<id>, # matching id sent by request

        "result": "<result>"|null, # JSON formatted result, null if any errors

        "error": "null"|<error_message> # JSON formatted error (if any)

    }

All responses from API will be `HTTP/1.0 200 OK`, if there's an error while

calling api *error* key from response will contain failure description

and result will be null.

API CLIENT

++++++++++

From version 1.4 RhodeCode adds a script that allows to easily

communicate with API. After installing RhodeCode a `rhodecode-api` script

will be available.

To get started quickly simply run::

  rhodecode-api _create_config --apikey=<youapikey> --apihost=<rhodecode host>

This will create a file named .config in the directory you executed it storing

json config file with credentials. You can skip this step and always provide

both of the arguments to be able to communicate with server

after that simply run any api command for example get_repo::

 rhodecode-api get_repo

 calling {"api_key": "<apikey>", "id": 75, "args": {}, "method": "get_repo"} to http://127.0.0.1:5000

 rhodecode said:

 {'error': 'Missing non optional `repoid` arg in JSON DATA',

  'id': 75,

  'result': None}

Ups looks like we forgot to add an argument

Let's try again now giving the repoid as parameters::

    rhodecode-api get_repo repoid:rhodecode   

    calling {"api_key": "<apikey>", "id": 39, "args": {"repoid": "rhodecode"}, "method": "get_repo"} to http://127.0.0.1:5000

    rhodecode said:

    {'error': None,

     'id': 39,

     'result': <json data...>}

API METHODS

+++++++++++

pull

----

Pulls given repo from remote location. Can be used to automatically keep

remote repos up to date. This command can be executed only using api_key

belonging to user with admin rights

INPUT::

    id : <id_for_response>

    api_key : "<api_key>"

    method :  "pull"

    args :    {

                "repoid" : "<reponame or repo_id>"

              }

OUTPUT::

    id : <id_given_in_input>

    result : "Pulled from `<reponame>`"

    error :  null

rescan_repos

------------

Dispatch rescan repositories action. If remove_obsolete is set

RhodeCode will delete repos that are in database but not in the filesystem.

This command can be executed only using api_key belonging to user with admin 

rights.

INPUT::

    id : <id_for_response>

    api_key : "<api_key>"

    method :  "rescan_repos"

    args :    {

                "remove_obsolete" : "<boolean = Optional(False)>"

              }

OUTPUT::

    id : <id_given_in_input>

    result : "{'added': [<list of names of added repos>], 

               'removed': [<list of names of removed repos>]}"

    error :  null

lock

----

Set locking state on given repository by given user.

This command can be executed only using api_key belonging to user with admin 

rights.

INPUT::

    id : <id_for_response>

    api_key : "<api_key>"

    method :  "lock"

    args :    {

                "repoid" : "<reponame or repo_id>"

                "userid" : "<user_id or username>",

                "locked" : "<bool true|false>"

              }

OUTPUT::

    id : <id_given_in_input>

    result : "User `<username>` set lock state for repo `<reponame>` to `true|false`"

    error :  null

get_user

--------

Get's an user by username or user_id, Returns empty result if user is not found.

This command can be executed only using api_key belonging to user with admin 

rights.

INPUT::

    id : <id_for_response>

    api_key : "<api_key>"

    method :  "get_user"

    args :    { 

                "userid" : "<username or user_id>"

              }

OUTPUT::

    id : <id_given_in_input>

    result: None if user does not exist or 

            {

                "user_id" :  "<user_id>",

                "username" : "<username>",

                "firstname": "<firstname>",

                "lastname" : "<lastname>",

                "email" :    "<email>",

                "emails":    "<list_of_all_additional_emails>",

                "active" :   "<bool>",

                "admin" :    "<bool>",

                "ldap_dn" :  "<ldap_dn>",

                "last_login": "<last_login>",

                "permissions": {

                    "global": ["hg.create.repository",

                               "repository.read",

                               "hg.register.manual_activate"],

                    "repositories": {"repo1": "repository.none"},

                    "repositories_groups": {"Group1": "group.read"}

                 },

            }

    error:  null

get_users

---------

Lists all existing users. This command can be executed only using api_key

belonging to user with admin rights.

INPUT::

    id : <id_for_response>

    api_key : "<api_key>"

    method :  "get_users"

    args :    { }

OUTPUT::

    id : <id_given_in_input>

    result: [

              {

                "user_id" :  "<user_id>",

                "username" : "<username>",

                "firstname": "<firstname>",

                "lastname" : "<lastname>",

                "email" :    "<email>",

                "emails":    "<list_of_all_additional_emails>",

                "active" :   "<bool>",

                "admin" :    "<bool>",

                "ldap_dn" :  "<ldap_dn>",

                "last_login": "<last_login>",

              },

    	      …

            ]

    error:  null

create_user

-----------

Creates new user. This command can 

be executed only using api_key belonging to user with admin rights.

INPUT::

    id : <id_for_response>

    api_key : "<api_key>"

    method :  "create_user"

    args :    {

                "username" :  "<username>",

                "email" :     "<useremail>",

                "password" :  "<password>",

                "firstname" : "<firstname> = Optional(None)",

                "lastname" :  "<lastname> = Optional(None)",

                "active" :    "<bool> = Optional(True)",

                "admin" :     "<bool> = Optional(False)",

                "ldap_dn" :   "<ldap_dn> = Optional(None)"

              }

OUTPUT::

    id : <id_given_in_input>

    result: {

              "msg" : "created new user `<username>`",

              "user": {

                "user_id" :  "<user_id>",

                "username" : "<username>",

                "firstname": "<firstname>",

                "lastname" : "<lastname>",

                "email" :    "<email>",

                "emails":    "<list_of_all_additional_emails>",

                "active" :   "<bool>",

                "admin" :    "<bool>",

                "ldap_dn" :  "<ldap_dn>",

                "last_login": "<last_login>",

              },

            }

    error:  null

update_user

-----------

updates given user if such user exists. This command can 

be executed only using api_key belonging to user with admin rights.

INPUT::

    id : <id_for_response>

    api_key : "<api_key>"

    method :  "update_user"

    args :    {

                "userid" : "<user_id or username>",

                "username" :  "<username> = Optional",

                "email" :     "<useremail> = Optional",

                "password" :  "<password> = Optional",

                "firstname" : "<firstname> = Optional",

                "lastname" :  "<lastname> = Optional",

                "active" :    "<bool> = Optional",

                "admin" :     "<bool> = Optional",

                "ldap_dn" :   "<ldap_dn> = Optional"

              }

OUTPUT::

    id : <id_given_in_input>

    result: {

              "msg" : "updated user ID:<userid> <username>",

              "user": {

                "user_id" :  "<user_id>",

                "username" : "<username>",

                "firstname": "<firstname>",

                "lastname" : "<lastname>",

                "email" :    "<email>",

                "emails":    "<list_of_all_additional_emails>",

                "active" :   "<bool>",

                "admin" :    "<bool>",

                "ldap_dn" :  "<ldap_dn>",

                "last_login": "<last_login>",

              },              

            }

    error:  null

delete_user

-----------

deletes givenuser if such user exists. This command can 

be executed only using api_key belonging to user with admin rights.

INPUT::

    id : <id_for_response>

    api_key : "<api_key>"

    method :  "delete_user"

    args :    {

                "userid" : "<user_id or username>",

              }

OUTPUT::

    id : <id_given_in_input>

    result: {

              "msg" : "deleted user ID:<userid> <username>",

              "user": null

            }

# -*- coding: utf-8 -*-

"""

    rhodecode.controllers.api

    ~~~~~~~~~~~~~~~~~~~~~~~~~

    JSON RPC controller

    :created_on: Aug 20, 2011

    :author: marcink

    :copyright: (C) 2011-2012 Marcin Kuzminski <marcin@python-works.com>

    :license: GPLv3, see COPYING for more details.

"""

# This program is free software; you can redistribute it and/or

# modify it under the terms of the GNU General Public License

# as published by the Free Software Foundation; version 2

# of the License or (at your opinion) any later version of the license.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program; if not, write to the Free Software

# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,

# MA  02110-1301, USA.

import inspect

import logging

import types

import urllib

import traceback

import time

from rhodecode.lib.compat import izip_longest, json

from paste.response import replace_header

from pylons.controllers import WSGIController

from webob.exc import HTTPNotFound, HTTPForbidden, HTTPInternalServerError, \

HTTPBadRequest, HTTPError

from rhodecode.model.db import User

from rhodecode.lib.auth import AuthUser, check_ip_access

from rhodecode.lib.base import _get_ip_addr, _get_access_path

from rhodecode.lib.utils2 import safe_unicode

log = logging.getLogger('JSONRPC')

class JSONRPCError(BaseException):

    def __init__(self, message):

        self.message = message

        super(JSONRPCError, self).__init__()

    def __str__(self):

        return str(self.message)

def jsonrpc_error(message, retid=None, code=None):

    """

    Generate a Response object with a JSON-RPC error body

    """

    from pylons.controllers.util import Response

    return Response(

            body=json.dumps(dict(id=retid, result=None, error=message)),

            status=code,

            content_type='application/json'

    )

class JSONRPCController(WSGIController):

    """

     A WSGI-speaking JSON-RPC controller class

     See the specification:

     <http://json-rpc.org/wiki/specification>`.

     Valid controller return values should be json-serializable objects.

     Sub-classes should catch their exceptions and raise JSONRPCError

     if they want to pass meaningful errors to the client.

     """

    def _get_method_args(self):

        """

        Return `self._rpc_args` to dispatched controller method

        chosen by __call__

        """

        return self._rpc_args

    def __call__(self, environ, start_response):

        """

        Parse the request body as JSON, look up the method on the

        controller and if it exists, dispatch to it.

        """

        start = time.time()

        self._req_id = None

        if 'CONTENT_LENGTH' not in environ:

            log.debug("No Content-Length")

            return jsonrpc_error(retid=self._req_id,

                                 message="No Content-Length in request")

        else:

            length = environ['CONTENT_LENGTH'] or 0

            length = int(environ['CONTENT_LENGTH'])

            log.debug('Content-Length: %s' % length)

        if length == 0:

            log.debug("Content-Length is 0")

            return jsonrpc_error(retid=self._req_id,

                                 message="Content-Length is 0")

        raw_body = environ['wsgi.input'].read(length)

        try:

            json_body = json.loads(urllib.unquote_plus(raw_body))

        except ValueError, e:

            # catch JSON errors Here

            return jsonrpc_error(retid=self._req_id,

                                 message="JSON parse error ERR:%s RAW:%r" \

                                 % (e, urllib.unquote_plus(raw_body)))

        # check AUTH based on API KEY

        try:

            self._req_api_key = json_body['api_key']

            self._req_id = json_body['id']

            self._req_method = json_body['method']

            self._request_params = json_body['args']

            log.debug(

                'method: %s, params: %s' % (self._req_method,

                                            self._request_params)

            )

        except KeyError, e:

            return jsonrpc_error(retid=self._req_id,

                                 message='Incorrect JSON query missing %s' % e)

        # check if we can find this session using api_key

        try:

            u = User.get_by_api_key(self._req_api_key)

            if u is None:

                return jsonrpc_error(retid=self._req_id,

                                     message='Invalid API KEY')

            #check if we are allowed to use this IP

            allowed_ips = AuthUser.get_allowed_ips(u.user_id)

            if check_ip_access(source_ip=ip_addr, allowed_ips=allowed_ips) is False:

                log.info('Access for IP:%s forbidden, '

                         'not in %s' % (ip_addr, allowed_ips))

                return jsonrpc_error(retid=self._req_id,

                        message='request from IP:%s not allowed' % (ip_addr))

            else:

                log.info('Access for IP:%s allowed' % (ip_addr))

            auth_u = AuthUser(u.user_id, self._req_api_key, ip_addr=ip_addr)

        except Exception, e:

            return jsonrpc_error(retid=self._req_id,

                                 message='Invalid API KEY')

        self._error = None

        try:

            self._func = self._find_method()

        except AttributeError, e:

            return jsonrpc_error(retid=self._req_id,

                                 message=str(e))

        # now that we have a method, add self._req_params to

        # self.kargs and dispatch control to WGIController

        argspec = inspect.getargspec(self._func)

        arglist = argspec[0][1:]

        defaults = map(type, argspec[3] or [])

        default_empty = types.NotImplementedType

        # kw arguments required by this method

        func_kwargs = dict(izip_longest(reversed(arglist), reversed(defaults),

                                        fillvalue=default_empty))

        # this is little trick to inject logged in user for

        # perms decorators to work they expect the controller class to have

        # rhodecode_user attribute set

        self.rhodecode_user = auth_u

        # This attribute will need to be first param of a method that uses

        # api_key, which is translated to instance of user at that name

        USER_SESSION_ATTR = 'apiuser'

        if USER_SESSION_ATTR not in arglist:

            return jsonrpc_error(

                retid=self._req_id,

                message='This method [%s] does not support '

                         'authentication (missing %s param)' % (

                                    self._func.__name__, USER_SESSION_ATTR)

            )

        # get our arglist and check if we provided them as args

        for arg, default in func_kwargs.iteritems():

            if arg == USER_SESSION_ATTR:

                # USER_SESSION_ATTR is something translated from api key and

                # this is checked before so we don't need validate it

                continue

            # skip the required param check if it's default value is

            # NotImplementedType (default_empty)

            if (default == default_empty and arg not in self._request_params):

                return jsonrpc_error(

                    retid=self._req_id,

                    message=(

                        'Missing non optional `%s` arg in JSON DATA' % arg

                    )

                )

        self._rpc_args = {USER_SESSION_ATTR: u}

        self._rpc_args.update(self._request_params)

        self._rpc_args['action'] = self._req_method

        self._rpc_args['environ'] = environ

        self._rpc_args['start_response'] = start_response

        status = []

        headers = []

        exc_info = []

        def change_content(new_status, new_headers, new_exc_info=None):

            status.append(new_status)

            headers.extend(new_headers)

            exc_info.append(new_exc_info)

        output = WSGIController.__call__(self, environ, change_content)

        output = list(output)

        headers.append(('Content-Length', str(len(output[0]))))

        replace_header(headers, 'Content-Type', 'application/json')

        start_response(status[0], headers, exc_info[0])

        log.info('IP: %s Request to %s time: %.3fs' % (

            _get_ip_addr(environ),

            safe_unicode(_get_access_path(environ)), time.time() - start)

        )

        return output

    def _dispatch_call(self):

        """

        Implement dispatch interface specified by WSGIController

        """

        try:

            raw_response = self._inspect_call(self._func)

            if isinstance(raw_response, HTTPError):

                self._error = str(raw_response)

        except JSONRPCError, e:

            self._error = str(e)

        except Exception, e:

            log.error('Encountered unhandled exception: %s' \

                      % traceback.format_exc())

            json_exc = JSONRPCError('Internal server error')

            self._error = str(json_exc)

        if self._error is not None:

            raw_response = None

        response = dict(id=self._req_id, result=raw_response,

                        error=self._error)

        try:

            return json.dumps(response)

        except TypeError, e:

            log.error('API FAILED. Error encoding response: %s' % e)

            return json.dumps(

                dict(

                    id=self._req_id,

                    result=None,

                    error="Error encoding response"

                )

            )

    def _find_method(self):

        """

        Return method named by `self._req_method` in controller if able

        """

        log.debug('Trying to find JSON-RPC method: %s' % self._req_method)

        if self._req_method.startswith('_'):

            raise AttributeError("Method not allowed")

        try:

            func = getattr(self, self._req_method, None)

        except UnicodeEncodeError:

            raise AttributeError("Problem decoding unicode in requested "

                                 "method name.")

        if isinstance(func, types.MethodType):

            return func

        else:

            raise AttributeError("No such method: %s" % self._req_method)

# -*- coding: utf-8 -*-

"""

    rhodecode.controllers.api

    ~~~~~~~~~~~~~~~~~~~~~~~~~

    API controller for RhodeCode

    :created_on: Aug 20, 2011

    :author: marcink

    :copyright: (C) 2011-2012 Marcin Kuzminski <marcin@python-works.com>

    :license: GPLv3, see COPYING for more details.

"""

# This program is free software; you can redistribute it and/or

# modify it under the terms of the GNU General Public License

# as published by the Free Software Foundation; version 2

# of the License or (at your opinion) any later version of the license.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program; if not, write to the Free Software

# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,

# MA  02110-1301, USA.

import traceback

import logging

from rhodecode.controllers.api import JSONRPCController, JSONRPCError

from rhodecode.lib.auth import HasPermissionAllDecorator, \

    HasPermissionAnyDecorator, PasswordGenerator, AuthUser

from rhodecode.lib.utils import map_groups, repo2db_mapper

from rhodecode.model.meta import Session

from rhodecode.model.scm import ScmModel

from rhodecode.model.repo import RepoModel

from rhodecode.model.user import UserModel

from rhodecode.model.users_group import UsersGroupModel

from rhodecode.model.permission import PermissionModel

log = logging.getLogger(__name__)

class Optional(object):

    """

    Defines an optional parameter::

        param = param.getval() if isinstance(param, Optional) else param

        param = param() if isinstance(param, Optional) else param

    is equivalent of::

        param = Optional.extract(param)

    """

    def __init__(self, type_):

        self.type_ = type_

    def __repr__(self):

        return '<Optional:%s>' % self.type_.__repr__()

    def __call__(self):

        return self.getval()

    def getval(self):

        """

        returns value from this Optional instance

        """

        return self.type_

    @classmethod

    def extract(cls, val):

        if isinstance(val, cls):

            return val.getval()

        return val

def get_user_or_error(userid):

    """

    Get user by id or name or return JsonRPCError if not found

    :param userid:

    """

    user = UserModel().get_user(userid)

    if user is None:

        raise JSONRPCError("user `%s` does not exist" % userid)

    return user

def get_repo_or_error(repoid):

    """

    Get repo by id or name or return JsonRPCError if not found

    :param userid:

    """

    repo = RepoModel().get_repo(repoid)

    if repo is None:

        raise JSONRPCError('repository `%s` does not exist' % (repoid))

    return repo

def get_users_group_or_error(usersgroupid):

    """

    Get users group by id or name or return JsonRPCError if not found

    :param userid:

    """

    users_group = UsersGroupModel().get_group(usersgroupid)

    if users_group is None:

        raise JSONRPCError('users group `%s` does not exist' % usersgroupid)

    return users_group

def get_perm_or_error(permid):

    """

    Get permission by id or name or return JsonRPCError if not found

    :param userid:

    """

    perm = PermissionModel().get_permission_by_name(permid)

    if perm is None:

        raise JSONRPCError('permission `%s` does not exist' % (permid))

    return perm

class ApiController(JSONRPCController):

    """

    API Controller

    Each method needs to have USER as argument this is then based on given

    API_KEY propagated as instance of user object

    Preferably this should be first argument also

    Each function should also **raise** JSONRPCError for any

    errors that happens

    """

    @HasPermissionAllDecorator('hg.admin')

    def pull(self, apiuser, repoid):

        """

        Dispatch pull action on given repo

        :param apiuser:

        :param repoid:

        """

        repo = get_repo_or_error(repoid)

        try:

            ScmModel().pull_changes(repo.repo_name,

                                    self.rhodecode_user.username)

            return 'Pulled from `%s`' % repo.repo_name

        except Exception:

            log.error(traceback.format_exc())